Praise be to Allah.
We put this question to Shaykh Muhammad ibn Saalih al-‘Uthaymeen, may Allaah preserve him, who answered as follows:
“This is not allowed, it is haraam.”
But the children need to be taught.
“He should bring them to the house of the wife whose day it is. And Allaah knows best.”