Praise be to Allah.Praise be to Allaah.
My view is that even in cases of necessity, only Muslim servants should be brought in, because the Prophet (peace and blessings of Allaah be upon him) commanded that the Jews and Christians be expelled from the Arabian Peninsula. If a servant is brough in, she should not be young and pretty, because she is a source of temptation, especially if there are young men in the family. For the Shaytaan flows through the son of Adam like blood. And no female servant should be brought without her mahram, because the Prophet (peace and blessings of Allaah be upon him) forbade women to travel without their mahrams.
If the servant has a mahram, then the problem mentioned in the question does not arise; her mahram can go for Hajj with her. But if she does not have a mahram, or if her mahram brought her then went back, then they should not take her for Hajj or travel with her; rather she should stay with someone who can be trusted to take care of her. If there is no one who can be trusted to take care of her, then she may go for Hajj with them because of necessity, and her Hajj will be valid.
And Allaah is the Source of strength.