Yes, you may not fast on the day on which you know you will be returning to your family.
The scholars differed as to whether it is permissible for a traveler not to fast if he knows that he will come back the next day. The majority of scholars (including Imam Abu Haneefah, Imam Maalik and Imam al-Shaafa’i – may Allaah have mercy on them) were of the view that it is permissible for him not to fast, because he is traveling and so is included in the verse in which Allaah says (interpretation of the meaning):
“and whoever is ill or on a journey, the same number [of days which one did not observe Sawm (fasts) must be made up] from other days”
Imam Ahmad (may Allaah have mercy on him) was of the view that he has to fast.
Ibn Muflih said in al-Furoo’ (3/24):
If the traveler knows that he will be coming back the next day, then he has to fast… and it was said that it is mustahabb, in accordance with the three imams (Abu Haneefah, Maalik and al-Shaafa’i), because there is a reason for a concession (allowing him not to fast).
See also al-Insaaf by al-Mirdaawi, 7/362.
Shaykh Ibn ‘Uthaymeen said in al-Sharh al-Mumti’ (6/210):
If the traveler knows that he will be coming back the next day, then he has to refrain from eating and drinking etc. This is the view (of Imam Ahmad). The correct view is that he does not have to refrain from eating and drinking.
With regard to the time of breaking your fast, you can break the fast at any time you wish so long as you are traveling, until you go back to your city. If you go back fasting then you have to complete your fast and it is haraam for you to break your fast in that case, because you no longer come under the heading of a traveler.
See al-Majmoo’, 6/173.
If you go back not fasting, then there is a difference of scholarly opinion as to whether you are obliged to refrain from eating and drinking or not.
In the answer to question “49008” we have already stated that it is not obligatory to refrain from eating and drinking.