Shaykh ‘Abd al-‘Azeez ibn Baaz (may Allaah have mercy on him) was asked a similar question and he replied:
What you mention is not a valid shar’i excuse for you to delay praying with the congregation. Rather what you must do is hasten to pray with your Muslim brothers in the houses of Allaah (the mosques), then rest and eat after that, because Allaah has made it obligatory for you to pray on time with your Muslim brothers in congregation. What you mention is not a legitimate shar’i excuse to delay it. Rather that is a trick of the Shaytaan and of the (human) self which is inclined towards evil; it is a sign of weak faith and a lack of fear of Allaah. So beware of your whims and desires and your shaytaan (devil), and your (human) self which is inclined towards evil; if you do that the consequences will be good and you will attain salvation and happiness in this world and in the Hereafter. May Allaah protect you from the evil of your own self and help you to avoid the traps of the Shaytaan.