If your situation is as you describe, then you are not obliged to go for Hajj, because you are not able to do it as defined in sharee’ah. Allaah says (interpretation of the meaning):
“And Hajj (pilgrimage to Makkah) to the House (Ka‘bah) is a duty that mankind owes to Allaah, those who can afford the expenses (for one’s conveyance, provision and residence)”
[Aal ‘Imraan 3:97]
“So keep your duty to Allaah and fear Him as much as you can”
“and [Allaah] has not laid upon you in religion any hardship”
And Allaah is the Source of strength.