Praise be to Allah.
If this loan is based on riba (usury, interest) , then it is haram and is one of the major sins and one of the seven sins that doom a person to Hell. All of the nations regarded it as forbidden, even the Greeks who were idol-worshippers. One of them, whose name was Solon said: Money is like a sterile hen; a drachma cannot give birth to a drachma.
In the Christian belief it says that one who consumes usury should not be shrouded if he dies, and even the Jews forbade usury.
Islam has forbidden it in a manner that leaves no room for doubt concerning its prohibition.
Allah says (interpretation of the meaning):
“Allah has permitted trading and forbidden Riba. So whosoever receives an admonition from his Lord and stops consuming Riba, shall not be punished for the past; his case is for Allah (to judge); but whoever returns (to Riba), such are the dwellers of the Fire — they will abide therein.” [al-Baqarah 2:275]
“O you who believe! Fear Allah and give up what remains (due to you) from Riba (from now onward) if you are (really) believers.” [al-Baqarah 2:278]
Abu Juhayfah (may Allah be pleased with him) narrated that the Messenger of Allah (peace and blessings of Allah be upon him) forbade the price of blood, the price of a dog and the earnings of a prostitute, and he cursed the woman who does tattoos and those who have them done, the one who consumes riba (usury) and the one who pays it, and he cursed the one who makes images. (Narrated by al-Bukhari, 2123)
‘Abd-Allah ibn Mas’ud (may Allah be pleased with him) said: The Messenger of Allah (peace and blessings of Allah be upon him) cursed the one who consumes riba and the one who pays it. (Narrated by Muslim, 1597)
Abu Hurayrah (may Allah be pleased with him) narrated that the Prophet (peace and blessings of Allah be upon him) said: “Avoid the seven sins that doom a person to Hell.” They said, “O Messenger of Allah, what are they?” He said: “Associating others in worship with Allah (shirk); witchcraft; killing a soul whom Allah has forbidden killing, unless that is done lawfully; consuming riba (usury, interest); consuming the property of orphans; running away from the battlefield; slandering innocent, chaste, believing women.” (Narrated by al-Bukhari, 2615; Muslim, 89)
Samurah ibn Jundub (may Allah be pleased with him) said: The Prophet (peace and blessings of Allah be upon him) said: “Last night I saw (in a dream) that two men came to me and took me out to a sacred land. They set out with me until we came to a river of blood in which a man was standing, and on the bank of the river there was a man with stones in front of him, facing the man who was in the river. Every time the man wanted to get out, the man (on the bank) threw a stone into his mouth and he went back to where he had been. I said: ‘What is this?’ They said, ‘The one whom you saw in the river is the one who consumed riba.’” (Narrated by al-Bukhari, 1979)
So you have to repent to Allah from this action.
But if the loan was a good kind of loan that did not involve riba, there is nothing wrong with it.
Going for Hajj or paying off the debt?
With regard to Hajj, the person who cannot spend on himself because he is of limited means is not obliged to go for Hajj, but which of the two is more important, going for Hajj or paying off the debt?
The most correct view is that paying off the debt is more important, because the debtor is not obliged to do Hajj, since one of the conditions of Hajj is that one should be able to do it.
If there is a conflict between your doing Hajj and your paying off your debt , then give priority to paying off the debt . But if there is no conflict, such as there being plenty of time for you to pay off your debt, or if the person to whom you owe money is patient and can wait, then the correct view is that there is nothing wrong with your doing Hajj or ‘Umrah.
Shaykh al-Islam Ibn Taymiyah (may Allah have mercy on him) said:
“It is permissible for a person who is in debt and facing financial hardship, either because he is unable to earn money or because the creditor is absent and he cannot pay him, to go for Hajj if someone else takes him for Hajj, provided that this does not affect his ability to pay the debt.” (Majmu’ al-Fatawa, 26/16)
All of that is subject to the condition that you are completely able to perform Hajj and that you pay off debts to anyone who asks you to pay him back when the debt becomes due, and that you are able to pay for the journey and its provisions, and whatever you need to take care of your affairs whilst you are travelling, without neglecting your family or those on whom you are obliged to spend.
So you should leave with them enough to cover their needs. If you do not do that then you are sinning and neglecting those whom Allah has commanded you to take care of.
Khaythamah said: “We were sitting with ‘Abd-Allah ibn ‘Amr when Qaharman came to him. He said, ‘Have you given the slaves their food?’ He said, ‘No.’ He said, ‘Go and give it to them. The Messenger of Allah (peace and blessings of Allah be upon him) said: “It is sufficient sin for a man if he withholds the food of his slave.”’ (Sahih Muslim, 996)
Ibn ‘Umar narrated that the Messenger of Allah (peace and blessings of Allah be upon him) said: “It is sufficient sin for a man if he neglects those whom he is responsible for.” (Narrated by Abu Dawud, 1692)
And Allah knows best.