Praise be to Allah.
From the point of view of sharee’ah, the validity of a person’s Hajj or ‘Umrah has nothing to do with their name. So long as there is no haraam meaning in your old name, it is permissible for you to keep it, although using an Arabic name that has an acceptable meaning is better. (See Question #1692). From the official point of view, when you go to the embassy to apply for the Hajj entry visa, it should be sufficient for you to take a certificate from the Islamic centre proving that you have embraced Islam; in sha Allaah there should be nothing to prevent you from coming to perform Hajj. You should hasten to do Hajj and ‘Umrah as soon as possible, accompanied by a Muslim mahram (husband, or close relative to whom marriage is permanently forbidden). And Allaah is the source of strength.