This question was put to Shaykh Muhammad ibn ‘Uthaymeen (may Allaah have mercy on him) and he said: “The ruling on this, as is well known from the shar’i ruling, is that everyone who worships Allaah in a manner that has not been prescribed by Allaah is an innovator. From this we know that going to this mountain to pray there and to touch it for blessings, and other similar things that are done by the masses, is bid’ah (an innovation). The one who does that is to be denounced and he should be told that there is nothing special about this mountain, apart from the fact that it is Sunnah to stand there on the day of ‘Arafah by the rocks, as the Prophet (peace and blessings of Allaah be upon him) stood there. The Prophet (peace and blessings of Allaah be upon him) stood by the rocks and said, “I stood here but all of ‘Arafah is the place of standing.”
Based on this, there is no need for people to be hard on themselves on the day of ‘Arafah by going to that mountain, maybe getting separated from their people and suffering from heat and thirst, and maybe sinning thereby, doing something which Allaah has not enjoined.