Looking deliberately is not permitted, because Allaah says (interpretation of the meaning):
“Tell the believing men to lower their gaze (from looking at forbidden things), and protect their private parts (from illegal sexual acts). That is purer for them”
Allaah has made the eye the mirror of the heart. If a person lowers his gaze, then his heart controls his desire and will, but if he lets his gaze wander freely, then his heart will give free rein to his desire. In al-Saheeh it is narrated that on the Day of Sacrifice, al-Fadl ibn ‘Abbaas (may Allaah be pleased with him) set off from Muzdalifah to Mina, and some women passed by and al-Fadl started looking at them. The Messenger of Allaah (peace and blessings of Allaah be upon him) turned his face to the other side. Ibn al-Qayyim said in Rawdat al-Muhibbeen: This shows that it is forbidden to look at non-mahram women and is a denunciation in practical terms. If looking were permissible, he would have approved of it. In al-Saheeh it is narrated that [the Prophet] (peace and blessings of Allaah be upon him) said: “Allaah has decreed for the son of Adam his share of zina which will inevitably come to him. The eye commits zina and its zina is looking. The tongue commits zina and its zina is speaking. The foot commits zina and its zina is walking. The hand commits zina and its zina is touching. The heart longs and wishes, and the private part confirms that or denies it.” He started with the zina of the eye because it is the basis of the zina of the hand, foot, heart and private part. And the zina of the mouth by kissing is implied by the reference to the zina of the mouth by speaking. And he described the private part as confirming that if the action is fulfilled, or denying it if it is not. He said: This hadeeth is one of the clearest signs that the eye can sin by looking, and that this is its zina. This is a refutation of those who permit letting the gaze wander freely. End quote.
As for looking unintentionally, one will not be punished for that if the heart did not intend it, but if he follows it with another look, then he is sinning. Muslim, Abu Dawood, al-Tirmidhi and al-Nasaa’i narrated that Jareer (may Allaah be pleased with him) said: I asked the Messenger of Allaah (peace and blessings of Allaah be upon him) about an accidental look and he commanded me to avert my gaze. Al-Tirmidhi said: This is a saheeh hasan hadeeth. Ahmad, Abu Dawood and al-Tirmidhi narrated from Buraydah (may Allaah be pleased with him) that he said: The Messenger of Allaah (peace and blessings of Allaah be upon him) said to ‘Ali: “O ‘Ali, do not follow one look with another, for you are allowed the former but you are not allowed the latter.” Al-Tirmidhi said: Hasan ghareeb. These two hadeeth indicate that if a person averts his gaze immediately, then there is no sin on him, but if he persists in looking then he is sinning. In Bab Nazr al-Faja’ah wa ma kuriha min al-Nazr in Kitaab al-Wara’ by Imam Ahmad ibn Hanbal, there is a report from Abu Bakr Ahmad ibn Muhammad al-Marwadhi which says: I said to Abu ‘Abd-Allaah (i.e., Imam Ahmad): A man has repented and says: Even if my back were struck with a whip I would not sin, but he could not stop looking. He said: What repentance is this? Jareer said: I asked the Messenger of Allaah (peace and blessings of Allaah be upon him) about an accidental look and he commanded me to avert my gaze. End quote.