Praise be to Allah.
You should realise that if you are trying your hardest not to look at women, but your gaze accidentally falls on women – either because there are so many improperly dressed women in public places, which means that your gaze falling on them is something that happens a great deal, or because your gaze happens to fall on a woman for some reason – you should try to lower your gaze after that as much as you can and strive to stop yourself looking at anything haraam. If the situation is as you describe, there is no blame on you for that in sha Allah, because of the report narrated by Jareer ibn ‘Abd-Allah who said: I asked the Messenger of Allah (blessings and peace of Allah be upon him) about an accidental glance and he told me to avert my gaze.
Narrated by Muslim, 2159.
Al-Nawawi (may Allah have mercy on him) said: What is meant by an accidental glance is when his gaze falls upon a non-mahram woman accidentally; there is no sin on him for the first glance but he has to avert his gaze immediately. If he averts his gaze immediately, there is no sin on him, but if he continues to look then he is sinning, because of this hadeeth. The Prophet (blessings and peace of Allah be upon him) instructed him to avert his gaze. Furthermore, Allah, may He be exalted, says (interpretation of the meaning): “Tell the believing men to lower their gaze (from looking at forbidden things)…” [al-Noor 24:30]. And Allah knows best.
End quote from Sharh Muslim, 14/139
The Messenger of Allah (blessings and peace of Allah be upon him) said: “O Ali, do not follow one glance with another, for you may be allowed the first but not the second.”
Narrated by al-Tirmidhi, 2701; classed as hasan by al-Albaani in Saheeh al-Jaami‘, 7953.
Al-Khattaabi (may Allah have mercy on him) said:
The first glance is only allowed and is not counted against him if it is accidental and unintentional and was not done deliberately. He does not have the right to follow it with a second glance or to look deliberately in the first lace.
End quote from Ma‘aalim al-Sunan, 3/222
Based on this, we advise you to try to lower your gaze as much as you can and to seek the help of Allah in doing that. One of the greatest means prescribed in Islam that will help you to lower your gaze is to get married, as advised by the Prophet (blessings and peace of Allah be upon him).
We ask Allah to help you to do all that is good and to divert temptation, both overt and hidden, away from you.