Praise be to Allah.
The scholars, may Allaah have mercy on them, agreed that covering the awrah is a condition of the validity of ones prayer. The evidence for this is the aayah (interpretation of the meaning): O Children of Adam! Take your adornment (by wearing your clean clothes) while praying [al-Araaf 7:31]
The conditions for clothes to cover the awrah properly are as follows:
They should not show the shape or contours of the body; if they do so, this is no good because it does not cover properly.
They should be clean (taahir). If the clothes are unclean or impure (naajis), then it is not correct to pray in them, because it is haraam to have any impurity (najaasah) on ones body or clothes when praying. Allaah says (interpretation of the meaning): And your garments purify! [al-Muddaththir 74:4]. It was reported that a baby boy who was not yet eating solid food was brought to the Prophet (peace and blessings of Allaah be upon him), and he sat the infant in his lap. The child urinated in his lap, and he called for water and poured it over the urine. The fact that the Prophet (peace and blessings of Allaah be upon him) hastened to wash the childs urine from his clothes indicates that it is obligatory to clean ones clothes of any impurity that may get on them.
They should be permissible, not haraam, whether they are haraam in and of themselves, such as silk garments, or because of the way in which they are made, such as garments that come below the ankles, or because of the way in which they were acquired, such as garments seized by force or stolen.
With regard to the matter of the awrah becoming uncovered during prayer, al-Shaafa'i, may Allaah have mercy on him, said: It is good enough for both men and women to pray with their awrah covered. The mans awrah is what I have already described (i.e., from the navel to theknee), and all of a woman is awrah, apart from her face and hands. The top of her feet is also awrah. If during prayer any part of the area between a mans navel and knees becomes uncovered, or any part of a womans hair, whether it is a lot or a little, or any part of her body apart from her face and hands and wrist and no more than that becomes uncovered, whether they realize it or not, then both (man and woman) must repeat the prayer unless it was uncovered by the wind, or it slipped and was immediately replaced without delay (no longer than it takes to pick up the garment and place it back immediately.) (Al-Shaafa'i, Kitaab al-Umm, Baab Kayfiyyat Libs al-Thiyaab fil-Salaah).
Ibn Qudaamah (may Allaah have mercy on him) said in his book al-Mughni: Section: if a little bit of the awrah becomes uncovered, this does not invalidate the prayer. This is the opinion of Ahmad and Abu Haneefah. Al-Shaafa'i said: It invalidates the prayer because it is a ruling that has to do with the awrah, and it is the same whether it is a little or a lot, as is the case with looking (at the awrah). We have what was reported by Abu Dawood, with his isnaad from Ayyoob, from Amr ibn Salamah al-Jarmi, who said: My father went with a group from his people to the Messenger of Allaah (peace and blessings of Allaah be upon him), and he taught them how to pray. He said, The one who reads Quraan best among you should lead you in prayer. I was the best at reading Quraan, so they appointed me. I used to lead them in prayer wearing a small yellow burdah (cloak or garment), and when I prostrated, my awrah became uncovered. One of the women said, Cover the awrah of your reciter for us! so they bought me an Omani shirt, and I never rejoiced over anything apart from Islam as much as I rejoiced over that. Reported by Abu Dawood. Al-Nasaa'i also reported from Aasim al-Ahwal, from Amr ibn Salamah, who said: I used to lead them in prayer wearing a burdah made of two attached parts in which there was a gap. When I prostrated in it, my backside showed. This was common and was not denounced, and we have not heard that the Prophet (peace and blessings of Allaah be upon him) or any of his Companions denounced it, because if prayer is still valid despite a severe problem where there is an excuse, then we should differentiate between whether the problem is severe or not where there is no excuse; it is like the case of walking during the prayer. Avoiding little matters may be too difficult, so a little is excused, as is the case with blood. Once this is clear, the definition of a lot is that which is offensive to look at, and in this case there is no differentiation between the private parts and other parts of the awrah. A little is that which is not offensive, and the point of reference is that which is customary, except that the two private parts are more offensive than anything else, so that is considered to be an obstacle to prayer This is a matter in which Islam did not define what is offensive, so the point of reference is what is customary, as also applies in the case of excessive movement during the prayer
If these holes show what is customarily regarded to be too much of the awrah, and it shows it for a while, or it is not possible to cover them because the garment is too small, for example, then the prayer is not valid, because covering the awrah is one of the conditions of prayer, and if one of the conditions is not met without a valid excuse such as inability to meet it, then the prayer is not valid. You have to repeat the prayers which you did wearing this garment. If you do not know how many they were, you should take what you are certain of. For example, if you think that they may have been four or three or five, but definitely not more than five, then what you can be certain of in this case is five, and so on. And Allaah knows best.