Praise be to Allah.
What gets onto underwear of the traces and smell of najaasah (impurity) due to sweating is usually very little, and can be overlooked, as is the view of the Hanafis and many of the scholars.
Al-Kaasaani said: Because a little najaasah cannot be completely avoided. Flies may land on something impure, then land on the clothes of the worshipper, and there will inevitably be some impurity on their wings and feet; if that were not overlooked, it would cause hardship for people.
End quote from Badaa’i‘ as-Sanaa’i‘, 1/79
Shaykh al-Islam Ibn Taymiyah said: A little najaasah may be overlooked, even mouse droppings and the like in food and so on. This is one view in the madhhab of Ahmad. If there is certainty that mud in the street is najis (impure), a little may be overlooked because it is too difficult to avoid it completely.
End quote from al-Fataawa al-Kubra, 5/313
Shaykh Ibn ‘Uthaymeen said: The correct view is that of Abu Haneefah and Shaykh al-Islam… Included in the small amounts of impurities that may be overlooked because it is too difficult to avoid it completely is a small trace of urine for one who suffers incontinence, but has done his best to protect himself from it as much as he can.
End quote from ash-Sharh al-Mumti‘, 1/447
Something that may make the matter more flexible with regard to what you mentioned is the fact that Islam allows a man to clean himself (after relieving himself) with stones or pebbles. It is well known that stones do not purify the place completely; rather there will inevitably still be some small traces of impurity, but this is something that may be overlooked.
Ibn Qudaamah said: Traces of impurity from urination and defecation may be overlooked because of where they are coming from in three cases, one of which is istinja’ (cleaning oneself after relieving oneself). Traces left after cleaning oneself with stones in the prescribed manner and number may be overlooked, and there is no difference of scholarly opinion concerning that, as far as we know.
End quote from al-Mughni, 2/486
Al-Qarraafi said: If a person sweats in his garment after cleaning himself with stones, it is overlooked, because this is a problem that everyone has… And because the Sahaabah (may Allah be pleased with them) used to clean themselves with stones and they used to sweat.
End quote from adh-Dhakheerah, 1/211
It says in Sharh Mukhtasar Khaleel by al-Kharashi (1/148): If a person sweats in that place, and it gets onto the garment, it does not matter.
Ibn Qudaamah said: Because the Sahaabah (may Allah be pleased with them) mostly used stones to clean themselves… and their land is hot, so it seems that they were not safe from sweating, but there is no report from them to suggest that they tried to avoid that or protect themselves from it.
End quote from al-Mughni, 1/119
Ibn al-Qayyim said: Most of the Sahaabah did not clean themselves with water (after relieving themselves); rather they used stones in the summer and in the winter; and usually a person sweats in his izaar or lower garment, but the Prophet (blessings and peace of Allah be upon him) did not instruct them to wash it although he was aware of the situation, and they did not do that, even though they were the best of generations and the ones who feared Allah most.
End quote from Badaa’i‘ al-Fawaa’id, 4/1490
Shihaab ad-Deen ar-Ramli ash-Shaafa‘i was asked about a person who cleaned himself with stones, then the tip of his penis touched a wet spot on his body whilst he was praying – did that invalidate his prayer, and did he have to clean himself with water and wash the area it got onto?
He replied: It does not invalidate his prayer, and he does not have to clean himself with water or wash whatever got onto the area that was cleaned with stones, because they (the scholars) said: Traces left after cleaning oneself with stones are overlooked, even if he sweats in the place and some slight contamination affected some other part.
End quote from Fataawa ar-Ramli, 1/33
Shaykh Muhammad Mukhtaar ash-Shinqeeti said:
Once it is established that a small amount of impurity in the anus or on the tip of the penis may be overlooked if a person cleans himself with stones then sweats, then that sweat will inevitably reach that place. If a place near it sweats, and this sweat reaches the garment or pants near that spot, this is overlooked, because if we were to rule that it is impure, this would cause hardship for the people the extent of which no one would know but Allah, may He be glorified and exalted.
End quote from Sharh Zaad al-Mustaqni‘, 23/4)
See also al-Mughni by Ibn Qudaamah, 1/219
To sum up: this slight trace that results from sweat does not matter, and your prayer is valid. All you need to do is make sure that you clean yourself properly when you clean yourself after relieving yourself, then do not worry about it any more than you should after that, so that you will not fall prey to waswaas.
And Allah knows best.