Praise be to Allah.
What appears to be the case is that what you mention is to be regarded as doubt or uncertainty, not as forgetfulness, because forgetfulness means omitting something as a result of losing focus and not being aware of what one is doing.
Al-Mawsoo‘ah al-Fiqhiyyah (16/198).
Based on that, forgetfulness is when you are certain that you omitted something as a result thereof.
As for being hesitant and uncertain as to whether you did something or not, this is doubt and uncertainty, such as being uncertain as to which of two scenarios occurred, without there being anything to make you think that one is more likely to have happened than the other, in the mind of the one who is uncertain.
The uncertainty to which the fuqaha’ refer includes being hesitant and uncertain as to which of two scenarios occurred, without there being anything to make you think that one is more likely to be the case; that includes doubt, as indicated by the meaning of the word, and conjecture.
Al-Bahooti said in Sharh al-Muntaha (3/142): Uncertainty, according to the scholars of usool, means being hesitant as to which of two scenarios occurred, when there is nothing to indicate which is more likely. This refers to being uncertain in a general sense as to whether what you are doubting took place or not, such as a talaaq (divorce), the number of talaaqs, and whether the condition of talaaq was met or not. That includes conjecture and speculation. End quote.
If there is a great deal of doubt or uncertainty, then this is waswasah (intrusive thoughts; whispers from the Shaytaan), and no attention is to be paid to it.
Al-Kaasaani (may Allah have mercy on him) said, quoting from Muhammad ibn al-Hasan (may Allah have mercy on him): If someone is uncertain about part of his wudoo’, and this is the first time that he has been uncertain, then he should wash the place concerning which he is unsure, because he is certain that that part was impure, but is uncertain as to whether he has washed it.
What is meant by his words “the first time that he has been uncertain” is that uncertainty has not become habitual in his case; it does not mean that he never experienced doubt and uncertainty before. But if that happens to him a great deal, he should pay no attention to it, because that is waswasah (intrusive thoughts). The way to deal with waswasah is to ignore it, because if he is distracted by it, that will lead to him never being able to have the time to pray, and that is not permissible.
End quote from Badaa’i‘ as-Sanaa’i‘ (1/33).
In al-Mawsoo‘ah al-Fiqhiyyah (14/233) it says: The one who is affected by waswasah is the one who is uncertain about his worship, and that happens to him a great deal, to the extent that he thinks that he has not done something when in fact he has done it.
In principle, when you are faced with doubt or uncertainty, you should go back to what you think you omitted (and do it), such as one who lifted his head and was not sure whether he had bowed or not; in that case, he has to bow, because in principle if he was not sure about something, that means that he did not do it. So he should proceed on the basis of what is certain.
If someone is not sure whether he prayed three or four rak‘ahs, he should assume that he had done three, and do another one, then do the prostration of forgetfulness.
But if he is affected by waswasah, he should not pay attention to these intrusive thoughts, because that would cause a great deal of hardship for him, and hardship is to be eliminated, according to Islamic teachings. Rather he should continue on the basis of what he thinks is most likely to be the case, so as to alleviate hardship and put an end to the intrusive thoughts. End quote.
What is prescribed in your case is to continue with your prayer, and not pay attention to the doubt or uncertainty. So you should not repeat the essential part or do the prostration of forgetfulness.
Al-Bahooti said in Sharh al-Muntaha (1/221): It is not prescribed to do the prostration of forgetfulness if there is so much uncertainty and doubt that it becomes like waswaas (intrusive thoughts), because that could lead to a kind of going to extremes and to doing additional things in the prayer, even though he is certain that he did the prayer properly. Therefore he should ignore this uncertainty and doubt, and pay no attention to it. End quote.
As-Saawi said: If the one who is not affected by intrusive thoughts is not sure about whether he washed some part of his body [in wudoo’], then he should wash it.
If the one who is not affected by intrusive thoughts is not sure about whether water reached some part of his body, then he must wash it by pouring water on it and rubbing it.
As for the one who is affected by intrusive thoughts – who is the one who suffers a great deal of uncertainty and doubt – he must ignore it, because succumbing to intrusive thoughts may undermine one’s religious commitment entirely – we seek refuge with Allah from that.
End quote from Haashiyat as-Saawi ‘ala ash-Sharh as-Sagheer (1/170).
See also the answer to question no. 145752.
And Allah knows best.