Utterances of divorce are of two types: explicit and implicit.
Explicit utterances are those which cannot be understood to mean anything but divorce in most cases, such as “I have divorced you” or “You are divorced” and so on. These phrases constitute divorce even if that was not the husband’s intention.
As for the implicit utterances, these are phrases which may be understood as referring to divorce or to something else, such as the words uttered by your husband (“Go by yourself to your family”). This may be understood as meaning that he intended a divorce, or it may be understood otherwise. The ruling on such words is that they do not constitute divorce unless the husband intended that.
If your husband intended divorce when he uttered these words, then divorce has taken place if the stipulated condition is met, which is your touching the mobile phone.
But if he did not intend a divorce, then nothing has happened.
The fact that he intended to scare you by saying that does not mean that he intended it also as a divorce, because a man may try to scare his wife with the threat of divorce so that she will obey him and not disobey him.
The husband has to beware of using the words of divorce, explicit and otherwise, and he should strive to solve problems in an atmosphere of love and understanding.
The wife has to obey her husband and strive to please him, and she does not have the right to disobey him or make him angry or take his money without his permission.
May Allaah help us and you to do that which He loves and which pleases Him.
And Allaah knows best.