There are abundant Islamic texts which indicate that it is haraam to befriend and love the kuffaar or take them as close friends, and honour them and show friendship to them, or be the first to greet them. But exceptions are made in some cases when that is for a purpose or need, such as greeting them with words other than salaam when necessary, or treating them kindly when calling them to Islam. That includes smiling at them.
Shaykh Ibn ‘Uthaymeen (may Allaah have mercy on him) was asked: What is the ruling on mixing with the kuffaar and treating them kindly in the hope that they will become Muslim?
He replied: Undoubtedly the Muslim should hate the enemies of Allaah and disavow them, because this is the way of the Messengers and their followers. Allaah says (interpretation of the meaning):
“Indeed there has been an excellent example for you in Ibraaheem (Abraham) and those with him, when they said to their people: Verily, we are free from you and whatever you worship besides Allaah, we have rejected you, and there has started between us and you, hostility and hatred for ever until you believe in Allaah Alone”
“You (O Muhammad) will not find any people who believe in Allaah and the Last Day, making friendship with those who oppose Allaah and His Messenger (Muhammad), even though they were their fathers or their sons or their brothers or their kindred (people). For such He has written Faith in their hearts, and strengthened them with Rooh (proofs, light and true guidance) from Himself”
Based on this, it is not permissible for the Muslim to feel any love in his heart towards the enemies of Allaah who are in fact his enemies too. Allaah says (interpretation of the meaning):
“O you who believe! Take not My enemies and your enemies (i.e. disbelievers and polytheists) as friends, showing affection towards them, while they have disbelieved in what has come to you of the truth”
But there is nothing wrong with a Muslim treating them kindly in the hope that they will become Muslims and believers, because that comes under the heading of softening their hearts towards Islam. But if he despairs of them becoming Muslim, then he should treat them as they deserve to be treated. This is discussed in detail in the books of the scholars, especially Ahkaam Ahl al-Dhimmah by Ibn al-Qayyim (may Allaah have mercy on him). End quote from Majmoo’ Fataawa Ibn ‘Uthaymeen (may Allaah have mercy on him) (3/13).
Ibn al-Qayyim (may Allaah have mercy on him) said in Zaad al-Ma’aad (2/424), concerning greeting the kuffaar first: A group (of scholars) said it is permissible to greet them first if that is for a purpose, such as a need or a fear of harm, or because they are relatives, or for any other reason. End quote.
It says in al-Mawsoo’ah al-Fiqhiyyah (25/168): The Hanafis are of the view that it is makrooh to greet ahl al-dhimmah with salaam, because that implies respecting them, but there is nothing wrong with greeting a dhimmi if one needs something from him, because in that case the salaam is because of the need, not in order to show respect. And it is permissible to say, “Peace be upon those who follow true guidance.” According to the Shaafa’is it is haraam to initiate the greeting of salaam with a dhimmi, but one may greet him in some other way, such as saying “May Allaah guide you” or “may Allaah bless your morning,” if he needs something from him. Otherwise he should not be the first to show any kind of respect to him, because that is an expression of friendship and love. Allaah says (interpretation of the meaning): “You (O Muhammad) will not find any people who believe in Allaah and the Last Day, making friendship with those who oppose Allaah and His Messenger (Muhammad).”.
Conclusion: it is permissible to be kind to a kaafir and smile at him, with the aim of calling him to Islam.
And Allaah knows best.