Yes, you have to offer expiation, and also pay the diyah (blood money) which must be given to his family. The evidence for that is the verse in which Allaah says (interpretation of the meaning):
“It is not for a believer to kill a believer except (that it be) by mistake; and whosoever kills a believer by mistake, (it is ordained that) he must set free a believing slave and a compensation (blood money, i.e. Diya) be given to the deceased’s family unless they remit it. If the deceased belonged to a people at war with you and he was a believer, the freeing of a believing slave (is prescribed); and if he belonged to a people with whom you have a treaty of mutual alliance, compensation (blood money — Diya) must be paid to his family, and a believing slave must be freed. And whoso finds this (the penance of freeing a slave) beyond his means, he must fast for two consecutive months in order to seek repentance from Allaah. And Allaah is Ever All-Knowing, All-Wise”
The majority of scholars are of the view that expiation must be offered by one who kills a kaafir who is protected by sharee’ah.
The kaafirs who are protected by sharee’ah are of three types:
1 – Al-dhimmi. This is one with whom we have a contract or treaty of al-dhimmah (i.e., one who lives in a Muslim state)
2 – Al-mu’aahad. This is one with whose people we have a peace treaty.
2 – Al-musta’man. This is one who has entered the Muslim land and has been guaranteed safety, such as those who come to do business, to work, to visit relatives, and so on.
Whoever kills a kaafir who is protected by sharee’ah has to do two things:
1 – Diyah (blood-money). He has to pay the diyah to the family of the deceased. This applies so long as his family are not muhaaribeen (i.e., belong to a people who are at war with the Muslim state). If his family are muhaaribeen, then they do not deserve the diyah, because their wealth and their blood are not sacred. Tafseer al-Sa’di, p. 277.
2 – Kafaarah (expiation). This is the view of the majority.
Ibn Qudaamah said in al-Mughni, 12/224:
(Expiation) must be offered for killing a kaafir who is protected by sharee’ah, whether he was a dhimmi or musta’man. This is the view of the majority of scholars. Al-Hasan and Maalik said that no expiation is to be offered, because Allaah says (interpretation of the meaning):
“and whosoever kills a believer by mistake, (it is ordained that) he must set free a believing slave”
So their understanding was that no expiation is required for a non-believer. But we note that Allaah says (interpretation of the meaning):
“and if he belonged to a people with whom you have a treaty of mutual alliance, compensation (blood money — Diyha) must be paid to his family, and a believing slave must be freed”
A dhimmi comes under the same category as those mentioned here, and this is what may be understood from this verse.
And because he is a human being who was killed wrongfully, so expiation must be offered for killing him, as in the case of a Muslim.
End of quote.
This view was favoured by a number of mufassireen, such as al-Tabari (9/43); al-Qurtubi (5/325); Ibn Katheer (2/376).
Ibn Jareer al-Tabari said (9/40-43):
Then the commentators differed concerning the description of the victim who belonged to a people with whom we have a treaty of mutual alliance – is he a believer or a kaafir? Some of them said, he is a kaafir but his killer has to pay the diyah (blood money), because he and his people have a treaty, so the diyah must be paid to his people because of the treaty between them and the believers; this is wealth that belongs to them and it is not permissible for the believers to take any of their wealth unless they give it willingly…
Then al-Tabari said: The better of the two views on the meaning of this verse is the view of those who said: what is meant by that is the victim from among ahl al-‘ahd [those with whom the Muslims have a treaty], because the verse is ambiguous, as it says, “and if he belonged to a people with whom you have a treaty of mutual alliance”, and it does not say, “and he is a believer,” as it says with regard to the case of one who was slain and was a believer and belonged to a people at war with the Muslims… The fact that it does not describe this person as a believer, as it does in the case of the two types mentioned previously, clearly indicates that what we are saying is correct.
It was narrated that Ibn ‘Abbaas said concerning the phrase “and if he belonged to a people with whom you have a treaty of mutual alliance”: If he was a kaafir who was under your protection and was killed, then his killer has to pay the diyah to his family, and set free a believing slave, or fast for two consecutive months.
This view was also favoured by Shaykh Ibn ‘Uthaymeen (may Allaah have mercy on him) in his commentary on Soorat al-Nisa’, tape no. 27, side 2.
And Allaah knows best.