The child – male or female – who can afford it has to spend on his parents if they are poor and needy.
Allaah says (interpretation of the meaning):
“And your Lord has decreed that you worship none but Him. And that you be dutiful to your parents”
Part of being dutiful is spending on them if they are in need.
Ibn al-Mundhir (may Allaah have mercy on him) said:
The scholars are agreed that spending on parents who are poor and have no income and no money is obligatory on the child.
The fact that you have brothers does not mean that you are not obliged to spend on your parents, because your brothers cannot afford to spend on them, as you mentioned.
See al-Mughni, 11/375-376.
Shaykh al-Islam Ibn Taymiyah said in al-Ikhtiyaaraat, p. 487:
The son who is well-off (rich) has to spend on his father if he is not well off (poor), and on his father’s wife, and on his younger brothers and sisters.
Based on this, what your husband says about this being the responsibility of your brothers only, is not correct.
Since your spending on them is obligatory, it is not permissible for you to obey your husband in this matter, because the Prophet (peace and blessings of Allaah be upon him) said: “There is no obedience if it involves disobedience to Allaah, rather obedience is only with regard to that which is right and proper.” Narrated by al-Bukhaari, 4340; Muslim, 1840.
At the same time, you must also be gentle with your husband and try to convince him about that, and convince him that this is what Allaah has enjoined on you, namely the duty to honour your parents and treat them kindly.
And Allaah knows best.