What is the ruling on marketing courses in return for commission that will increase when the number of students increases?

Praise be to Allah.

Firstly:

There is nothing wrong with marketing educational courses in return for a known commission, if the subject matter of the courses is permissible, because this is brokerage and a fee for something that is permissible.

Secondly:

For the fee to be valid for providing a service, it must be known.

In Al-Mawsu`ah Al-Fiqhiyyah (15/216), it says:

Charging a fee for providing a service, and what conditions are stipulated regarding that:

It must be known.

The Malikis, Shafa`is and Hanbalis said: In order for the service contract to be valid, it is stipulated that the fee must be wealth of a known type and amount, because if the fee is unknown, that undermines the purpose of the service contract, as hardly anyone would want to work when he does not know what he will get in return, let alone the fact that there is no reason not to know what fee is to be paid, in contrast to some tasks in which the effort required and who will do the work are both unknown  for a valid reason, in which case not knowing is forgivable.

Knowing the fee is done by seeing it or by describing it. (End quote)

In Al-Ma`ayir Ash-Shar`iyyah (p. 261) it says: It is stipulated that the fee should be known and it should be of value according to Islamic teachings, and be something that can be delivered. If the fee is not known, or it is something that is not permissible according to Islamic teachings, or it cannot be delivered, then the current standard fee for such work should be given. (End quote)

There is nothing wrong with the fee varying according to the number of students, or according to the number who subscribe during a month, so long as it is known beforehand.

There is nothing wrong with what you mentioned about receiving $13 for each student, then if ten students registered through you, you will receive $17 for any other students who subscribe, because the fee is known.

There is also nothing wrong with agreeing that if five students register in one month, you will receive $50 in addition to the commission. All of that is permissible, because the condition is that the fee or commission should be known, and this condition is met in this case.

And Allah knows best.