Hi @Lena-Nora
For my answer, I am assuming you are a German citizen.
As a result, your boyfriend will be father of a German child and be entitled to a residence permit on this basis.
It is not clear from your question what type of relationship you are planning between you, your boyfriend and his wife. A polyamorous relationship is not recognised under German immigration law.
If you and the father of the child are not in a relationship but are living together and co-parenting your child, then it is possible that the wife and children could receive a visa to join him. They can apply for a visa at the German embassy in Dakar, Senegal. It is absolutely necessary for this to prove that he has enough income to provide for himself, for your child with him, as well as his wife and two children.
Ultimately, it is the German embassy and your local Ausländerbehörde that will decide on granting a future visa application. I can say, however, that it will be difficult for his wife to prove to the authorities that she and her husband maintain a relationship if he is living together with you and you have a child together.
Best,
Éanna