It is difficult to give a clear answer on this. I have found information online where it states that with 19d is not a problem to travel to one’s home country. Residence permits are in different groups depending on the purpose of them. 19d is in group 4 in the Aufenthaltsgesetz. The permits in this group are given for work reasons and it is for this reason that travelling home is apparently not a problem.
The case is different with 25,5. This is in group 5 of the Aufenthaltsgesetz, which are residence permits given for humanitarian reasons. 25,5 was presumably issued to your wife and child on the basis on you having a residence permit. The basis of 25,5 is, however, that a person with a Duldung cannot return to their home country. If a person has 25,5 and then returns to their home country, then the very reason they were given it is removed and they could potentially lose their status. As I mentioned, I have read conflicting information on this.
Have you looked at the possibility of your wife and child changing their permit to paragraph 29? This is for family members of foreigners who have a residence permit. One of the main requirements for this is that you as a family have enough income to support yourselves financially. With paragraph 29 it is not a problem to visit the home country.