Dear @Lydia, 
The local "Jugendamt" (youth welfare office) is the right institution to involve here. You can contact the Jugendamt for individual counselling. As a measure of last resort and if it is the best for the children, they may also support you with applying for the sole custody at the family court. 
I hope this helps and please don't hesitate to get back to us with any further questions. 
All the best, 
Meike