Mr. Daniel sorry I am little bit confused if did you people officially a wife and husband. If the answer is YES so why Authorities are demanding your Passport? Is there any proof you have which can help you if you want to take this matter further.
Secondly,there is issue about you visiting Ghana High commision inorder to apply/obtain for new Passport.Was your Asyl case ended or still going on? What kind of case did you gave to BAMF (do not state you case here please) just think was it based on Politics(like you have problem with any Government department example police etc) or just family matters and so on.
If your case was based on political influence/issues and you decided to visit Ghana High Commision then its a little bit tricky. Perhaps Landrasamt want to punish you through this mistake.
I am going to be honeslyt and I want to make this very clear so people should not fall in this trap.Because why would I run away from POLICE in Ghana and few years or months later go to Ghanian Embassy looking for Passport? Are you not afraid the Ghanian Gov.anymore??? that is the question Landrasamt will be asking themself.
The issue of you getting a BLUE pass was bigger before you went to a Embassy,I am not saying you dont have a chance no.Perhaps a good lawyer can be able to bild a strong case on this.
The issue of deportation its hard one for me because I dont know if your Asyl case its still running or not.Also its depend on your legal Team how commited they are to defend you and your family.
About the chance of you getting your resident permit is there. Dont give in neither lose your focus.**** after your family and take it easy at this difficult time.What about Ghanian Nation ID card?