You have to hand in your passport at BAMF if you apply for asylum. They will forward it to Ausländerbehörde that will keep it until further notice. You usually get your passport back after your asylum procedure is over or if you leave the country again (e.g. when you are going back to your home country).
If you are granted asylum you will receive a blue passport. In this case it seems that you can keep it alongside with your home country passport (if you got it back by Ausländerbehörde).
Please check out the following links for further information about the blue passport:
As you can see in the threads linked above you can use the blue passport to travel to a lot of countries. I can't find information concerning the question if you can travel with the passport of your home country, but it seems to be intended that you use the blue passport for this.