Should I get back with my daughters father?

My daughters father and I have had extremely hard times the past few months since she's been born. He has anger issues (ADHD & Bi-polar) and could not control his anger so I broke up with him about 2 weeks ago. The other night he came over begging and pleading for me to give him another chance and would do whatever it takes to have us as a family together. It was very sincere and genuine, so I told him he could spend the night. History has a way of overcoming anger is about the most i can say! Anyways, I woke up in the morning and heard his phone go off. When I went to check it out, it was his exgirlfriend who he was with before me. After reading their shared texts, I saw her saying she loved him. I obviously confronted him, and he quickly told me he had no feelings for her. (The girls pregnant and with another man, she's not very smart and continuesly makes bad unhealthy choices while carrying a child.)He made it clear he did not want to be with her, and again seemed as though he was telling the truth. But then, about an hour later he confessed that they had messed around, not slept together but she lets say,gave him a release and he did nothing to her. I was extremely hurt and I asked him to leave. He began apologizing and actually crying because of his stupid actions. I still made him leave and for the past few days hes been very determined on apologizing about everything.. I'm not sure what to do because technically we were not in a relationship when this happened, but we were only broken up for 2 weeks. He says no guy whose been deprived and is overed that certain sexual deed cannot refuse, it was all sexual and no emotional.. What do you think I should do? I'm so confused because I do love him and wish to have a family but am unsure if this is forgivable.