That's a really good question.
A few years back (when I was high school), I filled in for someone in my school's jazz band for a couple weeks. I was playing 3rd and 4th trumpet parts, and was able to handle them pretty well, given that I hadn't played trumpet for a couple years (since my departure to horn). Anyways, later on in the day when Wind Ensemble class came up, and it was time to play horn, I noticed that the horn embouchure felt foreign, and could not attain it with out great difficulty. A lot of the same things that are happening to you happened to me. When I was finally done filling in for the missing guy in jazz band, it was only a matter of days before my horn embouchure and comfort came back to me.
So what did I learn from this?
(this is just my opinion)
I learned that since the diameter of the horn and trumpet mouthpieces are relatively similar, yet different, it was throwing off the muscles in my lips/embouchure. Just think about it: even though they are close in diameter, one rim is thick as opposed to the other, which is narrow. I concluded that if I could not do both, I would have to focus on one over the other. I stayed with the horn
Occasionally I will still pull out my trumpet and play a few things, but I try to stay away from it as much as possible.
On the other hand, I've got friends that double on both horn and trumpet and sound fantastic on both! So I guess you can draw that it varies from person to person.
Happy Independence Day!