We're on Gavage, baby is 1.5 months old, I hold him upright after feeding…

We're on Gavage, baby is 1.5 months old, I hold him upright after feeding with a bottle, immediately vomiting and some spitting up starts and then coughing, if I don't hold him upright after feeding, just lay him on his back in the crib, he doesn't spit up or cough, does this mean we shouldn't hold him upright or should we make sure there's spitting up?