I imagine being the other person in this scenario could get pretty aggravating. Steve's just trying to enjoy a movie or hang out with Bucky and Sam and Natasha or whatever and every five minutes there's a "are you sure you still want me here? Are you sure? I can just leave... Are you really sure?" Steve has the patience of a saint, of course, but I could see him getting a little snappy at Bucky after a few days of that and then just feeling like THE WORST PERSON ON THE PLANET.