I don't mind so much when you aren't. Usually means you're caught up in it all to remember manners.

[Not a bad thing, either. He liked making Bucky forget himself. Liked when politeness and civility between them was left at the door. It made it feel more real.]

And yes, of course it's all right.