Serendipitous in-office interactions are fun, but they have an adverse effect on the people who weren’t in that conversation.