Even being physically close to them is a danger
I think this could easily be turned on say Hollywood Celebs, even without horrible actual scandals, if they take pictures with fans of the opposite sex
They can be suspect of sexually exploiting them
They could have just made Joel NOT talk at all about his views and stick to voice acting and avoid socializing
My guess is this was a power play of sorts
Or just someone getting off on the opportunity to do it