As always the proper answer is "depends on the circumstances".

I don't enjoy much those locales, but if the hosts invite it will be rude not join them.