If we're having aversion, the person who needs Compassion is ourselves first. If we're feeding our aversion, it doesn't actually make a difference whatsoever to the people who are making the noise. They're probably having quite a nice time. But we're the ones who are suffering.
As far as having Compassion for people who are causing difficulties to others, understanding that they may be insensitive, more selfish, and don't understand how their actions may affect others means that they have perhaps a less developed level of Compassion. With a lower level of Compassion, they have less causes for the arising of spiritual happiness. This could end some of their craving for the worldly happiness that they are seeking and which may even get them into conflict with others. So understanding their ignorance and lack of ability to find more peace within themselves, helps us to have more Compassion for them.