The best way to balance compassion is the meditation on equanimity. Usually I don't talk too much about the fourth Brahma Vihara, equanimity, and systematic ways to develop this. This is because people who usually come to meditate in the beginning need to develop more compassion and we feel that if we teach meditations about equanimity then it may hinder compassion to the extent indifference may arise or increase. But a systematic way to develop more equanimity is doing the compassion reflection in a similar way, bringing people's difficulties and problems to mind and then instead of the Lovingkindness phrase we do an equanimity phrase "they are the owner of their own Kamma." This helps us to understand that all the Dukkha in the world is not our responsibility to heal.
We get overwhelmed when compassion is excessive because the wish to end Dukkha, brings up the desire to have the power to lessen all the Dukkha in the world.
Learning how to accept that Samsara is this way, the cycle of birth and death is this way, helps us to realize that it is not our responsibility and it is beyond our power to change. Even the Buddha could not change Samsara. He showed us ways to transcend Samsara, the cycle of birth and death. So within the realm of birth and death there is going to be Dukkha so we have to balance our compassion with equanimity so we understand that everything is not within our power to change.
Accepting that is sometimes very difficult for people who have a very strong caring nature and it took me sometime in the beginning to realize that I couldn't take away all the Dukkha of my students, too. Especially for the first year I started to get overwhelmed until I realized basically it wasn't my responsibility and even if I could say everything totally perfectly, that some people wouldn't get it. Even the Buddha couldn't make everyone understand and he was perfect, so what of me? Try to lessen our ego a little, try to understand it is not within our power or responsibility to try to change Samsara. We can only point the way. This is what the Buddha did, he only pointed the way. And direct your attention to doing more towards what you do have the power to change.