There is some of a family duty if you want to keep that family, then go to the family gatherings. Be friendly, be cordial, be a good daughter, be a good niece or son or whatever. And yet, you know, just realize that is part of your Kamma. And if you actually do not want to have anything to do with that family again - that's totally different.
As to your Dhamma family being more important - yes, I believe that's true. Because your Dhamma family thinks the same, your Dhamma family wants to go the same way, your Dhamma family watches where you are on a Dhamma level, not just on a materialistic level or on alcohol level or whatever else. So there is an importance here that is quite different to a blood family.
And if you have a chance to meet up with your Dhamma family - yeah - do it. And do it more often. Get a sitting group going wherever you live and call people on the phone or on skype - it's free! Make connections with other Dhamma people you know. And keep those connections going. For those of you I have known for many years - you have Dhamma friends. It's been great. You get support from them, you get excitement from them, you get interest from them. You don't feel so alone, even if you are in a town where you have no other Dhamma people living there. To be able to be in contact by Skype or whatever is great! So - treat your Dhamma family as special.