Don't worry about it. Random people get votekicked all the time.
It doesn't sound like you did anything wrong.
Hell, one time, someone called a kickvote on me, and even after they admitted that it was an accident, I got votekicked anyways.
There's nothing wrong with using the Adv Dragoon barbs unless you are using them against your base and your teammates. Other than that, you can shoot them at anything you want.