Make sure the bag is sitting down on the saddle bag support all the way front and back. Not sure if this is your entire problem but I could see it lending to the problem some. I had to rework the rubber isolators a little on mine so the bags would sit all the way down like they are supposed to.
What bags are they?
Mine was the same way. The bottoms are intentionally left long so that you can trim as needed. Mine also were different shape from side to side. I dont think that it will ever be noticed since you cant see both sides of the bike at the same time.