I'll assume you mean a reflex sight like the FastFireIII when you say a red dot, and not a tube scope style with an illuminated dot reticle. The big advantage with a reflex sight is that as long as you can see the dot, that is where your load is going. If you lift your head off the stock a bit with a bead you are likely to miss high, with a reflex sight it doesn't matter. I haven't had any issues with the zero shifting on my FF3, and I've given it some pretty heavy hits (unintentionally).
As for downsides? You could conceivably have the battery die but since battery life on a FF3 is 10,000+ hours, even if you could leave it on 24/7 (you can't, it auto shuts off after 8 hours) you could leave it on all through turkey season and not even make a real dent in the battery life. Just carry a spare battery and it's not even a concern. It is also an electronic device and some day it could fail but it's not something I'm going to spend time worrying about.
The FF3 and other reflex sights have a number of mounting options, you don't necessarily need a rail. If your gun needs to be drilled and tapped anyway you can set it up for pretty much whatever mount you want.