Anything beyond neutral in any shooting sport will affect accuracy. The way you hold the gun, the way you cant the gun, the angle you pull the trigger, pressure on the butt and pressure on the fore end, relaxed or tight when the gun goes off, and certainly whether or not you flinch.
Follow through in shooting is just as important as a proper squeeze. People can relax on a lead sled because they know they aren't going to get hammered, but tense up and flinch like crazy when shooting against their shoulder. Set your phone on slo-mo video and film yourself. I've seen some people completely lose their mind .25 seconds before the gun goes off, but they swear they made a good shot.
Some people notice no difference because they are disciplined shooters. If you are experiencing variation in different shooting positions, you need to work on your technique. It's as simple as that.