I had the same problem --- here is a good way to remember. the way you want the bullet or shot to move...
Front Opposite, Rear Same
FORS in short...
Example - if you are shooting to the right - move the rear sight in the direction you want to move the pattern - LEFT.
If your front sight could be adjusted - you would move it RIGHT in this case.