I soak an area that has very light rust with WD-40 and let it sit for some time and then use a soft cotton cloth to see if it will wipe off. Normally, some or all can be removed with this process but you may need to repeat it.
Another good method, if you're careful, is to use some "Mother's Mag & Aluminum Polish", Wal-Mart is a good source. It will remove very fine rust. I apply it with my finger, allow it to work and then very lightly polish with a soft cotton cloth.
I've also used 0000 steel wool (very, very carefully with a light touch and as stated, I keep it wet with WD-40) or very fine brass wool, which is what I prefer, if the other process does not remove it.
The real key to all the above is patience and time. I'll work on it for a while, then walk away from the job, so that I don't get carried away and cause more damage than I'm curing.
Thanks,
Clark
P.S. We can all prevent this problem by using some of the vapor technology products that are available several places under different names. You just put some in your gun cabinet, safe, etc. and allow it to protect your guns, inside and out.