Sometimes its just old brown oil stains.
My dad had a couple handguns that he used. He took good care of them. Kept them clean and lubed.
But, as he got older he seemed to not notice small/detailed things as well. I came home one weekend and he asked me to take a look at a semi-auto "pocket" pistol. He said it was "jamming." It was really gummed up with old oil. Brown looking stains on it and a buildup in places you could actually scrape off, or wipe away with a solvent wetted patch.
Same for his revolver. I grabbed it next and he'd been treating the stainless steel revolver just like the nickel plated pocket pistol. Clean and lubed, but a build up of years of old oil in spots. More solvent wetted patches and lots of wiping took care of that one (as well as removing the side plate and cleaning up the lube inside the frame, under the ejector, etc., etc., etc.
Prove it's rust by wiping with some Hoppes#9 or other powder solvent first.