All metal paperclips (antique and modern) are made of galvanized low carbon steel - low carbon because that kind of steel stays flexible. The coating material varies, but it will always respond to the electric current magnetizing the steel.
The zinc or whatever coating applied during galvanization never rusts, but it is very thin and can be scraped off through the friction of use. The thickness of the coating always varies some between production lots - the thinner the coating, the more likely there are microscopic places that failed to coat, particularly where the paperclip wire is in contact with itself.
Low carbon steel will always rust when exposed to water, and the rust will eat into the steel areas under remaining coating. Stainless steel (high carbon with enough chromium or other metals included in the metal to resist rusting) is always hard and rigid, impossible to bend like a paperclip, and the higher the carbon content, the more rigid the steel gets. Also, salt water is much more corrosive then fresh water. Once paperclips lose their coating, they must be kept completely dry to remain useful.
[I worked for 2 years in a plating plant, a fascinating process.]