From your symptoms I suspect your PSU is going. All of Apple's systems use a soft power switch. Basically a low voltage momentary button switch which via some logic switches on the AC line.
I don't think the mechanical switch it's self is defective here but the switching logic on the PSU. Follow this IFIXIT guide [guide|6291|iMac Intel 21.5" EMC 2389 Power Supply Replacement].
I would also double check your AC power service. Get a outlet tester to check ground and polarity.
Do you have a surge suppressor or UPS? If not I do recommend using them. All my personal systems are on UPS's and most of our systems at work are as well. UPS's offer the best protection and allow you to continue working when the power goes out allowing you to shutdown the system cleanly.