Ogling for women carries with it the implied risk of rape due to natural differences in physical strength, and women experience rape at a much greater rate than men, so that threat would have higher salience than any equivalent gender-based insecurity.
