Fewer cones may be better for low-light vision. Apparently mammals grew out of a evolutionary niche where this was important, and they generally only have two cones while other vertebrates have four. Remember how dogs have poor color vision? That's true of almost all mammals. Primates are unusual among mammals in having 3 types of cones and better color vision. Meanwhile reptiles, birds, amphibians, insects have 4 types of cones and much better color vision than us.
The eye has a limited amount of real estate for light sensitive cells, and animals have a limited amount of energy to dedicate towards eye growth. More cones generally means fewer rods, which means poorer night vision. You can see this for yourself at night, because cones are concentrated at the center of the field of vision and rods are more common at the periphery. A common trick in astronomy is to look next to a star rather than directly at a star, it is not too hard to find stars which are invisible when you look at them directly but which become visible when you look to the side. Various nocturnal and deep sea creatures also feature a number of different adaptations, such as the tapetum lucidum (which reflects light to pass through the retina a second time) and special inverted rods which increase light sensitivity. These structures come with their own various tradeoffs. For example, the tapetum lucidum reduces image sharpness.
In spite of this, humans have very good eyes, with a balance of good low-light vision, good color vision, and good image sharpness. There are other animals better at each of these, but humans are still pretty amazing.
