You’ve described an individual instance (“your IQ”) while considering a population variation. You’re right in the first instance, individual 2 point variation between tests, say, no big deal. But the second, yes, you would be concerned if the population’s mean IQ dropped 2 points over some period.