PyPI may exist and hold old packages, but even that will eventually bitrot, and the API that the old version of pip uses to download files may no longer exist in the future (as an example).

For those cases you will want to make sure you have a local mirror.