To be fair this behavior + extra-index-url definitely allowed for dependency confusion attacks. Now it's a bit silly for the CVE as written, but IIRC this was before you could use index-url to point exclusively to an internal package host so there were (and in other ecosystems still are) dependency confusion attacks going on in the wild where attackers are publishing "internal-package-a@999.9.9" on any public package host (NPM, PyPi, etc) to try to compromise companies that have internal packages of the same name.

For the attack you mentioned (reusing internal packages in a public repository) prefix reservation is one possible solution. Unfortunately PyPI does not support it.