Metadata inconsistent error message should contain all metadata caused the inconsistency

[larsoner]

• pip version: 20.3
• Python version: 3.9 or 3.8 (only two I've tested)
• OS: Linux or Windows (only two I've tested)

See numpy/numpy#17885 (comment) -- in 20.2.4 this works:

$ python3.9 -m pip install --progress-bar off --upgrade "pip!=20.3"
...
Successfully installed pip-20.2.4
$ python3.9 -m pip install --progress-bar off --upgrade --pre --only-binary ":all:" -i "https://pypi.anaconda.org/scipy-wheels-nightly/simple" numpy
...
Collecting numpy
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/numpy/1.21.0.dev0%2Bd0a457f/numpy-1.21.0.dev0%2B20201129035606_d0a457f-cp38-cp38-manylinux2010_x86_64.whl (15.4 MB)

Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.0.dev0+d3cffdc
    Uninstalling numpy-1.21.0.dev0+d3cffdc:
      Successfully uninstalled numpy-1.21.0.dev0+d3cffdc
Successfully installed numpy-1.21.0.dev0+d0a457f

But with 20.4:

$ python3.9 -m pip install --progress-bar off --upgrade --pre --only-binary ":all:" -i "https://pypi.anaconda.org/scipy-wheels-nightly/simple" numpy
...
Collecting numpy
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/numpy/1.21.0.dev0%2Bd0a457f/numpy-1.21.0.dev0%2B20201129040310_d0a457f-cp39-cp39-manylinux2010_x86_64.whl (15.4 MB)

ERROR: Requested numpy from https://pypi.anaconda.org/scipy-wheels-nightly/simple/numpy/1.21.0.dev0%2Bd0a457f/numpy-1.21.0.dev0%2B20201129040310_d0a457f-cp39-cp39-manylinux2010_x86_64.whl has different version in metadata: '1.21.0.dev0+d0a457f'

[dstufft]

I'm guessing a call to urlunquote needs added.

[uranusjr]

Weird, the new resolver does not reimplement any HTTP logic, it’s all shared with the legacy resolver.

[pradyunsg]

diff --git a/src/pip/_internal/resolution/resolvelib/candidates.py b/src/pip/_internal/resolution/resolvelib/candidates.py
index 1fc2ff479..a50e888c6 100644
--- a/src/pip/_internal/resolution/resolvelib/candidates.py
+++ b/src/pip/_internal/resolution/resolvelib/candidates.py
@@ -210,6 +210,8 @@ def _check_metadata_consistency(self, dist):
         if self._name is not None and self._name != name:
             raise MetadataInconsistent(self._ireq, "name", dist.project_name)
         version = dist.parsed_version
+        print(version)
+        print(self._version)
         if self._version is not None and self._version != version:
             raise MetadataInconsistent(self._ireq, "version", dist.version)

Gives:

1.21.0.dev0+d0a457f
1.21.0.dev0+20201129033659.d0a457f

I don't think there's a functionality bug here. The versions are actually different. We should probably present them both in the error message though, to avoid confusion like this.

[larsoner]

I don't think there's a functionality bug here. The versions are actually different. We should probably present them both in the error message though, to avoid confusion like this.

From this it's clear at least that in the pip internals somewhere they are different in a way that they must not have been (or that did not matter) in 20.2.4. Can you clarify where these two versions get parsed from the wheel/setuptools/etc. so that the NumPy wheels can be fixed at their end?

[charris]

The date of wheel creation is added to the wheel names so that pip will pick up the latest. The wheel generation itself is automated, are you suggesting that the date should be added when setup is run? This used to work, so the question we are asking is: bug or feature.

[pradyunsg]

@larsoner Sorry, I was very much in "triage all the notifications" mode when I looked at this and responded here.

pip's new resolver is stricter across the board for a lot of things related to a package's metadata. One of these is being stricter about the version information that pip sees about a package -- ensuring that what's provided from the METADATA file in that distribution (or as generated from that, in case of sdists) matches what the distribution can/may provide from other mechanisms -- in this case, the wheel filename.

What's happening here is that the wheel file (https://pypi.anaconda.org/scipy-wheels-nightly/simple/numpy/1.21.0.dev0%2Bd0a457f/numpy-1.21.0.dev0%2B20201129040310_d0a457f-cp39-cp39-manylinux2010_x86_64.whl) says the version is 1.21.0.dev0+20201129040310_d0a457f but all the metadata within the file says that it is 1.21.0.dev0+d0a457f.

The resolver is noticing this, and aborting since inconsistent metadata => can't rely on it => can't use it => AAAAAA. 😉

[pfmoore]

Also, just to be clear, this behaviour was present in 20.2.4, however it was opt-in only, as it's part of the new resolver. So the change here is actually just that the new resolver is now default. You should be able to demonstrate this by using the --use-feature=2020-resolver flag on pip 20.2.4, and you should see the same behaviour as 20.3. Conversely, you can (for a short period) opt out of the new resolver using --use-deprecated=legacy-resolver to get the old behaviour back. But this option will be getting dropped next release, so you should definitely only view this as a stopgap solution.