Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix logic and message reported for version if inaccessible. #701

Merged
merged 5 commits into from
Jul 14, 2020

Conversation

machawk1
Copy link
Member

This PR includes some handling of instances where pypi.org
is offline or inaccessible. I also cleaned up some of the
naming and introduced an exception dataclass to make handling
this issue more clear. Also also, I removed the dependence
on the six module to instead use the provided urllib
urlopen method.

Closes #700

This PR includes some handling of instances where pypi.org
is offline or inaccessible. I also cleaned up some of the
naming and introduced an exception dataclass to make handling
this issue more clear. Also also, I removed the dependence
on the six module to instead use the provided urllib
urlopen method.

Closes #700
@machawk1 machawk1 requested a review from ibnesayeed July 13, 2020 20:24
@codecov
Copy link

codecov bot commented Jul 13, 2020

Codecov Report

Merging #701 into master will increase coverage by 0.00%.
The diff coverage is 25.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #701   +/-   ##
=======================================
  Coverage   29.23%   29.24%           
=======================================
  Files          10       10           
  Lines        1245     1248    +3     
  Branches      184      185    +1     
=======================================
+ Hits          364      365    +1     
- Misses        858      860    +2     
  Partials       23       23           
Impacted Files Coverage Δ
ipwb/__main__.py 0.00% <0.00%> (ø)
ipwb/util.py 43.56% <26.31%> (-0.16%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee489ed...fa94066. Read the comment docs.

Copy link
Member

@ibnesayeed ibnesayeed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think reading installed version is less likely to every fail as it is already loaded in the memory and is available as an imported attribute, so no need to initialize it as None. Moreover, we do not need to overload the function that checks for the latest version to also compare it with the installed version. Let's do it something like this:

def get_latest_version():
    # Check PyPI online, parse the returned JSON, and return just the version number string
    # In case of any exception, return None

def check_for_update():
    latest = get_latest_version()
    if not latest:
        print("Failed to check for the latest version")
        return
    current = re.sub(r'\.0+', '.', ipwb_version)
    if latest == current:
        print(f"Installed version {current} is up to date")
    else:
        print("A new update is available\nRun `pip install --upgrade ipwb` to upgrade.")
        print(f"Latest version: {latest}\nInstalled version: {current}")

ipwb/util.py Outdated
return (currentVersion, latestVersion)
except Exception as e:
return (None, None)
latest_version = jResp['info']['version']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible that the returned JSON data is empty or has an error, in that case, this nested attribute access might throw another exception and the control main fail to reach to the next return statement.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, the exception will be handled by the caller, An unknown exception occurred. Perhaps we could/should handle it more directly. Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And that will be printed to the user something like:

An unknown error occurred.
Some ugly JSON parsing error message...

Which would be of no use of their as they cannot fix it. That's why I proposed a rather simpler approach to this simple task.

Co-authored-by: Sawood Alam <[email protected]>
@machawk1
Copy link
Member Author

I think reading installed version is less likely to every fail as it is already loaded in the memory and is available as an imported attribute, so no need to initialize it as None. Moreover, we do not need to overload the function that checks for the latest version to also compare it with the installed version. Let's do it something like this:

def get_latest_version():
    # Check PyPI online, parse the returned JSON, and return just the version number string
    # In case of any exception, return None

def check_for_update():
    latest = get_latest_version()
    if not latest:
        print("Failed to check for the latest version")
        return
    current = re.sub(r'\.0+', '.', ipwb_version)
    if latest == current:
        print(f"Installed version {current} is up to date")
    else:
        print("A new update is available\nRun `pip install --upgrade ipwb` to upgrade.")
        print(f"Latest version: {latest}\nInstalled version: {current}")

Can you submit this as a code improvement amending this PR? I agree with your cohesive improvement of getting the latest version for potential reuse elsewhere. We will still need to handle the scenario when the request to pypi fails, which is the crux of this PR/issue, and is implemented using an exception model not present in your proposal.

@ibnesayeed
Copy link
Member

Can you submit this as a code improvement amending this PR?

Sure!

We will still need to handle the scenario when the request to pypi fails, which is the crux of this PR/issue, and is implemented using an exception model not present in your proposal.

I did not illustrate that in a verbose manner, because I said it this way:

# Check PyPI online, parse the returned JSON, and return just the version number string
# In case of any exception, return None

Which was an indication to reuse the code of loading and response parsing you already put in place.

@machawk1 machawk1 merged commit 9334ace into master Jul 14, 2020
@machawk1 machawk1 deleted the issue-700 branch July 14, 2020 14:33
@ibnesayeed
Copy link
Member

@github-actions run

#! /bin/sh

export PS4="$ "
exec > comment.buffer 2>&1

echo "\`\`\`"
set -x

git checkout issue-700
pip install ./
ipwb -u

sed -i 's/2020/2019/' ipwb/__init__.py
pip install ./
ipwb -u

{ set +x; } 2>/dev/null
echo "\`\`\`"

@github-actions
Copy link

$ git checkout issue-700
error: pathspec 'issue-700' did not match any file(s) known to git
$ pip install ./
Processing /home/runner/work/ipwb/ipwb
Requirement already satisfied: warcio>=1.5.3 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (1.7.3)
Requirement already satisfied: ipfshttpclient>=0.6.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (0.6.0.post1)
Requirement already satisfied: Flask==1.1.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (1.1.1)
Requirement already satisfied: pycryptodome>=3.4.11 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (3.9.8)
Requirement already satisfied: requests>=2.19.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (2.24.0)
Requirement already satisfied: beautifulsoup4>=4.6.3 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (4.9.1)
Requirement already satisfied: six==1.11.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (1.11.0)
Requirement already satisfied: surt>=0.3.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2020.7.10.1854) (0.3.1)
Requirement already satisfied: multiaddr>=0.0.7 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipfshttpclient>=0.6.0->ipwb==0.2020.7.10.1854) (0.0.9)
Requirement already satisfied: itsdangerous>=0.24 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2020.7.10.1854) (1.1.0)
Requirement already satisfied: Werkzeug>=0.15 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2020.7.10.1854) (1.0.1)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2020.7.10.1854) (2.11.2)
Requirement already satisfied: click>=5.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2020.7.10.1854) (7.1.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2020.7.10.1854) (2020.6.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2020.7.10.1854) (1.25.9)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2020.7.10.1854) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2020.7.10.1854) (2.10)
Requirement already satisfied: soupsieve>1.2 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from beautifulsoup4>=4.6.3->ipwb==0.2020.7.10.1854) (2.0.1)
Requirement already satisfied: tldextract>=2.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from surt>=0.3.0->ipwb==0.2020.7.10.1854) (2.2.2)
Requirement already satisfied: varint in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2020.7.10.1854) (1.0.2)
Requirement already satisfied: netaddr in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2020.7.10.1854) (0.8.0)
Requirement already satisfied: base58 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2020.7.10.1854) (2.0.1)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Jinja2>=2.10.1->Flask==1.1.1->ipwb==0.2020.7.10.1854) (1.1.1)
Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from tldextract>=2.0->surt>=0.3.0->ipwb==0.2020.7.10.1854) (41.2.0)
Requirement already satisfied: requests-file>=1.4 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from tldextract>=2.0->surt>=0.3.0->ipwb==0.2020.7.10.1854) (1.5.1)
Using legacy setup.py install for ipwb, since package 'wheel' is not installed.
Installing collected packages: ipwb
  Attempting uninstall: ipwb
    Found existing installation: ipwb 0.2020.7.10.1854
    Uninstalling ipwb-0.2020.7.10.1854:
      Successfully uninstalled ipwb-0.2020.7.10.1854
    Running setup.py install for ipwb: started
    Running setup.py install for ipwb: finished with status 'done'
Successfully installed ipwb-0.2020.7.10.1854
$ ipwb -u
Installed version 0.2020.7.10.1854 is up to date.
$ sed -i s/2020/2019/ ipwb/__init__.py
$ pip install ./
Processing /home/runner/work/ipwb/ipwb
Requirement already satisfied: warcio>=1.5.3 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (1.7.3)
Requirement already satisfied: ipfshttpclient>=0.6.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (0.6.0.post1)
Requirement already satisfied: Flask==1.1.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (1.1.1)
Requirement already satisfied: pycryptodome>=3.4.11 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (3.9.8)
Requirement already satisfied: requests>=2.19.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (2.24.0)
Requirement already satisfied: beautifulsoup4>=4.6.3 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (4.9.1)
Requirement already satisfied: six==1.11.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (1.11.0)
Requirement already satisfied: surt>=0.3.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipwb==0.2019.7.10.1854) (0.3.1)
Requirement already satisfied: multiaddr>=0.0.7 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from ipfshttpclient>=0.6.0->ipwb==0.2019.7.10.1854) (0.0.9)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2019.7.10.1854) (2.11.2)
Requirement already satisfied: itsdangerous>=0.24 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2019.7.10.1854) (1.1.0)
Requirement already satisfied: click>=5.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2019.7.10.1854) (7.1.2)
Requirement already satisfied: Werkzeug>=0.15 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Flask==1.1.1->ipwb==0.2019.7.10.1854) (1.0.1)
Requirement already satisfied: idna<3,>=2.5 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2019.7.10.1854) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2019.7.10.1854) (1.25.9)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2019.7.10.1854) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from requests>=2.19.1->ipwb==0.2019.7.10.1854) (2020.6.20)
Requirement already satisfied: soupsieve>1.2 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from beautifulsoup4>=4.6.3->ipwb==0.2019.7.10.1854) (2.0.1)
Requirement already satisfied: tldextract>=2.0 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from surt>=0.3.0->ipwb==0.2019.7.10.1854) (2.2.2)
Requirement already satisfied: varint in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2019.7.10.1854) (1.0.2)
Requirement already satisfied: netaddr in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2019.7.10.1854) (0.8.0)
Requirement already satisfied: base58 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from multiaddr>=0.0.7->ipfshttpclient>=0.6.0->ipwb==0.2019.7.10.1854) (2.0.1)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from Jinja2>=2.10.1->Flask==1.1.1->ipwb==0.2019.7.10.1854) (1.1.1)
Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from tldextract>=2.0->surt>=0.3.0->ipwb==0.2019.7.10.1854) (41.2.0)
Requirement already satisfied: requests-file>=1.4 in /opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages (from tldextract>=2.0->surt>=0.3.0->ipwb==0.2019.7.10.1854) (1.5.1)
Using legacy setup.py install for ipwb, since package 'wheel' is not installed.
Installing collected packages: ipwb
  Attempting uninstall: ipwb
    Found existing installation: ipwb 0.2020.7.10.1854
    Uninstalling ipwb-0.2020.7.10.1854:
      Successfully uninstalled ipwb-0.2020.7.10.1854
    Running setup.py install for ipwb: started
    Running setup.py install for ipwb: finished with status 'done'
Successfully installed ipwb-0.2019.7.10.1854
$ ipwb -u
The installed version of ipwb is outdated.
* Installed: 0.2019.7.10.1854
* Latest:    0.2020.7.10.1854
Please run `pip install --upgrade ipwb` to upgrade.

@ibnesayeed
Copy link
Member

Well, the branch was deleted before I posted the above comment, but the code still worked.

@machawk1
Copy link
Member Author

machawk1 commented Jul 14, 2020

GitHub auto-deletes branches post-merge now.

(even though it says that I deleted it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update Check reports "None" if no net access is available
2 participants