Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle errors effectively #140

Closed
wants to merge 7 commits into from
Closed

Handle errors effectively #140

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
29815df
Merge pull request #1 from internetarchive/master
kumiDa Mar 11, 2019
c4ba8ed
Merge branch 'master' of https://github.com/internetarchive/openlibra…
kumiDa Mar 13, 2019
467aee2
Merge branch 'master' of https://github.com/internetarchive/openlibra…
kumiDa Mar 20, 2019
0d9aa2f
normalize response variable naming
kumiDa Mar 20, 2019
f8847aa
update README.md with updated CLI prompts
kumiDa Mar 20, 2019
5c4588f
handle errors with descriptive prompt
kumiDa Mar 21, 2019
c0d06e2
intermediate commit
kumiDa May 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 20 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,26 +120,31 @@ Installing the openlibrary-client library will also install the `ol` command lin
```
$ ol

usage: ol [-h] [-v] [--configure] [--get-work] [--get-book] [--get-olid]
[--olid OLID] [--isbn ISBN] [--create CREATE] [--title TITLE]
usage: ol [-h] [-v] [--configure] [--get-work] [--get-author-works]
[--get-book] [--get-olid] [--olid OLID] [--isbn ISBN]
[--create CREATE] [--title TITLE] [--author-name AUTHOR_NAME]
[--baseurl BASEURL] [--email EMAIL]

olclient

optional arguments:
-h, --help show this help message and exit
-v Displays the currently installed version of ol
--configure Configure ol client with credentials
--get-work Get a work by --title, --olid
--get-book Get a book by --isbn, --olid
--get-olid Get an olid by --title or --isbn
--olid OLID Specify an olid as an argument
--isbn ISBN Specify an isbn as an argument
--create CREATE Create a new work from json
--title TITLE Specify a title as an argument
--baseurl BASEURL Which OL backend to use
--email EMAIL An IA email for requests which require authentication.
You will be prompted discretely for a password
-h, --help show this help message and exit
-v Displays the currently installed version of ol
--configure Configure ol client with credentials
--get-work Get a work by --title, --olid
--get-author-works Get a works of an author providing author's --olid,
--author-name
--get-book Get a book by --isbn, --olid
--get-olid Get an olid by --title or --isbn
--olid OLID Specify an olid as an argument
--isbn ISBN Specify an isbn as an argument
--create CREATE Create a new work from json
--title TITLE Specify a title as an argument
--author-name AUTHOR_NAME
Specify an author as an argument
--baseurl BASEURL Which OL backend to use
--email EMAIL An IA email for requests which require authentication.
You will be prompted discretely for a password
```

You can create a new work from the command line using the following syntax. It's almost identical to the olclient.common.Book object construction, except instead of providing an Author object, you instead pass a key for "author" and a corresponding value:
Expand Down
58 changes: 41 additions & 17 deletions olclient/openlibrary.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,24 @@ def login(self, credentials):
@backoff.on_exception(on_giveup=err, **self.BACKOFF_KWARGS)
def _login(url, headers, data):
"""Makes best effort to perform request w/ exponential backoff"""
return self.session.post(url, data=data, headers=headers)
try:
return self.session.post(url, data=data, headers=headers)
except requests.exceptions.ConnectionError as e:
print(str(e) + '\n' + 'Login request failed to process due to connectivity issue.')
except requests.exceptions.Timeout as e:
print(str(e) + '\n' + 'Login request exceeded timeout limit to return response.')
except requests.exceptions.HTTPError as e:
if e.response.status_code == 404:
error_message = 'The generated URL does not seem to point to an available resource.'
else:
error_message = 'An invalid HTTP response was returned.'
print(str(e) + '\n' + error_message)


response = _login(url, headers, data)

if not self.session.cookies:
# if response:
if not (self.session.cookies and response):
raise ValueError("No cookie set")

def validate(self, doc, schema_name):
Expand Down Expand Up @@ -135,9 +148,20 @@ def save_many(self, docs, comment):
@backoff.on_exception(on_giveup=err, **BACKOFF_KWARGS)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

err = lambda e: logger.exception("Error retrieving OpenLibrary response: %s", e)
@rahul-kumi This is the exception that the backoff decorator eventually raises/d -- I think we want this to give a clearer idea of what happened, whether it was an 'ID not found', timeout, server error, or something else. I thought the intent of #139 was to avoid raising generic exceptions. The requests exceptions are generally pretty informative, but I think this original code lost some of that information. The messages you have added are good and informative, but I think they need to be encapsulated as custom exceptions (rather than prints) so they can be handled further up the chain.

def _get_ol_response(self, path):
"""Makes best effort to perform request w/ exponential backoff"""
response = self.session.get(self.base_url + path)
response.raise_for_status()
return response
try:
response = self.session.get(self.base_url + path)
response.raise_for_status()
return response
except requests.exceptions.ConnectionError as e:
print(str(e) + '\n' + 'Request failed to process due to connectivity issue.')
except requests.exceptions.Timeout as e:
print(str(e) + '\n' + 'Request exceeded timeout limit to return response.')
except requests.exceptions.HTTPError as e:
if e.response.status_code == 404:
error_message = 'The generated URL does not seem to point to an available resource.'
else:
error_message = 'An invalid HTTP response was returned.'
print(str(e) + '\n' + error_message)

@property
def Work(ol_self):
Expand Down Expand Up @@ -198,8 +222,8 @@ def editions(self):
"""
url = '%s/works/%s/editions.json' % (self.OL.base_url, self.olid)
try:
r = self.OL.session.get(url)
editions = r.json().get('entries', [])
response = self.OL.session.get(url)
editions = response.json().get('entries', [])
except Exception as e:
return []

Expand Down Expand Up @@ -246,12 +270,12 @@ def add_author(self, author):

def add_bookcover(self, url):
_url = '%s/works/%s/-/add-cover' % (self.OL.base_url, self.olid)
r = self.OL.session.post(_url, files={
response = self.OL.session.post(_url, files={
'file': '',
'url': url,
'upload': 'submit'
})
return r
return response

def add_subject(self, subject, comment=''):
return self.add_subjects([subject], comment)
Expand All @@ -267,8 +291,8 @@ def add_subjects(self, subjects, comment=''):

def rm_subjects(self, subjects, comment=''):
url = self.OL.base_url + "/works/" + self.olid + ".json"
r = self.OL.session.get(url)
data = r.json()
response = self.OL.session.get(url)
data = response.json()
data['_comment'] = comment or ('rm subjects: %s' % ', '.join(subjects))
data['subjects'] = list(set(data['subjects']) - set(subjects))
return self.OL.session.put(url, json.dumps(data))
Expand Down Expand Up @@ -296,8 +320,8 @@ def get(cls, olid):
>>> ol.Work.get('OL26278461W')
"""
path = '/works/%s.json' % olid
r = cls.OL._get_ol_response(path)
return cls(olid, **r.json())
response = cls.OL._get_ol_response(path)
return cls(olid, **response.json())

@classmethod
def search(cls, title=None, author=None):
Expand Down Expand Up @@ -430,12 +454,12 @@ def add_bookcover(self, url):
"""Adds a cover image to this edition"""
metadata = self.get_metadata('OLID', self.olid)
_url = '%s/add-cover' % metadata['preview_url']
r = self.OL.session.post(_url, files={
response = self.OL.session.post(_url, files={
'file': '',
'url': url,
'upload': 'submit'
})
return r
return response

def save(self, comment):
"""Saves this edition back to Open Library using the JSON API."""
Expand Down Expand Up @@ -719,9 +743,9 @@ def get(cls, olid):
>>> ol.Author.get('OL39307A')
"""
path = '/authors/%s.json' % olid
r = cls.OL._get_ol_response(path)
response = cls.OL._get_ol_response(path)
try:
data = r.json()
data = response.json()
olid = cls.OL._extract_olid_from_url(data.pop('key', u''),
url_type='authors')
except:
Expand Down