Skip to content

Commit

Permalink
feat(generic): let the importer try rdf before the json fallback
Browse files Browse the repository at this point in the history
This is what most of the projects will need and it doesn't cost much
computation time - if there is no RDF configuration for that model and
that URI the rdf parser raises an error and the importer will fall back
to json without doing an additional request.
  • Loading branch information
b1rger committed Jun 26, 2024
1 parent 9570bbd commit 97dba9d
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion apis_core/generic/importers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@
import urllib
from django.core.exceptions import ImproperlyConfigured
from apis_core.utils.normalize import clean_uri
from apis_core.utils.rdf import get_definition_and_attributes_from_uri


class GenericModelImporter:
"""
A generic importer class
It provides the standard methods for importing data from
an URI and creating a model instance of it.
By default it fetches a resource, tries to parse it using json and
By default it fetches a resource, first tries to parse it using
our rdf parser, if that fails tries to parse it using json and
then extracts the fields whose keys match the model field names.
Projects can inherit from this class and override the default
methods or simple write their own from scratch.
Expand All @@ -30,6 +32,14 @@ def clean_uri(self, uri):
return clean_uri(uri)

def request(self, uri):
# we first try to use the RDF parser
try:
defn, data = get_definition_and_attributes_from_uri(uri, self.model)
return data
except Exception:
pass
# if everything else fails, try parsing json
# if even that does not help, return an empty dict
try:
return json.loads(urllib.request.urlopen(uri).read())
except Exception:
Expand Down

0 comments on commit 97dba9d

Please sign in to comment.