gettyOverview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [The Getty Arts & Architecture Thesaurus](https://www.getty.edu/research/tools/vocabularies/aat/) (AAT) \n",
    "\n",
    "## Important Links or *How to find your way around in the Getty Maze*\n",
    "\n",
    "\n",
    "* [Vocabularies Downloads](https://www.getty.edu/research/tools/vocabularies/obtain/download.html)\n",
    "* [AAT Hierarchy](https://www.getty.edu/vow/AATHierarchy?find=&logic=AND&note=&english=N&subjectid=300000000)\n",
    "* [Technical Vocabularies Documentation](http://vocab.getty.edu/doc/)\n",
    "* [SPARQL UI](https://vocab.getty.edu/queries#Finding_Subjects)\n",
    "* [Semantic view](https://vocab.getty.edu/aat/) (Top of hierarchy)\n",
    "* [ODC-By license](https://opendatacommons.org/licenses/by/1-0/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is the AAT? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [**Getty Arts & Architecture Thesaurus**](https://www.getty.edu/research/tools/vocabularies/aat/) is a multilingual, semantically structured **controlled vocabulary**. It encompases terms, descriptions, and other information for generic concepts related to visual art, architecture, archaeology, and other cultural heritage. Importantly, the AAT contains generic terms, not iconographic subjects or proper names. In other words: [\"each concept is a case of many (a generic thing), not a case of one (a specific thing)\"](https://www.getty.edu/research/tools/vocabularies/aat/about.html#purpose). The full AAT database contains around 74,460 concept records (subject_id) and 503,230 terms (term_id). Each concept record contains one or more terms (e.g., singular/plural forms, spelling variations, translations). A record minimally contains a unique numeric id, a term, and an indication for the position in the structured hierarchy. Often it also contains a description of the term, a list of associated or equivalent terms and temporal information. The AAT is translated to Dutch by the Netherlands Institute for Art History. Note that the AAT is a compiled resource, so it's not a complete, definitive collection of concepts. It expands through community [contributions](https://www.getty.edu/research/tools/vocabularies/contributors.html) by domain experts. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Terminology\n",
    "Here you find a list of the most important terminology related to the AAT. The terms relate to the different layers of information in the thesaurus. \n",
    "\n",
    "- `Facet`: The major subdivision or upper layer of AAT's hierarchical structure, each containing classes of concepts. In total, there are 8 facets.  \n",
    "- `Hierarchy`: Groupings of terminology that are arranged within the facets.  \n",
    "- `Guide Term`: A placeholder for categories within the hierarchies. They serve to organize the hierarchies into logical segments. \n",
    "- `Record`: A single entry that contains information about a specific concept.  \n",
    "- `Term`: A word or phrase that represents the concept in the record. Includes singular/plural forms, spelling variations, and translations. A record can contain multiple terms.   \n",
    "\n",
    "\n",
    "\n",
    "This example illustrates how the different levels work: \n",
    "- `Facet`: [Agents](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264089)\n",
    "    - `Hierarchy`: [People (hierarchy name)](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300024978)\n",
    "        - `Guide Term`: [ < people by occupation > ](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300024980)\n",
    "            - `Record`: [taxonomists](https://www.getty.edu/vow/AATFullDisplay?find=beauty&logic=AND&note=&subjectid=300237758)\n",
    "                - `Term`: [taxonoom (Dutch)](https://www.getty.edu/vow/AATFullDisplay?find=beauty&logic=AND&note=&subjectid=300237758)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at this example record as seen from an end-user, and see some of the terminology in practice. \n",
    "\n",
    "<div>\n",
    "<img src=\"img/example_record.png\" width=\"600\"/>\n",
    "</div>\n",
    "\n",
    "\n",
    "[Image source](https://www.getty.edu/research/tools/vocabularies/aat/AAT-Users-Manual.pdf), p32. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What does the AAT contain?\n",
    "\n",
    "The AAT is structured as a hierarchical database of concepts, containing seven **facets** that each host a number of **hierarchies**. Below you can see an outline of the facets and hierarchies, including example terms. See [here](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&page=1&subjectid=300000000) for a collapsable hierarchy display on the Getty website. It allows you to browse through the structure and inspect the leaf nodes. You may see that each facet has its own internal logic to order the nodes (e.g. on function, geographical, temporal or cultural information). \n",
    "\n",
    "The root of the structure is called *Top of the AAT hierarchies*. Each facet in the table below contains a clickable link to their hierarchy on the Getty website. \n",
    "\n",
    "Most facets are relatively straightforward in their meaning. The final facet, *Brand Names*, allows for necessary additions by the conservation community, particularly where a material, process, or object does not have a generic name and the names are under trademark protection. The largest facet is the *Objects* facet. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [`Root`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300000000): **Top of the AAT hierarchies**\n",
    "\t- [`ASSOCIATED CONCEPT`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264086) FACET \\\n",
    "\t\t(e.g. *beauty*, *socialism*, *cultural pluralism*)\n",
    "\t\t- Associated Concepts \n",
    "\t- [`PHYSICAL ATTRIBUTES`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&page=1&subjectid=300264087) FACET \\\n",
    "\t\t(e.g. *borders, round, waterlogged*)\n",
    "\t\t- Attributes and Properties\n",
    "\t\t- Conditions and Effects\n",
    "\t\t- Design Elements\n",
    "\t\t- Color\n",
    "\t- [`STYLES AND PERIODS`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264088) FACET \\\n",
    "\t\t(e.g. *Abstract Expressionist, Yoruba*)\n",
    "\t\t- Styles and Periods\n",
    "\t- [`AGENTS`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&page=1&subjectid=300264089) FACET \\\n",
    "\t\t(e.g. *printmakers, landscape architects*)\n",
    "\t\t- People\n",
    "\t\t- Organizations\n",
    "\t\t- Living Organisms\n",
    "\t- [`ACTIVITIES`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264090) FACET \\\n",
    "\t\t(e.g. *archaeology, engineering, analyzing*)\n",
    "\t\t- Disciplines\n",
    "\t\t- Functions\n",
    "\t\t- Events\n",
    "\t\t- Physical and Mental Activities\n",
    "\t\t- Processes and Techniques\n",
    "\t- [`MATERIALS`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264091) FACET \\\n",
    "\t\t(e.g. *iron, clay, artificial ivory*)\n",
    "\t\t- Materials\n",
    "\t- [`OBJECTS`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300264092) FACET \\\n",
    "\t\t(e.g. *paintings, facades, cathedrals, chairs*)\n",
    "\t\t- Object Groupings and Systems\n",
    "\t\t- Object Genres \n",
    "\t\t- Components\n",
    "\t\t- Built Environment\n",
    "\t\t\t- Settlements and Landscapes\n",
    "\t\t\t- Built Complexes and Districts\n",
    "\t\t\t- Single Built Works\n",
    "\t\t\t- Open Spaces and Site Elements\n",
    "\t\t- Furnishings and Equipment\n",
    "\t\t\t- Furnishings\n",
    "\t\t\t- Costume\n",
    "\t\t\t- Tools and Equipment\n",
    "\t\t\t- Weapons and Ammunition\n",
    "\t\t\t- Measuring Devices\n",
    "\t\t\t- Containers\n",
    "\t\t\t- Sound Decides\n",
    "\t\t\t-  Recreational Artifacts\n",
    "\t\t\t- Transportation Vehicles\n",
    "\t\t- Visual and Verbal Communication\n",
    "\t\t\t- Visual Works\n",
    "\t\t\t- Exchange Media\n",
    "\t\t\t- Information Forms\n",
    "\t- [`BRAND NAMES`](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300343372) FACET \\\n",
    "\t\t(e.g. *Agfacolor, Arches paper*)\n",
    "\t\t- Brand Names\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Take a look under the hood: a Turtle file\n",
    "The AAT was originally released in XML and Relational Table formats. More recently, it has been made available as[ Linked Open Data (LOD)](https://www.getty.edu/research/tools/vocabularies/lod/index.html) in JSON, RDF, N3/Turtle, and N-Triples formats. It's recommended to not work with the XML or Relational Table formats because they may become obsolete in the future. The LOD options are the more robust choice. \n",
    "\n",
    "Let's take a look at what a single record can look like in [Turtle](https://en.wikipedia.org/wiki/Turtle_(syntax)) (Terse RDF Triple Language, or .ttl) format. Turtle is a syntax for expressing data in the Resource Description Framework (RDF), which is used to represent information about resources in the web and data interchange. Turtle is designed to be more human-readable compared to other RDF syntaxes, like XML. Turtle supports the use of URIs to uniquely identify concepts, improving data retrieval and linking related information. \n",
    "\n",
    "\n",
    "Open the [aat_300123559.ttl file](sparql/aat_300123559.ttl) in a new window and try to identify the following information: \n",
    "- The record identifier \n",
    "- The label name \n",
    "- The date of the most recent modification\n",
    "- The parent label\n",
    "- The Dutch translations\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may have found that the record identifier is *300123559*, the label name *Attributes and Properties (hierarchy name)*, the date of the most recent modification *2015-07-03*, and the parent label *Physical Attributes Facet*. The Dutch translations of the terms and the descriptions are accessible through the `@nl` tags "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AAT License: ODC-By\n",
    "> All Getty Vocabulary data found on this site are made available openly and free from fees under the  [Open Data Commons Attribution License (ODC-By) 1.0.](https://opendatacommons.org/licenses/by/1-0/).\n",
    "\n",
    "Source: https://www.getty.edu/research/tools/vocabularies/obtain/index.html#citing_vocab\n",
    "\n",
    "According to the [ODC-By license summary](https://opendatacommons.org/licenses/by/summary/):\n",
    "    \n",
    "You are free:\n",
    "\n",
    "* To share: To copy, distribute and use the database.\n",
    "* To create: To produce works from the database.\n",
    "* To adapt: To modify, transform and build upon the database.\n",
    "\n",
    "As long as you:\n",
    "\n",
    "Attribute: You must attribute any public use of the database, or works produced from the database, in the manner specified in the license. For any use or redistribution of the database, or works produced from it, you must make clear to others the license of the database and keep intact any notices on the original database.\n",
    "\n",
    "## How to cite AAT\n",
    "\n",
    ">  To attribute the Getty Vocabularies under the Open Data Commons Attribution License (ODC-By) 1.0, please use the following language:\n",
    "For AAT: *Contains information from the J. Paul Getty Trust, Getty Research Institute, the Art & Architecture Thesaurus, which is made available under the [ODC Attribution License](http://opendatacommons.org/licenses/by/1-0/).*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How can the AAT be used? \n",
    "The AAT can be downloaded in various formats for local use, but more integrated options are available, such as the SPARQL Endpoint and the API. See the sections below with information on how to use them. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### [SPARQL Endpoint](https://vocab.getty.edu/)\n",
    "With the SPARQL Endpoint you can take advantage of the linked data principles that are integrated in the AAT, and it allows you to perform complex queries. \n",
    "\n",
    "Below you find a Python implementation of a SPARQL query. It contains a simple query that retrieves all term labels that correspond to a given record. You can change the output by specifying different `subjectID`s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndentationError",
     "evalue": "unindent does not match any outer indentation level (<string>, line 29)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m<string>:29\u001b[0;36m\u001b[0m\n\u001b[0;31m    from SPARQLWrapper import SPARQLWrapper, JSON\u001b[0m\n\u001b[0m                                                 ^\u001b[0m\n\u001b[0;31mIndentationError\u001b[0m\u001b[0;31m:\u001b[0m unindent does not match any outer indentation level\n"
     ]
    }
   ],
   "source": [
    "from SPARQLWrapper import SPARQLWrapper, JSON\n",
    "\n",
    "def query_getty_aat(subjectID):\n",
    "    # Define the SPARQL endpoint\n",
    "    endpoint = \"http://vocab.getty.edu/sparql\"\n",
    "    \n",
    "    # Define the SPARQL query with the input subjectID\n",
    "    knows_query = f\"\"\"\n",
    "    PREFIX aat: <http://vocab.getty.edu/aat/>\n",
    "    PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
    "    SELECT * {{\n",
    "        aat:{subjectID} rdfs:label ?label .\n",
    "        FILTER (LANG(?label) = 'nl'  || LANG(?label) = 'en'   )  \n",
    "    }}\"\"\"\n",
    "    \n",
    "    # Initialize the SPARQL wrapper\n",
    "    sparql = SPARQLWrapper(endpoint)\n",
    "    sparql.setQuery(knows_query)\n",
    "```\n",
    "\n",
    "No labels there\n",
    "\n",
    "If we then check what statements are made in AATOut_2Terms.ttl we see statements made about different labels of aat:300008458\n",
    "\n",
    "Query\n",
    "```sparql\n",
    "SELECT * WHERE { <http://vocab.getty.edu/aat/300008475> ?p ?v}\n",
    "```\n",
    "\n",
    "Results: \n",
    "```txt\n",
    "-------------------------------------------------------------------------------------------------------------------------\n",
    "| p                                              | v                                                                    |\n",
    "=========================================================================================================================\n",
    "| <http://www.w3.org/2008/05/skos-xl#prefLabel>  | <http://vocab.getty.edu/aat/term/1000754126-zh-Latn-pinyin-x-notone> |\n",
    "| <http://www.w3.org/2008/05/skos-xl#prefLabel>  | <http://vocab.getty.edu/aat/term/1000754124-zh-Hant>                 |\n",
    "| <http://www.w3.org/2008/05/skos-xl#prefLabel>  | <http://vocab.getty.edu/aat/term/1000468934-nl>                      |\n",
    "| <http://www.w3.org/2008/05/skos-xl#prefLabel>  | <http://vocab.getty.edu/aat/term/1000008475-en>                      |\n",
    "\n",
    "```\n",
    "\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "---------------------------------------------------------------------------------------------------------------\n",
    "```\n",
    "Similar for http://vocab.getty.edu/aat/term/1000008475-en  `SELECT * WHERE {<http://vocab.getty.edu/aat/term/1000008475-en> ?p ?v}` we get the following results\n",
    "\n",
    "\n",
    "```txt\n",
    "===============================================================================================================\n",
    "| <http://purl.org/dc/elements/1.1/identifier>      | \"1000008475\"                                            |\n",
    "| <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> | <http://www.w3.org/2008/05/skos-xl#Label>               |\n",
    "| <http://vocab.getty.edu/ontology#termPOS>         | <http://vocab.getty.edu/term/POS/NeuterAdjectival>      |\n",
    "| <http://www.w3.org/2008/05/skos-xl#literalForm>   | \"<settlements by location: topographical>\"@en           |\n",
    "| <http://vocab.getty.edu/ontology#term>            | \"settlements by location: topographical\"@en             |\n",
    "| <http://vocab.getty.edu/ontology#termType>        | <http://vocab.getty.edu/term/type/Descriptor>           |\n",
    "\n",
    "```\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "\n",
    "concept aat:300008458\n",
    "        a gvp:Concept -> a skos:Concept\n",
    "        <http://purl.org/dc/terms/license>  <http://opendatacommons.org/licenses/by/1.0/>;\n",
    "        <http://www.w3.org/2004/02/skos/core#inScheme>  <http://vocab.getty.edu/aat/>;\n",
    "        skos:prefLabel ... @en\n",
    "        skos:prefLabel ... @nl\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "ntStringAbbrev>  \"cathedral centers, <settlements by function: administrative>, ... Objects Facet\";\n",
    "        <http://vocab.getty.edu/ontology#parentString>  \"cathedral centers, <settlements by function: administrative>, <settlements by function>, inhabited places, Settlements and Landscapes (hierarchy name), Built Environment (hierarchy name), Objects Facet\";\n",
    "the explicit.zip contains several .nl files:\n",
    "``` \n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "\n",
    "```\n",
    "\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "        <http://vocab.getty.edu/ontology#parentStringAbbrev>  \"cathedral centers, <settlements by function: administrative>, ... Objects Facet\";\n",
    "        <http://vocab.getty.edu/ontology#parentString>  \"cathedral centers, <settlements by function: administrative>, <settlements by function>, inhabited places, Settlements and Landscapes (hierarchy name), Built Environment (hierarchy name), Objects Facet\";\n",
    "the explicit.zip contains several .nl files:\n",
    "``` \n",
    "AATOut_1Subjects.nt\n",
    "AATOut_2Terms.nt\n",
    "AATOut_AssociativeRels.nt\n",
    "AATOut_ContribRels.nt\n",
    "AATOut_Contribs.nt\n",
    "AATOut_HierarchicalRels.nt\n",
    "AATOut_Lang_sameAs.nt\n",
    "AATOut_Contribs.nt\n",
    "AATOut_HierarchicalRels.nt\n",
    "AATOut_Lang_sameAs.nt\n",
    "AATOut_LCSHAlignment.nt\n",
    "        <http://purl.org/dc/elements/1.1/identifier>  \"300008458\";\n",
    "\n",
    "```\n",
    "\n",
    "No labels theremat. But in history and culture, dates are rarely precise.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "        a       <http://vocab.getty.edu/ontology#Concept> .\n",
    "\n",
    "    sparql.setReturnFormat(JSON)\n",
    "    \n",
    "    # Execute the query and get the results\n",
    "    results = sparql.query().convert()\n",
    "    \n",
    "    # Print the results\n",
    "    print(\"Subject ID: \", subjectID)\n",
    "    for result in results[\"results\"][\"bindings\"]:\n",
    "        #print(\"subjectID:\", subjectID)\n",
    "        print(\" \"*3, \"label: \", result[\"label\"][\"value\"])\n",
    "   from SPARQLWrapper import SPARQLWrapper, JSON\n",
    "\n",
    "def query_getty_aat(subjectID):\n",
    "    # Define the SPARQL endpoint\n",
    "    endpoint = \"http://vocab.getty.edu/sparql\"\n",
    "    \n",
    "    # Define the SPARQL query with the input subjectID\n",
    "    knows_query = f\"\"\"\n",
    "    PREFIX aat: <http://vocab.getty.edu/aat/>\n",
    "    PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
    "    SELECT * {{\n",
    "        aat:{subjectID} rdfs:label ?label .\n",
    "        FILTER (LANG(?label) = 'nl'  || LANG(?label) = 'en'   )  \n",
    "    }}\"\"\"\n",
    "    \n",
    "    # Initialize the SPARQL wrapper\n",
    "    sparql = SPARQLWrapper(endpoint)\n",
    "    sparql.setQuery(knows_query)\n",
    "    sparql.setReturnFormat(JSON)\n",
    "    \n",
    "    # Execute the query and get the results\n",
    "    results = sparql.query().convert()\n",
    "    \n",
    "    # Print the results\n",
    "    print(\"Subject ID: \", subjectID)\n",
    "    for result in results[\"results\"][\"bindings\"]:\n",
    "        #print(\"subjectID:\", subjectID)\n",
    "        print(\" \"*3, \"label: \", result[\"label\"][\"value\"])\n",
    "    \n",
    "    print()\n",
    "\n",
    "######## Change the subjectIDs here ########\n",
    "subjectIDs = [\"300008458\", \"300386018\", \"300055821\"]\n",
    "\n",
    "\n",
    "for subjectID in subjectIDs:\n",
    "    query_getty_aat(subjectID)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [API](http://vocabsservices.getty.edu/Docs/Getty_Vocabularies_Web_Services_Documentation_v2.pdf)\n",
    "\n",
    "API Documentation: http://vocabsservices.getty.edu/AATService.asmx\n",
    "\n",
    "The AAT also offers an **[API](http://vocabsservices.getty.edu/AATService.asmx)** (Application Programming Interface), a tool that allows developers to access and manipulate data programmatically. This makes it easier to integrate into applications and websites.\n",
    "\n",
    "The API provides a straightforward way to retrieve specific information, such as related terms or hierarchical structures. It supports [various operations](http://vocabsservices.getty.edu/AATService.asmx), including **GetChildren** and **GetParents**, which allow you to explore the relationships between concepts. The API supports different protocols like **SOAP** and **HTTP**. \n",
    "\n",
    "To get started with the API, you can try out the [**GetChildren**](http://vocabsservices.getty.edu/AATService.asmx?op=AATGetChildren) operation on their website. You can enter any subject ID, for example `300015646`.\n",
    "\n",
    "Alternatively, can run **HTTP GET** requests for the operators that are included in the current repositoey (see the files [GetParentLabel](http/getparentlabel.http) and [getSubjectTerms](http/getsubjectterms.http)). \n",
    "\n",
    "##### Negative aspects of Getty API?\n",
    "\n",
    "* responses are in XML (a bit antiquated and a pain to parse)\n",
    "* limited number of API methods\n",
    "* soon obsolete: according to statement in the [Getty LOD pages](https://www.getty.edu/research/tools/vocabularies/lod/index.html) it is stated (Dec.2024) that they will be updating Getty Vocabularies tech infrastructure, and retiring the XML (aka this API) at the end of 2025\n",
    "* information present in the API response, is rather poor, when compared to what the SPARQL-endpoint RDF files/dump return \n",
    "    * no Dutch(NL) label is returned\n",
    "    * no (immediately visible) information on the class of the term - is it a skos:Concept or is it a skos:Collention? ¯\\_(ツ)_/¯ \n",
    "\n",
    "Note: see an example response to the API method `AATGetSubject` by entering the AAT concept id ie. [300444999](http://vocab.getty.edu/page/aat/300444999) (label:'Post-Colonial'@en`) in http://vocabsservices.getty.edu/AATService.asmx/AATGetSubject you will see what I mean, comparing to the information present in the SPARQL-endpoint/RDF ([JSON](https://vocab.getty.edu/aat/300444999.json), [Semantic View](https://vocab.getty.edu/aat/300444999))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Language Labels in AAT\n",
    "\n",
    "AAT represent textual labels as SKOSXL nodes, and attach additional information to them. These nodes are values of the `skosxl:prefLabel` and `skosxl:altLabel` properties, declared for each concept.\n",
    "\n",
    "```txt\n",
    "aat:300000202  rdf:type               gvp:Subject , skos:Concept , gvp:Concept;\n",
    "\n",
    "        skosxl:altLabel               <http://vocab.getty.edu/aat/term/1000410399-es> , <http://vocab.getty.edu/aat/term/1000777859-pt> , <http://vocab.getty.edu/aat/term/1000651016-de> , <http://vocab.getty.edu/aat/term/1000651017-de> , <http://vocab.getty.edu/aat/term/1000651014-de> , <http://vocab.getty.edu/aat/term/1000887987-zh-Hant> , <http://vocab.getty.edu/aat/term/1000515217-en> , <http://vocab.getty.edu/aat/term/1000289816-nl> , <http://vocab.getty.edu/aat/term/1000289816-en>;\n",
    "\n",
    "        skosxl:prefLabel              <http://vocab.getty.edu/aat/term/1000410398-es> , <http://vocab.getty.edu/aat/term/1000651015-de> , <http://vocab.getty.edu/aat/term/1000462963-nl> , <http://vocab.getty.edu/aat/term/1000887990-zh-Latn-wadegile> , <http://vocab.getty.edu/aat/term/1000887988-zh-Latn-pinyin-x-notone> , <http://vocab.getty.edu/aat/term/1000887989-zh-Latn-pinyin-x-hanyu> , <http://vocab.getty.edu/aat/term/1000887986-zh-Hant> , <http://vocab.getty.edu/aat/term/1000000202-en>;\n",
    "\n",
    "```\n",
    "\n",
    "And each skosxl nodes, has the following set of property and value pairs, from which the most important for us are  `skosxl:literalForm` and `dc:language`\n",
    "\n",
    "```txt\n",
    "<http://vocab.getty.edu/aat/term/1000289816-nl>\n",
    "        rdf:type                     skosxl:Label;\n",
    "        dc11:identifier              \"1000289816\";\n",
    "        dc:contributor               <http://vocab.getty.edu/aat/contrib/10000205> , <http://vocab.getty.edu/aat/contrib/10000000>;\n",
    "        dc:language                  <http://vocab.getty.edu/language/nl> , aat:300388256;\n",
    "        dc:source                    <http://vocab.getty.edu/aat/source/2000099099> , <http://vocab.getty.edu/aat/source/2000113770> , <http://vocab.getty.edu/aat/source/2000114707>;\n",
    "        gvp:contributorNonPreferred  <http://vocab.getty.edu/aat/contrib/10000205> , <http://vocab.getty.edu/aat/contrib/10000000>;\n",
    "        gvp:displayOrder             \"2\"^^xsd:positiveInteger;\n",
    "        gvp:qualifier                \"gebouw\"@nl;\n",
    "        gvp:sourceNonPreferred       <http://vocab.getty.edu/aat/source/2000113770> , <http://vocab.getty.edu/aat/source/2000114707>;\n",
    "        gvp:term                     \"complex\"@nl;\n",
    "        gvp:termType                 <http://vocab.getty.edu/term/type/AlternateDescriptor>;\n",
    "        skosxl:literalForm           \"complex (gebouw)\"@nl .\n",
    "```\n",
    "\n",
    "Note: on SPARQL queries from https://vocab.getty.edu/sparql you might see concepts textual labels under `rdfs:label` property. However these are not clearly state, but only inferred by Getty SPARQL processor. This inferred values, will not be present when we perform remote SPARQL queries.\n",
    "\n",
    "\n",
    "\n",
    "If we take an example the concept aat_300000202.ttl *complexes (buildings and sites)* and query its `skosxl:prefLabel`\n",
    "\n",
    "\n",
    "```txt\n",
    "SELECT * {\n",
    "  aat:300000202 rdfs:label ?label \n",
    "}\n",
    "```\n",
    "\n",
    "```txt\n",
    "SELECT * {\n",
    "  aat:300000202 skosxl:prefLabel ?prelabel \n",
    "}\n",
    "```\n",
    "\n",
    "COME BACK TO THIS, ONCE UNDERSTOOD\n",
    "\n",
    "Do I get the same number of results if I query:\n",
    "- the concepts with rdfs:label in @en and @nl?\n",
    "- the concepts with skosxl:prefLabel in @en and @nl?\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "```txt\n",
    "<http://vocab.getty.edu/aat/300008458>\n",
    "        <http://vocab.getty.edu/ontology#displayOrder>  \"1\"^^<http://www.w3.org/2001/XMLSchema#positiveInteger>;\n",
    "        <http://vocab.getty.edu/ontology#parentStringAbbrev>  \"cathedral centers, <settlements by function: administrative>, ... Objects Facet\";\n",
    "        <http://vocab.getty.edu/ontology#parentString>  \"cathedral centers, <settlements by function: administrative>, <settlements by function>, inhabited places, Settlements and Landscapes (hierarchy name), Built Environment (hierarchy name), Objects Facet\";\n",
    "        <http://purl.org/dc/elements/1.1/identifier>  \"300008458\";\n",
    "        <http://creativecommons.org/ns#license>  <http://opendatacommons.org/licenses/by/1.0/>;\n",
    "        <http://www.w3.org/2004/02/skos/core#inScheme>  <http://vocab.getty.edu/aat/>;\n",
    "        <http://rdfs.org/ns/void#inDataset>  <http://vocab.getty.edu/dataset/aat>;\n",
    "        <http://purl.org/dc/terms/license>  <http://opendatacommons.org/licenses/by/1.0/>;\n",
    "        a       <http://vocab.getty.edu/ontology#Concept> .\n",
    "\n",
    "```\n",
    "\n",
    "There are no labels.\n",
    "\n",
    "Given that dating information about people, places, and things is oftelations, if we enable inference ([see query](https://vocab.getty.edu/queries?toc=&query=%23+find+subject+with+skos%3Anarrower+relationships+to+other+subject+%0D%0A%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0A%0D%0ASELECT+*%0D%0AWHERE+%7B%0D%0A++++%3Fs+skos%3Anarrower+%3Fo+.%0D%0A%7D%0D%0ALIMIT+10&implicit=true&equivalent=false#Finding_Subjects)). If querying the Getty AAT SPARQL endpoint remotely, like we try in [sparql/aat_narrower.ttl](sparql/aat_narrower.ttl), we cannot rely on inferred statements, hence we get zero results from that query.\n",
    "\n",
    "\n",
    "\n",
    "# Hierarchical Representations in AAT \n",
    "\n",
    "**The Getty AAT does not explicitly use the common SKOS properties `skos:boarder` and `skos:narrower` to denote the hierarchical relations** between its concepts. Instead it uses `gpv:narrower`and `gpv:broader`. However skos hierarchies are possible to query via [Getty SPAQL web UI](https://vocab.getty.edu)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The AAT in other projects\n",
    "\n",
    "- **[Termennetwerk](https://termennetwerk.netwerkdigitaalerfgoed.nl/)**: This project presents a Dutch search engine for terms and links them to their URIs in various thesauri, including the AAT.\n",
    "\n",
    "- **[Europeana](https://www.europeana.eu/en)**: An online information portal that provides access to millions of cultural heritage resources by aggregating metadata from museums, libraries, and archives across Europe. A part of their records are encoded with AAT URIs, which they use to retrieve translations of the records [(source)](https://doi.org/10.7152/nasko.v5i1.15179 ).  \n",
    "\n",
    "- **[A Methodology for Semantic Enrichment of Cultural Heritage Images Using Artificial Intelligence Technologies](https://doi.org/10.3390/jimaging7080121)**: This paper proposes a methodology for analyzing and enriching a collection of cultural images, specifically focusing on food in a cultural context. It combines the use of ontologies, including the AAT, to consistently represent concepts alongside Computer Vision tools to enhance image descriptions. --> expand, read method, what part of getty? how did they select it? \n",
    "\n",
    "- **[\"Linking HBIM Graphical and Semantic Information Through the Getty AAT\"](https://doi.org/10.1088/1757-899X/364/1/012100)**: This study explores the integration of graphical and semantic information in Historic Building Information Modeling (HBIM) using the AAT. It develops a method that automatically links elements from a 3D model of a Spanish castle to concepts in the Getty AAT. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other Getty Vocabularies\n",
    "\n",
    "### [ULAN](https://www.getty.edu/research/tools/vocabularies/ulan/about.html)\n",
    "The AAT is one example of the vocabularies that are developed by the Getty Institute. There's [more](https://www.getty.edu/research/tools/vocabularies/index.html), for example the Union List of Artist Names (ULAN). It contains records for artists and agents in the cultural landscape, specifically the visual arts. This includes [names, relationships, and biographical information for makers and other people and corporate bodies](https://www.getty.edu/research/tools/vocabularies/ulan/about.html#scope).\n",
    "\n",
    "Similar to the AAT, the ULAN consists or *records*, in this case referring to a unique person, institute or corporation. A record can contain mutliple *terms* that may capture [given names, pseudonyms, variant spellings, names in multiple languages, and names that have changed over time](https://www.getty.edu/research/tools/vocabularies/ulan/about.html#scope). A minimal record contains the following fields: record type, name, name source, display biography, nationality, role, birth date and death date. \n",
    "\n",
    "See an overview of the ULAN Facets below. Information is taken from the [ULAN documentation](https://www.getty.edu/research/tools/vocabularies/ulan/about.html#scope)\n",
    "\n",
    "\n",
    "- `PERSONS, ARTIST` Facet: represents information about individuals involved in the creation or production of works of art or architecture (e.g., *Rembrandt van Rijn*)\n",
    "- `CORPORATE BODIES` Facet: represents information about corporate bodies, defined as two or more people working together to create or produce art or architecture (e.g., *Adler and Sullivan*)\n",
    "- `NON-ARTISTS` Facet: mostly represents patrons, who often had input in the creative process, and occasionally donors, sitters, and others whose names are required for indexing visual works but who are themselves not artists.  \n",
    "- `UNKNOWN PEOPLE BY CULTURE` Facet: refers to the generic culture in which a work was created (e.g. *unknown Aztec*, or simply *Aztec*)\n",
    "- `UNIDENTIFIED NAMED PEOPLE ` Facet: people or corporate bodies where the identity is knowable, but has not yet been thoroughly researched\n",
    "\n",
    "Records may be linked through associative relationships, including professional relationships (e.g. *assistant of*, *influenced*) and familial relationships (e.g. *child of*, *sibling of*). See [here](https://www.getty.edu/vow/ULANFullDisplay?find=van+gogh&role=&nation=&page=1&subjectid=500115588) for an example of an ULAN record. \n",
    "\n",
    "The ULAN is also available as Linked Open Data, and can be accessed through the Sparql Endpoint. Most of the ULAN records are in English, but notes and descriptions are sometimes translated to other languages, including Dutch. \n",
    "\n",
    "\n",
    "### [Iconography Authority]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AAT and Data Station SSH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the context of Data Station SSH, the AAT can be useful to provide record names as keywords. However, curation should probably happen some way to select terms at a relevant granularity. The section below outlines challenges, recommendations, and some concrete properties that can be helpful in this curation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Challenges\n",
    "- **Selecting the right granularity**: The main challenge would be to find a way to curate which concepts should be included or excluded. \n",
    "  - **Size**: the AAT contains more than 70.000 records that all refer to unique concepts. If we would use the Dutch terms for each concept, it would mean we have 50.000 keywords to choose from. Many of these terms may be topically irrelevant, too specific or too broad for our purposes. \n",
    "  - **Lack of generalizability across facets**: each facet has its own internal logic. A path that may lead to useful terms in one facet might not translate to another facet. \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### Recommendations \n",
    "- Curation per facet: establish a path to a layer that yields keyword at the desired level of granularity for each facet. For example, in the *Associated concepts* facet, the [leaf nodes](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300055938) generally seem useful terms to function as keywords, whereas the hierarchy names are quite generic. In contrast, in the *Physical attribute* facet, some of the [leaf nodes](https://www.getty.edu/vow/AATHierarchy?find=beauty&logic=AND&note=&subjectid=300185404) seem too specific to be useful keywords (e.g. *halation, oxidative-reductive deterioration*). \n",
    "- Some facets may be excluded entirely (e.g. *Brand Names*, possibly *Activities*)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following properties are useful for the process of selecting the records: \n",
    "- `Subject_ID`: This is the unique identifier for a concept record.  \n",
    "- `Parent_ID`: This allows for tracing the source of a record \n",
    "- `Term_Language`: Allows for the possibly to only keep Dutch and English entries\n",
    "- `Term_ID`: The unique identifier for a term\n",
    "- `Term_Text`: The text or description belonging to a term\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each node belongs to one of the following `Record_Type` categories: \n",
    "- Facet\n",
    "- Hierarchy\n",
    "- Concept\n",
    "- GuideTerm (placeholder to create a level in the hierarchy)\n",
    "- (ScopeNote)\n",
    "- (ObsoleteSubject)\n",
    "\n",
    "These labels can help filter out unnecessary records. For example, Facet names or GuideTerms will rarely, if ever, be useful as keywords. \n",
    "\n",
    "Moreover, the SKOS predicates such as `broader` can be used to define the desired paths.  \n",
    "\n",
    "![example of the use of different terms](img/008-complex-hierarchy.png)\n",
    "\n",
    "[image source](https://vocab.getty.edu/doc/#Term)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}