Malformed strings with replacement characters (�)

[mdeagen]

Somewhere in data ETL to the RDF triple store, special characters are replaced with the replacement character, �.

The issue also occurs when saving a Vega-Lite chart spec containing any Unicode escape string (e.g., \u00b0 for ° symbol).

Below is a SPARQL query that returns many (~600) of these malformed strings, which include article titles, author names, and definitions of terms:

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dct: <http://purl.org/dc/terms/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX sio: <http://semanticscience.org/resource/>
SELECT * WHERE {
  ?sub ?pred ?MalformedString
  FILTER(REGEX(STR(?MalformedString), "�"))
}
VALUES ?pred { dct:title foaf:name rdfs:label skos:definition dcat:keyword dct:description skos:altLabel skos:notation }
# NOTE: sio:hasValue also has ~1500 malformed strings as objects, but query time gets increased by ~120 seconds