InconsistentTriplestoreDataException/has no creator error #1488

gfoo · 2019-10-23T09:29:54Z

I get this error on a gravsearch query:

webapi_1   | 2019-10-23 09:12:05 | ERROR | Dispatcher | Entity http://rdfh.ch/0113/9h2huqLgS4aG9d3xLgXcIQ/values/-szOyfUkRnGYbF0UULjnUg has no creator
webapi_1   | org.knora.webapi.InconsistentTriplestoreDataException: Entity http://rdfh.ch/0113/9h2huqLgS4aG9d3xLgXcIQ/values/-szOyfUkRnGYbF0UULjnUg has no creator
webapi_1   |    at org.knora.webapi.util.PermissionUtilADM$.$anonfun$getUserPermissionFromAssertionsADM$1(PermissionUtilADM.scala:414)
webapi_1   |    at scala.collection.MapLike.getOrElse(MapLike.scala:131)
webapi_1   |    at scala.collection.MapLike.getOrElse$(MapLike.scala:129)
webapi_1   |    at scala.collection.AbstractMap.getOrElse(Map.scala:63)
webapi_1   |    at org.knora.webapi.util.PermissionUtilADM$.getUserPermissionFromAssertionsADM(PermissionUtilADM.scala:414)
webapi_1   |    at org.knora.webapi.util.PermissionUtilADM$.getUserPermissionFromConstructAssertionsADM(PermissionUtilADM.scala:389)
webapi_1   |    at org.knora.webapi.util.ConstructResponseUtilV2$.$anonfun$splitMainResourcesAndValueRdfData$16(ConstructResponseUtilV2.scala:411)
webapi_1   |    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
webapi_1   |    at scala.collection.immutable.List.foreach(List.scala:392)
webapi_1   |    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
webapi_1   |    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
webapi_1   |    at scala.collection.immutable.List.map(List.scala:298)
webapi_1   |    at org.knora.webapi.util.ConstructResponseUtilV2$.$anonfun$splitMainResourcesAndValueRdfData$15(ConstructResponseUtilV2.scala:395)
webapi_1   |    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
webapi_1   |    at scala.collection.immutable.Map$Map3.foreach(Map.scala:195)
webapi_1   |    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
webapi_1   |    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
webapi_1   |    at scala.collection.AbstractTraversable.map(Traversable.scala:108)
webapi_1   |    at org.knora.webapi.util.ConstructResponseUtilV2$.$anonfun$splitMainResourcesAndValueRdfData$6(ConstructResponseUtilV2.scala:391)
webapi_1   |    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
webapi_1   |    at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:234)
webapi_1   |    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:465)
webapi_1   |    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:465)
webapi_1   |    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:465)
webapi_1   |    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
webapi_1   |    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
webapi_1   |    at scala.collection.AbstractTraversable.map(Traversable.scala:108)
webapi_1   |    at org.knora.webapi.util.ConstructResponseUtilV2$.splitMainResourcesAndValueRdfData(ConstructResponseUtilV2.scala:330)
webapi_1   |    at org.knora.webapi.responders.v2.search.MainQueryResultProcessor$.getMainQueryResultsWithFullGraphPattern(MainQueryResultProcessor.scala:221)
webapi_1   |    at org.knora.webapi.responders.v2.SearchResponderV2.$anonfun$gravsearchV2$13(SearchResponderV2.scala:539)
webapi_1   |    at scala.util.Success.$anonfun$map$1(Try.scala:255)
webapi_1   |    at scala.util.Success.map(Try.scala:213)
webapi_1   |    at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
webapi_1   |    at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
webapi_1   |    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
webapi_1   |    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
webapi_1   |    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
webapi_1   |    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
webapi_1   |    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
webapi_1   |    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
webapi_1   |    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
webapi_1   |    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
webapi_1   |    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
webapi_1   |    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
webapi_1   |    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
webapi_1   |    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
webapi_1   |    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

The text was updated successfully, but these errors were encountered:

gfoo · 2019-10-23T09:31:23Z

@benjamingeer and @tobiasschweizer I shared with you on switchdrive the file export-LL-23.10.2019.trig.zip (x-trig export of my graphdb repos)

gfoo · 2019-10-23T09:33:53Z

To reproduce with gravsearch (being the user [email protected] / 9e1601a8) :

PREFIX knora-api: <http://api.knora.org/ontology/knora-api/v2#>
PREFIX knora-simple-api: <http://api.knora.org/ontology/knora-api/simple/v2#>
PREFIX ll: <http://0.0.0.0:3333/ontology/0113/lumieres-lausanne/v2#>

CONSTRUCT {
  ?Collection knora-api:isMainResource true .
  ?Collection ll:collectionHasTitle ?collectionHasTitle .
?Collection ll:collectionHasUrlElement ?collectionHasUrlElement .
?Collection ll:collectionHasDescription ?collectionHasDescription .
?Collection ll:hasBibliographicNoticeElement ?hasBibliographicNoticeElement .
?Collection ll:hasTranscriptionElement ?hasTranscriptionElement .
?Collection ll:hasPersonElement ?hasPersonElement .
} WHERE {
    ?Collection a ll:Collection .
  ?Collection ll:collectionHasTitle ?collectionHasTitle .
?Collection ll:collectionHasUrlElement ?collectionHasUrlElement .
OPTIONAL {
?Collection ll:collectionHasDescription ?collectionHasDescription .
}
OPTIONAL {
?Collection ll:hasBibliographicNoticeElement ?hasBibliographicNoticeElement .
?hasBibliographicNoticeElement a ll:BibliographicNotice .
}
OPTIONAL {
?Collection ll:hasTranscriptionElement ?hasTranscriptionElement .
?hasTranscriptionElement a ll:Transcription .
}
OPTIONAL {
?Collection ll:hasPersonElement ?hasPersonElement .
?hasPersonElement a ll:Person .
}
}
ORDER BY ASC(?collectionHasTitle)
 OFFSET 2

gfoo · 2019-10-23T09:36:45Z

The related resource and its value reported by the error seem to be fine:
.../v2/resources/http%3A%2F%2Frdfh.ch%2F0113%2F9h2huqLgS4aG9d3xLgXcIQ

   {
      "@id": "http://rdfh.ch/0113/9h2huqLgS4aG9d3xLgXcIQ/values/-szOyfUkRnGYbF0UULjnUg",
      "@type": "knora-api:LinkValue",
      "knora-api:arkUrl": {
        "@type": "xsd:anyURI",
        "@value": "https://ark.dasch.swiss/ark:/72163/1/0113/9h2huqLgS4aG9d3xLgXcIQ8/4TyOg7WDQFyuRQ2vYMUuBQo"
      },
      "knora-api:attachedToUser": {
        "@id": "http://rdfh.ch/users/lumieres-lausanne-fantin_reichler"
      },
      "knora-api:hasPermissions": "CR http://rdfh.ch/groups/0113/lumieres-lausanne-administrator,http://rdfh.ch/groups/0113/lumieres-lausanne-director|D knora-admin:Creator|M http://rdfh.ch/groups/0113/lumieres-lausanne-phdstudent,http://rdfh.ch/groups/0113/lumieres-lausanne-researcher,http://rdfh.ch/groups/0113/lumieres-lausanne-student|V http://rdfh.ch/groups/0113/lumieres-lausanne-user,knora-admin:KnownUser,knora-admin:UnknownUser",
      "knora-api:linkValueHasTarget": {
        "@id": "http://rdfh.ch/0113/oDFFjxYWQ-iz6WBKwhtnCA",
        "@type": "lumieres-lausanne:BibliographicNotice",
        "knora-api:arkUrl": {
          "@type": "xsd:anyURI",
          "@value": "https://ark.dasch.swiss/ark:/72163/1/0113/oDFFjxYWQ=iz6WBKwhtnCAY"
        },
        "knora-api:attachedToProject": {
          "@id": "http://rdfh.ch/projects/0113"
        },
        "knora-api:attachedToUser": {
          "@id": "http://rdfh.ch/users/lumieres-lausanne-fantin_reichler"
        },
        "knora-api:creationDate": {
          "@type": "xsd:dateTimeStamp",
          "@value": "2017-03-07T12:23:41Z"
        },
        "knora-api:hasPermissions": "CR http://rdfh.ch/groups/0113/lumieres-lausanne-administrator,http://rdfh.ch/groups/0113/lumieres-lausanne-director|D knora-admin:Creator|M http://rdfh.ch/groups/0113/lumieres-lausanne-phdstudent,http://rdfh.ch/groups/0113/lumieres-lausanne-researcher|V http://rdfh.ch/groups/0113/lumieres-lausanne-student,http://rdfh.ch/groups/0113/lumieres-lausanne-user,knora-admin:KnownUser,knora-admin:UnknownUser",
        "knora-api:userHasPermission": "D",
        "knora-api:versionArkUrl": {
          "@type": "xsd:anyURI",
          "@value": "https://ark.dasch.swiss/ark:/72163/1/0113/oDFFjxYWQ=iz6WBKwhtnCAY.20170307T122341Z"
        },
        "rdfs:label": "Qu'est-ce qu'un médecin des Lumières? Portraits et discours croisés de quelques contemporains de Tissot"
      },
      "knora-api:userHasPermission": "D",
      "knora-api:valueCreationDate": {
        "@type": "xsd:dateTimeStamp",
        "@value": "2019-10-22T15:30:12.399105Z"
      },
      "knora-api:valueHasUUID": "4TyOg7WDQFyuRQ2vYMUuBQ",
      "knora-api:versionArkUrl": {
        "@type": "xsd:anyURI",
        "@value": "https://ark.dasch.swiss/ark:/72163/1/0113/9h2huqLgS4aG9d3xLgXcIQ8/4TyOg7WDQFyuRQ2vYMUuBQo.20191022T153012399105Z"
      }
    },

benjamingeer · 2019-10-23T09:47:16Z

Is the resource very large? If so, it could be that the SPARQL query result is truncated. @tobiasschweizer would you have time to look at this?

gfoo · 2019-10-23T10:09:47Z

Is the resource very large?

res.json.zip

tobiasschweizer · 2019-10-23T11:47:57Z

If so, it could be that the SPARQL query result is truncated.

Yes, I think we had such problems before. But you removed the size limit, didn't you?

benjamingeer · 2019-10-23T11:50:17Z

It's still there in the scripts in src/webapi/scripts:

owlim:query-limit-results "50000";

tobiasschweizer · 2019-10-23T11:54:50Z

@gfoo Could you just try to make the limit higher and see if this fixes the problem? You have to change it it the template file, e.g., webapi/scripts/graphdb-se-knora-prod-repository-config.ttl.tmpl.

I thought it would be good to have both a time and size limit to protect agains overload.

Of course, it is hard so say what a reasonable limit is. Could we make some recommendations in terms of how many properties a resource should have at max.? I really do not want to implement paging for resource requests.

benjamingeer · 2019-10-23T11:58:43Z

Relevant discussion with @mrivoal here:

#1320 (comment)

tobiasschweizer · 2019-10-23T12:01:48Z

In the past I have spotted problems when GraphDB couldn't serialize a response because it was too large. This could also be problem with fewer properties that have large values like texts, see https://github.com/dhlab-basel/knora-large-texts.

benjamingeer · 2019-10-23T12:04:19Z

In the past I have spotted problems when GraphDB couldn't serialize a response because it was too large. This could also be problem with fewer properties that have large values like texts

Yes, I think I've seen that too, but haven't had time to investigate it yet. I wonder if there's a configurable response size limit in GraphDB (in KB rather than in triples).

tobiasschweizer · 2019-10-23T12:07:20Z

I wonder if there's a configurable response size limit in GraphDB (in KB rather than in triples).

Good question. I couldn't find anything like this on http://graphdb.ontotext.com/documentation/free/configuring-a-repository.html#configure-a-repository-programmatically

benjamingeer · 2019-10-23T12:08:32Z

It would be nice to know how many bytes GraphDB is actually returning in this case. That could give us a clue.

benjamingeer · 2019-10-23T12:09:14Z

(and how many statements)

tobiasschweizer · 2019-10-23T12:09:37Z

It would be nice to know how many bytes GraphDB is actually returning in this case. That could give us a clue.

Haven't you already written some kind of logic that could be used for this in the code that handles the communication with the triplestore?

benjamingeer · 2019-10-23T12:16:36Z

In HttpTriplestoreConnector.getSparqlHttpResponse, you could print the length of the response string before returning it:

https://github.com/dasch-swiss/knora-api/blob/f089f4b7820f47bbda93c9af33de88e5570cd8d3/webapi/src/main/scala/org/knora/webapi/store/triplestore/http/HttpTriplestoreConnector.scala#L677

If it's an extended construct response, it will be parsed by ConstructResponseTurtleHandler, which could count the statements in handleStatement and print out the statement count in endRDF:

https://github.com/dasch-swiss/knora-api/blob/f089f4b7820f47bbda93c9af33de88e5570cd8d3/webapi/src/main/scala/org/knora/webapi/messages/store/triplestoremessages/TriplestoreMessages.scala#L192

gfoo · 2019-10-23T13:37:15Z

@gfoo Could you just try to make the limit higher and see if this fixes the problem? You have to change it it the template file, e.g., webapi/scripts/graphdb-se-knora-prod-repository-config.ttl.tmpl.

yes it works with owlim:query-limit-results "100000"

benjamingeer · 2019-10-23T13:51:02Z

@gfoo Good to know. The problem with owlim:query-limit-results is that it just truncates the result if it's too big. It would be better to have a setting that returned an error if the result was too big.

tobiasschweizer · 2019-10-23T13:51:59Z

yes it works with owlim:query-limit-results "100000"

So, where's your problem then? ;-)

gfoo · 2019-10-23T13:52:16Z

Anyway, in my particular case ~~I could also to not ask~~ I could also avoid asking has*Element properties to really limit data retrieved.

The main problem there is to understand the error :)

gfoo added API/V2 Gravsearch labels Oct 23, 2019

subotic added this to the Backlog milestone Feb 7, 2020

irinaschubert removed this from the Backlog milestone Dec 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

InconsistentTriplestoreDataException/has no creator error #1488

InconsistentTriplestoreDataException/has no creator error #1488

gfoo commented Oct 23, 2019

gfoo commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading

benjamingeer commented Oct 23, 2019

gfoo commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019 •

edited

Loading

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading

InconsistentTriplestoreDataException/has no creator error #1488

InconsistentTriplestoreDataException/has no creator error #1488

Comments

gfoo commented Oct 23, 2019

gfoo commented Oct 23, 2019 • edited Loading

gfoo commented Oct 23, 2019 • edited Loading

gfoo commented Oct 23, 2019 • edited Loading

benjamingeer commented Oct 23, 2019

gfoo commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019 • edited Loading

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019

benjamingeer commented Oct 23, 2019 • edited Loading

gfoo commented Oct 23, 2019

benjamingeer commented Oct 23, 2019

tobiasschweizer commented Oct 23, 2019 • edited Loading

gfoo commented Oct 23, 2019 • edited Loading

gfoo commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading

tobiasschweizer commented Oct 23, 2019 •

edited

Loading

benjamingeer commented Oct 23, 2019 •

edited

Loading

tobiasschweizer commented Oct 23, 2019 •

edited

Loading

gfoo commented Oct 23, 2019 •

edited

Loading