Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add t value to pattern resolution #199

Open
quoll opened this issue Jul 14, 2021 · 1 comment
Open

Add t value to pattern resolution #199

quoll opened this issue Jul 14, 2021 · 1 comment

Comments

@quoll
Copy link
Contributor

quoll commented Jul 14, 2021

Datomic expects to bind a 4th element to a transaction "t" value.
e.g.
[:entity :attr ?value ?t]

To do this, the basic graph patterns will need to result in a t-value lookup in the indexes. These are currently stored in the durable system in another index (and will require a join), and not stored at all in the in-memory index (ticket #198).

The current graph API has resolve-triple with 4 arguments (graph, subject, predicate, object).

We should consider new arities here, though a simple extension to a 5th argument will cause problems with considering bindings of statement IDs. The new arity may need to accept 6 arguments.

When considering the new arities with respect to #200 we are looking for the following mappings of BGPs:

User Syntax Internal Representation
[?e ?a ?v] [?e ?a ?v _ _]
[?e ?a ?v ?t] [?e ?a ?v _ ?t]
[?e ?a ?v :as ?stmt] [?e ?a ?v ?stmt _]
[?e ?a ?v ?t :as ?stmt] [?e ?a ?v ?stmt ?t]

There are 3 cases to consider:

  • The t value is variable: Lookup the value and return in bindings.
  • The t is set, along with other values: The t value is then used as a filter (directly on statements for in-memory, and via the tSPO index for durable).
  • The t is set, and everything else is variable: Filter for in-memory, but use the tSPO index for durable storage.

Depends on t values from #198

@quoll
Copy link
Contributor Author

quoll commented Jul 23, 2021

Initial data

(def d1 [[:a :p 1] [:b :p 2]])  ;; transaction 1
(def d2 [[:c :p 3] [:d :p 4]])  ;; transaction 2
(transact c {:tx-triples d1})
(transact c {:tx-triples d2})

T value is variable

Querying for transaction IDs:

=> (q '[:find ?e ?v ?t :where [?e :p ?v ?t]] (as-of (db c) 1))
([:a 1 1] [:b 2 1])
=> (q '[:find ?e ?v ?t :where [?e :p ?v ?t]] (as-of (db c) 2))
([:a 1 1] [:b 2 1] [:c 3 2] [:d 4 2])

Performed by looking up the queries as [?e :p ?v], then:

  • in memory: emit the :t value
  • durable: lookup the tspo index, and emit column 0

T value is set. Other values set

Querying using transaction IDs. Predicate is set to :p:

=> (q '[:find ?e ?v :where [?e :p ?v 1]] (db c))
([:a 1] [:b 2])
=> (q '[:find ?e ?v :where [?e :p ?v 2]] (db c))
([:c 3] [:d 4])
=> (q '[:find ?e ?v :where [?e :p ?v 2]] (as-of (db c) 1))
()

Performed by looking up the queries as [?e :p ?v], then:

  • in memory: filter by :t value
  • durable: lookup the tspo index, filter by column 0

T value is set. All other values variable

Querying using transaction IDs.

=> (q '[:find ?e ?p ?v :where [?e ?p ?v 1]] (db c))
([:a :p 1] [:b :p 2])
=> (q '[:find ?e ?p ?v :where [?e ?p ?v 2]] (db c))
([:c :p 3] [:d :p 4])
=> (q '[:find ?e ?v :where [?e :p ?v 2]] (as-of (db c) 1))
()

Performed by:

  • in memory: Iterate over every triple in SPO. Filter by transaction value
  • durable: Binary search the tspo index on column 0 for the t value, and t+1.
    This gives 2 longs: lower-bound, upper-bound.
    Result is: (map (fn [id] (subvec (get-record tspo id) 1)) (range lower-bound upper-bound))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant