feat: search projects #1330

tpluscode · 2022-11-15T10:16:02Z

search.mp4

changeset-bot · 2022-11-15T10:16:06Z

🦋 Changeset detected

Latest commit: bbdce1d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@cube-creator/core-api	Minor
@cube-creator/ui	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

codecov-commenter · 2022-11-15T10:25:28Z

Codecov Report

Merging #1330 (51c4b91) into master (b715403) will decrease coverage by 0.38%.
The diff coverage is 9.85%.

@@            Coverage Diff             @@
##           master    #1330      +/-   ##
==========================================
- Coverage   81.41%   81.03%   -0.39%     
==========================================
  Files         194      195       +1     
  Lines       13453    13509      +56     
  Branches      753      754       +1     
==========================================
- Hits        10953    10947       -6     
- Misses       2492     2554      +62     
  Partials        8        8

Impacted Files	Coverage Δ
apis/core/lib/auth.ts	`0.00% <0.00%> (ø)`
apis/core/lib/domain/cube-projects/search.ts	`0.00% <0.00%> (ø)`
apis/core/lib/domain/cube-projects/create.ts	`84.11% <100.00%> (-0.37%)`	⬇️
apis/core/lib/domain/cube-projects/import.ts	`98.19% <100.00%> (-0.03%)`	⬇️
packages/core/namespaces/shapes.ts	`96.55% <100.00%> (+0.12%)`	⬆️
packages/model/Project.ts	`100.00% <100.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

re zazuko/kopflos#105

tpluscode · 2022-11-16T12:21:00Z

apis/core/bootstrap/cube-projects.ts

+<cube-projects/status> {
+  <cube-projects/status> a ${hydra.Collection}, ${schema.DefinedTermSet} ;
+    ${hydra.member} ${Draft}, ${Published} ;
+  .
+
+  ${Published} a ${schema.DefinedTerm} ;
+    ${rdfs.label} "Published" .
+
+  ${Draft} a ${schema.DefinedTerm} ;
+    ${rdfs.label} "Draft" .
+}


A static collection of statuses to populate the search dropdown

tpluscode · 2022-11-16T12:21:20Z

apis/core/bootstrap/shapes/cube-project.ts

+${shape('cube-project/search')} {
+  ${shape('cube-project/search')} a ${sh.NodeShape}, ${hydra.Resource} ;
+    ${sh.property} [
+      ${sh.name} "Keyword Search" ;
+      ${sh.order} 10 ;
+      ${sh.path} ${hydra.freetextQuery} ;
+      ${sh.minCount} 1 ;
+      ${sh.maxCount} 1 ;
+    ],
+    [
+      ${sh.name} "Author" ;
+      ${sh.path} ${dcterms.creator} ;
+      ${sh.order} 20 ;
+      ${sh.minCount} 1 ;
+      ${sh.maxCount} 1 ;
+      ${dash.editor} ${dash.InstancesSelectEditor} ;
+      ${hydra.collection} </users> ;
+    ],
+    [
+      ${sh.name} "Status" ;
+      ${sh.path} ${schema.creativeWorkStatus} ;
+      ${sh.order} 30 ;
+      ${sh.minCount} 1 ;
+      ${sh.maxCount} 1 ;
+      ${dash.editor} ${dash.InstancesSelectEditor} ;
+      ${hydra.collection} </cube-projects/status> ;
+    ] ;
+  .


Shape which defines the fields for search form

tpluscode · 2022-11-16T12:22:56Z

apis/core/bootstrap/users.ts

+<users> {
+  <users> a ${hydra.Collection} ;
+    ${hydra.manages} [
+      ${hydra.property} ${rdf.type} ;
+      ${hydra.object} ${schema.Person}
+    ] .
+}
+`


This reflects a bigger change where users actually become independent resources in their own named graphs. Here, the collection is added to retrieve them

I love this change!

tpluscode · 2022-11-16T12:26:21Z

apis/core/lib/auth.ts

+function updateUserResource(): RequestHandler {
+  const users = new TermSet()
+
+  return (req, res, next) => {
+    if (req.user?.id && !users.has(req.user.id)) {
+      const { id, name, sub } = req.user
+      users.add(id)
+
+      req.once('end', () => {
+        DELETE`
+          GRAPH ${id} {
+            ${id} ${schema.name} ?name .
+          }
+        `.INSERT`
+          GRAPH ${id} {
+            ${id} ${schema.name} "${name || sub}"; a ${schema.Person} .
+          }
+        `.WHERE`
+            OPTIONAL {
+              GRAPH ${id} {
+                ${id} ${schema.name} ?name .
+              }
+            }
+        `.execute(req.labyrinth.sparql.query)
+          .catch(warning)
+      })
+    }
+    next()
+  }
+}


Since we want users as resources, they need to be created at some point.

I added this middleware to run after authentication, so that any one that successfully logs in will have their user graph created automatically.

Importantly, the users set ensures that as long the API process is running, every user will be created only once.

A future improvement could be to actually load existing users in the beginning. On the other hand, this way will also keep the names updated, shall they change on the auth provider...

tpluscode · 2022-11-16T12:27:19Z

apis/core/lib/domain/cube-projects/import.ts

@@ -103,7 +101,7 @@ export async function importProject({
  const projectNode = await store.createMember(projectsCollection.term, id.cubeProject(label))

  const project = createMinimalProject(projectNode, {
-    creator: { id: user, name: userName },
+    creator: user,


Since we now have graphs for users, only the URI will be stored with the project now, unlike the lazy duplication we've had so far

tpluscode · 2022-11-16T12:28:01Z

apis/core/lib/domain/cube-projects/search.ts

+export function byFreeText({ subject, object }: Pattern) {
+  return sparql`
+    ${subject} ${rdfs.label}|${schema.alternateName}|${schema.disambiguatingDescription} ?freetextSearched .
+
+    FILTER(REGEX(?freetextSearched, "${object.value}", "i"))
+  `
+}
+
+export function byStatus({ subject, predicate, object }: Pattern) {
+  return sparql`
+    ${subject} ${cc.dataset} ?dataset .
+
+    graph ?dataset {
+      ?dataset ${predicate} ${object.term} .
+    }
+  `
+}
+
+export function byUser({ subject, predicate, object }: Pattern) {
+  return sparql`
+    ${subject} ${predicate} ${object.term} .
+  `
+}


Functions which provide the SPARQL patterns to search by respective query params (see apis/core/hydra/projects.ttl)

tpluscode · 2022-11-16T12:29:13Z

patches/hydra-box+0.6.6.patch

+      }))
+     })
+
+     req.dataset = () => {


A temporary patch to hydra-box which fixes handling of + in query string values. See zazuko/kopflos#104

Could you provide me a link to learn more about these kind of patches? I haven't seen this before

This is done using patch-package

When you modify a library in node_modules/foo-bar, you the run npx patch-package foo-bar. That will create the patch file.

Running patch-package in postinstall script applies the package when installed

tpluscode · 2022-11-16T12:31:18Z

ui/src/store/serializers.ts


 export const displayLanguage = ['en', 'de', 'fr', '']

 export function serializeProjectsCollection (collection: ProjectsCollection): ProjectsCollection {
  return Object.freeze({
    ...serializeResource(collection),
+    searchParams: clone(collection.pointer),


The server echoes the query string params as RDF. This makes a copy so that the form does modify the representation retrieved from the API

tpluscode · 2022-11-16T12:36:47Z

ui/src/views/CubeProjects.vue

+    onSearch (e: CustomEvent) {
+      const template = this.operation?.get<IriTemplate>(hydraBox.variables)
+      if (template && e.detail?.value) {
+        const url = template.expand(e.detail.value)
+        const search = new URLSearchParams(url.substring(url.indexOf('?')))
+        const entries = [...search.entries()].filter(([, value]) => !!value && value !== '""')
+
+        this.$router.push({
+          query: Object.fromEntries(entries),
+        })
+      }
+    }


I don't exactly like this.

The search is triggered by router query param change. In a different scenario I would construct the URL from template and then use that as a whole to change the "route"

My best solution here is to round-trip the expanded template back into the query params and push them to the router

yeah, it's difficult to understand. Perhaps you could factorize it in a function with a name ('pushSearchRoute'?) to explain the intent?

All right, I moved that logic to a function exported from @/router.

A bit of a mouthful. That might be the longest function name in this project now 🙈

well, they don't charge for number of chars in function names ;)

Else Java people would be richest :D

tpluscode · 2022-11-16T13:24:00Z

@l00mi a bigger change to be aware of is that users become their own resources. Until now the user names were simply stored with projects themselves

graph <project> {
  <project> dcterms:creator <user> .
  <user> schema:name "John Doe" .
}

Now it will be like

graph <project> {
  <project> dcterms:creator <user> .
}

graph <user> {
  <user> schema:name "John Doe" ; a schema:Person , hydra:Resource .
}

Projects already referred to users with URIs so I don't find this a breaking change per se.
User resources will be automatically created as users sign in. For existing users, I would run an update like this on every environment:

prefix dcterms: <http://purl.org/dc/terms/>
prefix schema: <http://schema.org/>
prefix cc: <https://cube-creator.zazuko.com/vocab#>
prefix hydra: <http://www.w3.org/ns/hydra/core#>

DELETE {
  GRAPH ?project {
    ?user schema:name ?name
  }
}
INSERT {
  GRAPH ?user {
    ?user schema:name ?name ; a schema:Person , hydra:Resource .
  }
} WHERE {
  GRAPH ?project {
    ?project a cc:CubeProject ; dcterms:creator ?user .
    ?user schema:name ?name .
  }
}

cristianvasquez · 2022-11-17T12:29:59Z

apis/core/lib/auth.ts

+      req.once('end', () => {
+        DELETE`
+          GRAPH ${id} {
+            ${id} ${schema.name} ?name .


If users live in their own named graph, wouldn't it be cleaner to delete the graph as a whole?

Well, this is but not deleting the whole graph. Only schema:name :)

cristianvasquez · 2022-11-17T12:48:17Z

ui/src/components/FormSubmitCancel.vue

@@ -17,6 +17,11 @@
          Cancel
        </o-button>
      </div>
+      <div class="control" v-if="showClear">


UI-wise, I think it would look better with an X instead of a button 'clear', (optional)

Yes, I expected you'd say that. Unfortunately, oruga select does not appear to allow that

cristianvasquez

Looks ok to me!

Only (optional) comments

feat: search projects

fb7eaed

Base automatically changed from project-meta to master November 16, 2022 10:37

tpluscode added 6 commits November 16, 2022 11:42

style: lint fix

b9d58f2

fix: text search pattern

e1ffaae

refactor: field rename

f9c26a0

fix: apply initial query

1c06999

fix: run search when form cleared

65fb0d7

fix: handing spaces in query string

d886998

re zazuko/kopflos#105

tpluscode marked this pull request as ready for review November 16, 2022 11:54

tpluscode requested a review from cristianvasquez November 16, 2022 11:54

fix: clear did not work on initial load

e6739a8

tpluscode commented Nov 16, 2022

View reviewed changes

tpluscode added 2 commits November 16, 2022 14:57

test: fix link check

6db47d3

test: make user a hydra:Resource

51c4b91

cristianvasquez reviewed Nov 17, 2022

View reviewed changes

cristianvasquez approved these changes Nov 17, 2022

View reviewed changes

tpluscode added 3 commits November 17, 2022 16:30

refactor: extract method

8cffcd6

test: guard for e2e tests

ec8aea2

test: must more precisely select the right form

bbdce1d

tpluscode merged commit 913570f into master Nov 21, 2022

tpluscode deleted the project-search branch November 21, 2022 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: search projects #1330

feat: search projects #1330

tpluscode commented Nov 15, 2022 •

edited

Loading

changeset-bot bot commented Nov 15, 2022 •

edited

Loading

codecov-commenter commented Nov 15, 2022 •

edited

Loading

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

cristianvasquez Nov 17, 2022

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

cristianvasquez Nov 17, 2022

tpluscode Nov 17, 2022

tpluscode Nov 16, 2022

tpluscode Nov 16, 2022

cristianvasquez Nov 17, 2022

tpluscode Nov 17, 2022

cristianvasquez Nov 17, 2022

tpluscode Nov 17, 2022

tpluscode commented Nov 16, 2022 •

edited

Loading

cristianvasquez Nov 17, 2022

tpluscode Nov 17, 2022

cristianvasquez Nov 17, 2022

tpluscode Nov 17, 2022

cristianvasquez left a comment

feat: search projects #1330

feat: search projects #1330

Conversation

tpluscode commented Nov 15, 2022 • edited Loading

changeset-bot bot commented Nov 15, 2022 • edited Loading

🦋 Changeset detected

codecov-commenter commented Nov 15, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tpluscode commented Nov 16, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cristianvasquez left a comment

Choose a reason for hiding this comment

tpluscode commented Nov 15, 2022 •

edited

Loading

changeset-bot bot commented Nov 15, 2022 •

edited

Loading

codecov-commenter commented Nov 15, 2022 •

edited

Loading

tpluscode commented Nov 16, 2022 •

edited

Loading