-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft of CLI/API design #1
Comments
What's the reason for a purpose specific CLI for documents? So I can't manage other types of content? That can't be right. |
The new ISO 19135 already provides the underlying model for concept class, concept, register item class and register item. These are the concept classes to implement:
These are the register item classes to implement:
I.e you need to create data definitions for these things above. We need a generic way of handling these, not hard coding their attributes in code. Please refer to how data definitions are expressed and implemented in lutaml-model. |
Bad name I guess. It's a temporary name. Lemme change that. |
Ruby Paneron Register currently only has concepts of register item class and register item, which as I understand, realize "concept classes" and "concepts" from ISO 19135. If we want to keep the generality of the tools, my thoughts go to something like this: > ctl --concept-model-map ./document_with_stages_mapping_to_register_model.lutaml \
init document_register
> cd document_register
> ctl --concept-model-map ../document_with_stages_mapping_to_register_model.lutaml \
init-dataset my-dataset
> ctl --concept-model-map ../document_with_stages_mapping_to_register_model.lutaml \
upload my-dataset/Doc-00001 files... Does this sound like the right direction? EDIT: I am getting some similar vibe from here: lutaml/lutaml-model#165 |
Further ideas & goals
|
These issues may make defining register data models easier: |
Brainstorming how a data model definition may look like: ---
register:
concepts:
- name: Document
attributes:
id:
definition: The unique identifier of the document
type: string
year:
definition: The year the document was published
type: year
doctype:
definition: The Metanorma document type
type: string
doc_class:
definition: The Metanorma document class
type: string
relations:
- target: DocumentStage
relationship:
source:
type: aggregation
attributes:
document:
cardinality:
min: 1
max: "*"
target:
type: direct
attributes:
documentStage:
cardinality:
min: 1
max: "*"
action:
verb: comprises
direction: target
- name: DocumentStage
attributes:
stage:
definition: The stage of the document
type: string
relations:
- target: DocumentVersion
relationship:
source:
type: aggregation
attributes:
documentStage:
cardinality:
min: 1
max: "*"
target:
type: direct
attributes:
documentVersion:
cardinality:
min: 1
max: "*"
action:
verb: comprises
direction: target
- name: DocumentVersion
attributes:
version:
definition: The version of the document
type: string
data:
item_classes:
values: null # No concrete item classes, since we are sorting by Document ID
# register items with the same Document ID will be grouped together
# in an item class
group_by:
name: Document
attribute: id
sort_by:
attribute: year
order: desc
register_items:
attributes:
# Here, we map each register item attribute to the corresponding concept
id:
definition: The document identifier
concept: Document
attribute: id
stage:
definition: The stage of the document
concept: DocumentStage
attribute: stage
version:
definition: The version of the document
concept: DocumentVersion
attribute: version
year:
definition: The year the document was published
concept: Document
attribute: year
title:
definition: The title of the document
type: string
actions:
# Here, list out the actions that can be performed on the
# item classes and register items,
# and map them to the corresponding functions...
- name: create_document_version
use: create_register_item
parameters:
# ... TBD
|
I think it is worthwhile to brainstorm some possible concrete interactions before hashing out the full data models, to get a better sense of the requirements: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model # remote
# or
> export CTL_DATA_DEFINITION=/home/username/register-data-model # local
> ctl create_document_version my-data-set/cc-10001:2022 ./compiled/artifacts/cc-10001-2022/
Using data definition from github:CalConnect/register-data-model...
You are creating a new "document version".
(keys: <esc> to go back one step, <enter> to continue)
# A note on the default values provided: these could be configured
# to be parsed from the documents to be uploaded, e.g. to specify that
# document.rxl be read (via a programmatic API)
Input "id" (default: cc-10001:2022): <enter>
Input "stage" (<space> to select):
o published
o final-draft
x draft
Input "version" (default: 1): <enter>
Input "year" (default: 2022): <enter>
Input "title" (default: CalConnect Directive: Standardization and publication): <enter>
Confirm the following details
(keys: <esc> to go back one step, <enter> to continue)
id: cc-10001:2022
stage: draft
version: 1
year: 2022
title: CalConnect Directive: Standardization and publication
continue? <enter>
Copying 4 files... done
Creating commit in register... done
Syncing... done
Summary
- A new document version has been created (my-data-set/cc-10001:2022). |
Overwriting an existing version: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl create_document_version my-data-set/cc-10001:2022 ./compiled/artifacts/cc-10001-2022/
Using data definition from github:CalConnect/register-data-model...
You are creating a new "document version".
(keys: <esc> to go back one step, <enter> to continue)
Input "id" (default: cc-10001:2022): <enter>
Input "title" (default: CalConnect Directive: Standardization and publication): <enter>
Confirm the following details
Input "year" (default: 2022): <enter>
Input "stage" (<space> to select):
o published
o final-draft
x draft
# The "version" field follows the auto-increment rule defined in the data model,
# which now provides an updated default value.
Input "version" (default: 2): 1<enter>
(keys: <esc> to go back one step, <enter> to continue)
id: cc-10001:2022
title: CalConnect Directive: Standardization and publication
year: 2022
stage: draft
version: 1
continue? <enter>
*** A "document version" with the same metadata already exists.
*** Overwrite? (Y/n) Y <enter>
Copying 4 files... done
Creating commit in register... done
Syncing... done
Summary
- A "document version" has been overwritten (my-data-set/cc-10001:2022). Listing document versions: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl list_document_versions my-data-set/cc-10001:2022
Using data definition from github:CalConnect/register-data-model...
Listing all "document version" for (my-data-set/cc-10001:2022)...
cc-10001:2022, CalConnect Directive: Standardiz..., 2022, draft, 1, cc10001-2022.pdf, cc10001-2022.rxl, ...
Summary
- 1 "document version" found for (my-data-set/cc-10001:2022). List document versions (with multiple results): > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl list_document_versions my-data-set/cc-10002:2019
Using data definition from github:CalConnect/register-data-model...
Listing all "document version" for (my-data-set/cc-10002:2019)...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 4, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 5, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 6, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
Summary
- 11 "document version" found for (my-data-set/cc-10002:2019). List document versions with filtering: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl list_document_versions my-data-set/cc-10002:2019 stage:published
Using data definition from github:CalConnect/register-data-model...
Listing all "document version" for (my-data-set/cc-10002:2019)...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
Summary
- 2 "document version" found for (my-data-set/cc-10002:2019). Removing a document version (by default selects the last version): > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl delete_document_version my-data-set/cc-10001:2022
Using data definition from github:CalConnect/register-data-model...
You are deleting a "document version" (my-data-set/cc-10001:2022).
(keys: <esc> to go back one step, <enter> to continue)
id: cc-10001:2022
title: CalConnect Directive: Standardization and publication
year: 2022
stage: draft
version: 1
continue? <enter>
Removing 4 files... done
Creating commit in register... done
Syncing... done
Summary
- A "document version" has been deleted (my-data-set/cc-10001:2022). Removing a specific document version with filtering: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl delete_document_version my-data-set/cc-10002:2019 stage:published
Using data definition from github:CalConnect/register-data-model...
You are deleting a "document version" (my-data-set/cc-10002:2019).
(keys: <esc> to go back one step, <enter> to continue)
id: cc-10002:2019
title: CalConnect Directive: Standard and documents
year: 2019
stage: published
version: 2
continue? <enter>
Removing 4 files... done
Creating commit in register... done
Syncing... done
Summary
- A "document version" has been deleted (my-data-set/cc-10002:2019). > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl delete_document_version my-data-set/cc-10002:2019 stage:published
Using data definition from github:CalConnect/register-data-model...
You are deleting a "document version" (my-data-set/cc-10002:2019).
(keys: <esc> to go back one step, <enter> to continue)
id: cc-10002:2019
title: CalConnect Directive: Standard and documents
year: 2019
stage: published
version: 1
continue? <enter>
Removing 4 files... done
Creating commit in register... done
Syncing... done
Summary
- A "document version" has been deleted (my-data-set/cc-10002:2019). Removing a non-existent document version: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl delete_document_version my-data-set/cc-10001:2022 version:3
Using data definition from github:CalConnect/register-data-model...
You are deleting a "document version" (my-data-set/cc-10001:2022).
Error: There is no existing "document version" with the following metadata. Aborting.
id: cc-10001:2022
version: 3
Summary
- No "document version" has been deleted (my-data-set/cc-10001:2022). List all documents: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl list_documents my-data-set
Using data definition from github:CalConnect/register-data-model...
Listing all "document" from data set (my-data-set)...
cc-10001:2022, CalConnect Directive: Standardiz..., 2022, draft, 1, cc10001-2022.pdf, cc10001-2022.rxl, ...
cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
Summary
- 2 "document" found for data set (my-data-set). Show details of a specific document version: > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl show_document_version my-data-set/cc-10002:2019
Using data definition from github:CalConnect/register-data-model...
Select the "document version" for (my-data-set/cc-10002:2019)...
(<space> to select):
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 4, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 5, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 6, cc10002-2019.pdf, cc10002-2019.rxl, ...
x cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
id: cc-10002:2019
title: CalConnect Directive: Standard document requirements
year: 2019
stage: final-draft
version: 1
artifacts:
- cc10002-2019.html
- cc10002-2019.pdf
- cc10002-2019.rxl
- cc10002-2019.xml Download artifacts for a specific document version (from a remote source): > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl get_document_version github:CalConnect/cc-document-register#my-data-set/cc-10002:2019
Using data definition from github:CalConnect/register-data-model...
Using remote register from github:CalConnect/cc-document-register...
Select the "document version" for (my-data-set/cc-10002:2019)...
(<space> to select):
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 4, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 5, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, draft, 6, cc10002-2019.pdf, cc10002-2019.rxl, ...
x cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, final-draft, 3, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 1, cc10002-2019.pdf, cc10002-2019.rxl, ...
o cc-10002:2019, CalConnect Directive: Standard d..., 2019, published, 2, cc10002-2019.pdf, cc10002-2019.rxl, ...
id: cc-10002:2019
title: CalConnect Directive: Standard document requirements
year: 2019
stage: final-draft
version: 1
artifacts:
- cc10002-2019.html
- cc10002-2019.pdf
- cc10002-2019.rxl
- cc10002-2019.xml
Save files to (default: .)? <enter>
Copying 4 files... done
Summary
- 4 "artifacts" items for a "document version" have been downloaded (github:CalConnect/cc-document-register#my-data-set/cc-10002:2019). Show all available actions (as defined by the data definition): > export CTL_DATA_DEFINITION=github:CalConnect/register-data-model
> ctl help
Using data definition from github:CalConnect/register-data-model...
Available actions:
- create_document_version
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title (default: parsed from rxl file in item_class_path)
- year (default: parsed from rxl file in item_class_path)
- stage (default: parsed from rxl file in item_class_path)
- version (default: auto-increment)
- artifacts (type: file list)
- delete_document_version
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title (default: parsed from rxl file in item_class_path)
- year (default: parsed from rxl file in item_class_path)
- stage (default: parsed from rxl file in item_class_path)
- version (default: last, sorted in ascending order)
- get_document_version
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title (default: parsed from rxl file in item_class_path)
- year (default: parsed from rxl file in item_class_path)
- stage (default: parsed from rxl file in item_class_path)
- version (default: last, sorted in ascending order)
returns:
- artifacts
- list_document_versions
parameters:
- item_class_path
returns:
- list of document version items
- list_documents
parameters:
- data_set_path
returns:
- list of document items
- show_document_version
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title
- year
- stage
- version
returns:
- details of a document version item
- add_artifacts
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title
- year
- stage
- version
- artifacts (type: file list)
- delete_artifacts
parameters:
- item_class_path
- id (default: parsed from item_class_path)
- title
- year
- stage
- version
- filename of artifacts (type: string list)
Another register might have a different set of actions: > export CTL_DATA_DEFINITION=github:isogr/register-data-model
> ctl help
Using data definition from github:isogr/register-data-model...
Available actions:
- create_proposal
parameters:
- title
returns:
- id of proposal
- delete_proposal
parameters:
- id of proposal
- remove_from_proposal
parameters:
- id of proposal
- list of item_path
- submit_proposal
parameters:
- id of proposal
- approve_proposal
parameters:
- id of proposal
- reject_proposal
parameters:
- id of proposal
- create_extent
parameters:
- id of proposal
- name
- ...
returns:
- id of extent
- create_transformation
parameters:
- id of proposal
- id of extent
- name
- ... |
@ribose-jeffreylau I've written up a specification for the CLI here. Let me know what you think? |
Originally posted by @ronaldtse in #3 |
This issue is to gather inputs from potential users of this tool.
Assuming the CLI is called
doctl
:Initialize a documents register
Set register metadata
Initialize dataset
A dataset groups together a set of documents.
It is up to the documents register user to decide which documents to be grouped as one.
One document can only belong to one dataset.
List datasets
Set dataset metadata
Upload a new document
Under the hood, UNIQUE_EXTERNAL_DOCUMENT_ID will be modelled as an Item Class (ISO 19135).
Modify document metadata
List document versions
List document files of a particular version
Add a new document version
> doctl upload my-documents/UNIQUE_EXTERNAL_DOCUMENT_ID FILES...
Override a document version
> doctl upload --force my-documents/UNIQUE_EXTERNAL_DOCUMENT_ID#VERSION_STRING FILES...
Remove a document version
> doctl rm my-documents/UNIQUE_EXTERNAL_DOCUMENT_ID#VERSION_STRING
Getting help
The text was updated successfully, but these errors were encountered: