Notice: Firelight compatibility with upcoming release post-delivery

[ronaldtse]

This is an advance notice to the team that the upcoming release on Monday will be incompatible with the current version of Firelight.

As we are preparing for delivery on 2025-02-28, we have frozen the current XML file to the current version, which has the XML root of ("old version"):

<plateau-standard xmlns="https://www.metanorma.org/ns/plateau" type="presentation" version="0.1.15" schema-version="v1.4.1">

On 2025-03-03 (Monday), the new root tag will be ("new version"):

<metanorma ... >

Firelight will process the "old version" until it is updated for the "new version" (after 2025-02-28).

FYI @ReesePlews @strogonoff

[strogonoff]

The tag change is already supported, and it should support most of the other changes except a couple.

[strogonoff]

@ronaldtse according to Reese, the deliverable is static (though we also deliver the build process, it’s not the main part), and build regressions after delivery are normal while we don’t have a stable/unstable release channels and don’t have dependency versions pinned.

[ronaldtse]

There are 2 things we have to do because the upcoming Monday Metanorma release will be incompatible with the current Firelight.

1. We will lock the Docker image on this repository to 1.11.5 until Firelight is updated:

mn-samples-plateau/.github/workflows/docker.yml

Line 23 in ad00275

image: metanorma/metanorma:latest

The docker image for 1.11.5 is:

• docker pull metanorma/metanorma:ubuntu-1.11.5

2. Firelight needs to be updated for Metanorma 1.11.5. Relevant code:

mn-samples-plateau/.github/workflows/docker.yml

Lines 71 to 97 in ad00275

	- name: "Firelight: prepare config for document ${{ matrix.document_dir }}"
	run: |
	cat <<'EOF' >> anafero-config.json
	{
	"version": "0.1",
	"entryPoint": "file:documents/${{ matrix.document_dir }}-v5/document.presentation.xml",
	"storeAdapters": [
	"git+https://github.com/metanorma/firelight#main/packages/metanorma-xml-store"
	],
	"contentAdapters": [
	"git+https://github.com/metanorma/firelight#main/packages/metanorma-site-content"
	],
	"resourceLayouts": [
	"git+https://github.com/metanorma/firelight#main/packages/plateau-layout"
	]
	}
	EOF
	git add anafero-config.json
	git commit -m "Add build config for ${{ matrix.document_dir }}"
	- name: "Firelight: generate HTML for doc ${{ matrix.document_dir }}"
	run: |
	npx --node-options='--experimental-vm-modules' \
	-y @riboseinc/[email protected] \
	--target-dir dist/${{ matrix.document_dir }} \
	--path-prefix /mn-samples-plateau/${{ matrix.document_dir }} \
	--current-rev main

[ReesePlews]

hello @ronaldtse thank you for this additional information, however i have a question about the mn version. why is the version 1.11.5 and not 0.2.1? the current metanorma-plateau version is 0.2.1

my understanding on 0.2.1 is that release incorporated a number of small but very important fixes from late january to mid-february (listed here: metanorma/metanorma-plateau#158 ). i have confirmed from multiple checks of github output if we use the 1.11.5 version these fixes are not present.

the client and i are using Path: C:/tools/ruby33/lib/ruby/gems/3.3.0/gems/metanorma-2.1.2 for our current local builds; making revisions then committing and merging on the main branch, then building the document again locally.

i dont recall a docker release built for metanorma 2.1.2. is there a reason this was not produced? it may be the speed issue?

@strogonoff has worked very hard to finish all of the feedback issues by Feb 28th, as requested by the client. this date was the final for addressing issues but the final delivery of the project is described here. from March 1 to March 7 the client will be addressing feedback from M2 and finalizing the content of the documents.

therefore the week of March 3 the static artifacts of the viewer builds matching the document content the client has updated are needed. i have a meeting with the client on Feb 28th. i will supply more information thereafter.

happy to discuss options/work arounds. thank you.

[ronaldtse]

1.11.5 refers to the docker version, which uses the same version as the "metanorma-cli" gem. The versioning number is separate from the "metanorma" or the "metanorma-plateau" gem. You can consider it the "package version.

[strogonoff]

There are 2 things we have to do because the upcoming Monday Metanorma release will be incompatible with the current Firelight.

1. We will lock the Docker image on this repository to 1.11.5 until Firelight is updated:

mn-samples-plateau/.github/workflows/docker.yml

Line 23 in ad00275

image: metanorma/metanorma:latest
The docker image for 1.11.5 is:

• docker pull metanorma/metanorma:ubuntu-1.11.5

2. Firelight needs to be updated for Metanorma 1.11.5. Relevant code:

mn-samples-plateau/.github/workflows/docker.yml

Lines 71 to 97 in ad00275

   - name: "Firelight: prepare config for document ${{ matrix.document_dir }}" 
     run: | 
       cat <<'EOF' >> anafero-config.json 
       { 
         "version": "0.1", 
         "entryPoint": "file:documents/${{ matrix.document_dir }}-v5/document.presentation.xml", 
         "storeAdapters": [ 
           "git+https://github.com/metanorma/firelight#main/packages/metanorma-xml-store" 
         ], 
         "contentAdapters": [ 
           "git+https://github.com/metanorma/firelight#main/packages/metanorma-site-content" 
         ], 
         "resourceLayouts": [ 
           "git+https://github.com/metanorma/firelight#main/packages/plateau-layout" 
         ] 
       } 
       EOF 
       git add anafero-config.json 
       git commit -m "Add build config for ${{ matrix.document_dir }}" 

   - name: "Firelight: generate HTML for doc ${{ matrix.document_dir }}" 
     run: | 
       npx --node-options='--experimental-vm-modules' \ 
         -y @riboseinc/[email protected] \ 
         --target-dir dist/${{ matrix.document_dir }} \ 
         --path-prefix /mn-samples-plateau/${{ matrix.document_dir }} \ 
         --current-rev main

Yes, it would be a good idea to pin versions. New root tag is not a problem, I updated to support it, but new xrefs and other structural changes will need some time to support.

[ReesePlews]

hello @ronaldtse thank you for this information.

1.11.5 refers to the docker version, which uses the same version as the "metanorma-cli" gem. The versioning number is separate from the "metanorma" or the "metanorma-plateau" gem. You can consider it the "package version.

i am sorry i still dont quite understand.

these are the metanorma related versions we are using at this time during our local builds:

  * lutaml (0.9.27)
  * lutaml-model (0.6.3)
  * lutaml-path (0.1.0)
...
  * metanorma (2.1.2)
  * metanorma-bipm (2.6.0)
  * metanorma-cc (2.6.0)
  * metanorma-cli (1.12.0)
  * metanorma-csa (2.6.0)
  * metanorma-generic (3.0.0)
  * metanorma-iec (2.6.0)
  * metanorma-ieee (1.4.0)
  * metanorma-ietf (3.5.0)
  * metanorma-ietf-data (0.2.0)
  * metanorma-iho (1.1.0)
  * metanorma-iso (3.0.0)
  * metanorma-itu (2.6.0)
  * metanorma-jis (0.5.0)
  * metanorma-ogc (2.7.0)
  * metanorma-plateau (0.2.1)
  * metanorma-plugin-datastruct (0.3.0)
  * metanorma-plugin-glossarist (0.2.3)
  * metanorma-plugin-lutaml (0.7.21)
  * metanorma-standoc (3.0.0)
  * metanorma-utils (1.10.2)
 ...
  * mn-requirements (0.5.0)

in the discussion above will this "locked" version support these releases (like: metanorma 2.1.2, metanorma-cli 1.12.0, and metanorma-plateau 0.2.1)?

in my understanding the fixes listed here: metanorma/metanorma-plateau#158 are mostly in lutaml, metanorma-plateau, and mn-requirements.

i tried to explain some issues about the new release to the client today. they are concerned that updates and fixes mentioned above may not be stable on github and that the github mn build may not be synchronized with the github firelight build. the client asked for instructions how to setup firelight tool chain locally so they can use the above releases to generate the firelight "packages"

[strogonoff]

Firelight is mostly processing new XML fine. One confusion is with links, it’s unclear what’s the deal with <xref> (since there’s <fmt-xref>, can xref be ignored now? if it’s not ignored, there will be duplicate links) and <eref> (there’s no <fmt-eref>, it looks like <eref> can be ignored and just the inner <fmt-link> is important).

[strogonoff]

One thing I’d like to get a hold of is latest MN XML for both documents, since it won’t be available in GHA if we pin it to stable version. I’ll try running Ruby toolchain and see if it works…

[opoudjis]

One confusion is with links, it’s unclear what’s the deal with (since there’s , can xref be ignored now?

https://github.com/metanorma/metanorma-model-iso/blob/main/grammars/isodoc-presentation.rnc

## convert eref to xref (pointing to bibliography anchor, xref/@target = eref/@bibitemid),
## or link (if a URI for the reference is supplied in the bibitem being referenced).
## The locality text (locality, sourcelocality), if present, is inserted as part of the cross-referenced text.
## If the locality is an anchor, and the eref is converted to link, append the anchor to the URI.
## If there is no text in the Semantic XML eref, it is populated with eref/@citeas
eref &= semx
# semx = ( fmt-eref | fmt-xref | fmt-link )

metanorma/isodoc#641

Markup has become:

      <xref target="N1" id="A"/>
      <semx element="xref" source="A">
         <fmt-xref target="N1">
            <semx element="introduction" source="intro">Introduction</semx>
         </fmt-xref>
      </semx>

xref, eref, origin, link are now ignored in favour of fmt-xref, fmt-eref, fmt-origin, fmt-link

metanorma/isodoc#649

A ticket you were notified of:

Presentation XML refactor: eref, xref, link, origin, erefstack are to be ignored. They are replaced by fmt-eref, fmt-xref, fmt-link, fmt-origin. The attributes of these new tags are the same as the tags they are replacing.

[opoudjis]

@strogonoff It is very important that you review ALL Presentation XML tickets in the sprint that you have missed: they each have a PDF counterpart ticket, which summarised the changes made. The fact that your compilation does not break is neither here nor there: all these containers have changed to use fmt-* counterpart elements, containing the text to be rendered, and they all mean that the sibling or parent Semantic XML node is to be ignored in rendering.

https://github.com/metanorma/isodoc/releases/tag/v3.0.0

Presentation XML refactor, identifier: metanorma/mn-requirements#35
Presentation XML, dates: metanorma/isodoc#642
Presentation XML, stem: metanorma/isodoc#642
Presentation XML, concept: metanorma/isodoc#637
eref, xref, origin, link Presentation XML: metanorma/isodoc#641
svgmap Presentation XML: metanorma/isodoc#641
Presentation XML: sourcecode: metanorma/isodoc#640
quote/source and amend PresentationXML: metanorma/isodoc#643

[strogonoff]

I was able to compile the docs and continuing to update Firelight Metanorma integration to new schema version.

@opoudjis Appreciated link to refs grammar. Regarding pings, feel free to not ping me in gem tickets unless it’s a ticket I am participating in. As schema consumer I think it’s probably enough to have one ping with rundown of all breaking changes / new features per version update, like in your latest comment[0]. Or just making it part of changelog would sort it, I suppose. Will circle back to it later.

[0] Though in a more digestible form—Metanorma gem tickets are your work process and normally much of it is not understandable/relevant for outsiders or just people not directly working on the gems. It could be useful when troubleshooting, but is too much if it’s just routine upgrade…

[strogonoff]

Compatibility with new schema is worked on in https://github.com/metanorma/firelight/tree/metanorma-schema-v2.0.0.