Scheme for specification prose

[dhower-qc]

We've tried two methods to conditionally apply asciidoc source depending on parameters or equations.

First, we tried using ERB. It works fine but requires some basic knowledge of Ruby.

Second, we tried using a [when,"condition"] Asciidoc style. It works for asciidoctor.js because I made a plugin, but doesn't work with asciidoctor(-pdf) because there is no plugin there yet.

I'm not sold on either approach. More discussion is needed.

[dhower-qc]

@edolnx, @kersten1

Stepping up a level, there are several things we want to identify about each statement in the prose.

• The type of statement (normative, non-normative)
• Any conditions that apply for the statement
• A unique identifier for the statement that can be referenced throughout the database (e.g., for test plans, executable spec, etc.)

Given that, here is a strawman for how we might represent prose in the database:

For the prose:

The mimpid CSR provides a unique encoding of the version of the processor implementation. This register must be readable in any implementation, but a value of 0 can be returned to indicate that the field is not implemented. The Implementation value should reflect the design of the RISC-V processor itself and not any surrounding system.

The mimpid CSR is MXLEN bits long.

[NOTE] The format of this field is left to the provider of the architecture source code, but will often be printed by standard tools as a hexadecimal string without any leading or trailing zeros, so the Implementation value can be left-justified (i.e., filled in from most-significant nibble down) with subfields aligned on nibble boundaries to ease human readability.

description:
- type: spec # short for normative
  text: |
    The `mimpid` CSR provides a unique encoding of the version of the processor implementation.
  id: misa-01
- type: spec
  text: |
    This register must be readable in any implementation,
    but a value of 0 can be returned to indicate that the field is not implemented.
  misa-02
- type: spec
  text: |
    The Implementation value should reflect the design of the RISC-V processor itself
    and not any surrounding system.
  id: misa-03
- type: spec
  text: |
    The `mimpid` CSR is MXLEN (32) bits long.
  cond: XLEN == 32
  id: misa-04
- type: spec
  text: |
    The `mimpid` CSR is MXLEN (64) bits long.
  cond: XLEN == 64
  id: misa-05
- type: info # short for non-normative
  text: |
    The format of this field is left to the provider of the architecture source code,
    but will often be printed by standard tools as a hexadecimal string without any leading
    or trailing zeros, so the Implementation value can be left-justified (i.e., filled in from
    most-significant nibble down) with subfields aligned on nibble boundaries to ease human
    readability.
  id: misa-06

Things I like:

• Can generate config-specific docs from this data
• Can build robust testplans & implementations from this data

Things I don't like:

• Pretty verbose

Note that we can probably come up with a way to generate the ids so you would never have to hand-type those.

Thoughts?

[ThinkOpenly]

Those "id" fields might become a pain to maintain. Do they need to persist, forever associated with the text? If a statement gets split during editing, does the original "id" need to continue to be associated with each new statement? I guess there could be some CI magic that tried to maintain order, but even that seems challenging. (Maybe it's good that editing the spec will become very difficult, and require great attention to detail?)

[dhower-qc]

Yes, the ids will likely be a pain. We could lessen the pain somewhat by generating most of the IDs with CI, but that won't solve everything.

The idea is that the ids would be immutable and forever tied to a single specification requirement/note/etc. The benefit (which to me seems substantial) is that it gives us the ability to reference a statement unambiguously across spec versions. So, for example, we can "this certification test tests Requirement misa-05" rather than "this certification test tests the second sentence of the third paragraph of Priv Section 3.1.4 in manual release 20240412."

As such, if a statement gets split for some reason, I think that would generate two new ids. The ids would not imply any order; misa-01 does not necessarily come before misa-02 in the prose, giving us some flexibility to reorder the spec without touching ids.

I understand it is unreasonable to encode the spec in this form during development. That's why I propose that it be an encoding step after freeze. And after encoding/ratification, statement changes are hopefully rare.

[james-ball-qualcomm]

I'm thinking that we'd add the tags into the adoc source using some standard format and then extract them from the adoc into the database. I'm meeting with @kbroch-rivosinc and @edolnx today at 2pm Pacific to discuss this. In particular, I'm wondering if adoc supports something like a macro that we can type into the adoc and then extract with something like the Ruby asciidoctor library.

[kbroch-rivosinc]

Yes asciidoc spec definitely has the means: https://docs.asciidoctor.org/asciidoc/latest/directives/include-tagged-regions/

How we create all those is the challenge (and as @ThinkOpenly brought up the maintenance burden). It might be possible to use asciidoctor and AST it generates to create tagged regions automatically as a baseline (but that could also be more trouble than it's worth).

[dhower-qc]

Yes, it will be a challenge. Asciidoc tags are block-oriented. I'm not sure you can tag a since sentence in a paragraph, which we'd have to do to get the resolution James is looking for to get CSC requirement pointers. Or restructure the manual so each statement is a block/paragraph.

FWIW, I've found the asciidoctor AST difficult to work with. There is no way (that I can find) to dump and reload the AST with modifications, so it'd be hard to programmatically add metadata to the AST without a custom version of asciidoctor.

[james-ball-qualcomm]

I was imagining something like this:

  MUL performs an XLEN-bitxXLEN-bit multiplication of `rs1` by `rs2` and places the lower
  XLEN bits in the destination register.
  Any overflow is thrown away.

  [NOTE]
  If both the high and low bits of the same product are required, then the recommended code
  sequence is:
  MULH[[S]U] rdh, rs1, rs2; MUL rdl, rs1, rs2
  (source register specifiers must be in same order and rdh cannot be the same as rs1 or rs2).
  Microarchitectures can then fuse these into a single multiply operation instead of
  performing two separate multiplies.

  [REQ, id=ext-M-inst-mul-001, version="1.0"]
  MUL performs an XLEN-bitxXLEN-bit multiplication of `rs1` by `rs2` and places the lower
  XLEN bits in the destination register.

  [REQ, id=ext-M-inst-mul-002, version="1.1"]
  Any overflow is thrown away.

[james-ball-qualcomm]

I'm assuming the first paragraph and NOTE paragraph are the current adoc source in the M extension chapter. The REQ entries are new and in this case just duplicate the first 2 sentences in the first paragraph but could be re-written as needed to be a nice crisp testable requirement.

[james-ball-qualcomm]

We could add additional meta-data into the REQ entries.

[dhower-qc]

As you've written it, you are using the role syntax. It's possible to write an asciidoctor extension to handle the role how we want, but then we are tied to that extension to generate the documentation correctly. Also note that you've only written one role/tag here. Since there is no blank line between the requirements, it's going to be parsed as a single text block. (and if you put the blank line in, the statements will be generated as a separate paragraphs)

[james-ball-qualcomm]

Added blank line

[james-ball-qualcomm]

After brainstorming with Carl and Kevin, we think we have a plan. CTP (Certification Test Plans) will refer to rules (AKA assertions, AKA functional requirements) defined in the UDB appendices (extensions, instructions, CSRs) and then UDB will have links to the ISA manual and/or IDL. We use the standard adoc facilities for tagging (see riscv/riscv-isa-manual#1397) so that tags are only visible in the ISA manual adoc source. The tags provide static IDs. In the case when one rule in UDB needs to point to multiple tags in the ISA manual, we'll allow a UDB rule to be a list of tags (this is the one to many case). The inverse is also possible where one ISA manual tag is shared by several UDB rules (the many to one case). In either case, this is a 1:1 relationship between CTP coverage points and UDB rules. So, in summary, use UDB rules as a level of indirection between the CTP and ISA manuals. We also want to allow UDB rules to point to IDL code in addition to ISA manual adoc tags. So, I can imagine adding a section to the UDB appendices for a list of rules associated with its corresponding content (extension, instruction, or CSR) that then each have one or more ISA manual adoc tags and IDL tags. We'll have to invent a way to add tags to the IDL that get ignored when converting to things like C++ models but can become adoc tags in the UDB-generated appendices. I'll mock all this up for the M extension and then eventually MC100.

[james-ball-qualcomm]

Forgot to say that I'd like to add the IDL functions that are called as appendices since I'll probably want to link to that IDL code too. Derek, how hard is it to add this? Is this information available in UDB? Do you have an example of a backend that generates this information? I'm talking about IDL functions that are called from the instruction appendices such as "implemented?".

[james-ball-qualcomm]

Never mind. I figured this out and added it to portfolios.

[james-ball-qualcomm]

See the discussion in riscv/riscv-isa-manual#1397 for nuances about how to add anchors and cross references in AsciiDoc that I'm summarizing.