Skip to content

Commit

Permalink
Adds sections for defining macros/modules, renames void
Browse files Browse the repository at this point in the history
  • Loading branch information
zslayton committed Sep 3, 2024
1 parent f858c7a commit 3952607
Show file tree
Hide file tree
Showing 12 changed files with 499 additions and 164 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,6 @@ tags

### Java Builds (to test grammar) copy the grammar to here
grammar/IonText.g4

### `mdbook` build target
_books/**/book/
7 changes: 5 additions & 2 deletions _books/ion-1-1/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@

- [Introduction](./introduction.md)
- [What's new](./whats_new.md)
- [Macros by example](macros_by_example.md)
- [Macros](macros.md)
- [Defining macros](macros/defining_macros.md)
- [Macros by example](macros/macros_by_example.md)
- [Modules](modules.md)
- [System Module](modules/system_module.md)
- [Encoding module](modules/encoding_module.md)
- [System module](modules/system_module.md)
- [Binary encoding](binary/encoding.md)
- [Encoding primitives](binary/primitives.md)
- [`FlexUInt`](binary/primitives/flex_uint.md)
Expand Down
4 changes: 2 additions & 2 deletions _books/ion-1-1/src/binary/e_expressions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
## Encoding Expressions

> [!NOTE]
> This chapter focuses on the binary encoding of e-expressions. [_Macros by example_](../macros_by_example.md) explains what they are and how they are used.
> This chapter focuses on the binary encoding of e-expressions. [_Macros by example_](../macros/macros_by_example.md) explains what they are and how they are used.
### E-expression with the address in the opcode

Expand Down Expand Up @@ -285,7 +285,7 @@ considered "shapes" rather than types because while their encoding is always sta
produced by their expansion is not. A single macro can produce streams of varying length and containing values of
different Ion types depending on the arguments provided in the invocation.

See [Macro Shapes](macros_by_example.md#macro-shapes) for more information.
See [Macro Shapes](../macros/macros_by_example.md#macro-shapes) for more information.

### Encoding E-expressions With Multiple Arguments

Expand Down
37 changes: 37 additions & 0 deletions _books/ion-1-1/src/macros.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
## Macros

Like other self-describing formats, Ion 1.0 makes it possible to write a stream with truly arbitrary content--no formal schema required. However, in practice all applications have a de facto schema, with each stream sharing large amounts of predictable structure and recurring values. This means that Ion readers and writers often spend substantial resources processing undifferentiated data.

Consider this example excerpt from a webserver's log file:

```ion
{method: GET, statusCode: 200, status: "OK", protocol: https, clientIp: ip_addr::"192.168.1.100", resource: "index.html"}
{method: GET, statusCode: 200, status: "OK", protocol: https, clientIp: ip_addr::"192.168.1.100", resource: "images/funny.jpg"}
{method: GET, statusCode: 200, status: "OK", protocol: https, clientIp: ip_addr::"192.168.1.101", resource: "index.html"}
```

_Macros_ allow users to define fill-in-the-blank templates for their data. This enables applications to focus on encoding and decoding the parts of the data that are distinctive, eliding the work needed to encode the boilerplate.

Using this macro definition:
```ion
(macro getOk ($clientIp $resource)
{
method: GET,
statusCode: 200,
status: "OK",
protocol: (literal https),
clientIp: (annotate "ip_addr" $clientIp),
resource: "index.html"
})
```

The same webserver log file could be written like this:
```ion
(:getOk "192.168.1.100" "index.html")
(:getOk "192.168.1.100" "images/funny.jpg")
(:getOk "192.168.1.101" "index.html")
```

Macros are an encoding-level concern, and their use in the data stream is invisible to consuming applications. For writers, macros are always optional--a writer can always elect to write their data using value literals instead.

For a guided walkthrough of what macros can do, see [Macros by example](macros/macros_by_example.md).
76 changes: 76 additions & 0 deletions _books/ion-1-1/src/macros/defining_macros.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
## Defining macros

A macro is defined using a `macro` clause within a [module](../modules.md)'s [`macro_table` clause](../modules.md#macro_table).

### Syntax
```ion
(macro name signature template)
```

| Argument | Description |
|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------|
| [`name`](#macro-names) | A unique name assigned to the macro or--to construct an anonymous macro--`null`. |
| [`signature`](#macro-signatures) | An s-expression enumerating the parameters this macro accepts. |
| [`template`](#template-definition-language-tdl) | A template definition language (TDL) expression that can be evaluated to produce zero or more Ion values. |

### Macro names

Syntactically, macro names are [identifiers](../modules.md#identifiers). Each macro name in a macro table must be unique.

In some circumstances, it may not make sense to name a macro. (For example, when the macro is generated automatically.) In such cases, authors may set the macro name to `null` or `null.symbol` to indicate that the macro does not have a name. Anonymous macros can only be referenced by their address in the macro table.

### Macro Parameters

Macros accept zero or more parameters.

Each parameter is comprised of three elements:
1. A name
2. An encoding
3. A cardinality

#### Parameter names

A parameter's name is an [identifier](../modules.md#identifiers). The name is required; any non-identifier (including `null`, quoted symbols, `$0`, or a non-symbol) found in parameter-name position will cause the reader to raise an error.

All of a macro's parameters must have unique names.

#### Parameter encodings

In binary Ion, the default encoding for all parameters is _tagged_. Each argument passed into the macro from the callsite is prefixed by an [opcode](../binary/opcodes.md) (or "tag") that indicates the argument's type and length.

Parameters may choose to specify an alternative encoding to make the corresponding arguments' binary representation more compact and/or fixed width. These "tagless" encodings do not begin with an opcode, an arrangement which saves space but also limits the domain of values they can each represent. Arguments passed to tagless parameters cannot be `null`, cannot be annotated, and may have additional range restrictions.

To specify an encoding, the [parameter name](#parameter-names) is annotated with one of the following tokens:

| Tagless encodings | Description |
|--------------------------------------|-------------------------------------------------------------------|
| `flex_int` | Variable-width signed int |
| `flex_uint` | Variable-width unsigned int |
| `int8` `int16` `int32` `int64` | Fixed-width signed int |
| `uint8` `uint16` `uint32` `uint64` | Fixed-width unsigned int |
| `float16` `float32` `float64` | Fixed-width float |
| `flex_symbol` | [`FlexSym`](../binary/primitives/flex_sym.md)-encoded SID or text |


#### Parameter cardinalities

A parameter name may optionally be followed by a _cardinality modifier_. This is a sigil that indicates how many values the parameter expects the corresponding argument expression to produce when it is evaluated.

| Modifier | Cardinality |
|:--------:|---------------------|
| `!` | exactly-one value |
| `?` | zero-or-one value |
| `+` | one-or-more values |
| `*` | zero-or-more values |

If no modifier is specified, the parameter's cardinality will default to exactly-one.

If an argument expression expands to a number of values that the cardinality forbids, the reader must raise an error.

### Macro signatures

A macro signature is an s-expression containing a series of parameter definitions.

### Template definition language (TDL)

<!-- TODO -->
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ spliced into the surrounding container:
(first (:values left right) last) ⇒ (first left right last)
```

This also applies wherever a [tagged type](binary/values.md) can appear inside an E-expression:
This also applies wherever a [tagged type](../binary/values.md) can appear inside an E-expression:

```ion
(first (:values (:values left right) (:values)) last) ⇒ (first left right last)
Expand Down Expand Up @@ -455,37 +455,34 @@ becomes empty.
> This termination rule is under discussion; see <https://github.com/amazon-ion/ion-docs/issues/201>

### Empty Streams: `void`
### Empty Streams: `none`

The empty stream is an important edge case that requires careful handling and communication.
We'll use the term _void_ to mean “empty stream”. We’ll even mint the word _voidable_ to
describe parameters that can accept empty streams, like the ``*``s above.

Correspondingly, the built-in macro `void` accepts no values and produces an empty stream:
The built-in macro `none` accepts no values and produces an empty stream:

```ion
(:int_list (:void)) ⇒ []
(:int_list 1 (:void) 2) ⇒ [1, 2]
[(:void)] ⇒ []
{a:(:void)} ⇒ {}
(:int_list (:none)) ⇒ []
(:int_list 1 (:none) 2) ⇒ [1, 2]
[(:none)] ⇒ []
{a:(:none)} ⇒ {}
```

When used as a macro argument, a `void` invocation (like any other expression) counts as one
When used as a macro argument, a `none` invocation (like any other expression) counts as one
argument:

```ion
(:pi (:void)) ⇒ _error: 'pi' expects 0 arguments, given 1_
(:pi (:none)) ⇒ _error: 'pi' expects 0 arguments, given 1_
```

The special form `(:)` is an [empty argument group](todo.md), similar to
`(:void)` but used specifically to express the absence of an argument:
The special form `(:)` is an [empty argument group](../todo.md), similar to
`(:none)` but used specifically to express the absence of an argument:

```ion
(:int_list (:)) ⇒ []
(:int_list 1 (:) 2) ⇒ [1, 2]
```

TIP: While `void` and `values` both produce the empty stream, the former is preferred for
TIP: While `none` and `values` both produce the empty stream, the former is preferred for
clarity of intent and terminology.


Expand Down Expand Up @@ -515,7 +512,7 @@ might refer to them as _singleton streams_ or just _singletons_ colloquially.
#### Zero-or-One

A parameter with the modifier `?` has _zero-or-one cardinality_, which is much like
exactly-one cardinality, except the parameter is voidable. That is, it accepts an empty-stream
exactly-one cardinality, except the parameter accepts an empty-stream
argument as a way to denote an absent parameter.

```ion
Expand All @@ -524,7 +521,7 @@ argument as a way to denote an absent parameter.
{degrees: degrees, scale: scale})
```

Since the scale is voidable, we can pass it void:
Since the scale accepts the empty stream, we can pass it an empty argument group:

```ion
(:temperature 96 F) ⇒ {degrees:96, scale:F}
Expand All @@ -533,28 +530,28 @@ Since the scale is voidable, we can pass it void:

Note that the result’s `scale` field has disappeared because no value was provided. It would be
more useful to fill in a default value, and to do that we introduce a special form that can
detect void:
detect the empty stream:

```ion
(macro temperature
(decimal::degrees symbol::scale?)
{degrees: degrees, scale: (if_void scale (literal K) scale)})
{degrees: degrees, scale: (if_none scale (literal K) scale)})
```
```ion
(:temperature 96 F) ⇒ {degrees:96, scale:F}
(:temperature 283 (:)) ⇒ {degrees:283, scale:K}
```

The `if_empty` form is if/then/else syntax testing stream emptiness. It has three sub-expressions,
the first being a stream to check. If and only if that stream is void (it produces no
values), the second sub-expression is expanded and its results are returned by the `if_empty`
The `if_none` form is if/then/else syntax testing stream emptiness. It has three sub-expressions,
the first being a stream to check. If and only if that stream is empty (it produces no
values), the second sub-expression is expanded and its results are returned by the `if_none`
expression. Otherwise, the third sub-expression is expanded and returned.

> [!NOTE]
> Exactly one branch is expanded, because otherwise the void stream might be used in a context
> Exactly one branch is expanded, because otherwise the empty stream might be used in a context
> that requires a value, resulting in an errant expansion error.
To refine things a bit further, trailing voidable arguments can be omitted entirely:
To refine things a bit further, trailing arguments that accept the empty stream can be omitted entirely:

```ion
(:temperature 283) ⇒ {degrees:283, scale:K}
Expand Down Expand Up @@ -583,7 +580,7 @@ expression that produces the desired values:
(:prices (: 10 9.99) GBP) ⇒ {amount:10, currency:GBP} {amount:9.99, currency:GBP}
```

Here we use a non-empty [empty argument group](todo.md) `(: ...)` to delimit
Here we use a non-empty [empty argument group](../todo.md) `(: ...)` to delimit
the multiple elements of the `amount` stream.


Expand Down Expand Up @@ -631,7 +628,7 @@ Thank you to my Patreon supporters:
The non-rest versions of multi-value parameters require some kind of delimiting
syntax to contain the applicable sub-expressions. For the tagged-type parameters we've seen
so far, you _could_ use `:values` or some other macro to produce the stream, but that doesn't
work for [tagless types](todo.md).
work for [tagless types](../todo.md).
The preferred syntax, supporting all argument types, is a special delimiting form
called an _argument group_. Here is a macro to illustrate:

Expand Down Expand Up @@ -687,7 +684,7 @@ As usual, the text format mirrors this constraint.

### Optional Arguments

When a trailing parameter is voidable, an invocation can omit its corresponding argument expression,
When a trailing parameter accepts the empty stream, an invocation can omit its corresponding argument expression,
as long as no following parameter is being given an expression. We’ve seen
this as applied to final `*` parameters, but it also applies to `?`
parameters:
Expand All @@ -698,14 +695,14 @@ parameters:
(make_list a b c d e f))
```

Since `d`, `e`, and `f` are all voidable, they can be omitted by invokers. But `c` is required so
Since `d`, `e`, and `f` all accept the empty stream, they can be omitted by invokers. But `c` is required so
`a` and `b` must always be present, at least as an empty group:

```ion
(:optionals (:) (:) "value for c") ⇒ ["value for c"]
```

Now `c` receives the string `"value for c"` while the other parameters are all void.
Now `c` receives the string `"value for c"` while the other parameters are all empty.
If we want to provide `e`, then we must also provide a group for `d`:

```ion
Expand Down
Loading

0 comments on commit 3952607

Please sign in to comment.