Add documentation on Shadow Paths (#758)

* Add documentatin on Shadow Paths * improve * add documentation reference to generator.go's help message * Update docs/shadow_paths.md Co-authored-by: Rob Shakir <[email protected]> * Update docs/shadow_paths.md Co-authored-by: Rob Shakir <[email protected]> * improve english Co-authored-by: Rob Shakir <[email protected]>
openconfig · Jan 10, 2023 · 950f793 · 950f793
1 parent f58f5df
commit 950f793
Show file tree

Hide file tree

Showing 2 changed files with 105 additions and 1 deletion.
diff --git a/docs/shadow_paths.md b/docs/shadow_paths.md
@@ -0,0 +1,104 @@
+# GoStruct Shadow Paths
+
+## Introduction
+
+GoStruct shadow paths refer to `shadow-path` annotations currently generated via
+the flag `-ignore_shadow_schema_paths`:
+
+```go
+// Interface represents the /openconfig-interfaces/interfaces/interface YANG schema element.
+type Interface struct {
+    Description      *string                            `path:"state/description" module:"openconfig-interfaces/openconfig-interfaces" shadow-path:"config/description" shadow-module:"openconfig-interfaces/openconfig-interfaces"`
+    Enabled          *bool                              `path:"state/enabled" module:"openconfig-interfaces/openconfig-interfaces" shadow-path:"config/enabled" shadow-module:"openconfig-interfaces/openconfig-interfaces"`
+    Mtu              *uint16                            `path:"state/mtu" module:"openconfig-interfaces/openconfig-interfaces" shadow-path:"config/mtu" shadow-module:"openconfig-interfaces/openconfig-interfaces"`
+    Name             *string                            `path:"state/name|name" module:"openconfig-interfaces/openconfig-interfaces|openconfig-interfaces" shadow-path:"config/name|name" shadow-module:"openconfig-interfaces/openconfig-interfaces|openconfig-interfaces"`
+    OperStatus       E_Interface_OperStatus             `path:"state/oper-status" module:"openconfig-interfaces/openconfig-interfaces"`
+}
+```
+
+The aim of this document is to clarify what shadow paths are and why they exist
+as a generation option.
+
+## Shadow paths: compressed-out "config" or "state" YANG leaves
+
+https://datatracker.ietf.org/doc/html/draft-openconfig-netmod-opstate-01#section-2
+contains a diagram of the relationship between intended config (`/config`) and
+applied config (`/state`) leaves:
+
+```
+          +---------+
+          |         |    transition intended
+          |intended |    to applied
+          | config  +---------+
+          |         |         |
+          +---------+         |
+              ^               |         config: true
+   +----------|------------------------------------+
+              |               |         config: false
+              |               |
+              |               |
+              |       +-----------------------------+
+              |       |       | operational state   |
+              |       |  +----v----+ +-----------+  |
+              |       |  |         | |           |  |
+              +       |  | applied | |  derived  |  |   operational:true
+            same +------>| config  | |   state   |<-------+
+            leaves    |  |         | |           |  |
+                      |  |         | |           |  |
+                      |  +---------+ +-----------+  |
+                      +-----------------------------+
+```
+
+In this diagram, applied config are "config false", or `/state` leaves that
+mirror intended config leaves, which are "config true", or `/config` leaves. Per
+[design.md](design.md#openconfig-path-compression), one of these are compressed
+out depending on the value of the `-prefer_operational_state` generation flag.
+
+Shadow paths are relevant only for compressed GoStructs, and indicate the
+compressed-out `/config` or `/state` YANG `leaf` nodes.
+
+## Problems with Path Compression and how Shadow Paths Help
+
+Path compression leads to the GoStruct that ygot generates not being able to
+represent both intended config and applied config at the same time. This leads
+to two problems:
+
+1.  When subscribing to a non-leaf path, some gNMI clients want to silently
+    ignore the compressed-out paths rather than erroring out due to an
+    unrecognized path.
+2.  Some use cases (e.g. [ygnmi](https://github.com/openconfig/ygnmi#queries))
+    use the same compressed GoStruct for representing either the "config view"
+    (intended config+derived state) combination, or the "state view" (applied
+    config+derived state) combination of leaves. We want to allow switching
+    between "config views" and "state views" for certain ygot utilities (e.g.
+    marshalling/unmarshalling).
+
+ygot address these issues by,
+
+1.  Always ignoring paths that match a `shadow-path` tag when doing
+    unmarshalling. For example, if a gNMI update for
+    `/interfaces/interface[name="foo"]/config/mtu` is unmarshalled into the
+    `Interface` GoStruct at the beginning of this documentation, then the field
+    will not be populated since it is a shadow path.
+2.  Supporting a `PreferShadowPath` option for some utilities (see section
+    below). `PreferShadowPath` means that the "shadow" path will be used in
+    preference to the "primary" path annotation.
+
+## Preferring Shadow Paths
+
+`PreferShadowPath` is behavioural option used to describe utilities preferring
+the `shadow-path` tag instead of the `path` tag in the generated GoStructs when
+they both exist on a field, and is therefore used to switch the meaning of the
+GoStruct from a "config view" (intended config+derived state) to a "state view"
+(applied config+derived state) or vice-versa (depending on the value of the
+`-prefer_operational_state` generation flag).
+
+For example, say we're using the utility `ytypes.SetNode` to unmarshal a gNMI
+update for `/interfaces/interface[name="foo"]/config/mtu`. Recall that this
+update is ignored when unmarshalled into the `Interface` GoStruct at the
+beginning of this documentation. This is because ygot sees that it is a shadow
+path (or alternatively, it does not exist in the "state view" of the YANG
+subtree corresponding to the GoStruct). However, if the `ygot.PreferShadowPath`
+option is used, then `ytypes.SetNode` will now populate the field using this
+update since it prefers the shadow path (or alternatively, it exists in the
+"config view" which `ygot.PreferShadowPath` indicated).
diff --git a/generator/generator.go b/generator/generator.go
@@ -70,7 +70,7 @@ var (
 	excludeState                         = flag.Bool("exclude_state", false, "If set to true, state (config false) fields in the YANG schema are not included in the generated Go code.")
 	skipEnumDedup                        = flag.Bool("skip_enum_deduplication", false, "If set to true, all leaves of type enumeration will have a unique enum output for them, rather than sharing a common type (default behaviour).")
 	preferOperationalState               = flag.Bool("prefer_operational_state", false, "If set to true, state (config false) fields in the YANG schema are preferred over intended config leaves in the generated Go code with compressed schema paths. This flag is only valid for compress_paths=true and exclude_state=false.")
-	ignoreShadowSchemaPaths              = flag.Bool("ignore_shadow_schema_paths", false, "If set to true when compress_paths=true, the shadowed schema path will be ignored while unmarshalling instead of causing an error. A shadow schema path is a config or state path which is selected over the other during schema compression when both config and state versions of the node exist.")
+	ignoreShadowSchemaPaths              = flag.Bool("ignore_shadow_schema_paths", false, "If set to true when compress_paths=true, the shadowed schema path will be ignored while unmarshalling instead of causing an error. A shadow schema path is a config or state path which is selected over the other during schema compression when both config and state versions of the node exist. See https://github.com/openconfig/ygot/blob/master/docs/shadow_paths.md for more information on shadow paths.")
 	shortenEnumLeafNames                 = flag.Bool("shorten_enum_leaf_names", false, "If also set to true when compress_paths=true, all leaves of type enumeration will by default not be prefixed with the name of its residing module.")
 	useDefiningModuleForTypedefEnumNames = flag.Bool("typedef_enum_with_defmod", false, "If set to true, all typedefs of type enumeration or identity will be prefixed with the name of its module of definition instead of its residing module.")
 	appendEnumSuffixForSimpleUnionEnums  = flag.Bool("enum_suffix_for_simple_union_enums", false, "If set to true when typedef_enum_with_defmod is also true, all inlined enumerations within unions will be suffixed with \"Enum\", instead of adding the suffix only for inlined enumerations within typedef unions.")