Add documentation for DirectiveBreakdown

[matthew-richerson]

Document the DirectiveBreakdown resource and how the WLM should interpret its fields.

[bdevcich]

Suggested change

* `colocation` specifies how two or more allocations influence the location of each other. The colocation constraint has two fields, `type` and `key`. Currently, the only value for `type` is `exlusive`. `key` can be any value. This constraint means that the allocations from an allocation set with the colocation constraint can't be placed on an NNF node with another allocation whose allocation set has a colocation constraint with the same key. Allocations from allocation sets with colocation constraints with different keys or allocation sets without the colocation constraint are okay to put on the same NNF node.

* `colocation` specifies how two or more allocations influence the location of each other. The colocation constraint has two fields, `type` and `key`. Currently, the only value for `type` is `exclusive`. `key` can be any value. This constraint means that the allocations from an allocation set with the colocation constraint can't be placed on an NNF node with another allocation whose allocation set has a colocation constraint with the same key. Allocations from allocation sets with colocation constraints with different keys or allocation sets without the colocation constraint are okay to put on the same NNF node.

[bdevcich]

Suggested change

	* `count` this field specified the number of allocations to make when `status.storage.allocationSets.allocationStrategy` is `AllocateAcrossServers`
	* `count` this field specifies the number of allocations to make when `status.storage.allocationSets.allocationStrategy` is `AllocateAcrossServers`

[roehrich-hpe]

Matt, when does an admin create a storage profile that uses 'scale' rather than 'count'? What is the admin trying to accomplish when they do that? The webhook does not allow both to be specified together.

[matthew-richerson]

count gives you consistent performance of the file system regardless of how big your job is. scale gives you file system performance that scales with the number of computes.

I think different workload types will benefit from setting the parameters differently.

• Single shared file jobs might only need a single MDT since metadata requirements are low.
• File per process jobs might want OSTs to scale with the number of computes.
• Creating a persistent storage allocation would benefit from setting count for both OSTs and MDTs since there's no compute job, so the compute node count will be 1 (or 0 if flux adds support for jobs with no compute resources)

I think scale will be the more used option, but count is good for the situations where you want to do something specific and want full control.

[roehrich-hpe]

All of that sounds like useful info to pass on to the WLM developer.

[matthew-richerson]

Yeah, I think you're right. I think I'll add it to the documentation for the corresponding fields in the NnfStorageProfile, though (from the year old PR). That's more of the admin/user facing documentation. That's who would be interested in knowing why you'd want to choose one value over another. The WLM documentation is more about how to take the information NNF provides in the DirectiveBreakdown and use that for scheduling computes and Rabbits.

I'm adding the contents of that old PR to the existing documentation for storage profiles. When that's done, I'll put a link to that section here.

[roehrich-hpe]

"This document describes how to interpret..." should that be "how the WLM should interpret" ?

[roehrich-hpe]

Suggested change

The DWS `DirectiveBreakdown` contains all the information to inform the WLM how to pick storage and compute nodes for a job. The `DirectiveBreakdown` resource is created by the NNF software during the `proposal` phase of the DWS workflow. The `spec` section of the `DirectiveBreakdown` is filled in with the `#DW` directive by the NNF software, and the `status` section contains the information for the WLM. The WLM should wait until the `status.ready` field is true before interpreting the rest of the `status` fields.

The DWS `DirectiveBreakdown` contains all the information necessary to inform the WLM about how to pick storage and compute nodes for a job. The `DirectiveBreakdown` resource is created by the NNF software during the `Proposal` phase of the DWS workflow. The `spec` section of the `DirectiveBreakdown` is filled in with the `#DW` directive by the NNF software, and the `status` section contains the information for the WLM. The WLM should wait until the `status.ready` field is true before interpreting the rest of the `status` fields.