Merge branch 'mlcommons:master' into master

cm-mlops/script/get-dataset-cognata/README.md

@@ -0,0 +1,187 @@
+<details>
+<summary>Click here to see the table of contents.</summary>
+
+* [About](#about)
+* [Summary](#summary)
+* [Reuse this script in your project](#reuse-this-script-in-your-project)
+  * [ Install CM automation language](#install-cm-automation-language)
+  * [ Check CM script flags](#check-cm-script-flags)
+  * [ Run this script from command line](#run-this-script-from-command-line)
+  * [ Run this script from Python](#run-this-script-from-python)
+  * [ Run this script via GUI](#run-this-script-via-gui)
+  * [ Run this script via Docker (beta)](#run-this-script-via-docker-(beta))
+* [Customization](#customization)
+  * [ Variations](#variations)
+  * [ Default environment](#default-environment)
+* [Script workflow, dependencies and native scripts](#script-workflow-dependencies-and-native-scripts)
+* [Script output](#script-output)
+* [New environment keys (filter)](#new-environment-keys-(filter))
+* [New environment keys auto-detected from customize](#new-environment-keys-auto-detected-from-customize)
+* [Maintainers](#maintainers)
+
+</details>
+
+*Note that this README is automatically generated - don't edit!*
+
+### About
+
+#### Summary
+
+* Category: *AI/ML datasets.*
+* CM GitHub repository: *[mlcommons@ck](https://github.com/mlcommons/ck/tree/master/cm-mlops)*
+* GitHub directory for this script: *[GitHub](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata)*
+* CM meta description for this script: *[_cm.json](_cm.json)*
+* CM "database" tags to find this script: *get,dataset,cognata,object-detection,original*
+* Output cached? *True*
+___
+### Reuse this script in your project
+
+#### Install CM automation language
+
+* [Installation guide](https://github.com/mlcommons/ck/blob/master/docs/installation.md)
+* [CM intro](https://doi.org/10.5281/zenodo.8105339)
+
+#### Pull CM repository with this automation
+
+```cm pull repo mlcommons@ck```
+
+
+#### Run this script from command line
+
+1. `cm run script --tags=get,dataset,cognata,object-detection,original[,variations] `
+
+2. `cmr "get dataset cognata object-detection original[ variations]" `
+
+* `variations` can be seen [here](#variations)
+
+#### Run this script from Python
+
+<details>
+<summary>Click here to expand this section.</summary>
+
+```python
+
+import cmind
+
+r = cmind.access({'action':'run'
+                  'automation':'script',
+                  'tags':'get,dataset,cognata,object-detection,original'
+                  'out':'con',
+                  ...
+                  (other input keys for this script)
+                  ...
+                 })
+
+if r['return']>0:
+    print (r['error'])
+
+```
+
+</details>
+
+
+#### Run this script via GUI
+
+```cmr "cm gui" --script="get,dataset,cognata,object-detection,original"```
+
+Use this [online GUI](https://cKnowledge.org/cm-gui/?tags=get,dataset,cognata,object-detection,original) to generate CM CMD.
+
+#### Run this script via Docker (beta)
+
+`cm docker script "get dataset cognata object-detection original[ variations]" `
+
+___
+### Customization
+
+
+#### Variations
+
+  * Group "**dataset-type**"
+    <details>
+    <summary>Click here to expand this section.</summary>
+
+    * `_calibration`
+      - Workflow:
+    * **`_validation`** (default)
+      - Environment variables:
+        - *CM_DATASET_CALIBRATION*: `no`
+      - Workflow:
+
+    </details>
+
+
+  * Group "**size**"
+    <details>
+    <summary>Click here to expand this section.</summary>
+
+    * **`_50`** (default)
+      - Environment variables:
+        - *CM_DATASET_SIZE*: `50`
+      - Workflow:
+    * `_500`
+      - Environment variables:
+        - *CM_DATASET_SIZE*: `500`
+      - Workflow:
+    * `_full`
+      - Environment variables:
+        - *CM_DATASET_SIZE*: ``
+      - Workflow:
+    * `_size.#`
+      - Environment variables:
+        - *CM_DATASET_SIZE*: `#`
+      - Workflow:
+
+    </details>
+
+
+#### Default variations
+
+`_50,_validation`
+#### Default environment
+
+<details>
+<summary>Click here to expand this section.</summary>
+
+These keys can be updated via `--env.KEY=VALUE` or `env` dictionary in `@input.json` or using script flags.
+
+* CM_DATASET_CALIBRATION: `no`
+
+</details>
+
+___
+### Script workflow, dependencies and native scripts
+
+<details>
+<summary>Click here to expand this section.</summary>
+
+  1. ***Read "deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/_cm.json)***
+     * get,python3
+       * CM names: `--adr.['python', 'python3']...`
+       - CM script: [get-python3](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-python3)
+  1. ***Run "preprocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/customize.py)***
+  1. Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/_cm.json)
+  1. ***Run native script if exists***
+  1. Read "posthook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/_cm.json)
+  1. ***Run "postrocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/customize.py)***
+  1. Read "post_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cognata/_cm.json)
+</details>
+
+___
+### Script output
+`cmr "get dataset cognata object-detection original[,variations]"  -j`
+#### New environment keys (filter)
+
+* `CM_CALIBRATION_DATASET_PATH`
+* `CM_DATASET_ANNOTATIONS_DIR_PATH`
+* `CM_DATASET_ANNOTATIONS_FILE_PATH`
+* `CM_DATASET_PATH`
+* `CM_DATASET_PATH_ROOT`
+#### New environment keys auto-detected from customize
+
+* `CM_CALIBRATION_DATASET_PATH`
+* `CM_DATASET_PATH`
+* `CM_DATASET_PATH_ROOT`
+___
+### Maintainers
+
+* [Open MLCommons taskforce on automation and reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md)

cm-mlops/script/reproduce-mlperf-inference-nvidia/README.md

@@ -272,6 +272,8 @@ ___
              - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
            * get,generic-python-lib,_onnx-graphsurgeon
              - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
+           * get,generic-python-lib,_package.scikit-learn
+             - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
     * `_gptj_`
       - Workflow:
         1. ***Read "deps" on other CM scripts***