Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extra docs clean ups #1081

Merged
merged 7 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,18 @@

### About

Collective Mind (CM) is a human-friendly interface to help everyone run, manage and reuse
a [growing number of MLPerf, MLOps and DevOps scripts](https://github.com/mlcommons/ck/tree/master/docs/list_of_scripts.md)
from [MLCommons projects](https://mlcommons.org) and [research papers](https://cTuning.org/ae)
in a unified way on any operating system with any software and hardware
either natively or inside containers.
Collective Mind (CM) is a [collection of portable, reusable and extensible
automation recipes](https://github.com/mlcommons/ck/tree/master/docs/list_of_scripts.md)
from [MLCommons projects](https://mlcommons.org) and [research papers](https://www.youtube.com/watch?v=7zpeIVwICa4)
with a human-friendly interface to help everyone assemble, run, benchmark
and optimize complex AI/ML applications and systems
in a unified way on any operating system with any software and hardware.

Here are a few most commonly used examples from the CM users
that should run in the same way on Linux, MacOS, Windows and other platforms
(see [Getting Started Guide](docs/getting-started.md) to understand
how they work and how to reuse them in your projects):
with any hardware (see the [Getting Started Guide](docs/getting-started.md)
to understand how they work, how to reuse and extend them for your projects
and how to share your own ones):

```bash
pip install cmind
Expand Down Expand Up @@ -100,11 +102,9 @@ output=cmind.access({'action':'run', 'automation':'script',
if output['return']==0: print (output)
```


Collective Mind is a community project being developed by the
[MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md)
with great help from [MLCommons (70+ AI organizations)](https://mlcommons.org/,
[research community]( https://www.youtube.com/watch?v=7zpeIVwICa4 )
with great help from [MLCommons (70+ AI organizations)](https://mlcommons.org),
and [individual contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md) -
we want to have a simple, non-intrusive, technology-agnostic, portable and easily-extensible interface
to automate all our manual and repetitive tasks including
Expand Down
45 changes: 1 addition & 44 deletions cm-mlops/script/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This is a source code of portable and reusable automation recipes
from MLCommons projects with a [human-friendly CM interface](https://github.com/mlcommons/ck) -
you can find more details [here](../../docs/list_of_scripts.md).
you can find a human-readable catalog of these automation recipes [here](../../docs/list_of_scripts.md).

### License

Expand All @@ -11,46 +11,3 @@ you can find more details [here](../../docs/list_of_scripts.md).
### Copyright

2022-2024 [MLCommons](https://mlcommons.org)

### Examples

```bash
pip install cmind

cm pull repo mlcommons@ck

cm run script "python app image-classification onnx"

cm run script "download file _wget" --url=https://cKnowledge.org/ai/data/computer_mouse.jpg --verify=no --env.CM_DOWNLOAD_CHECKSUM=45ae5c940233892c2f860efdf0b66e7e

cm run script "python app image-classification onnx" --input=computer_mouse.jpg

cm docker script "python app image-classification onnx" --input=computer_mouse.jpg
cm docker script "python app image-classification onnx" --input=computer_mouse.jpg -j -docker_it

cm run script "get generic-python-lib _package.onnxruntime"
cm run script "get coco dataset _val _2014"
cm run script "get ml-model stable-diffusion"
cm run script "get ml-model huggingface zoo _model-stub.alpindale/Llama-2-13b-ONNX" --model_filename=FP32/LlamaV2_13B_float32.onnx --skip_cache

cm show cache
cm show cache "get ml-model stable-diffusion"

cm run script "run common mlperf inference" --implementation=nvidia --model=bert-99 --category=datacenter --division=closed
cm find script "run common mlperf inference"

cm pull repo ctuning@cm-reproduce-research-projects
cmr "reproduce paper micro-2023 victima _install_deps"
cmr "reproduce paper micro-2023 victima _run"

...

```

```python
import cmind
output=cmind.access({'action':'run', 'automation':'script',
'tags':'python,app,image-classification,onnx',
'input':'computer_mouse.jpg'})
if output['return']==0: print (output)
```
5 changes: 5 additions & 0 deletions cm-mlops/script/install-cuda-prebuilt/_cm.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@
"CM_CUDA_LINUX_FILENAME": "cuda_12.0.0_525.60.13_linux.run"
}
},
"12.1.1": {
"env": {
"CM_CUDA_LINUX_FILENAME": "cuda_12.1.1_530.30.02_linux.run"
}
},
"12.2.0": {
"env": {
"CM_CUDA_LINUX_FILENAME": "cuda_12.2.0_535.54.03_linux.run"
Expand Down
2 changes: 1 addition & 1 deletion cm-mlops/script/install-cuda-prebuilt/customize.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ def preprocess(i):
automation = i['automation']
version = env.get('CM_VERSION')
if version not in env.get('CM_CUDA_LINUX_FILENAME', ''):
return {'return': 1, 'error': "Only CUDA versions 11.7.0, 11.8.0, 12.0.0 and 12.2.0 are supported now!"}
return {'return': 1, 'error': "Only CUDA versions 11.7.0, 11.8.0, 12.0.0, 12.1.1 and 12.2.3 are supported now!"}

recursion_spaces = i['recursion_spaces']
nvcc_bin = "nvcc"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ deps:
########################################################################
# Install MLPerf inference dependencies

- tags: get,mlperf,inference,results
- tags: get,mlperf,inference,results,_ctuning
names:
inference-results
version: v3.1
Expand Down
18 changes: 18 additions & 0 deletions cm-mlops/script/reproduce-mlperf-inference-nvidia/_cm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,24 @@ variations:
env:
CM_MODEL: dlrm-v2-99.9

gptj_:
deps:
- tags: get,generic-python-lib,_torch

gptj-99:
group: model
base:
- gptj_
env:
CM_MODEL: gptj-99

gptj-99.9:
group: model
base:
- gptj_
env:
CM_MODEL: gptj-99.9

batch_size.#:
group: batch-size
env:
Expand Down
6 changes: 3 additions & 3 deletions docs/taskforce.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

## Mission

* Develop [reusable automation recipes and workflows for MLOps and DevOps](https://github.com/mlcommons/ck/tree/master/cm-mlops/script)
* Develop [reusable automation recipes and workflows](https://github.com/mlcommons/ck/tree/master/cm-mlops/script)
with [a common and human-friendly interface (Collective Mind aka CM)](https://github.com/mlcommons/ck)
to help everyone run, reproduce, customize and optimize different implementations of MLPerf benchmarks
to support MLCommons projects and help everyone assemble, run, reproduce, customize and optimize ML(Perf) benchmarks
in a unified and automated way across diverse models, data sets, software and hardware from different vendors.
* Gradually add the MLCommons CM interface to [all MLPerf inference submissions](https://github.com/mlcommons/ck/issues/1052) starting from v3.1.
* Gradually extend a unified MLCommons CM interface to automate [all MLPerf inference submissions](https://github.com/mlcommons/ck/issues/1052) starting from v3.1.
* Continuously encode MLPerf rules and best practices in the [CM automation recipes and workflows for MLPerf](https://github.com/mlcommons/ck/tree/master/cm-mlops/script)
to reduce the burden for submitters to go through numerous README files
and track all the latest changes and updates.
Expand Down
Loading