diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/404.html b/404.html new file mode 100644 index 00000000..baf5a82e --- /dev/null +++ b/404.html @@ -0,0 +1,2689 @@ + + + + + + + + + + + + + + + + + + + + + + AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ +

404 - Not found

+ +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/CNAME b/CNAME new file mode 100644 index 00000000..9f3a0846 --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +ai-on-openshift.io \ No newline at end of file diff --git a/assets/ai-on-openshift-title.svg b/assets/ai-on-openshift-title.svg new file mode 100644 index 00000000..ffb9735d --- /dev/null +++ b/assets/ai-on-openshift-title.svg @@ -0,0 +1,697 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/assets/home-robot.png b/assets/home-robot.png new file mode 100644 index 00000000..0e7123cb Binary files /dev/null and b/assets/home-robot.png differ diff --git a/assets/images/favicon.png b/assets/images/favicon.png new file mode 100644 index 00000000..1cf13b9f Binary files /dev/null and b/assets/images/favicon.png differ diff --git a/assets/images/social/demos/codellama-continue/codellama-continue.png b/assets/images/social/demos/codellama-continue/codellama-continue.png new file mode 100644 index 00000000..9453c32f Binary files /dev/null and b/assets/images/social/demos/codellama-continue/codellama-continue.png differ diff --git a/assets/images/social/demos/credit-card-fraud-detection-mlflow/credit-card-fraud.png b/assets/images/social/demos/credit-card-fraud-detection-mlflow/credit-card-fraud.png new file mode 100644 index 00000000..66105f6f Binary files /dev/null and b/assets/images/social/demos/credit-card-fraud-detection-mlflow/credit-card-fraud.png differ diff --git a/assets/images/social/demos/financial-fraud-detection/financial-fraud-detection.png b/assets/images/social/demos/financial-fraud-detection/financial-fraud-detection.png new file mode 100644 index 00000000..72742fbc Binary files /dev/null and b/assets/images/social/demos/financial-fraud-detection/financial-fraud-detection.png differ diff --git a/assets/images/social/demos/llama2-finetune/llama2-finetune.png b/assets/images/social/demos/llama2-finetune/llama2-finetune.png new file mode 100644 index 00000000..3a315386 Binary files /dev/null and b/assets/images/social/demos/llama2-finetune/llama2-finetune.png differ diff --git a/assets/images/social/demos/llm-chat-doc/llm-chat-doc.png b/assets/images/social/demos/llm-chat-doc/llm-chat-doc.png new file mode 100644 index 00000000..6c9d53ef Binary files /dev/null and b/assets/images/social/demos/llm-chat-doc/llm-chat-doc.png differ diff --git a/assets/images/social/demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai.png b/assets/images/social/demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai.png new file mode 100644 index 00000000..c2b4a029 Binary files /dev/null and b/assets/images/social/demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai.png differ diff --git a/assets/images/social/demos/retail-object-detection/retail-object-detection.png b/assets/images/social/demos/retail-object-detection/retail-object-detection.png new file mode 100644 index 00000000..a3ca2654 Binary files /dev/null and b/assets/images/social/demos/retail-object-detection/retail-object-detection.png differ diff --git a/assets/images/social/demos/smart-city/smart-city.png b/assets/images/social/demos/smart-city/smart-city.png new file mode 100644 index 00000000..4d6e7451 Binary files /dev/null and b/assets/images/social/demos/smart-city/smart-city.png differ diff --git a/assets/images/social/demos/stable-diffusion/stable-diffusion.png b/assets/images/social/demos/stable-diffusion/stable-diffusion.png new file mode 100644 index 00000000..5e25bc44 Binary files /dev/null and b/assets/images/social/demos/stable-diffusion/stable-diffusion.png differ diff --git a/assets/images/social/demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow.png b/assets/images/social/demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow.png new file mode 100644 index 00000000..775aebb8 Binary files /dev/null and b/assets/images/social/demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow.png differ diff --git a/assets/images/social/demos/water-pump-failure-prediction/water-pump-failure-prediction.png b/assets/images/social/demos/water-pump-failure-prediction/water-pump-failure-prediction.png new file mode 100644 index 00000000..e947998b Binary files /dev/null and b/assets/images/social/demos/water-pump-failure-prediction/water-pump-failure-prediction.png differ diff --git a/assets/images/social/demos/xray-pipeline/xray-pipeline.png b/assets/images/social/demos/xray-pipeline/xray-pipeline.png new file mode 100644 index 00000000..329a6e88 Binary files /dev/null and b/assets/images/social/demos/xray-pipeline/xray-pipeline.png differ diff --git a/assets/images/social/demos/yolov5-training-serving/yolov5-training-serving.png b/assets/images/social/demos/yolov5-training-serving/yolov5-training-serving.png new file mode 100644 index 00000000..c095c08c Binary files /dev/null and b/assets/images/social/demos/yolov5-training-serving/yolov5-training-serving.png differ diff --git a/assets/images/social/generative-ai/llm-serving.png b/assets/images/social/generative-ai/llm-serving.png new file mode 100644 index 00000000..981eb716 Binary files /dev/null and b/assets/images/social/generative-ai/llm-serving.png differ diff --git a/assets/images/social/generative-ai/what-is-generative-ai.png b/assets/images/social/generative-ai/what-is-generative-ai.png new file mode 100644 index 00000000..605efadc Binary files /dev/null and b/assets/images/social/generative-ai/what-is-generative-ai.png differ diff --git a/assets/images/social/getting-started/opendatahub.png b/assets/images/social/getting-started/opendatahub.png new file mode 100644 index 00000000..0afe5ac3 Binary files /dev/null and b/assets/images/social/getting-started/opendatahub.png differ diff --git a/assets/images/social/getting-started/openshift-ai.png b/assets/images/social/getting-started/openshift-ai.png new file mode 100644 index 00000000..3822c1fa Binary files /dev/null and b/assets/images/social/getting-started/openshift-ai.png differ diff --git a/assets/images/social/getting-started/openshift.png b/assets/images/social/getting-started/openshift.png new file mode 100644 index 00000000..af0909c3 Binary files /dev/null and b/assets/images/social/getting-started/openshift.png differ diff --git a/assets/images/social/getting-started/why-this-site.png b/assets/images/social/getting-started/why-this-site.png new file mode 100644 index 00000000..8260faeb Binary files /dev/null and b/assets/images/social/getting-started/why-this-site.png differ diff --git a/assets/images/social/index.png b/assets/images/social/index.png new file mode 100644 index 00000000..26bb281a Binary files /dev/null and b/assets/images/social/index.png differ diff --git a/assets/images/social/odh-rhoai/accelerator-profiles.png b/assets/images/social/odh-rhoai/accelerator-profiles.png new file mode 100644 index 00000000..d4705abd Binary files /dev/null and b/assets/images/social/odh-rhoai/accelerator-profiles.png differ diff --git a/assets/images/social/odh-rhoai/configuration.png b/assets/images/social/odh-rhoai/configuration.png new file mode 100644 index 00000000..5e59b39e Binary files /dev/null and b/assets/images/social/odh-rhoai/configuration.png differ diff --git a/assets/images/social/odh-rhoai/connect-vscode-to-rhoai-wb.png b/assets/images/social/odh-rhoai/connect-vscode-to-rhoai-wb.png new file mode 100644 index 00000000..00ea52cc Binary files /dev/null and b/assets/images/social/odh-rhoai/connect-vscode-to-rhoai-wb.png differ diff --git a/assets/images/social/odh-rhoai/custom-notebooks.png b/assets/images/social/odh-rhoai/custom-notebooks.png new file mode 100644 index 00000000..edf15bbf Binary files /dev/null and b/assets/images/social/odh-rhoai/custom-notebooks.png differ diff --git a/assets/images/social/odh-rhoai/custom-runtime-triton.png b/assets/images/social/odh-rhoai/custom-runtime-triton.png new file mode 100644 index 00000000..a7da897b Binary files /dev/null and b/assets/images/social/odh-rhoai/custom-runtime-triton.png differ diff --git a/assets/images/social/odh-rhoai/from-zero-to-workbench/using-cli.png b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-cli.png new file mode 100644 index 00000000..370e004c Binary files /dev/null and b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-cli.png differ diff --git a/assets/images/social/odh-rhoai/from-zero-to-workbench/using-developer-hub.png b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-developer-hub.png new file mode 100644 index 00000000..f88ab429 Binary files /dev/null and b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-developer-hub.png differ diff --git a/assets/images/social/odh-rhoai/from-zero-to-workbench/using-ui.png b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-ui.png new file mode 100644 index 00000000..cea8e36e Binary files /dev/null and b/assets/images/social/odh-rhoai/from-zero-to-workbench/using-ui.png differ diff --git a/assets/images/social/odh-rhoai/gitops.png b/assets/images/social/odh-rhoai/gitops.png new file mode 100644 index 00000000..2725e089 Binary files /dev/null and b/assets/images/social/odh-rhoai/gitops.png differ diff --git a/assets/images/social/odh-rhoai/gpu-pruner.png b/assets/images/social/odh-rhoai/gpu-pruner.png new file mode 100644 index 00000000..40bd6eb6 Binary files /dev/null and b/assets/images/social/odh-rhoai/gpu-pruner.png differ diff --git a/assets/images/social/odh-rhoai/kserve-timeout.png b/assets/images/social/odh-rhoai/kserve-timeout.png new file mode 100644 index 00000000..98899e58 Binary files /dev/null and b/assets/images/social/odh-rhoai/kserve-timeout.png differ diff --git a/assets/images/social/odh-rhoai/kserve-uwm-dashboard-metrics.png b/assets/images/social/odh-rhoai/kserve-uwm-dashboard-metrics.png new file mode 100644 index 00000000..772484bb Binary files /dev/null and b/assets/images/social/odh-rhoai/kserve-uwm-dashboard-metrics.png differ diff --git a/assets/images/social/odh-rhoai/kueue-preemption/readme.png b/assets/images/social/odh-rhoai/kueue-preemption/readme.png new file mode 100644 index 00000000..e608d2b6 Binary files /dev/null and b/assets/images/social/odh-rhoai/kueue-preemption/readme.png differ diff --git a/assets/images/social/odh-rhoai/model-serving-type-modification.png b/assets/images/social/odh-rhoai/model-serving-type-modification.png new file mode 100644 index 00000000..3c233a4a Binary files /dev/null and b/assets/images/social/odh-rhoai/model-serving-type-modification.png differ diff --git a/assets/images/social/odh-rhoai/nvidia-gpus.png b/assets/images/social/odh-rhoai/nvidia-gpus.png new file mode 100644 index 00000000..58c58abc Binary files /dev/null and b/assets/images/social/odh-rhoai/nvidia-gpus.png differ diff --git a/assets/images/social/odh-rhoai/odh-tools-and-extensions-companion.png b/assets/images/social/odh-rhoai/odh-tools-and-extensions-companion.png new file mode 100644 index 00000000..d91c7c57 Binary files /dev/null and b/assets/images/social/odh-rhoai/odh-tools-and-extensions-companion.png differ diff --git a/assets/images/social/odh-rhoai/openshift-group-management.png b/assets/images/social/odh-rhoai/openshift-group-management.png new file mode 100644 index 00000000..373c1731 Binary files /dev/null and b/assets/images/social/odh-rhoai/openshift-group-management.png differ diff --git a/assets/images/social/odh-rhoai/single-stack-serving-certificate.png b/assets/images/social/odh-rhoai/single-stack-serving-certificate.png new file mode 100644 index 00000000..40b2ac1c Binary files /dev/null and b/assets/images/social/odh-rhoai/single-stack-serving-certificate.png differ diff --git a/assets/images/social/patterns/bucket-notifications/bucket-notifications.png b/assets/images/social/patterns/bucket-notifications/bucket-notifications.png new file mode 100644 index 00000000..f614c9e6 Binary files /dev/null and b/assets/images/social/patterns/bucket-notifications/bucket-notifications.png differ diff --git a/assets/images/social/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage.png b/assets/images/social/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage.png new file mode 100644 index 00000000..f500469f Binary files /dev/null and b/assets/images/social/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage.png differ diff --git a/assets/images/social/patterns/kafka/kafka-to-serverless/kafka-to-serverless.png b/assets/images/social/patterns/kafka/kafka-to-serverless/kafka-to-serverless.png new file mode 100644 index 00000000..8e3f4ea2 Binary files /dev/null and b/assets/images/social/patterns/kafka/kafka-to-serverless/kafka-to-serverless.png differ diff --git a/assets/images/social/patterns/starproxy/starproxy.png b/assets/images/social/patterns/starproxy/starproxy.png new file mode 100644 index 00000000..1000e7dd Binary files /dev/null and b/assets/images/social/patterns/starproxy/starproxy.png differ diff --git a/assets/images/social/predictive-ai/what-is-predictive-ai.png b/assets/images/social/predictive-ai/what-is-predictive-ai.png new file mode 100644 index 00000000..6770bd95 Binary files /dev/null and b/assets/images/social/predictive-ai/what-is-predictive-ai.png differ diff --git a/assets/images/social/tools-and-applications/airflow/airflow.png b/assets/images/social/tools-and-applications/airflow/airflow.png new file mode 100644 index 00000000..6a68bd1a Binary files /dev/null and b/assets/images/social/tools-and-applications/airflow/airflow.png differ diff --git a/assets/images/social/tools-and-applications/apache-nifi/apache-nifi.png b/assets/images/social/tools-and-applications/apache-nifi/apache-nifi.png new file mode 100644 index 00000000..d6910c88 Binary files /dev/null and b/assets/images/social/tools-and-applications/apache-nifi/apache-nifi.png differ diff --git a/assets/images/social/tools-and-applications/apache-spark/apache-spark.png b/assets/images/social/tools-and-applications/apache-spark/apache-spark.png new file mode 100644 index 00000000..39ad4d17 Binary files /dev/null and b/assets/images/social/tools-and-applications/apache-spark/apache-spark.png differ diff --git a/assets/images/social/tools-and-applications/ensemble-serving/ensemble-serving.png b/assets/images/social/tools-and-applications/ensemble-serving/ensemble-serving.png new file mode 100644 index 00000000..94eaea7c Binary files /dev/null and b/assets/images/social/tools-and-applications/ensemble-serving/ensemble-serving.png differ diff --git a/assets/images/social/tools-and-applications/minio/minio.png b/assets/images/social/tools-and-applications/minio/minio.png new file mode 100644 index 00000000..78cbfe5d Binary files /dev/null and b/assets/images/social/tools-and-applications/minio/minio.png differ diff --git a/assets/images/social/tools-and-applications/mlflow/mlflow.png b/assets/images/social/tools-and-applications/mlflow/mlflow.png new file mode 100644 index 00000000..207118fe Binary files /dev/null and b/assets/images/social/tools-and-applications/mlflow/mlflow.png differ diff --git a/assets/images/social/tools-and-applications/rclone/rclone.png b/assets/images/social/tools-and-applications/rclone/rclone.png new file mode 100644 index 00000000..f1838a52 Binary files /dev/null and b/assets/images/social/tools-and-applications/rclone/rclone.png differ diff --git a/assets/images/social/tools-and-applications/riva/riva.png b/assets/images/social/tools-and-applications/riva/riva.png new file mode 100644 index 00000000..c8b3ba8e Binary files /dev/null and b/assets/images/social/tools-and-applications/riva/riva.png differ diff --git a/assets/images/social/whats-new/whats-new.png b/assets/images/social/whats-new/whats-new.png new file mode 100644 index 00000000..06c6696c Binary files /dev/null and b/assets/images/social/whats-new/whats-new.png differ diff --git a/assets/javascripts/bundle.83f73b43.min.js b/assets/javascripts/bundle.83f73b43.min.js new file mode 100644 index 00000000..43d8b70f --- /dev/null +++ b/assets/javascripts/bundle.83f73b43.min.js @@ -0,0 +1,16 @@ +"use strict";(()=>{var Wi=Object.create;var gr=Object.defineProperty;var Di=Object.getOwnPropertyDescriptor;var Vi=Object.getOwnPropertyNames,Vt=Object.getOwnPropertySymbols,Ni=Object.getPrototypeOf,yr=Object.prototype.hasOwnProperty,ao=Object.prototype.propertyIsEnumerable;var io=(e,t,r)=>t in e?gr(e,t,{enumerable:!0,configurable:!0,writable:!0,value:r}):e[t]=r,$=(e,t)=>{for(var r in t||(t={}))yr.call(t,r)&&io(e,r,t[r]);if(Vt)for(var r of Vt(t))ao.call(t,r)&&io(e,r,t[r]);return e};var so=(e,t)=>{var r={};for(var o in e)yr.call(e,o)&&t.indexOf(o)<0&&(r[o]=e[o]);if(e!=null&&Vt)for(var o of Vt(e))t.indexOf(o)<0&&ao.call(e,o)&&(r[o]=e[o]);return r};var xr=(e,t)=>()=>(t||e((t={exports:{}}).exports,t),t.exports);var zi=(e,t,r,o)=>{if(t&&typeof t=="object"||typeof t=="function")for(let n of Vi(t))!yr.call(e,n)&&n!==r&&gr(e,n,{get:()=>t[n],enumerable:!(o=Di(t,n))||o.enumerable});return e};var Mt=(e,t,r)=>(r=e!=null?Wi(Ni(e)):{},zi(t||!e||!e.__esModule?gr(r,"default",{value:e,enumerable:!0}):r,e));var co=(e,t,r)=>new Promise((o,n)=>{var i=p=>{try{s(r.next(p))}catch(c){n(c)}},a=p=>{try{s(r.throw(p))}catch(c){n(c)}},s=p=>p.done?o(p.value):Promise.resolve(p.value).then(i,a);s((r=r.apply(e,t)).next())});var lo=xr((Er,po)=>{(function(e,t){typeof Er=="object"&&typeof po!="undefined"?t():typeof define=="function"&&define.amd?define(t):t()})(Er,function(){"use strict";function e(r){var o=!0,n=!1,i=null,a={text:!0,search:!0,url:!0,tel:!0,email:!0,password:!0,number:!0,date:!0,month:!0,week:!0,time:!0,datetime:!0,"datetime-local":!0};function s(k){return!!(k&&k!==document&&k.nodeName!=="HTML"&&k.nodeName!=="BODY"&&"classList"in k&&"contains"in k.classList)}function p(k){var ft=k.type,qe=k.tagName;return!!(qe==="INPUT"&&a[ft]&&!k.readOnly||qe==="TEXTAREA"&&!k.readOnly||k.isContentEditable)}function c(k){k.classList.contains("focus-visible")||(k.classList.add("focus-visible"),k.setAttribute("data-focus-visible-added",""))}function l(k){k.hasAttribute("data-focus-visible-added")&&(k.classList.remove("focus-visible"),k.removeAttribute("data-focus-visible-added"))}function f(k){k.metaKey||k.altKey||k.ctrlKey||(s(r.activeElement)&&c(r.activeElement),o=!0)}function u(k){o=!1}function d(k){s(k.target)&&(o||p(k.target))&&c(k.target)}function y(k){s(k.target)&&(k.target.classList.contains("focus-visible")||k.target.hasAttribute("data-focus-visible-added"))&&(n=!0,window.clearTimeout(i),i=window.setTimeout(function(){n=!1},100),l(k.target))}function L(k){document.visibilityState==="hidden"&&(n&&(o=!0),X())}function X(){document.addEventListener("mousemove",J),document.addEventListener("mousedown",J),document.addEventListener("mouseup",J),document.addEventListener("pointermove",J),document.addEventListener("pointerdown",J),document.addEventListener("pointerup",J),document.addEventListener("touchmove",J),document.addEventListener("touchstart",J),document.addEventListener("touchend",J)}function te(){document.removeEventListener("mousemove",J),document.removeEventListener("mousedown",J),document.removeEventListener("mouseup",J),document.removeEventListener("pointermove",J),document.removeEventListener("pointerdown",J),document.removeEventListener("pointerup",J),document.removeEventListener("touchmove",J),document.removeEventListener("touchstart",J),document.removeEventListener("touchend",J)}function J(k){k.target.nodeName&&k.target.nodeName.toLowerCase()==="html"||(o=!1,te())}document.addEventListener("keydown",f,!0),document.addEventListener("mousedown",u,!0),document.addEventListener("pointerdown",u,!0),document.addEventListener("touchstart",u,!0),document.addEventListener("visibilitychange",L,!0),X(),r.addEventListener("focus",d,!0),r.addEventListener("blur",y,!0),r.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&r.host?r.host.setAttribute("data-js-focus-visible",""):r.nodeType===Node.DOCUMENT_NODE&&(document.documentElement.classList.add("js-focus-visible"),document.documentElement.setAttribute("data-js-focus-visible",""))}if(typeof window!="undefined"&&typeof document!="undefined"){window.applyFocusVisiblePolyfill=e;var t;try{t=new CustomEvent("focus-visible-polyfill-ready")}catch(r){t=document.createEvent("CustomEvent"),t.initCustomEvent("focus-visible-polyfill-ready",!1,!1,{})}window.dispatchEvent(t)}typeof document!="undefined"&&e(document)})});var qr=xr((hy,On)=>{"use strict";/*! + * escape-html + * Copyright(c) 2012-2013 TJ Holowaychuk + * Copyright(c) 2015 Andreas Lubbe + * Copyright(c) 2015 Tiancheng "Timothy" Gu + * MIT Licensed + */var $a=/["'&<>]/;On.exports=Pa;function Pa(e){var t=""+e,r=$a.exec(t);if(!r)return t;var o,n="",i=0,a=0;for(i=r.index;i{/*! + * clipboard.js v2.0.11 + * https://clipboardjs.com/ + * + * Licensed MIT © Zeno Rocha + */(function(t,r){typeof It=="object"&&typeof Yr=="object"?Yr.exports=r():typeof define=="function"&&define.amd?define([],r):typeof It=="object"?It.ClipboardJS=r():t.ClipboardJS=r()})(It,function(){return function(){var e={686:function(o,n,i){"use strict";i.d(n,{default:function(){return Ui}});var a=i(279),s=i.n(a),p=i(370),c=i.n(p),l=i(817),f=i.n(l);function u(V){try{return document.execCommand(V)}catch(A){return!1}}var d=function(A){var M=f()(A);return u("cut"),M},y=d;function L(V){var A=document.documentElement.getAttribute("dir")==="rtl",M=document.createElement("textarea");M.style.fontSize="12pt",M.style.border="0",M.style.padding="0",M.style.margin="0",M.style.position="absolute",M.style[A?"right":"left"]="-9999px";var F=window.pageYOffset||document.documentElement.scrollTop;return M.style.top="".concat(F,"px"),M.setAttribute("readonly",""),M.value=V,M}var X=function(A,M){var F=L(A);M.container.appendChild(F);var D=f()(F);return u("copy"),F.remove(),D},te=function(A){var M=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body},F="";return typeof A=="string"?F=X(A,M):A instanceof HTMLInputElement&&!["text","search","url","tel","password"].includes(A==null?void 0:A.type)?F=X(A.value,M):(F=f()(A),u("copy")),F},J=te;function k(V){"@babel/helpers - typeof";return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?k=function(M){return typeof M}:k=function(M){return M&&typeof Symbol=="function"&&M.constructor===Symbol&&M!==Symbol.prototype?"symbol":typeof M},k(V)}var ft=function(){var A=arguments.length>0&&arguments[0]!==void 0?arguments[0]:{},M=A.action,F=M===void 0?"copy":M,D=A.container,Y=A.target,$e=A.text;if(F!=="copy"&&F!=="cut")throw new Error('Invalid "action" value, use either "copy" or "cut"');if(Y!==void 0)if(Y&&k(Y)==="object"&&Y.nodeType===1){if(F==="copy"&&Y.hasAttribute("disabled"))throw new Error('Invalid "target" attribute. Please use "readonly" instead of "disabled" attribute');if(F==="cut"&&(Y.hasAttribute("readonly")||Y.hasAttribute("disabled")))throw new Error(`Invalid "target" attribute. You can't cut text from elements with "readonly" or "disabled" attributes`)}else throw new Error('Invalid "target" value, use a valid Element');if($e)return J($e,{container:D});if(Y)return F==="cut"?y(Y):J(Y,{container:D})},qe=ft;function Fe(V){"@babel/helpers - typeof";return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?Fe=function(M){return typeof M}:Fe=function(M){return M&&typeof Symbol=="function"&&M.constructor===Symbol&&M!==Symbol.prototype?"symbol":typeof M},Fe(V)}function ki(V,A){if(!(V instanceof A))throw new TypeError("Cannot call a class as a function")}function no(V,A){for(var M=0;M0&&arguments[0]!==void 0?arguments[0]:{};this.action=typeof D.action=="function"?D.action:this.defaultAction,this.target=typeof D.target=="function"?D.target:this.defaultTarget,this.text=typeof D.text=="function"?D.text:this.defaultText,this.container=Fe(D.container)==="object"?D.container:document.body}},{key:"listenClick",value:function(D){var Y=this;this.listener=c()(D,"click",function($e){return Y.onClick($e)})}},{key:"onClick",value:function(D){var Y=D.delegateTarget||D.currentTarget,$e=this.action(Y)||"copy",Dt=qe({action:$e,container:this.container,target:this.target(Y),text:this.text(Y)});this.emit(Dt?"success":"error",{action:$e,text:Dt,trigger:Y,clearSelection:function(){Y&&Y.focus(),window.getSelection().removeAllRanges()}})}},{key:"defaultAction",value:function(D){return vr("action",D)}},{key:"defaultTarget",value:function(D){var Y=vr("target",D);if(Y)return document.querySelector(Y)}},{key:"defaultText",value:function(D){return vr("text",D)}},{key:"destroy",value:function(){this.listener.destroy()}}],[{key:"copy",value:function(D){var Y=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body};return J(D,Y)}},{key:"cut",value:function(D){return y(D)}},{key:"isSupported",value:function(){var D=arguments.length>0&&arguments[0]!==void 0?arguments[0]:["copy","cut"],Y=typeof D=="string"?[D]:D,$e=!!document.queryCommandSupported;return Y.forEach(function(Dt){$e=$e&&!!document.queryCommandSupported(Dt)}),$e}}]),M}(s()),Ui=Fi},828:function(o){var n=9;if(typeof Element!="undefined"&&!Element.prototype.matches){var i=Element.prototype;i.matches=i.matchesSelector||i.mozMatchesSelector||i.msMatchesSelector||i.oMatchesSelector||i.webkitMatchesSelector}function a(s,p){for(;s&&s.nodeType!==n;){if(typeof s.matches=="function"&&s.matches(p))return s;s=s.parentNode}}o.exports=a},438:function(o,n,i){var a=i(828);function s(l,f,u,d,y){var L=c.apply(this,arguments);return l.addEventListener(u,L,y),{destroy:function(){l.removeEventListener(u,L,y)}}}function p(l,f,u,d,y){return typeof l.addEventListener=="function"?s.apply(null,arguments):typeof u=="function"?s.bind(null,document).apply(null,arguments):(typeof l=="string"&&(l=document.querySelectorAll(l)),Array.prototype.map.call(l,function(L){return s(L,f,u,d,y)}))}function c(l,f,u,d){return function(y){y.delegateTarget=a(y.target,f),y.delegateTarget&&d.call(l,y)}}o.exports=p},879:function(o,n){n.node=function(i){return i!==void 0&&i instanceof HTMLElement&&i.nodeType===1},n.nodeList=function(i){var a=Object.prototype.toString.call(i);return i!==void 0&&(a==="[object NodeList]"||a==="[object HTMLCollection]")&&"length"in i&&(i.length===0||n.node(i[0]))},n.string=function(i){return typeof i=="string"||i instanceof String},n.fn=function(i){var a=Object.prototype.toString.call(i);return a==="[object Function]"}},370:function(o,n,i){var a=i(879),s=i(438);function p(u,d,y){if(!u&&!d&&!y)throw new Error("Missing required arguments");if(!a.string(d))throw new TypeError("Second argument must be a String");if(!a.fn(y))throw new TypeError("Third argument must be a Function");if(a.node(u))return c(u,d,y);if(a.nodeList(u))return l(u,d,y);if(a.string(u))return f(u,d,y);throw new TypeError("First argument must be a String, HTMLElement, HTMLCollection, or NodeList")}function c(u,d,y){return u.addEventListener(d,y),{destroy:function(){u.removeEventListener(d,y)}}}function l(u,d,y){return Array.prototype.forEach.call(u,function(L){L.addEventListener(d,y)}),{destroy:function(){Array.prototype.forEach.call(u,function(L){L.removeEventListener(d,y)})}}}function f(u,d,y){return s(document.body,u,d,y)}o.exports=p},817:function(o){function n(i){var a;if(i.nodeName==="SELECT")i.focus(),a=i.value;else if(i.nodeName==="INPUT"||i.nodeName==="TEXTAREA"){var s=i.hasAttribute("readonly");s||i.setAttribute("readonly",""),i.select(),i.setSelectionRange(0,i.value.length),s||i.removeAttribute("readonly"),a=i.value}else{i.hasAttribute("contenteditable")&&i.focus();var p=window.getSelection(),c=document.createRange();c.selectNodeContents(i),p.removeAllRanges(),p.addRange(c),a=p.toString()}return a}o.exports=n},279:function(o){function n(){}n.prototype={on:function(i,a,s){var p=this.e||(this.e={});return(p[i]||(p[i]=[])).push({fn:a,ctx:s}),this},once:function(i,a,s){var p=this;function c(){p.off(i,c),a.apply(s,arguments)}return c._=a,this.on(i,c,s)},emit:function(i){var a=[].slice.call(arguments,1),s=((this.e||(this.e={}))[i]||[]).slice(),p=0,c=s.length;for(p;p0&&i[i.length-1])&&(c[0]===6||c[0]===2)){r=0;continue}if(c[0]===3&&(!i||c[1]>i[0]&&c[1]=e.length&&(e=void 0),{value:e&&e[o++],done:!e}}};throw new TypeError(t?"Object is not iterable.":"Symbol.iterator is not defined.")}function N(e,t){var r=typeof Symbol=="function"&&e[Symbol.iterator];if(!r)return e;var o=r.call(e),n,i=[],a;try{for(;(t===void 0||t-- >0)&&!(n=o.next()).done;)i.push(n.value)}catch(s){a={error:s}}finally{try{n&&!n.done&&(r=o.return)&&r.call(o)}finally{if(a)throw a.error}}return i}function q(e,t,r){if(r||arguments.length===2)for(var o=0,n=t.length,i;o1||p(d,L)})},y&&(n[d]=y(n[d])))}function p(d,y){try{c(o[d](y))}catch(L){u(i[0][3],L)}}function c(d){d.value instanceof nt?Promise.resolve(d.value.v).then(l,f):u(i[0][2],d)}function l(d){p("next",d)}function f(d){p("throw",d)}function u(d,y){d(y),i.shift(),i.length&&p(i[0][0],i[0][1])}}function uo(e){if(!Symbol.asyncIterator)throw new TypeError("Symbol.asyncIterator is not defined.");var t=e[Symbol.asyncIterator],r;return t?t.call(e):(e=typeof he=="function"?he(e):e[Symbol.iterator](),r={},o("next"),o("throw"),o("return"),r[Symbol.asyncIterator]=function(){return this},r);function o(i){r[i]=e[i]&&function(a){return new Promise(function(s,p){a=e[i](a),n(s,p,a.done,a.value)})}}function n(i,a,s,p){Promise.resolve(p).then(function(c){i({value:c,done:s})},a)}}function H(e){return typeof e=="function"}function ut(e){var t=function(o){Error.call(o),o.stack=new Error().stack},r=e(t);return r.prototype=Object.create(Error.prototype),r.prototype.constructor=r,r}var zt=ut(function(e){return function(r){e(this),this.message=r?r.length+` errors occurred during unsubscription: +`+r.map(function(o,n){return n+1+") "+o.toString()}).join(` + `):"",this.name="UnsubscriptionError",this.errors=r}});function Qe(e,t){if(e){var r=e.indexOf(t);0<=r&&e.splice(r,1)}}var Ue=function(){function e(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}return e.prototype.unsubscribe=function(){var t,r,o,n,i;if(!this.closed){this.closed=!0;var a=this._parentage;if(a)if(this._parentage=null,Array.isArray(a))try{for(var s=he(a),p=s.next();!p.done;p=s.next()){var c=p.value;c.remove(this)}}catch(L){t={error:L}}finally{try{p&&!p.done&&(r=s.return)&&r.call(s)}finally{if(t)throw t.error}}else a.remove(this);var l=this.initialTeardown;if(H(l))try{l()}catch(L){i=L instanceof zt?L.errors:[L]}var f=this._finalizers;if(f){this._finalizers=null;try{for(var u=he(f),d=u.next();!d.done;d=u.next()){var y=d.value;try{ho(y)}catch(L){i=i!=null?i:[],L instanceof zt?i=q(q([],N(i)),N(L.errors)):i.push(L)}}}catch(L){o={error:L}}finally{try{d&&!d.done&&(n=u.return)&&n.call(u)}finally{if(o)throw o.error}}}if(i)throw new zt(i)}},e.prototype.add=function(t){var r;if(t&&t!==this)if(this.closed)ho(t);else{if(t instanceof e){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(r=this._finalizers)!==null&&r!==void 0?r:[]).push(t)}},e.prototype._hasParent=function(t){var r=this._parentage;return r===t||Array.isArray(r)&&r.includes(t)},e.prototype._addParent=function(t){var r=this._parentage;this._parentage=Array.isArray(r)?(r.push(t),r):r?[r,t]:t},e.prototype._removeParent=function(t){var r=this._parentage;r===t?this._parentage=null:Array.isArray(r)&&Qe(r,t)},e.prototype.remove=function(t){var r=this._finalizers;r&&Qe(r,t),t instanceof e&&t._removeParent(this)},e.EMPTY=function(){var t=new e;return t.closed=!0,t}(),e}();var Tr=Ue.EMPTY;function qt(e){return e instanceof Ue||e&&"closed"in e&&H(e.remove)&&H(e.add)&&H(e.unsubscribe)}function ho(e){H(e)?e():e.unsubscribe()}var Pe={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var dt={setTimeout:function(e,t){for(var r=[],o=2;o0},enumerable:!1,configurable:!0}),t.prototype._trySubscribe=function(r){return this._throwIfClosed(),e.prototype._trySubscribe.call(this,r)},t.prototype._subscribe=function(r){return this._throwIfClosed(),this._checkFinalizedStatuses(r),this._innerSubscribe(r)},t.prototype._innerSubscribe=function(r){var o=this,n=this,i=n.hasError,a=n.isStopped,s=n.observers;return i||a?Tr:(this.currentObservers=null,s.push(r),new Ue(function(){o.currentObservers=null,Qe(s,r)}))},t.prototype._checkFinalizedStatuses=function(r){var o=this,n=o.hasError,i=o.thrownError,a=o.isStopped;n?r.error(i):a&&r.complete()},t.prototype.asObservable=function(){var r=new j;return r.source=this,r},t.create=function(r,o){return new To(r,o)},t}(j);var To=function(e){oe(t,e);function t(r,o){var n=e.call(this)||this;return n.destination=r,n.source=o,n}return t.prototype.next=function(r){var o,n;(n=(o=this.destination)===null||o===void 0?void 0:o.next)===null||n===void 0||n.call(o,r)},t.prototype.error=function(r){var o,n;(n=(o=this.destination)===null||o===void 0?void 0:o.error)===null||n===void 0||n.call(o,r)},t.prototype.complete=function(){var r,o;(o=(r=this.destination)===null||r===void 0?void 0:r.complete)===null||o===void 0||o.call(r)},t.prototype._subscribe=function(r){var o,n;return(n=(o=this.source)===null||o===void 0?void 0:o.subscribe(r))!==null&&n!==void 0?n:Tr},t}(g);var _r=function(e){oe(t,e);function t(r){var o=e.call(this)||this;return o._value=r,o}return Object.defineProperty(t.prototype,"value",{get:function(){return this.getValue()},enumerable:!1,configurable:!0}),t.prototype._subscribe=function(r){var o=e.prototype._subscribe.call(this,r);return!o.closed&&r.next(this._value),o},t.prototype.getValue=function(){var r=this,o=r.hasError,n=r.thrownError,i=r._value;if(o)throw n;return this._throwIfClosed(),i},t.prototype.next=function(r){e.prototype.next.call(this,this._value=r)},t}(g);var At={now:function(){return(At.delegate||Date).now()},delegate:void 0};var Ct=function(e){oe(t,e);function t(r,o,n){r===void 0&&(r=1/0),o===void 0&&(o=1/0),n===void 0&&(n=At);var i=e.call(this)||this;return i._bufferSize=r,i._windowTime=o,i._timestampProvider=n,i._buffer=[],i._infiniteTimeWindow=!0,i._infiniteTimeWindow=o===1/0,i._bufferSize=Math.max(1,r),i._windowTime=Math.max(1,o),i}return t.prototype.next=function(r){var o=this,n=o.isStopped,i=o._buffer,a=o._infiniteTimeWindow,s=o._timestampProvider,p=o._windowTime;n||(i.push(r),!a&&i.push(s.now()+p)),this._trimBuffer(),e.prototype.next.call(this,r)},t.prototype._subscribe=function(r){this._throwIfClosed(),this._trimBuffer();for(var o=this._innerSubscribe(r),n=this,i=n._infiniteTimeWindow,a=n._buffer,s=a.slice(),p=0;p0?e.prototype.schedule.call(this,r,o):(this.delay=o,this.state=r,this.scheduler.flush(this),this)},t.prototype.execute=function(r,o){return o>0||this.closed?e.prototype.execute.call(this,r,o):this._execute(r,o)},t.prototype.requestAsyncId=function(r,o,n){return n===void 0&&(n=0),n!=null&&n>0||n==null&&this.delay>0?e.prototype.requestAsyncId.call(this,r,o,n):(r.flush(this),0)},t}(gt);var Lo=function(e){oe(t,e);function t(){return e!==null&&e.apply(this,arguments)||this}return t}(yt);var kr=new Lo(Oo);var Mo=function(e){oe(t,e);function t(r,o){var n=e.call(this,r,o)||this;return n.scheduler=r,n.work=o,n}return t.prototype.requestAsyncId=function(r,o,n){return n===void 0&&(n=0),n!==null&&n>0?e.prototype.requestAsyncId.call(this,r,o,n):(r.actions.push(this),r._scheduled||(r._scheduled=vt.requestAnimationFrame(function(){return r.flush(void 0)})))},t.prototype.recycleAsyncId=function(r,o,n){var i;if(n===void 0&&(n=0),n!=null?n>0:this.delay>0)return e.prototype.recycleAsyncId.call(this,r,o,n);var a=r.actions;o!=null&&((i=a[a.length-1])===null||i===void 0?void 0:i.id)!==o&&(vt.cancelAnimationFrame(o),r._scheduled=void 0)},t}(gt);var _o=function(e){oe(t,e);function t(){return e!==null&&e.apply(this,arguments)||this}return t.prototype.flush=function(r){this._active=!0;var o=this._scheduled;this._scheduled=void 0;var n=this.actions,i;r=r||n.shift();do if(i=r.execute(r.state,r.delay))break;while((r=n[0])&&r.id===o&&n.shift());if(this._active=!1,i){for(;(r=n[0])&&r.id===o&&n.shift();)r.unsubscribe();throw i}},t}(yt);var me=new _o(Mo);var S=new j(function(e){return e.complete()});function Yt(e){return e&&H(e.schedule)}function Hr(e){return e[e.length-1]}function Xe(e){return H(Hr(e))?e.pop():void 0}function ke(e){return Yt(Hr(e))?e.pop():void 0}function Bt(e,t){return typeof Hr(e)=="number"?e.pop():t}var xt=function(e){return e&&typeof e.length=="number"&&typeof e!="function"};function Gt(e){return H(e==null?void 0:e.then)}function Jt(e){return H(e[bt])}function Xt(e){return Symbol.asyncIterator&&H(e==null?void 0:e[Symbol.asyncIterator])}function Zt(e){return new TypeError("You provided "+(e!==null&&typeof e=="object"?"an invalid object":"'"+e+"'")+" where a stream was expected. You can provide an Observable, Promise, ReadableStream, Array, AsyncIterable, or Iterable.")}function Zi(){return typeof Symbol!="function"||!Symbol.iterator?"@@iterator":Symbol.iterator}var er=Zi();function tr(e){return H(e==null?void 0:e[er])}function rr(e){return fo(this,arguments,function(){var r,o,n,i;return Nt(this,function(a){switch(a.label){case 0:r=e.getReader(),a.label=1;case 1:a.trys.push([1,,9,10]),a.label=2;case 2:return[4,nt(r.read())];case 3:return o=a.sent(),n=o.value,i=o.done,i?[4,nt(void 0)]:[3,5];case 4:return[2,a.sent()];case 5:return[4,nt(n)];case 6:return[4,a.sent()];case 7:return a.sent(),[3,2];case 8:return[3,10];case 9:return r.releaseLock(),[7];case 10:return[2]}})})}function or(e){return H(e==null?void 0:e.getReader)}function U(e){if(e instanceof j)return e;if(e!=null){if(Jt(e))return ea(e);if(xt(e))return ta(e);if(Gt(e))return ra(e);if(Xt(e))return Ao(e);if(tr(e))return oa(e);if(or(e))return na(e)}throw Zt(e)}function ea(e){return new j(function(t){var r=e[bt]();if(H(r.subscribe))return r.subscribe(t);throw new TypeError("Provided object does not correctly implement Symbol.observable")})}function ta(e){return new j(function(t){for(var r=0;r=2;return function(o){return o.pipe(e?b(function(n,i){return e(n,i,o)}):le,Te(1),r?De(t):Qo(function(){return new ir}))}}function jr(e){return e<=0?function(){return S}:E(function(t,r){var o=[];t.subscribe(T(r,function(n){o.push(n),e=2,!0))}function pe(e){e===void 0&&(e={});var t=e.connector,r=t===void 0?function(){return new g}:t,o=e.resetOnError,n=o===void 0?!0:o,i=e.resetOnComplete,a=i===void 0?!0:i,s=e.resetOnRefCountZero,p=s===void 0?!0:s;return function(c){var l,f,u,d=0,y=!1,L=!1,X=function(){f==null||f.unsubscribe(),f=void 0},te=function(){X(),l=u=void 0,y=L=!1},J=function(){var k=l;te(),k==null||k.unsubscribe()};return E(function(k,ft){d++,!L&&!y&&X();var qe=u=u!=null?u:r();ft.add(function(){d--,d===0&&!L&&!y&&(f=Ur(J,p))}),qe.subscribe(ft),!l&&d>0&&(l=new at({next:function(Fe){return qe.next(Fe)},error:function(Fe){L=!0,X(),f=Ur(te,n,Fe),qe.error(Fe)},complete:function(){y=!0,X(),f=Ur(te,a),qe.complete()}}),U(k).subscribe(l))})(c)}}function Ur(e,t){for(var r=[],o=2;oe.next(document)),e}function P(e,t=document){return Array.from(t.querySelectorAll(e))}function R(e,t=document){let r=fe(e,t);if(typeof r=="undefined")throw new ReferenceError(`Missing element: expected "${e}" to be present`);return r}function fe(e,t=document){return t.querySelector(e)||void 0}function Ie(){var e,t,r,o;return(o=(r=(t=(e=document.activeElement)==null?void 0:e.shadowRoot)==null?void 0:t.activeElement)!=null?r:document.activeElement)!=null?o:void 0}var wa=O(h(document.body,"focusin"),h(document.body,"focusout")).pipe(_e(1),Q(void 0),m(()=>Ie()||document.body),G(1));function et(e){return wa.pipe(m(t=>e.contains(t)),K())}function $t(e,t){return C(()=>O(h(e,"mouseenter").pipe(m(()=>!0)),h(e,"mouseleave").pipe(m(()=>!1))).pipe(t?Ht(r=>Le(+!r*t)):le,Q(e.matches(":hover"))))}function Jo(e,t){if(typeof t=="string"||typeof t=="number")e.innerHTML+=t.toString();else if(t instanceof Node)e.appendChild(t);else if(Array.isArray(t))for(let r of t)Jo(e,r)}function x(e,t,...r){let o=document.createElement(e);if(t)for(let n of Object.keys(t))typeof t[n]!="undefined"&&(typeof t[n]!="boolean"?o.setAttribute(n,t[n]):o.setAttribute(n,""));for(let n of r)Jo(o,n);return o}function sr(e){if(e>999){let t=+((e-950)%1e3>99);return`${((e+1e-6)/1e3).toFixed(t)}k`}else return e.toString()}function Tt(e){let t=x("script",{src:e});return C(()=>(document.head.appendChild(t),O(h(t,"load"),h(t,"error").pipe(v(()=>$r(()=>new ReferenceError(`Invalid script: ${e}`))))).pipe(m(()=>{}),_(()=>document.head.removeChild(t)),Te(1))))}var Xo=new g,Ta=C(()=>typeof ResizeObserver=="undefined"?Tt("https://unpkg.com/resize-observer-polyfill"):I(void 0)).pipe(m(()=>new ResizeObserver(e=>e.forEach(t=>Xo.next(t)))),v(e=>O(Ye,I(e)).pipe(_(()=>e.disconnect()))),G(1));function ce(e){return{width:e.offsetWidth,height:e.offsetHeight}}function ge(e){let t=e;for(;t.clientWidth===0&&t.parentElement;)t=t.parentElement;return Ta.pipe(w(r=>r.observe(t)),v(r=>Xo.pipe(b(o=>o.target===t),_(()=>r.unobserve(t)))),m(()=>ce(e)),Q(ce(e)))}function St(e){return{width:e.scrollWidth,height:e.scrollHeight}}function cr(e){let t=e.parentElement;for(;t&&(e.scrollWidth<=t.scrollWidth&&e.scrollHeight<=t.scrollHeight);)t=(e=t).parentElement;return t?e:void 0}function Zo(e){let t=[],r=e.parentElement;for(;r;)(e.clientWidth>r.clientWidth||e.clientHeight>r.clientHeight)&&t.push(r),r=(e=r).parentElement;return t.length===0&&t.push(document.documentElement),t}function Ve(e){return{x:e.offsetLeft,y:e.offsetTop}}function en(e){let t=e.getBoundingClientRect();return{x:t.x+window.scrollX,y:t.y+window.scrollY}}function tn(e){return O(h(window,"load"),h(window,"resize")).pipe(Me(0,me),m(()=>Ve(e)),Q(Ve(e)))}function pr(e){return{x:e.scrollLeft,y:e.scrollTop}}function Ne(e){return O(h(e,"scroll"),h(window,"scroll"),h(window,"resize")).pipe(Me(0,me),m(()=>pr(e)),Q(pr(e)))}var rn=new g,Sa=C(()=>I(new IntersectionObserver(e=>{for(let t of e)rn.next(t)},{threshold:0}))).pipe(v(e=>O(Ye,I(e)).pipe(_(()=>e.disconnect()))),G(1));function tt(e){return Sa.pipe(w(t=>t.observe(e)),v(t=>rn.pipe(b(({target:r})=>r===e),_(()=>t.unobserve(e)),m(({isIntersecting:r})=>r))))}function on(e,t=16){return Ne(e).pipe(m(({y:r})=>{let o=ce(e),n=St(e);return r>=n.height-o.height-t}),K())}var lr={drawer:R("[data-md-toggle=drawer]"),search:R("[data-md-toggle=search]")};function nn(e){return lr[e].checked}function Je(e,t){lr[e].checked!==t&&lr[e].click()}function ze(e){let t=lr[e];return h(t,"change").pipe(m(()=>t.checked),Q(t.checked))}function Oa(e,t){switch(e.constructor){case HTMLInputElement:return e.type==="radio"?/^Arrow/.test(t):!0;case HTMLSelectElement:case HTMLTextAreaElement:return!0;default:return e.isContentEditable}}function La(){return O(h(window,"compositionstart").pipe(m(()=>!0)),h(window,"compositionend").pipe(m(()=>!1))).pipe(Q(!1))}function an(){let e=h(window,"keydown").pipe(b(t=>!(t.metaKey||t.ctrlKey)),m(t=>({mode:nn("search")?"search":"global",type:t.key,claim(){t.preventDefault(),t.stopPropagation()}})),b(({mode:t,type:r})=>{if(t==="global"){let o=Ie();if(typeof o!="undefined")return!Oa(o,r)}return!0}),pe());return La().pipe(v(t=>t?S:e))}function ye(){return new URL(location.href)}function lt(e,t=!1){if(B("navigation.instant")&&!t){let r=x("a",{href:e.href});document.body.appendChild(r),r.click(),r.remove()}else location.href=e.href}function sn(){return new g}function cn(){return location.hash.slice(1)}function pn(e){let t=x("a",{href:e});t.addEventListener("click",r=>r.stopPropagation()),t.click()}function Ma(e){return O(h(window,"hashchange"),e).pipe(m(cn),Q(cn()),b(t=>t.length>0),G(1))}function ln(e){return Ma(e).pipe(m(t=>fe(`[id="${t}"]`)),b(t=>typeof t!="undefined"))}function Pt(e){let t=matchMedia(e);return ar(r=>t.addListener(()=>r(t.matches))).pipe(Q(t.matches))}function mn(){let e=matchMedia("print");return O(h(window,"beforeprint").pipe(m(()=>!0)),h(window,"afterprint").pipe(m(()=>!1))).pipe(Q(e.matches))}function Nr(e,t){return e.pipe(v(r=>r?t():S))}function zr(e,t){return new j(r=>{let o=new XMLHttpRequest;return o.open("GET",`${e}`),o.responseType="blob",o.addEventListener("load",()=>{o.status>=200&&o.status<300?(r.next(o.response),r.complete()):r.error(new Error(o.statusText))}),o.addEventListener("error",()=>{r.error(new Error("Network error"))}),o.addEventListener("abort",()=>{r.complete()}),typeof(t==null?void 0:t.progress$)!="undefined"&&(o.addEventListener("progress",n=>{var i;if(n.lengthComputable)t.progress$.next(n.loaded/n.total*100);else{let a=(i=o.getResponseHeader("Content-Length"))!=null?i:0;t.progress$.next(n.loaded/+a*100)}}),t.progress$.next(5)),o.send(),()=>o.abort()})}function je(e,t){return zr(e,t).pipe(v(r=>r.text()),m(r=>JSON.parse(r)),G(1))}function fn(e,t){let r=new DOMParser;return zr(e,t).pipe(v(o=>o.text()),m(o=>r.parseFromString(o,"text/html")),G(1))}function un(e,t){let r=new DOMParser;return zr(e,t).pipe(v(o=>o.text()),m(o=>r.parseFromString(o,"text/xml")),G(1))}function dn(){return{x:Math.max(0,scrollX),y:Math.max(0,scrollY)}}function hn(){return O(h(window,"scroll",{passive:!0}),h(window,"resize",{passive:!0})).pipe(m(dn),Q(dn()))}function bn(){return{width:innerWidth,height:innerHeight}}function vn(){return h(window,"resize",{passive:!0}).pipe(m(bn),Q(bn()))}function gn(){return z([hn(),vn()]).pipe(m(([e,t])=>({offset:e,size:t})),G(1))}function mr(e,{viewport$:t,header$:r}){let o=t.pipe(ee("size")),n=z([o,r]).pipe(m(()=>Ve(e)));return z([r,t,n]).pipe(m(([{height:i},{offset:a,size:s},{x:p,y:c}])=>({offset:{x:a.x-p,y:a.y-c+i},size:s})))}function _a(e){return h(e,"message",t=>t.data)}function Aa(e){let t=new g;return t.subscribe(r=>e.postMessage(r)),t}function yn(e,t=new Worker(e)){let r=_a(t),o=Aa(t),n=new g;n.subscribe(o);let i=o.pipe(Z(),ie(!0));return n.pipe(Z(),Re(r.pipe(W(i))),pe())}var Ca=R("#__config"),Ot=JSON.parse(Ca.textContent);Ot.base=`${new URL(Ot.base,ye())}`;function xe(){return Ot}function B(e){return Ot.features.includes(e)}function Ee(e,t){return typeof t!="undefined"?Ot.translations[e].replace("#",t.toString()):Ot.translations[e]}function Se(e,t=document){return R(`[data-md-component=${e}]`,t)}function ae(e,t=document){return P(`[data-md-component=${e}]`,t)}function ka(e){let t=R(".md-typeset > :first-child",e);return h(t,"click",{once:!0}).pipe(m(()=>R(".md-typeset",e)),m(r=>({hash:__md_hash(r.innerHTML)})))}function xn(e){if(!B("announce.dismiss")||!e.childElementCount)return S;if(!e.hidden){let t=R(".md-typeset",e);__md_hash(t.innerHTML)===__md_get("__announce")&&(e.hidden=!0)}return C(()=>{let t=new g;return t.subscribe(({hash:r})=>{e.hidden=!0,__md_set("__announce",r)}),ka(e).pipe(w(r=>t.next(r)),_(()=>t.complete()),m(r=>$({ref:e},r)))})}function Ha(e,{target$:t}){return t.pipe(m(r=>({hidden:r!==e})))}function En(e,t){let r=new g;return r.subscribe(({hidden:o})=>{e.hidden=o}),Ha(e,t).pipe(w(o=>r.next(o)),_(()=>r.complete()),m(o=>$({ref:e},o)))}function Rt(e,t){return t==="inline"?x("div",{class:"md-tooltip md-tooltip--inline",id:e,role:"tooltip"},x("div",{class:"md-tooltip__inner md-typeset"})):x("div",{class:"md-tooltip",id:e,role:"tooltip"},x("div",{class:"md-tooltip__inner md-typeset"}))}function wn(...e){return x("div",{class:"md-tooltip2",role:"tooltip"},x("div",{class:"md-tooltip2__inner md-typeset"},e))}function Tn(e,t){if(t=t?`${t}_annotation_${e}`:void 0,t){let r=t?`#${t}`:void 0;return x("aside",{class:"md-annotation",tabIndex:0},Rt(t),x("a",{href:r,class:"md-annotation__index",tabIndex:-1},x("span",{"data-md-annotation-id":e})))}else return x("aside",{class:"md-annotation",tabIndex:0},Rt(t),x("span",{class:"md-annotation__index",tabIndex:-1},x("span",{"data-md-annotation-id":e})))}function Sn(e){return x("button",{class:"md-clipboard md-icon",title:Ee("clipboard.copy"),"data-clipboard-target":`#${e} > code`})}var Ln=Mt(qr());function Qr(e,t){let r=t&2,o=t&1,n=Object.keys(e.terms).filter(p=>!e.terms[p]).reduce((p,c)=>[...p,x("del",null,(0,Ln.default)(c))," "],[]).slice(0,-1),i=xe(),a=new URL(e.location,i.base);B("search.highlight")&&a.searchParams.set("h",Object.entries(e.terms).filter(([,p])=>p).reduce((p,[c])=>`${p} ${c}`.trim(),""));let{tags:s}=xe();return x("a",{href:`${a}`,class:"md-search-result__link",tabIndex:-1},x("article",{class:"md-search-result__article md-typeset","data-md-score":e.score.toFixed(2)},r>0&&x("div",{class:"md-search-result__icon md-icon"}),r>0&&x("h1",null,e.title),r<=0&&x("h2",null,e.title),o>0&&e.text.length>0&&e.text,e.tags&&x("nav",{class:"md-tags"},e.tags.map(p=>{let c=s?p in s?`md-tag-icon md-tag--${s[p]}`:"md-tag-icon":"";return x("span",{class:`md-tag ${c}`},p)})),o>0&&n.length>0&&x("p",{class:"md-search-result__terms"},Ee("search.result.term.missing"),": ",...n)))}function Mn(e){let t=e[0].score,r=[...e],o=xe(),n=r.findIndex(l=>!`${new URL(l.location,o.base)}`.includes("#")),[i]=r.splice(n,1),a=r.findIndex(l=>l.scoreQr(l,1)),...p.length?[x("details",{class:"md-search-result__more"},x("summary",{tabIndex:-1},x("div",null,p.length>0&&p.length===1?Ee("search.result.more.one"):Ee("search.result.more.other",p.length))),...p.map(l=>Qr(l,1)))]:[]];return x("li",{class:"md-search-result__item"},c)}function _n(e){return x("ul",{class:"md-source__facts"},Object.entries(e).map(([t,r])=>x("li",{class:`md-source__fact md-source__fact--${t}`},typeof r=="number"?sr(r):r)))}function Kr(e){let t=`tabbed-control tabbed-control--${e}`;return x("div",{class:t,hidden:!0},x("button",{class:"tabbed-button",tabIndex:-1,"aria-hidden":"true"}))}function An(e){return x("div",{class:"md-typeset__scrollwrap"},x("div",{class:"md-typeset__table"},e))}function Ra(e){var o;let t=xe(),r=new URL(`../${e.version}/`,t.base);return x("li",{class:"md-version__item"},x("a",{href:`${r}`,class:"md-version__link"},e.title,((o=t.version)==null?void 0:o.alias)&&e.aliases.length>0&&x("span",{class:"md-version__alias"},e.aliases[0])))}function Cn(e,t){var o;let r=xe();return e=e.filter(n=>{var i;return!((i=n.properties)!=null&&i.hidden)}),x("div",{class:"md-version"},x("button",{class:"md-version__current","aria-label":Ee("select.version")},t.title,((o=r.version)==null?void 0:o.alias)&&t.aliases.length>0&&x("span",{class:"md-version__alias"},t.aliases[0])),x("ul",{class:"md-version__list"},e.map(Ra)))}var Ia=0;function ja(e){let t=z([et(e),$t(e)]).pipe(m(([o,n])=>o||n),K()),r=C(()=>Zo(e)).pipe(ne(Ne),pt(1),He(t),m(()=>en(e)));return t.pipe(Ae(o=>o),v(()=>z([t,r])),m(([o,n])=>({active:o,offset:n})),pe())}function Fa(e,t){let{content$:r,viewport$:o}=t,n=`__tooltip2_${Ia++}`;return C(()=>{let i=new g,a=new _r(!1);i.pipe(Z(),ie(!1)).subscribe(a);let s=a.pipe(Ht(c=>Le(+!c*250,kr)),K(),v(c=>c?r:S),w(c=>c.id=n),pe());z([i.pipe(m(({active:c})=>c)),s.pipe(v(c=>$t(c,250)),Q(!1))]).pipe(m(c=>c.some(l=>l))).subscribe(a);let p=a.pipe(b(c=>c),re(s,o),m(([c,l,{size:f}])=>{let u=e.getBoundingClientRect(),d=u.width/2;if(l.role==="tooltip")return{x:d,y:8+u.height};if(u.y>=f.height/2){let{height:y}=ce(l);return{x:d,y:-16-y}}else return{x:d,y:16+u.height}}));return z([s,i,p]).subscribe(([c,{offset:l},f])=>{c.style.setProperty("--md-tooltip-host-x",`${l.x}px`),c.style.setProperty("--md-tooltip-host-y",`${l.y}px`),c.style.setProperty("--md-tooltip-x",`${f.x}px`),c.style.setProperty("--md-tooltip-y",`${f.y}px`),c.classList.toggle("md-tooltip2--top",f.y<0),c.classList.toggle("md-tooltip2--bottom",f.y>=0)}),a.pipe(b(c=>c),re(s,(c,l)=>l),b(c=>c.role==="tooltip")).subscribe(c=>{let l=ce(R(":scope > *",c));c.style.setProperty("--md-tooltip-width",`${l.width}px`),c.style.setProperty("--md-tooltip-tail","0px")}),a.pipe(K(),ve(me),re(s)).subscribe(([c,l])=>{l.classList.toggle("md-tooltip2--active",c)}),z([a.pipe(b(c=>c)),s]).subscribe(([c,l])=>{l.role==="dialog"?(e.setAttribute("aria-controls",n),e.setAttribute("aria-haspopup","dialog")):e.setAttribute("aria-describedby",n)}),a.pipe(b(c=>!c)).subscribe(()=>{e.removeAttribute("aria-controls"),e.removeAttribute("aria-describedby"),e.removeAttribute("aria-haspopup")}),ja(e).pipe(w(c=>i.next(c)),_(()=>i.complete()),m(c=>$({ref:e},c)))})}function mt(e,{viewport$:t},r=document.body){return Fa(e,{content$:new j(o=>{let n=e.title,i=wn(n);return o.next(i),e.removeAttribute("title"),r.append(i),()=>{i.remove(),e.setAttribute("title",n)}}),viewport$:t})}function Ua(e,t){let r=C(()=>z([tn(e),Ne(t)])).pipe(m(([{x:o,y:n},i])=>{let{width:a,height:s}=ce(e);return{x:o-i.x+a/2,y:n-i.y+s/2}}));return et(e).pipe(v(o=>r.pipe(m(n=>({active:o,offset:n})),Te(+!o||1/0))))}function kn(e,t,{target$:r}){let[o,n]=Array.from(e.children);return C(()=>{let i=new g,a=i.pipe(Z(),ie(!0));return i.subscribe({next({offset:s}){e.style.setProperty("--md-tooltip-x",`${s.x}px`),e.style.setProperty("--md-tooltip-y",`${s.y}px`)},complete(){e.style.removeProperty("--md-tooltip-x"),e.style.removeProperty("--md-tooltip-y")}}),tt(e).pipe(W(a)).subscribe(s=>{e.toggleAttribute("data-md-visible",s)}),O(i.pipe(b(({active:s})=>s)),i.pipe(_e(250),b(({active:s})=>!s))).subscribe({next({active:s}){s?e.prepend(o):o.remove()},complete(){e.prepend(o)}}),i.pipe(Me(16,me)).subscribe(({active:s})=>{o.classList.toggle("md-tooltip--active",s)}),i.pipe(pt(125,me),b(()=>!!e.offsetParent),m(()=>e.offsetParent.getBoundingClientRect()),m(({x:s})=>s)).subscribe({next(s){s?e.style.setProperty("--md-tooltip-0",`${-s}px`):e.style.removeProperty("--md-tooltip-0")},complete(){e.style.removeProperty("--md-tooltip-0")}}),h(n,"click").pipe(W(a),b(s=>!(s.metaKey||s.ctrlKey))).subscribe(s=>{s.stopPropagation(),s.preventDefault()}),h(n,"mousedown").pipe(W(a),re(i)).subscribe(([s,{active:p}])=>{var c;if(s.button!==0||s.metaKey||s.ctrlKey)s.preventDefault();else if(p){s.preventDefault();let l=e.parentElement.closest(".md-annotation");l instanceof HTMLElement?l.focus():(c=Ie())==null||c.blur()}}),r.pipe(W(a),b(s=>s===o),Ge(125)).subscribe(()=>e.focus()),Ua(e,t).pipe(w(s=>i.next(s)),_(()=>i.complete()),m(s=>$({ref:e},s)))})}function Wa(e){return e.tagName==="CODE"?P(".c, .c1, .cm",e):[e]}function Da(e){let t=[];for(let r of Wa(e)){let o=[],n=document.createNodeIterator(r,NodeFilter.SHOW_TEXT);for(let i=n.nextNode();i;i=n.nextNode())o.push(i);for(let i of o){let a;for(;a=/(\(\d+\))(!)?/.exec(i.textContent);){let[,s,p]=a;if(typeof p=="undefined"){let c=i.splitText(a.index);i=c.splitText(s.length),t.push(c)}else{i.textContent=s,t.push(i);break}}}}return t}function Hn(e,t){t.append(...Array.from(e.childNodes))}function fr(e,t,{target$:r,print$:o}){let n=t.closest("[id]"),i=n==null?void 0:n.id,a=new Map;for(let s of Da(t)){let[,p]=s.textContent.match(/\((\d+)\)/);fe(`:scope > li:nth-child(${p})`,e)&&(a.set(p,Tn(p,i)),s.replaceWith(a.get(p)))}return a.size===0?S:C(()=>{let s=new g,p=s.pipe(Z(),ie(!0)),c=[];for(let[l,f]of a)c.push([R(".md-typeset",f),R(`:scope > li:nth-child(${l})`,e)]);return o.pipe(W(p)).subscribe(l=>{e.hidden=!l,e.classList.toggle("md-annotation-list",l);for(let[f,u]of c)l?Hn(f,u):Hn(u,f)}),O(...[...a].map(([,l])=>kn(l,t,{target$:r}))).pipe(_(()=>s.complete()),pe())})}function $n(e){if(e.nextElementSibling){let t=e.nextElementSibling;if(t.tagName==="OL")return t;if(t.tagName==="P"&&!t.children.length)return $n(t)}}function Pn(e,t){return C(()=>{let r=$n(e);return typeof r!="undefined"?fr(r,e,t):S})}var Rn=Mt(Br());var Va=0;function In(e){if(e.nextElementSibling){let t=e.nextElementSibling;if(t.tagName==="OL")return t;if(t.tagName==="P"&&!t.children.length)return In(t)}}function Na(e){return ge(e).pipe(m(({width:t})=>({scrollable:St(e).width>t})),ee("scrollable"))}function jn(e,t){let{matches:r}=matchMedia("(hover)"),o=C(()=>{let n=new g,i=n.pipe(jr(1));n.subscribe(({scrollable:c})=>{c&&r?e.setAttribute("tabindex","0"):e.removeAttribute("tabindex")});let a=[];if(Rn.default.isSupported()&&(e.closest(".copy")||B("content.code.copy")&&!e.closest(".no-copy"))){let c=e.closest("pre");c.id=`__code_${Va++}`;let l=Sn(c.id);c.insertBefore(l,e),B("content.tooltips")&&a.push(mt(l,{viewport$}))}let s=e.closest(".highlight");if(s instanceof HTMLElement){let c=In(s);if(typeof c!="undefined"&&(s.classList.contains("annotate")||B("content.code.annotate"))){let l=fr(c,e,t);a.push(ge(s).pipe(W(i),m(({width:f,height:u})=>f&&u),K(),v(f=>f?l:S)))}}return P(":scope > span[id]",e).length&&e.classList.add("md-code__content"),Na(e).pipe(w(c=>n.next(c)),_(()=>n.complete()),m(c=>$({ref:e},c)),Re(...a))});return B("content.lazy")?tt(e).pipe(b(n=>n),Te(1),v(()=>o)):o}function za(e,{target$:t,print$:r}){let o=!0;return O(t.pipe(m(n=>n.closest("details:not([open])")),b(n=>e===n),m(()=>({action:"open",reveal:!0}))),r.pipe(b(n=>n||!o),w(()=>o=e.open),m(n=>({action:n?"open":"close"}))))}function Fn(e,t){return C(()=>{let r=new g;return r.subscribe(({action:o,reveal:n})=>{e.toggleAttribute("open",o==="open"),n&&e.scrollIntoView()}),za(e,t).pipe(w(o=>r.next(o)),_(()=>r.complete()),m(o=>$({ref:e},o)))})}var Un=".node circle,.node ellipse,.node path,.node polygon,.node rect{fill:var(--md-mermaid-node-bg-color);stroke:var(--md-mermaid-node-fg-color)}marker{fill:var(--md-mermaid-edge-color)!important}.edgeLabel .label rect{fill:#0000}.label{color:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}.label foreignObject{line-height:normal;overflow:visible}.label div .edgeLabel{color:var(--md-mermaid-label-fg-color)}.edgeLabel,.edgeLabel p,.label div .edgeLabel{background-color:var(--md-mermaid-label-bg-color)}.edgeLabel,.edgeLabel p{fill:var(--md-mermaid-label-bg-color);color:var(--md-mermaid-edge-color)}.edgePath .path,.flowchart-link{stroke:var(--md-mermaid-edge-color);stroke-width:.05rem}.edgePath .arrowheadPath{fill:var(--md-mermaid-edge-color);stroke:none}.cluster rect{fill:var(--md-default-fg-color--lightest);stroke:var(--md-default-fg-color--lighter)}.cluster span{color:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}g #flowchart-circleEnd,g #flowchart-circleStart,g #flowchart-crossEnd,g #flowchart-crossStart,g #flowchart-pointEnd,g #flowchart-pointStart{stroke:none}g.classGroup line,g.classGroup rect{fill:var(--md-mermaid-node-bg-color);stroke:var(--md-mermaid-node-fg-color)}g.classGroup text{fill:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}.classLabel .box{fill:var(--md-mermaid-label-bg-color);background-color:var(--md-mermaid-label-bg-color);opacity:1}.classLabel .label{fill:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}.node .divider{stroke:var(--md-mermaid-node-fg-color)}.relation{stroke:var(--md-mermaid-edge-color)}.cardinality{fill:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}.cardinality text{fill:inherit!important}defs #classDiagram-compositionEnd,defs #classDiagram-compositionStart,defs #classDiagram-dependencyEnd,defs #classDiagram-dependencyStart,defs #classDiagram-extensionEnd,defs #classDiagram-extensionStart{fill:var(--md-mermaid-edge-color)!important;stroke:var(--md-mermaid-edge-color)!important}defs #classDiagram-aggregationEnd,defs #classDiagram-aggregationStart{fill:var(--md-mermaid-label-bg-color)!important;stroke:var(--md-mermaid-edge-color)!important}g.stateGroup rect{fill:var(--md-mermaid-node-bg-color);stroke:var(--md-mermaid-node-fg-color)}g.stateGroup .state-title{fill:var(--md-mermaid-label-fg-color)!important;font-family:var(--md-mermaid-font-family)}g.stateGroup .composit{fill:var(--md-mermaid-label-bg-color)}.nodeLabel,.nodeLabel p{color:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}a .nodeLabel{text-decoration:underline}.node circle.state-end,.node circle.state-start,.start-state{fill:var(--md-mermaid-edge-color);stroke:none}.end-state-inner,.end-state-outer{fill:var(--md-mermaid-edge-color)}.end-state-inner,.node circle.state-end{stroke:var(--md-mermaid-label-bg-color)}.transition{stroke:var(--md-mermaid-edge-color)}[id^=state-fork] rect,[id^=state-join] rect{fill:var(--md-mermaid-edge-color)!important;stroke:none!important}.statediagram-cluster.statediagram-cluster .inner{fill:var(--md-default-bg-color)}.statediagram-cluster rect{fill:var(--md-mermaid-node-bg-color);stroke:var(--md-mermaid-node-fg-color)}.statediagram-state rect.divider{fill:var(--md-default-fg-color--lightest);stroke:var(--md-default-fg-color--lighter)}defs #statediagram-barbEnd{stroke:var(--md-mermaid-edge-color)}.attributeBoxEven,.attributeBoxOdd{fill:var(--md-mermaid-node-bg-color);stroke:var(--md-mermaid-node-fg-color)}.entityBox{fill:var(--md-mermaid-label-bg-color);stroke:var(--md-mermaid-node-fg-color)}.entityLabel{fill:var(--md-mermaid-label-fg-color);font-family:var(--md-mermaid-font-family)}.relationshipLabelBox{fill:var(--md-mermaid-label-bg-color);fill-opacity:1;background-color:var(--md-mermaid-label-bg-color);opacity:1}.relationshipLabel{fill:var(--md-mermaid-label-fg-color)}.relationshipLine{stroke:var(--md-mermaid-edge-color)}defs #ONE_OR_MORE_END *,defs #ONE_OR_MORE_START *,defs #ONLY_ONE_END *,defs #ONLY_ONE_START *,defs #ZERO_OR_MORE_END *,defs #ZERO_OR_MORE_START *,defs #ZERO_OR_ONE_END *,defs #ZERO_OR_ONE_START *{stroke:var(--md-mermaid-edge-color)!important}defs #ZERO_OR_MORE_END circle,defs #ZERO_OR_MORE_START circle{fill:var(--md-mermaid-label-bg-color)}.actor{fill:var(--md-mermaid-sequence-actor-bg-color);stroke:var(--md-mermaid-sequence-actor-border-color)}text.actor>tspan{fill:var(--md-mermaid-sequence-actor-fg-color);font-family:var(--md-mermaid-font-family)}line{stroke:var(--md-mermaid-sequence-actor-line-color)}.actor-man circle,.actor-man line{fill:var(--md-mermaid-sequence-actorman-bg-color);stroke:var(--md-mermaid-sequence-actorman-line-color)}.messageLine0,.messageLine1{stroke:var(--md-mermaid-sequence-message-line-color)}.note{fill:var(--md-mermaid-sequence-note-bg-color);stroke:var(--md-mermaid-sequence-note-border-color)}.loopText,.loopText>tspan,.messageText,.noteText>tspan{stroke:none;font-family:var(--md-mermaid-font-family)!important}.messageText{fill:var(--md-mermaid-sequence-message-fg-color)}.loopText,.loopText>tspan{fill:var(--md-mermaid-sequence-loop-fg-color)}.noteText>tspan{fill:var(--md-mermaid-sequence-note-fg-color)}#arrowhead path{fill:var(--md-mermaid-sequence-message-line-color);stroke:none}.loopLine{fill:var(--md-mermaid-sequence-loop-bg-color);stroke:var(--md-mermaid-sequence-loop-border-color)}.labelBox{fill:var(--md-mermaid-sequence-label-bg-color);stroke:none}.labelText,.labelText>span{fill:var(--md-mermaid-sequence-label-fg-color);font-family:var(--md-mermaid-font-family)}.sequenceNumber{fill:var(--md-mermaid-sequence-number-fg-color)}rect.rect{fill:var(--md-mermaid-sequence-box-bg-color);stroke:none}rect.rect+text.text{fill:var(--md-mermaid-sequence-box-fg-color)}defs #sequencenumber{fill:var(--md-mermaid-sequence-number-bg-color)!important}";var Gr,Qa=0;function Ka(){return typeof mermaid=="undefined"||mermaid instanceof Element?Tt("https://unpkg.com/mermaid@11/dist/mermaid.min.js"):I(void 0)}function Wn(e){return e.classList.remove("mermaid"),Gr||(Gr=Ka().pipe(w(()=>mermaid.initialize({startOnLoad:!1,themeCSS:Un,sequence:{actorFontSize:"16px",messageFontSize:"16px",noteFontSize:"16px"}})),m(()=>{}),G(1))),Gr.subscribe(()=>co(this,null,function*(){e.classList.add("mermaid");let t=`__mermaid_${Qa++}`,r=x("div",{class:"mermaid"}),o=e.textContent,{svg:n,fn:i}=yield mermaid.render(t,o),a=r.attachShadow({mode:"closed"});a.innerHTML=n,e.replaceWith(r),i==null||i(a)})),Gr.pipe(m(()=>({ref:e})))}var Dn=x("table");function Vn(e){return e.replaceWith(Dn),Dn.replaceWith(An(e)),I({ref:e})}function Ya(e){let t=e.find(r=>r.checked)||e[0];return O(...e.map(r=>h(r,"change").pipe(m(()=>R(`label[for="${r.id}"]`))))).pipe(Q(R(`label[for="${t.id}"]`)),m(r=>({active:r})))}function Nn(e,{viewport$:t,target$:r}){let o=R(".tabbed-labels",e),n=P(":scope > input",e),i=Kr("prev");e.append(i);let a=Kr("next");return e.append(a),C(()=>{let s=new g,p=s.pipe(Z(),ie(!0));z([s,ge(e),tt(e)]).pipe(W(p),Me(1,me)).subscribe({next([{active:c},l]){let f=Ve(c),{width:u}=ce(c);e.style.setProperty("--md-indicator-x",`${f.x}px`),e.style.setProperty("--md-indicator-width",`${u}px`);let d=pr(o);(f.xd.x+l.width)&&o.scrollTo({left:Math.max(0,f.x-16),behavior:"smooth"})},complete(){e.style.removeProperty("--md-indicator-x"),e.style.removeProperty("--md-indicator-width")}}),z([Ne(o),ge(o)]).pipe(W(p)).subscribe(([c,l])=>{let f=St(o);i.hidden=c.x<16,a.hidden=c.x>f.width-l.width-16}),O(h(i,"click").pipe(m(()=>-1)),h(a,"click").pipe(m(()=>1))).pipe(W(p)).subscribe(c=>{let{width:l}=ce(o);o.scrollBy({left:l*c,behavior:"smooth"})}),r.pipe(W(p),b(c=>n.includes(c))).subscribe(c=>c.click()),o.classList.add("tabbed-labels--linked");for(let c of n){let l=R(`label[for="${c.id}"]`);l.replaceChildren(x("a",{href:`#${l.htmlFor}`,tabIndex:-1},...Array.from(l.childNodes))),h(l.firstElementChild,"click").pipe(W(p),b(f=>!(f.metaKey||f.ctrlKey)),w(f=>{f.preventDefault(),f.stopPropagation()})).subscribe(()=>{history.replaceState({},"",`#${l.htmlFor}`),l.click()})}return B("content.tabs.link")&&s.pipe(Ce(1),re(t)).subscribe(([{active:c},{offset:l}])=>{let f=c.innerText.trim();if(c.hasAttribute("data-md-switching"))c.removeAttribute("data-md-switching");else{let u=e.offsetTop-l.y;for(let y of P("[data-tabs]"))for(let L of P(":scope > input",y)){let X=R(`label[for="${L.id}"]`);if(X!==c&&X.innerText.trim()===f){X.setAttribute("data-md-switching",""),L.click();break}}window.scrollTo({top:e.offsetTop-u});let d=__md_get("__tabs")||[];__md_set("__tabs",[...new Set([f,...d])])}}),s.pipe(W(p)).subscribe(()=>{for(let c of P("audio, video",e))c.pause()}),Ya(n).pipe(w(c=>s.next(c)),_(()=>s.complete()),m(c=>$({ref:e},c)))}).pipe(Ke(se))}function zn(e,{viewport$:t,target$:r,print$:o}){return O(...P(".annotate:not(.highlight)",e).map(n=>Pn(n,{target$:r,print$:o})),...P("pre:not(.mermaid) > code",e).map(n=>jn(n,{target$:r,print$:o})),...P("pre.mermaid",e).map(n=>Wn(n)),...P("table:not([class])",e).map(n=>Vn(n)),...P("details",e).map(n=>Fn(n,{target$:r,print$:o})),...P("[data-tabs]",e).map(n=>Nn(n,{viewport$:t,target$:r})),...P("[title]",e).filter(()=>B("content.tooltips")).map(n=>mt(n,{viewport$:t})))}function Ba(e,{alert$:t}){return t.pipe(v(r=>O(I(!0),I(!1).pipe(Ge(2e3))).pipe(m(o=>({message:r,active:o})))))}function qn(e,t){let r=R(".md-typeset",e);return C(()=>{let o=new g;return o.subscribe(({message:n,active:i})=>{e.classList.toggle("md-dialog--active",i),r.textContent=n}),Ba(e,t).pipe(w(n=>o.next(n)),_(()=>o.complete()),m(n=>$({ref:e},n)))})}var Ga=0;function Ja(e,t){document.body.append(e);let{width:r}=ce(e);e.style.setProperty("--md-tooltip-width",`${r}px`),e.remove();let o=cr(t),n=typeof o!="undefined"?Ne(o):I({x:0,y:0}),i=O(et(t),$t(t)).pipe(K());return z([i,n]).pipe(m(([a,s])=>{let{x:p,y:c}=Ve(t),l=ce(t),f=t.closest("table");return f&&t.parentElement&&(p+=f.offsetLeft+t.parentElement.offsetLeft,c+=f.offsetTop+t.parentElement.offsetTop),{active:a,offset:{x:p-s.x+l.width/2-r/2,y:c-s.y+l.height+8}}}))}function Qn(e){let t=e.title;if(!t.length)return S;let r=`__tooltip_${Ga++}`,o=Rt(r,"inline"),n=R(".md-typeset",o);return n.innerHTML=t,C(()=>{let i=new g;return i.subscribe({next({offset:a}){o.style.setProperty("--md-tooltip-x",`${a.x}px`),o.style.setProperty("--md-tooltip-y",`${a.y}px`)},complete(){o.style.removeProperty("--md-tooltip-x"),o.style.removeProperty("--md-tooltip-y")}}),O(i.pipe(b(({active:a})=>a)),i.pipe(_e(250),b(({active:a})=>!a))).subscribe({next({active:a}){a?(e.insertAdjacentElement("afterend",o),e.setAttribute("aria-describedby",r),e.removeAttribute("title")):(o.remove(),e.removeAttribute("aria-describedby"),e.setAttribute("title",t))},complete(){o.remove(),e.removeAttribute("aria-describedby"),e.setAttribute("title",t)}}),i.pipe(Me(16,me)).subscribe(({active:a})=>{o.classList.toggle("md-tooltip--active",a)}),i.pipe(pt(125,me),b(()=>!!e.offsetParent),m(()=>e.offsetParent.getBoundingClientRect()),m(({x:a})=>a)).subscribe({next(a){a?o.style.setProperty("--md-tooltip-0",`${-a}px`):o.style.removeProperty("--md-tooltip-0")},complete(){o.style.removeProperty("--md-tooltip-0")}}),Ja(o,e).pipe(w(a=>i.next(a)),_(()=>i.complete()),m(a=>$({ref:e},a)))}).pipe(Ke(se))}function Xa({viewport$:e}){if(!B("header.autohide"))return I(!1);let t=e.pipe(m(({offset:{y:n}})=>n),Be(2,1),m(([n,i])=>[nMath.abs(i-n.y)>100),m(([,[n]])=>n),K()),o=ze("search");return z([e,o]).pipe(m(([{offset:n},i])=>n.y>400&&!i),K(),v(n=>n?r:I(!1)),Q(!1))}function Kn(e,t){return C(()=>z([ge(e),Xa(t)])).pipe(m(([{height:r},o])=>({height:r,hidden:o})),K((r,o)=>r.height===o.height&&r.hidden===o.hidden),G(1))}function Yn(e,{header$:t,main$:r}){return C(()=>{let o=new g,n=o.pipe(Z(),ie(!0));o.pipe(ee("active"),He(t)).subscribe(([{active:a},{hidden:s}])=>{e.classList.toggle("md-header--shadow",a&&!s),e.hidden=s});let i=ue(P("[title]",e)).pipe(b(()=>B("content.tooltips")),ne(a=>Qn(a)));return r.subscribe(o),t.pipe(W(n),m(a=>$({ref:e},a)),Re(i.pipe(W(n))))})}function Za(e,{viewport$:t,header$:r}){return mr(e,{viewport$:t,header$:r}).pipe(m(({offset:{y:o}})=>{let{height:n}=ce(e);return{active:o>=n}}),ee("active"))}function Bn(e,t){return C(()=>{let r=new g;r.subscribe({next({active:n}){e.classList.toggle("md-header__title--active",n)},complete(){e.classList.remove("md-header__title--active")}});let o=fe(".md-content h1");return typeof o=="undefined"?S:Za(o,t).pipe(w(n=>r.next(n)),_(()=>r.complete()),m(n=>$({ref:e},n)))})}function Gn(e,{viewport$:t,header$:r}){let o=r.pipe(m(({height:i})=>i),K()),n=o.pipe(v(()=>ge(e).pipe(m(({height:i})=>({top:e.offsetTop,bottom:e.offsetTop+i})),ee("bottom"))));return z([o,n,t]).pipe(m(([i,{top:a,bottom:s},{offset:{y:p},size:{height:c}}])=>(c=Math.max(0,c-Math.max(0,a-p,i)-Math.max(0,c+p-s)),{offset:a-i,height:c,active:a-i<=p})),K((i,a)=>i.offset===a.offset&&i.height===a.height&&i.active===a.active))}function es(e){let t=__md_get("__palette")||{index:e.findIndex(o=>matchMedia(o.getAttribute("data-md-color-media")).matches)},r=Math.max(0,Math.min(t.index,e.length-1));return I(...e).pipe(ne(o=>h(o,"change").pipe(m(()=>o))),Q(e[r]),m(o=>({index:e.indexOf(o),color:{media:o.getAttribute("data-md-color-media"),scheme:o.getAttribute("data-md-color-scheme"),primary:o.getAttribute("data-md-color-primary"),accent:o.getAttribute("data-md-color-accent")}})),G(1))}function Jn(e){let t=P("input",e),r=x("meta",{name:"theme-color"});document.head.appendChild(r);let o=x("meta",{name:"color-scheme"});document.head.appendChild(o);let n=Pt("(prefers-color-scheme: light)");return C(()=>{let i=new g;return i.subscribe(a=>{if(document.body.setAttribute("data-md-color-switching",""),a.color.media==="(prefers-color-scheme)"){let s=matchMedia("(prefers-color-scheme: light)"),p=document.querySelector(s.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");a.color.scheme=p.getAttribute("data-md-color-scheme"),a.color.primary=p.getAttribute("data-md-color-primary"),a.color.accent=p.getAttribute("data-md-color-accent")}for(let[s,p]of Object.entries(a.color))document.body.setAttribute(`data-md-color-${s}`,p);for(let s=0;sa.key==="Enter"),re(i,(a,s)=>s)).subscribe(({index:a})=>{a=(a+1)%t.length,t[a].click(),t[a].focus()}),i.pipe(m(()=>{let a=Se("header"),s=window.getComputedStyle(a);return o.content=s.colorScheme,s.backgroundColor.match(/\d+/g).map(p=>(+p).toString(16).padStart(2,"0")).join("")})).subscribe(a=>r.content=`#${a}`),i.pipe(ve(se)).subscribe(()=>{document.body.removeAttribute("data-md-color-switching")}),es(t).pipe(W(n.pipe(Ce(1))),ct(),w(a=>i.next(a)),_(()=>i.complete()),m(a=>$({ref:e},a)))})}function Xn(e,{progress$:t}){return C(()=>{let r=new g;return r.subscribe(({value:o})=>{e.style.setProperty("--md-progress-value",`${o}`)}),t.pipe(w(o=>r.next({value:o})),_(()=>r.complete()),m(o=>({ref:e,value:o})))})}var Jr=Mt(Br());function ts(e){e.setAttribute("data-md-copying","");let t=e.closest("[data-copy]"),r=t?t.getAttribute("data-copy"):e.innerText;return e.removeAttribute("data-md-copying"),r.trimEnd()}function Zn({alert$:e}){Jr.default.isSupported()&&new j(t=>{new Jr.default("[data-clipboard-target], [data-clipboard-text]",{text:r=>r.getAttribute("data-clipboard-text")||ts(R(r.getAttribute("data-clipboard-target")))}).on("success",r=>t.next(r))}).pipe(w(t=>{t.trigger.focus()}),m(()=>Ee("clipboard.copied"))).subscribe(e)}function ei(e,t){return e.protocol=t.protocol,e.hostname=t.hostname,e}function rs(e,t){let r=new Map;for(let o of P("url",e)){let n=R("loc",o),i=[ei(new URL(n.textContent),t)];r.set(`${i[0]}`,i);for(let a of P("[rel=alternate]",o)){let s=a.getAttribute("href");s!=null&&i.push(ei(new URL(s),t))}}return r}function ur(e){return un(new URL("sitemap.xml",e)).pipe(m(t=>rs(t,new URL(e))),de(()=>I(new Map)))}function os(e,t){if(!(e.target instanceof Element))return S;let r=e.target.closest("a");if(r===null)return S;if(r.target||e.metaKey||e.ctrlKey)return S;let o=new URL(r.href);return o.search=o.hash="",t.has(`${o}`)?(e.preventDefault(),I(new URL(r.href))):S}function ti(e){let t=new Map;for(let r of P(":scope > *",e.head))t.set(r.outerHTML,r);return t}function ri(e){for(let t of P("[href], [src]",e))for(let r of["href","src"]){let o=t.getAttribute(r);if(o&&!/^(?:[a-z]+:)?\/\//i.test(o)){t[r]=t[r];break}}return I(e)}function ns(e){for(let o of["[data-md-component=announce]","[data-md-component=container]","[data-md-component=header-topic]","[data-md-component=outdated]","[data-md-component=logo]","[data-md-component=skip]",...B("navigation.tabs.sticky")?["[data-md-component=tabs]"]:[]]){let n=fe(o),i=fe(o,e);typeof n!="undefined"&&typeof i!="undefined"&&n.replaceWith(i)}let t=ti(document);for(let[o,n]of ti(e))t.has(o)?t.delete(o):document.head.appendChild(n);for(let o of t.values()){let n=o.getAttribute("name");n!=="theme-color"&&n!=="color-scheme"&&o.remove()}let r=Se("container");return We(P("script",r)).pipe(v(o=>{let n=e.createElement("script");if(o.src){for(let i of o.getAttributeNames())n.setAttribute(i,o.getAttribute(i));return o.replaceWith(n),new j(i=>{n.onload=()=>i.complete()})}else return n.textContent=o.textContent,o.replaceWith(n),S}),Z(),ie(document))}function oi({location$:e,viewport$:t,progress$:r}){let o=xe();if(location.protocol==="file:")return S;let n=ur(o.base);I(document).subscribe(ri);let i=h(document.body,"click").pipe(He(n),v(([p,c])=>os(p,c)),pe()),a=h(window,"popstate").pipe(m(ye),pe());i.pipe(re(t)).subscribe(([p,{offset:c}])=>{history.replaceState(c,""),history.pushState(null,"",p)}),O(i,a).subscribe(e);let s=e.pipe(ee("pathname"),v(p=>fn(p,{progress$:r}).pipe(de(()=>(lt(p,!0),S)))),v(ri),v(ns),pe());return O(s.pipe(re(e,(p,c)=>c)),s.pipe(v(()=>e),ee("pathname"),v(()=>e),ee("hash")),e.pipe(K((p,c)=>p.pathname===c.pathname&&p.hash===c.hash),v(()=>i),w(()=>history.back()))).subscribe(p=>{var c,l;history.state!==null||!p.hash?window.scrollTo(0,(l=(c=history.state)==null?void 0:c.y)!=null?l:0):(history.scrollRestoration="auto",pn(p.hash),history.scrollRestoration="manual")}),e.subscribe(()=>{history.scrollRestoration="manual"}),h(window,"beforeunload").subscribe(()=>{history.scrollRestoration="auto"}),t.pipe(ee("offset"),_e(100)).subscribe(({offset:p})=>{history.replaceState(p,"")}),s}var ni=Mt(qr());function ii(e){let t=e.separator.split("|").map(n=>n.replace(/(\(\?[!=<][^)]+\))/g,"").length===0?"\uFFFD":n).join("|"),r=new RegExp(t,"img"),o=(n,i,a)=>`${i}${a}`;return n=>{n=n.replace(/[\s*+\-:~^]+/g," ").trim();let i=new RegExp(`(^|${e.separator}|)(${n.replace(/[|\\{}()[\]^$+*?.-]/g,"\\$&").replace(r,"|")})`,"img");return a=>(0,ni.default)(a).replace(i,o).replace(/<\/mark>(\s+)]*>/img,"$1")}}function jt(e){return e.type===1}function dr(e){return e.type===3}function ai(e,t){let r=yn(e);return O(I(location.protocol!=="file:"),ze("search")).pipe(Ae(o=>o),v(()=>t)).subscribe(({config:o,docs:n})=>r.next({type:0,data:{config:o,docs:n,options:{suggest:B("search.suggest")}}})),r}function si(e){var l;let{selectedVersionSitemap:t,selectedVersionBaseURL:r,currentLocation:o,currentBaseURL:n}=e,i=(l=Xr(n))==null?void 0:l.pathname;if(i===void 0)return;let a=ss(o.pathname,i);if(a===void 0)return;let s=ps(t.keys());if(!t.has(s))return;let p=Xr(a,s);if(!p||!t.has(p.href))return;let c=Xr(a,r);if(c)return c.hash=o.hash,c.search=o.search,c}function Xr(e,t){try{return new URL(e,t)}catch(r){return}}function ss(e,t){if(e.startsWith(t))return e.slice(t.length)}function cs(e,t){let r=Math.min(e.length,t.length),o;for(o=0;oS)),o=r.pipe(m(n=>{let[,i]=t.base.match(/([^/]+)\/?$/);return n.find(({version:a,aliases:s})=>a===i||s.includes(i))||n[0]}));r.pipe(m(n=>new Map(n.map(i=>[`${new URL(`../${i.version}/`,t.base)}`,i]))),v(n=>h(document.body,"click").pipe(b(i=>!i.metaKey&&!i.ctrlKey),re(o),v(([i,a])=>{if(i.target instanceof Element){let s=i.target.closest("a");if(s&&!s.target&&n.has(s.href)){let p=s.href;return!i.target.closest(".md-version")&&n.get(p)===a?S:(i.preventDefault(),I(new URL(p)))}}return S}),v(i=>ur(i).pipe(m(a=>{var s;return(s=si({selectedVersionSitemap:a,selectedVersionBaseURL:i,currentLocation:ye(),currentBaseURL:t.base}))!=null?s:i})))))).subscribe(n=>lt(n,!0)),z([r,o]).subscribe(([n,i])=>{R(".md-header__topic").appendChild(Cn(n,i))}),e.pipe(v(()=>o)).subscribe(n=>{var a;let i=__md_get("__outdated",sessionStorage);if(i===null){i=!0;let s=((a=t.version)==null?void 0:a.default)||"latest";Array.isArray(s)||(s=[s]);e:for(let p of s)for(let c of n.aliases.concat(n.version))if(new RegExp(p,"i").test(c)){i=!1;break e}__md_set("__outdated",i,sessionStorage)}if(i)for(let s of ae("outdated"))s.hidden=!1})}function ls(e,{worker$:t}){let{searchParams:r}=ye();r.has("q")&&(Je("search",!0),e.value=r.get("q"),e.focus(),ze("search").pipe(Ae(i=>!i)).subscribe(()=>{let i=ye();i.searchParams.delete("q"),history.replaceState({},"",`${i}`)}));let o=et(e),n=O(t.pipe(Ae(jt)),h(e,"keyup"),o).pipe(m(()=>e.value),K());return z([n,o]).pipe(m(([i,a])=>({value:i,focus:a})),G(1))}function pi(e,{worker$:t}){let r=new g,o=r.pipe(Z(),ie(!0));z([t.pipe(Ae(jt)),r],(i,a)=>a).pipe(ee("value")).subscribe(({value:i})=>t.next({type:2,data:i})),r.pipe(ee("focus")).subscribe(({focus:i})=>{i&&Je("search",i)}),h(e.form,"reset").pipe(W(o)).subscribe(()=>e.focus());let n=R("header [for=__search]");return h(n,"click").subscribe(()=>e.focus()),ls(e,{worker$:t}).pipe(w(i=>r.next(i)),_(()=>r.complete()),m(i=>$({ref:e},i)),G(1))}function li(e,{worker$:t,query$:r}){let o=new g,n=on(e.parentElement).pipe(b(Boolean)),i=e.parentElement,a=R(":scope > :first-child",e),s=R(":scope > :last-child",e);ze("search").subscribe(l=>s.setAttribute("role",l?"list":"presentation")),o.pipe(re(r),Wr(t.pipe(Ae(jt)))).subscribe(([{items:l},{value:f}])=>{switch(l.length){case 0:a.textContent=f.length?Ee("search.result.none"):Ee("search.result.placeholder");break;case 1:a.textContent=Ee("search.result.one");break;default:let u=sr(l.length);a.textContent=Ee("search.result.other",u)}});let p=o.pipe(w(()=>s.innerHTML=""),v(({items:l})=>O(I(...l.slice(0,10)),I(...l.slice(10)).pipe(Be(4),Vr(n),v(([f])=>f)))),m(Mn),pe());return p.subscribe(l=>s.appendChild(l)),p.pipe(ne(l=>{let f=fe("details",l);return typeof f=="undefined"?S:h(f,"toggle").pipe(W(o),m(()=>f))})).subscribe(l=>{l.open===!1&&l.offsetTop<=i.scrollTop&&i.scrollTo({top:l.offsetTop})}),t.pipe(b(dr),m(({data:l})=>l)).pipe(w(l=>o.next(l)),_(()=>o.complete()),m(l=>$({ref:e},l)))}function ms(e,{query$:t}){return t.pipe(m(({value:r})=>{let o=ye();return o.hash="",r=r.replace(/\s+/g,"+").replace(/&/g,"%26").replace(/=/g,"%3D"),o.search=`q=${r}`,{url:o}}))}function mi(e,t){let r=new g,o=r.pipe(Z(),ie(!0));return r.subscribe(({url:n})=>{e.setAttribute("data-clipboard-text",e.href),e.href=`${n}`}),h(e,"click").pipe(W(o)).subscribe(n=>n.preventDefault()),ms(e,t).pipe(w(n=>r.next(n)),_(()=>r.complete()),m(n=>$({ref:e},n)))}function fi(e,{worker$:t,keyboard$:r}){let o=new g,n=Se("search-query"),i=O(h(n,"keydown"),h(n,"focus")).pipe(ve(se),m(()=>n.value),K());return o.pipe(He(i),m(([{suggest:s},p])=>{let c=p.split(/([\s-]+)/);if(s!=null&&s.length&&c[c.length-1]){let l=s[s.length-1];l.startsWith(c[c.length-1])&&(c[c.length-1]=l)}else c.length=0;return c})).subscribe(s=>e.innerHTML=s.join("").replace(/\s/g," ")),r.pipe(b(({mode:s})=>s==="search")).subscribe(s=>{switch(s.type){case"ArrowRight":e.innerText.length&&n.selectionStart===n.value.length&&(n.value=e.innerText);break}}),t.pipe(b(dr),m(({data:s})=>s)).pipe(w(s=>o.next(s)),_(()=>o.complete()),m(()=>({ref:e})))}function ui(e,{index$:t,keyboard$:r}){let o=xe();try{let n=ai(o.search,t),i=Se("search-query",e),a=Se("search-result",e);h(e,"click").pipe(b(({target:p})=>p instanceof Element&&!!p.closest("a"))).subscribe(()=>Je("search",!1)),r.pipe(b(({mode:p})=>p==="search")).subscribe(p=>{let c=Ie();switch(p.type){case"Enter":if(c===i){let l=new Map;for(let f of P(":first-child [href]",a)){let u=f.firstElementChild;l.set(f,parseFloat(u.getAttribute("data-md-score")))}if(l.size){let[[f]]=[...l].sort(([,u],[,d])=>d-u);f.click()}p.claim()}break;case"Escape":case"Tab":Je("search",!1),i.blur();break;case"ArrowUp":case"ArrowDown":if(typeof c=="undefined")i.focus();else{let l=[i,...P(":not(details) > [href], summary, details[open] [href]",a)],f=Math.max(0,(Math.max(0,l.indexOf(c))+l.length+(p.type==="ArrowUp"?-1:1))%l.length);l[f].focus()}p.claim();break;default:i!==Ie()&&i.focus()}}),r.pipe(b(({mode:p})=>p==="global")).subscribe(p=>{switch(p.type){case"f":case"s":case"/":i.focus(),i.select(),p.claim();break}});let s=pi(i,{worker$:n});return O(s,li(a,{worker$:n,query$:s})).pipe(Re(...ae("search-share",e).map(p=>mi(p,{query$:s})),...ae("search-suggest",e).map(p=>fi(p,{worker$:n,keyboard$:r}))))}catch(n){return e.hidden=!0,Ye}}function di(e,{index$:t,location$:r}){return z([t,r.pipe(Q(ye()),b(o=>!!o.searchParams.get("h")))]).pipe(m(([o,n])=>ii(o.config)(n.searchParams.get("h"))),m(o=>{var a;let n=new Map,i=document.createNodeIterator(e,NodeFilter.SHOW_TEXT);for(let s=i.nextNode();s;s=i.nextNode())if((a=s.parentElement)!=null&&a.offsetHeight){let p=s.textContent,c=o(p);c.length>p.length&&n.set(s,c)}for(let[s,p]of n){let{childNodes:c}=x("span",null,p);s.replaceWith(...Array.from(c))}return{ref:e,nodes:n}}))}function fs(e,{viewport$:t,main$:r}){let o=e.closest(".md-grid"),n=o.offsetTop-o.parentElement.offsetTop;return z([r,t]).pipe(m(([{offset:i,height:a},{offset:{y:s}}])=>(a=a+Math.min(n,Math.max(0,s-i))-n,{height:a,locked:s>=i+n})),K((i,a)=>i.height===a.height&&i.locked===a.locked))}function Zr(e,o){var n=o,{header$:t}=n,r=so(n,["header$"]);let i=R(".md-sidebar__scrollwrap",e),{y:a}=Ve(i);return C(()=>{let s=new g,p=s.pipe(Z(),ie(!0)),c=s.pipe(Me(0,me));return c.pipe(re(t)).subscribe({next([{height:l},{height:f}]){i.style.height=`${l-2*a}px`,e.style.top=`${f}px`},complete(){i.style.height="",e.style.top=""}}),c.pipe(Ae()).subscribe(()=>{for(let l of P(".md-nav__link--active[href]",e)){if(!l.clientHeight)continue;let f=l.closest(".md-sidebar__scrollwrap");if(typeof f!="undefined"){let u=l.offsetTop-f.offsetTop,{height:d}=ce(f);f.scrollTo({top:u-d/2})}}}),ue(P("label[tabindex]",e)).pipe(ne(l=>h(l,"click").pipe(ve(se),m(()=>l),W(p)))).subscribe(l=>{let f=R(`[id="${l.htmlFor}"]`);R(`[aria-labelledby="${l.id}"]`).setAttribute("aria-expanded",`${f.checked}`)}),fs(e,r).pipe(w(l=>s.next(l)),_(()=>s.complete()),m(l=>$({ref:e},l)))})}function hi(e,t){if(typeof t!="undefined"){let r=`https://api.github.com/repos/${e}/${t}`;return st(je(`${r}/releases/latest`).pipe(de(()=>S),m(o=>({version:o.tag_name})),De({})),je(r).pipe(de(()=>S),m(o=>({stars:o.stargazers_count,forks:o.forks_count})),De({}))).pipe(m(([o,n])=>$($({},o),n)))}else{let r=`https://api.github.com/users/${e}`;return je(r).pipe(m(o=>({repositories:o.public_repos})),De({}))}}function bi(e,t){let r=`https://${e}/api/v4/projects/${encodeURIComponent(t)}`;return st(je(`${r}/releases/permalink/latest`).pipe(de(()=>S),m(({tag_name:o})=>({version:o})),De({})),je(r).pipe(de(()=>S),m(({star_count:o,forks_count:n})=>({stars:o,forks:n})),De({}))).pipe(m(([o,n])=>$($({},o),n)))}function vi(e){let t=e.match(/^.+github\.com\/([^/]+)\/?([^/]+)?/i);if(t){let[,r,o]=t;return hi(r,o)}if(t=e.match(/^.+?([^/]*gitlab[^/]+)\/(.+?)\/?$/i),t){let[,r,o]=t;return bi(r,o)}return S}var us;function ds(e){return us||(us=C(()=>{let t=__md_get("__source",sessionStorage);if(t)return I(t);if(ae("consent").length){let o=__md_get("__consent");if(!(o&&o.github))return S}return vi(e.href).pipe(w(o=>__md_set("__source",o,sessionStorage)))}).pipe(de(()=>S),b(t=>Object.keys(t).length>0),m(t=>({facts:t})),G(1)))}function gi(e){let t=R(":scope > :last-child",e);return C(()=>{let r=new g;return r.subscribe(({facts:o})=>{t.appendChild(_n(o)),t.classList.add("md-source__repository--active")}),ds(e).pipe(w(o=>r.next(o)),_(()=>r.complete()),m(o=>$({ref:e},o)))})}function hs(e,{viewport$:t,header$:r}){return ge(document.body).pipe(v(()=>mr(e,{header$:r,viewport$:t})),m(({offset:{y:o}})=>({hidden:o>=10})),ee("hidden"))}function yi(e,t){return C(()=>{let r=new g;return r.subscribe({next({hidden:o}){e.hidden=o},complete(){e.hidden=!1}}),(B("navigation.tabs.sticky")?I({hidden:!1}):hs(e,t)).pipe(w(o=>r.next(o)),_(()=>r.complete()),m(o=>$({ref:e},o)))})}function bs(e,{viewport$:t,header$:r}){let o=new Map,n=P(".md-nav__link",e);for(let s of n){let p=decodeURIComponent(s.hash.substring(1)),c=fe(`[id="${p}"]`);typeof c!="undefined"&&o.set(s,c)}let i=r.pipe(ee("height"),m(({height:s})=>{let p=Se("main"),c=R(":scope > :first-child",p);return s+.8*(c.offsetTop-p.offsetTop)}),pe());return ge(document.body).pipe(ee("height"),v(s=>C(()=>{let p=[];return I([...o].reduce((c,[l,f])=>{for(;p.length&&o.get(p[p.length-1]).tagName>=f.tagName;)p.pop();let u=f.offsetTop;for(;!u&&f.parentElement;)f=f.parentElement,u=f.offsetTop;let d=f.offsetParent;for(;d;d=d.offsetParent)u+=d.offsetTop;return c.set([...p=[...p,l]].reverse(),u)},new Map))}).pipe(m(p=>new Map([...p].sort(([,c],[,l])=>c-l))),He(i),v(([p,c])=>t.pipe(Fr(([l,f],{offset:{y:u},size:d})=>{let y=u+d.height>=Math.floor(s.height);for(;f.length;){let[,L]=f[0];if(L-c=u&&!y)f=[l.pop(),...f];else break}return[l,f]},[[],[...p]]),K((l,f)=>l[0]===f[0]&&l[1]===f[1])))))).pipe(m(([s,p])=>({prev:s.map(([c])=>c),next:p.map(([c])=>c)})),Q({prev:[],next:[]}),Be(2,1),m(([s,p])=>s.prev.length{let i=new g,a=i.pipe(Z(),ie(!0));if(i.subscribe(({prev:s,next:p})=>{for(let[c]of p)c.classList.remove("md-nav__link--passed"),c.classList.remove("md-nav__link--active");for(let[c,[l]]of s.entries())l.classList.add("md-nav__link--passed"),l.classList.toggle("md-nav__link--active",c===s.length-1)}),B("toc.follow")){let s=O(t.pipe(_e(1),m(()=>{})),t.pipe(_e(250),m(()=>"smooth")));i.pipe(b(({prev:p})=>p.length>0),He(o.pipe(ve(se))),re(s)).subscribe(([[{prev:p}],c])=>{let[l]=p[p.length-1];if(l.offsetHeight){let f=cr(l);if(typeof f!="undefined"){let u=l.offsetTop-f.offsetTop,{height:d}=ce(f);f.scrollTo({top:u-d/2,behavior:c})}}})}return B("navigation.tracking")&&t.pipe(W(a),ee("offset"),_e(250),Ce(1),W(n.pipe(Ce(1))),ct({delay:250}),re(i)).subscribe(([,{prev:s}])=>{let p=ye(),c=s[s.length-1];if(c&&c.length){let[l]=c,{hash:f}=new URL(l.href);p.hash!==f&&(p.hash=f,history.replaceState({},"",`${p}`))}else p.hash="",history.replaceState({},"",`${p}`)}),bs(e,{viewport$:t,header$:r}).pipe(w(s=>i.next(s)),_(()=>i.complete()),m(s=>$({ref:e},s)))})}function vs(e,{viewport$:t,main$:r,target$:o}){let n=t.pipe(m(({offset:{y:a}})=>a),Be(2,1),m(([a,s])=>a>s&&s>0),K()),i=r.pipe(m(({active:a})=>a));return z([i,n]).pipe(m(([a,s])=>!(a&&s)),K(),W(o.pipe(Ce(1))),ie(!0),ct({delay:250}),m(a=>({hidden:a})))}function Ei(e,{viewport$:t,header$:r,main$:o,target$:n}){let i=new g,a=i.pipe(Z(),ie(!0));return i.subscribe({next({hidden:s}){e.hidden=s,s?(e.setAttribute("tabindex","-1"),e.blur()):e.removeAttribute("tabindex")},complete(){e.style.top="",e.hidden=!0,e.removeAttribute("tabindex")}}),r.pipe(W(a),ee("height")).subscribe(({height:s})=>{e.style.top=`${s+16}px`}),h(e,"click").subscribe(s=>{s.preventDefault(),window.scrollTo({top:0})}),vs(e,{viewport$:t,main$:o,target$:n}).pipe(w(s=>i.next(s)),_(()=>i.complete()),m(s=>$({ref:e},s)))}function wi({document$:e,viewport$:t}){e.pipe(v(()=>P(".md-ellipsis")),ne(r=>tt(r).pipe(W(e.pipe(Ce(1))),b(o=>o),m(()=>r),Te(1))),b(r=>r.offsetWidth{let o=r.innerText,n=r.closest("a")||r;return n.title=o,B("content.tooltips")?mt(n,{viewport$:t}).pipe(W(e.pipe(Ce(1))),_(()=>n.removeAttribute("title"))):S})).subscribe(),B("content.tooltips")&&e.pipe(v(()=>P(".md-status")),ne(r=>mt(r,{viewport$:t}))).subscribe()}function Ti({document$:e,tablet$:t}){e.pipe(v(()=>P(".md-toggle--indeterminate")),w(r=>{r.indeterminate=!0,r.checked=!1}),ne(r=>h(r,"change").pipe(Dr(()=>r.classList.contains("md-toggle--indeterminate")),m(()=>r))),re(t)).subscribe(([r,o])=>{r.classList.remove("md-toggle--indeterminate"),o&&(r.checked=!1)})}function gs(){return/(iPad|iPhone|iPod)/.test(navigator.userAgent)}function Si({document$:e}){e.pipe(v(()=>P("[data-md-scrollfix]")),w(t=>t.removeAttribute("data-md-scrollfix")),b(gs),ne(t=>h(t,"touchstart").pipe(m(()=>t)))).subscribe(t=>{let r=t.scrollTop;r===0?t.scrollTop=1:r+t.offsetHeight===t.scrollHeight&&(t.scrollTop=r-1)})}function Oi({viewport$:e,tablet$:t}){z([ze("search"),t]).pipe(m(([r,o])=>r&&!o),v(r=>I(r).pipe(Ge(r?400:100))),re(e)).subscribe(([r,{offset:{y:o}}])=>{if(r)document.body.setAttribute("data-md-scrolllock",""),document.body.style.top=`-${o}px`;else{let n=-1*parseInt(document.body.style.top,10);document.body.removeAttribute("data-md-scrolllock"),document.body.style.top="",n&&window.scrollTo(0,n)}})}Object.entries||(Object.entries=function(e){let t=[];for(let r of Object.keys(e))t.push([r,e[r]]);return t});Object.values||(Object.values=function(e){let t=[];for(let r of Object.keys(e))t.push(e[r]);return t});typeof Element!="undefined"&&(Element.prototype.scrollTo||(Element.prototype.scrollTo=function(e,t){typeof e=="object"?(this.scrollLeft=e.left,this.scrollTop=e.top):(this.scrollLeft=e,this.scrollTop=t)}),Element.prototype.replaceWith||(Element.prototype.replaceWith=function(...e){let t=this.parentNode;if(t){e.length===0&&t.removeChild(this);for(let r=e.length-1;r>=0;r--){let o=e[r];typeof o=="string"?o=document.createTextNode(o):o.parentNode&&o.parentNode.removeChild(o),r?t.insertBefore(this.previousSibling,o):t.replaceChild(o,this)}}}));function ys(){return location.protocol==="file:"?Tt(`${new URL("search/search_index.js",eo.base)}`).pipe(m(()=>__index),G(1)):je(new URL("search/search_index.json",eo.base))}document.documentElement.classList.remove("no-js");document.documentElement.classList.add("js");var ot=Go(),Ut=sn(),Lt=ln(Ut),to=an(),Oe=gn(),hr=Pt("(min-width: 960px)"),Mi=Pt("(min-width: 1220px)"),_i=mn(),eo=xe(),Ai=document.forms.namedItem("search")?ys():Ye,ro=new g;Zn({alert$:ro});var oo=new g;B("navigation.instant")&&oi({location$:Ut,viewport$:Oe,progress$:oo}).subscribe(ot);var Li;((Li=eo.version)==null?void 0:Li.provider)==="mike"&&ci({document$:ot});O(Ut,Lt).pipe(Ge(125)).subscribe(()=>{Je("drawer",!1),Je("search",!1)});to.pipe(b(({mode:e})=>e==="global")).subscribe(e=>{switch(e.type){case"p":case",":let t=fe("link[rel=prev]");typeof t!="undefined"&<(t);break;case"n":case".":let r=fe("link[rel=next]");typeof r!="undefined"&<(r);break;case"Enter":let o=Ie();o instanceof HTMLLabelElement&&o.click()}});wi({viewport$:Oe,document$:ot});Ti({document$:ot,tablet$:hr});Si({document$:ot});Oi({viewport$:Oe,tablet$:hr});var rt=Kn(Se("header"),{viewport$:Oe}),Ft=ot.pipe(m(()=>Se("main")),v(e=>Gn(e,{viewport$:Oe,header$:rt})),G(1)),xs=O(...ae("consent").map(e=>En(e,{target$:Lt})),...ae("dialog").map(e=>qn(e,{alert$:ro})),...ae("palette").map(e=>Jn(e)),...ae("progress").map(e=>Xn(e,{progress$:oo})),...ae("search").map(e=>ui(e,{index$:Ai,keyboard$:to})),...ae("source").map(e=>gi(e))),Es=C(()=>O(...ae("announce").map(e=>xn(e)),...ae("content").map(e=>zn(e,{viewport$:Oe,target$:Lt,print$:_i})),...ae("content").map(e=>B("search.highlight")?di(e,{index$:Ai,location$:Ut}):S),...ae("header").map(e=>Yn(e,{viewport$:Oe,header$:rt,main$:Ft})),...ae("header-title").map(e=>Bn(e,{viewport$:Oe,header$:rt})),...ae("sidebar").map(e=>e.getAttribute("data-md-type")==="navigation"?Nr(Mi,()=>Zr(e,{viewport$:Oe,header$:rt,main$:Ft})):Nr(hr,()=>Zr(e,{viewport$:Oe,header$:rt,main$:Ft}))),...ae("tabs").map(e=>yi(e,{viewport$:Oe,header$:rt})),...ae("toc").map(e=>xi(e,{viewport$:Oe,header$:rt,main$:Ft,target$:Lt})),...ae("top").map(e=>Ei(e,{viewport$:Oe,header$:rt,main$:Ft,target$:Lt})))),Ci=ot.pipe(v(()=>Es),Re(xs),G(1));Ci.subscribe();window.document$=ot;window.location$=Ut;window.target$=Lt;window.keyboard$=to;window.viewport$=Oe;window.tablet$=hr;window.screen$=Mi;window.print$=_i;window.alert$=ro;window.progress$=oo;window.component$=Ci;})(); +//# sourceMappingURL=bundle.83f73b43.min.js.map + diff --git a/assets/javascripts/bundle.83f73b43.min.js.map b/assets/javascripts/bundle.83f73b43.min.js.map new file mode 100644 index 00000000..fe920b7d --- /dev/null +++ b/assets/javascripts/bundle.83f73b43.min.js.map @@ -0,0 +1,7 @@ +{ + "version": 3, + "sources": ["node_modules/focus-visible/dist/focus-visible.js", "node_modules/escape-html/index.js", "node_modules/clipboard/dist/clipboard.js", "src/templates/assets/javascripts/bundle.ts", "node_modules/tslib/tslib.es6.mjs", "node_modules/rxjs/src/internal/util/isFunction.ts", "node_modules/rxjs/src/internal/util/createErrorClass.ts", "node_modules/rxjs/src/internal/util/UnsubscriptionError.ts", "node_modules/rxjs/src/internal/util/arrRemove.ts", "node_modules/rxjs/src/internal/Subscription.ts", "node_modules/rxjs/src/internal/config.ts", "node_modules/rxjs/src/internal/scheduler/timeoutProvider.ts", "node_modules/rxjs/src/internal/util/reportUnhandledError.ts", "node_modules/rxjs/src/internal/util/noop.ts", "node_modules/rxjs/src/internal/NotificationFactories.ts", "node_modules/rxjs/src/internal/util/errorContext.ts", "node_modules/rxjs/src/internal/Subscriber.ts", "node_modules/rxjs/src/internal/symbol/observable.ts", "node_modules/rxjs/src/internal/util/identity.ts", "node_modules/rxjs/src/internal/util/pipe.ts", "node_modules/rxjs/src/internal/Observable.ts", "node_modules/rxjs/src/internal/util/lift.ts", "node_modules/rxjs/src/internal/operators/OperatorSubscriber.ts", "node_modules/rxjs/src/internal/scheduler/animationFrameProvider.ts", "node_modules/rxjs/src/internal/util/ObjectUnsubscribedError.ts", "node_modules/rxjs/src/internal/Subject.ts", "node_modules/rxjs/src/internal/BehaviorSubject.ts", "node_modules/rxjs/src/internal/scheduler/dateTimestampProvider.ts", "node_modules/rxjs/src/internal/ReplaySubject.ts", "node_modules/rxjs/src/internal/scheduler/Action.ts", "node_modules/rxjs/src/internal/scheduler/intervalProvider.ts", "node_modules/rxjs/src/internal/scheduler/AsyncAction.ts", "node_modules/rxjs/src/internal/Scheduler.ts", "node_modules/rxjs/src/internal/scheduler/AsyncScheduler.ts", "node_modules/rxjs/src/internal/scheduler/async.ts", "node_modules/rxjs/src/internal/scheduler/QueueAction.ts", "node_modules/rxjs/src/internal/scheduler/QueueScheduler.ts", "node_modules/rxjs/src/internal/scheduler/queue.ts", "node_modules/rxjs/src/internal/scheduler/AnimationFrameAction.ts", "node_modules/rxjs/src/internal/scheduler/AnimationFrameScheduler.ts", "node_modules/rxjs/src/internal/scheduler/animationFrame.ts", "node_modules/rxjs/src/internal/observable/empty.ts", "node_modules/rxjs/src/internal/util/isScheduler.ts", "node_modules/rxjs/src/internal/util/args.ts", "node_modules/rxjs/src/internal/util/isArrayLike.ts", "node_modules/rxjs/src/internal/util/isPromise.ts", "node_modules/rxjs/src/internal/util/isInteropObservable.ts", "node_modules/rxjs/src/internal/util/isAsyncIterable.ts", "node_modules/rxjs/src/internal/util/throwUnobservableError.ts", "node_modules/rxjs/src/internal/symbol/iterator.ts", "node_modules/rxjs/src/internal/util/isIterable.ts", "node_modules/rxjs/src/internal/util/isReadableStreamLike.ts", "node_modules/rxjs/src/internal/observable/innerFrom.ts", "node_modules/rxjs/src/internal/util/executeSchedule.ts", "node_modules/rxjs/src/internal/operators/observeOn.ts", "node_modules/rxjs/src/internal/operators/subscribeOn.ts", "node_modules/rxjs/src/internal/scheduled/scheduleObservable.ts", "node_modules/rxjs/src/internal/scheduled/schedulePromise.ts", "node_modules/rxjs/src/internal/scheduled/scheduleArray.ts", "node_modules/rxjs/src/internal/scheduled/scheduleIterable.ts", "node_modules/rxjs/src/internal/scheduled/scheduleAsyncIterable.ts", "node_modules/rxjs/src/internal/scheduled/scheduleReadableStreamLike.ts", "node_modules/rxjs/src/internal/scheduled/scheduled.ts", "node_modules/rxjs/src/internal/observable/from.ts", "node_modules/rxjs/src/internal/observable/of.ts", "node_modules/rxjs/src/internal/observable/throwError.ts", "node_modules/rxjs/src/internal/util/EmptyError.ts", "node_modules/rxjs/src/internal/util/isDate.ts", "node_modules/rxjs/src/internal/operators/map.ts", "node_modules/rxjs/src/internal/util/mapOneOrManyArgs.ts", "node_modules/rxjs/src/internal/util/argsArgArrayOrObject.ts", "node_modules/rxjs/src/internal/util/createObject.ts", "node_modules/rxjs/src/internal/observable/combineLatest.ts", "node_modules/rxjs/src/internal/operators/mergeInternals.ts", "node_modules/rxjs/src/internal/operators/mergeMap.ts", "node_modules/rxjs/src/internal/operators/mergeAll.ts", "node_modules/rxjs/src/internal/operators/concatAll.ts", "node_modules/rxjs/src/internal/observable/concat.ts", "node_modules/rxjs/src/internal/observable/defer.ts", "node_modules/rxjs/src/internal/observable/fromEvent.ts", "node_modules/rxjs/src/internal/observable/fromEventPattern.ts", "node_modules/rxjs/src/internal/observable/timer.ts", "node_modules/rxjs/src/internal/observable/merge.ts", "node_modules/rxjs/src/internal/observable/never.ts", "node_modules/rxjs/src/internal/util/argsOrArgArray.ts", "node_modules/rxjs/src/internal/operators/filter.ts", "node_modules/rxjs/src/internal/observable/zip.ts", "node_modules/rxjs/src/internal/operators/audit.ts", "node_modules/rxjs/src/internal/operators/auditTime.ts", "node_modules/rxjs/src/internal/operators/bufferCount.ts", "node_modules/rxjs/src/internal/operators/catchError.ts", "node_modules/rxjs/src/internal/operators/scanInternals.ts", "node_modules/rxjs/src/internal/operators/combineLatest.ts", "node_modules/rxjs/src/internal/operators/combineLatestWith.ts", "node_modules/rxjs/src/internal/operators/debounce.ts", "node_modules/rxjs/src/internal/operators/debounceTime.ts", "node_modules/rxjs/src/internal/operators/defaultIfEmpty.ts", "node_modules/rxjs/src/internal/operators/take.ts", "node_modules/rxjs/src/internal/operators/ignoreElements.ts", "node_modules/rxjs/src/internal/operators/mapTo.ts", "node_modules/rxjs/src/internal/operators/delayWhen.ts", "node_modules/rxjs/src/internal/operators/delay.ts", "node_modules/rxjs/src/internal/operators/distinctUntilChanged.ts", "node_modules/rxjs/src/internal/operators/distinctUntilKeyChanged.ts", "node_modules/rxjs/src/internal/operators/throwIfEmpty.ts", "node_modules/rxjs/src/internal/operators/endWith.ts", "node_modules/rxjs/src/internal/operators/finalize.ts", "node_modules/rxjs/src/internal/operators/first.ts", "node_modules/rxjs/src/internal/operators/takeLast.ts", "node_modules/rxjs/src/internal/operators/merge.ts", "node_modules/rxjs/src/internal/operators/mergeWith.ts", "node_modules/rxjs/src/internal/operators/repeat.ts", "node_modules/rxjs/src/internal/operators/scan.ts", "node_modules/rxjs/src/internal/operators/share.ts", "node_modules/rxjs/src/internal/operators/shareReplay.ts", "node_modules/rxjs/src/internal/operators/skip.ts", "node_modules/rxjs/src/internal/operators/skipUntil.ts", "node_modules/rxjs/src/internal/operators/startWith.ts", "node_modules/rxjs/src/internal/operators/switchMap.ts", "node_modules/rxjs/src/internal/operators/takeUntil.ts", "node_modules/rxjs/src/internal/operators/takeWhile.ts", "node_modules/rxjs/src/internal/operators/tap.ts", "node_modules/rxjs/src/internal/operators/throttle.ts", "node_modules/rxjs/src/internal/operators/throttleTime.ts", "node_modules/rxjs/src/internal/operators/withLatestFrom.ts", "node_modules/rxjs/src/internal/operators/zip.ts", "node_modules/rxjs/src/internal/operators/zipWith.ts", "src/templates/assets/javascripts/browser/document/index.ts", "src/templates/assets/javascripts/browser/element/_/index.ts", "src/templates/assets/javascripts/browser/element/focus/index.ts", "src/templates/assets/javascripts/browser/element/hover/index.ts", "src/templates/assets/javascripts/utilities/h/index.ts", "src/templates/assets/javascripts/utilities/round/index.ts", "src/templates/assets/javascripts/browser/script/index.ts", "src/templates/assets/javascripts/browser/element/size/_/index.ts", "src/templates/assets/javascripts/browser/element/size/content/index.ts", "src/templates/assets/javascripts/browser/element/offset/_/index.ts", "src/templates/assets/javascripts/browser/element/offset/content/index.ts", "src/templates/assets/javascripts/browser/element/visibility/index.ts", "src/templates/assets/javascripts/browser/toggle/index.ts", "src/templates/assets/javascripts/browser/keyboard/index.ts", "src/templates/assets/javascripts/browser/location/_/index.ts", "src/templates/assets/javascripts/browser/location/hash/index.ts", "src/templates/assets/javascripts/browser/media/index.ts", "src/templates/assets/javascripts/browser/request/index.ts", "src/templates/assets/javascripts/browser/viewport/offset/index.ts", "src/templates/assets/javascripts/browser/viewport/size/index.ts", "src/templates/assets/javascripts/browser/viewport/_/index.ts", "src/templates/assets/javascripts/browser/viewport/at/index.ts", "src/templates/assets/javascripts/browser/worker/index.ts", "src/templates/assets/javascripts/_/index.ts", "src/templates/assets/javascripts/components/_/index.ts", "src/templates/assets/javascripts/components/announce/index.ts", "src/templates/assets/javascripts/components/consent/index.ts", "src/templates/assets/javascripts/templates/tooltip/index.tsx", "src/templates/assets/javascripts/templates/annotation/index.tsx", "src/templates/assets/javascripts/templates/clipboard/index.tsx", "src/templates/assets/javascripts/templates/search/index.tsx", "src/templates/assets/javascripts/templates/source/index.tsx", "src/templates/assets/javascripts/templates/tabbed/index.tsx", "src/templates/assets/javascripts/templates/table/index.tsx", "src/templates/assets/javascripts/templates/version/index.tsx", "src/templates/assets/javascripts/components/tooltip2/index.ts", "src/templates/assets/javascripts/components/content/annotation/_/index.ts", "src/templates/assets/javascripts/components/content/annotation/list/index.ts", "src/templates/assets/javascripts/components/content/annotation/block/index.ts", "src/templates/assets/javascripts/components/content/code/_/index.ts", "src/templates/assets/javascripts/components/content/details/index.ts", "src/templates/assets/javascripts/components/content/mermaid/index.css", "src/templates/assets/javascripts/components/content/mermaid/index.ts", "src/templates/assets/javascripts/components/content/table/index.ts", "src/templates/assets/javascripts/components/content/tabs/index.ts", "src/templates/assets/javascripts/components/content/_/index.ts", "src/templates/assets/javascripts/components/dialog/index.ts", "src/templates/assets/javascripts/components/tooltip/index.ts", "src/templates/assets/javascripts/components/header/_/index.ts", "src/templates/assets/javascripts/components/header/title/index.ts", "src/templates/assets/javascripts/components/main/index.ts", "src/templates/assets/javascripts/components/palette/index.ts", "src/templates/assets/javascripts/components/progress/index.ts", "src/templates/assets/javascripts/integrations/clipboard/index.ts", "src/templates/assets/javascripts/integrations/sitemap/index.ts", "src/templates/assets/javascripts/integrations/instant/index.ts", "src/templates/assets/javascripts/integrations/search/highlighter/index.ts", "src/templates/assets/javascripts/integrations/search/worker/message/index.ts", "src/templates/assets/javascripts/integrations/search/worker/_/index.ts", "src/templates/assets/javascripts/integrations/version/findurl/index.ts", "src/templates/assets/javascripts/integrations/version/index.ts", "src/templates/assets/javascripts/components/search/query/index.ts", "src/templates/assets/javascripts/components/search/result/index.ts", "src/templates/assets/javascripts/components/search/share/index.ts", "src/templates/assets/javascripts/components/search/suggest/index.ts", "src/templates/assets/javascripts/components/search/_/index.ts", "src/templates/assets/javascripts/components/search/highlight/index.ts", "src/templates/assets/javascripts/components/sidebar/index.ts", "src/templates/assets/javascripts/components/source/facts/github/index.ts", "src/templates/assets/javascripts/components/source/facts/gitlab/index.ts", "src/templates/assets/javascripts/components/source/facts/_/index.ts", "src/templates/assets/javascripts/components/source/_/index.ts", "src/templates/assets/javascripts/components/tabs/index.ts", "src/templates/assets/javascripts/components/toc/index.ts", "src/templates/assets/javascripts/components/top/index.ts", "src/templates/assets/javascripts/patches/ellipsis/index.ts", "src/templates/assets/javascripts/patches/indeterminate/index.ts", "src/templates/assets/javascripts/patches/scrollfix/index.ts", "src/templates/assets/javascripts/patches/scrolllock/index.ts", "src/templates/assets/javascripts/polyfills/index.ts"], + "sourcesContent": ["(function (global, factory) {\n typeof exports === 'object' && typeof module !== 'undefined' ? factory() :\n typeof define === 'function' && define.amd ? define(factory) :\n (factory());\n}(this, (function () { 'use strict';\n\n /**\n * Applies the :focus-visible polyfill at the given scope.\n * A scope in this case is either the top-level Document or a Shadow Root.\n *\n * @param {(Document|ShadowRoot)} scope\n * @see https://github.com/WICG/focus-visible\n */\n function applyFocusVisiblePolyfill(scope) {\n var hadKeyboardEvent = true;\n var hadFocusVisibleRecently = false;\n var hadFocusVisibleRecentlyTimeout = null;\n\n var inputTypesAllowlist = {\n text: true,\n search: true,\n url: true,\n tel: true,\n email: true,\n password: true,\n number: true,\n date: true,\n month: true,\n week: true,\n time: true,\n datetime: true,\n 'datetime-local': true\n };\n\n /**\n * Helper function for legacy browsers and iframes which sometimes focus\n * elements like document, body, and non-interactive SVG.\n * @param {Element} el\n */\n function isValidFocusTarget(el) {\n if (\n el &&\n el !== document &&\n el.nodeName !== 'HTML' &&\n el.nodeName !== 'BODY' &&\n 'classList' in el &&\n 'contains' in el.classList\n ) {\n return true;\n }\n return false;\n }\n\n /**\n * Computes whether the given element should automatically trigger the\n * `focus-visible` class being added, i.e. whether it should always match\n * `:focus-visible` when focused.\n * @param {Element} el\n * @return {boolean}\n */\n function focusTriggersKeyboardModality(el) {\n var type = el.type;\n var tagName = el.tagName;\n\n if (tagName === 'INPUT' && inputTypesAllowlist[type] && !el.readOnly) {\n return true;\n }\n\n if (tagName === 'TEXTAREA' && !el.readOnly) {\n return true;\n }\n\n if (el.isContentEditable) {\n return true;\n }\n\n return false;\n }\n\n /**\n * Add the `focus-visible` class to the given element if it was not added by\n * the author.\n * @param {Element} el\n */\n function addFocusVisibleClass(el) {\n if (el.classList.contains('focus-visible')) {\n return;\n }\n el.classList.add('focus-visible');\n el.setAttribute('data-focus-visible-added', '');\n }\n\n /**\n * Remove the `focus-visible` class from the given element if it was not\n * originally added by the author.\n * @param {Element} el\n */\n function removeFocusVisibleClass(el) {\n if (!el.hasAttribute('data-focus-visible-added')) {\n return;\n }\n el.classList.remove('focus-visible');\n el.removeAttribute('data-focus-visible-added');\n }\n\n /**\n * If the most recent user interaction was via the keyboard;\n * and the key press did not include a meta, alt/option, or control key;\n * then the modality is keyboard. Otherwise, the modality is not keyboard.\n * Apply `focus-visible` to any current active element and keep track\n * of our keyboard modality state with `hadKeyboardEvent`.\n * @param {KeyboardEvent} e\n */\n function onKeyDown(e) {\n if (e.metaKey || e.altKey || e.ctrlKey) {\n return;\n }\n\n if (isValidFocusTarget(scope.activeElement)) {\n addFocusVisibleClass(scope.activeElement);\n }\n\n hadKeyboardEvent = true;\n }\n\n /**\n * If at any point a user clicks with a pointing device, ensure that we change\n * the modality away from keyboard.\n * This avoids the situation where a user presses a key on an already focused\n * element, and then clicks on a different element, focusing it with a\n * pointing device, while we still think we're in keyboard modality.\n * @param {Event} e\n */\n function onPointerDown(e) {\n hadKeyboardEvent = false;\n }\n\n /**\n * On `focus`, add the `focus-visible` class to the target if:\n * - the target received focus as a result of keyboard navigation, or\n * - the event target is an element that will likely require interaction\n * via the keyboard (e.g. a text box)\n * @param {Event} e\n */\n function onFocus(e) {\n // Prevent IE from focusing the document or HTML element.\n if (!isValidFocusTarget(e.target)) {\n return;\n }\n\n if (hadKeyboardEvent || focusTriggersKeyboardModality(e.target)) {\n addFocusVisibleClass(e.target);\n }\n }\n\n /**\n * On `blur`, remove the `focus-visible` class from the target.\n * @param {Event} e\n */\n function onBlur(e) {\n if (!isValidFocusTarget(e.target)) {\n return;\n }\n\n if (\n e.target.classList.contains('focus-visible') ||\n e.target.hasAttribute('data-focus-visible-added')\n ) {\n // To detect a tab/window switch, we look for a blur event followed\n // rapidly by a visibility change.\n // If we don't see a visibility change within 100ms, it's probably a\n // regular focus change.\n hadFocusVisibleRecently = true;\n window.clearTimeout(hadFocusVisibleRecentlyTimeout);\n hadFocusVisibleRecentlyTimeout = window.setTimeout(function() {\n hadFocusVisibleRecently = false;\n }, 100);\n removeFocusVisibleClass(e.target);\n }\n }\n\n /**\n * If the user changes tabs, keep track of whether or not the previously\n * focused element had .focus-visible.\n * @param {Event} e\n */\n function onVisibilityChange(e) {\n if (document.visibilityState === 'hidden') {\n // If the tab becomes active again, the browser will handle calling focus\n // on the element (Safari actually calls it twice).\n // If this tab change caused a blur on an element with focus-visible,\n // re-apply the class when the user switches back to the tab.\n if (hadFocusVisibleRecently) {\n hadKeyboardEvent = true;\n }\n addInitialPointerMoveListeners();\n }\n }\n\n /**\n * Add a group of listeners to detect usage of any pointing devices.\n * These listeners will be added when the polyfill first loads, and anytime\n * the window is blurred, so that they are active when the window regains\n * focus.\n */\n function addInitialPointerMoveListeners() {\n document.addEventListener('mousemove', onInitialPointerMove);\n document.addEventListener('mousedown', onInitialPointerMove);\n document.addEventListener('mouseup', onInitialPointerMove);\n document.addEventListener('pointermove', onInitialPointerMove);\n document.addEventListener('pointerdown', onInitialPointerMove);\n document.addEventListener('pointerup', onInitialPointerMove);\n document.addEventListener('touchmove', onInitialPointerMove);\n document.addEventListener('touchstart', onInitialPointerMove);\n document.addEventListener('touchend', onInitialPointerMove);\n }\n\n function removeInitialPointerMoveListeners() {\n document.removeEventListener('mousemove', onInitialPointerMove);\n document.removeEventListener('mousedown', onInitialPointerMove);\n document.removeEventListener('mouseup', onInitialPointerMove);\n document.removeEventListener('pointermove', onInitialPointerMove);\n document.removeEventListener('pointerdown', onInitialPointerMove);\n document.removeEventListener('pointerup', onInitialPointerMove);\n document.removeEventListener('touchmove', onInitialPointerMove);\n document.removeEventListener('touchstart', onInitialPointerMove);\n document.removeEventListener('touchend', onInitialPointerMove);\n }\n\n /**\n * When the polfyill first loads, assume the user is in keyboard modality.\n * If any event is received from a pointing device (e.g. mouse, pointer,\n * touch), turn off keyboard modality.\n * This accounts for situations where focus enters the page from the URL bar.\n * @param {Event} e\n */\n function onInitialPointerMove(e) {\n // Work around a Safari quirk that fires a mousemove on whenever the\n // window blurs, even if you're tabbing out of the page. \u00AF\\_(\u30C4)_/\u00AF\n if (e.target.nodeName && e.target.nodeName.toLowerCase() === 'html') {\n return;\n }\n\n hadKeyboardEvent = false;\n removeInitialPointerMoveListeners();\n }\n\n // For some kinds of state, we are interested in changes at the global scope\n // only. For example, global pointer input, global key presses and global\n // visibility change should affect the state at every scope:\n document.addEventListener('keydown', onKeyDown, true);\n document.addEventListener('mousedown', onPointerDown, true);\n document.addEventListener('pointerdown', onPointerDown, true);\n document.addEventListener('touchstart', onPointerDown, true);\n document.addEventListener('visibilitychange', onVisibilityChange, true);\n\n addInitialPointerMoveListeners();\n\n // For focus and blur, we specifically care about state changes in the local\n // scope. This is because focus / blur events that originate from within a\n // shadow root are not re-dispatched from the host element if it was already\n // the active element in its own scope:\n scope.addEventListener('focus', onFocus, true);\n scope.addEventListener('blur', onBlur, true);\n\n // We detect that a node is a ShadowRoot by ensuring that it is a\n // DocumentFragment and also has a host property. This check covers native\n // implementation and polyfill implementation transparently. If we only cared\n // about the native implementation, we could just check if the scope was\n // an instance of a ShadowRoot.\n if (scope.nodeType === Node.DOCUMENT_FRAGMENT_NODE && scope.host) {\n // Since a ShadowRoot is a special kind of DocumentFragment, it does not\n // have a root element to add a class to. So, we add this attribute to the\n // host element instead:\n scope.host.setAttribute('data-js-focus-visible', '');\n } else if (scope.nodeType === Node.DOCUMENT_NODE) {\n document.documentElement.classList.add('js-focus-visible');\n document.documentElement.setAttribute('data-js-focus-visible', '');\n }\n }\n\n // It is important to wrap all references to global window and document in\n // these checks to support server-side rendering use cases\n // @see https://github.com/WICG/focus-visible/issues/199\n if (typeof window !== 'undefined' && typeof document !== 'undefined') {\n // Make the polyfill helper globally available. This can be used as a signal\n // to interested libraries that wish to coordinate with the polyfill for e.g.,\n // applying the polyfill to a shadow root:\n window.applyFocusVisiblePolyfill = applyFocusVisiblePolyfill;\n\n // Notify interested libraries of the polyfill's presence, in case the\n // polyfill was loaded lazily:\n var event;\n\n try {\n event = new CustomEvent('focus-visible-polyfill-ready');\n } catch (error) {\n // IE11 does not support using CustomEvent as a constructor directly:\n event = document.createEvent('CustomEvent');\n event.initCustomEvent('focus-visible-polyfill-ready', false, false, {});\n }\n\n window.dispatchEvent(event);\n }\n\n if (typeof document !== 'undefined') {\n // Apply the polyfill to the global document, so that no JavaScript\n // coordination is required to use the polyfill in the top-level document:\n applyFocusVisiblePolyfill(document);\n }\n\n})));\n", "/*!\n * escape-html\n * Copyright(c) 2012-2013 TJ Holowaychuk\n * Copyright(c) 2015 Andreas Lubbe\n * Copyright(c) 2015 Tiancheng \"Timothy\" Gu\n * MIT Licensed\n */\n\n'use strict';\n\n/**\n * Module variables.\n * @private\n */\n\nvar matchHtmlRegExp = /[\"'&<>]/;\n\n/**\n * Module exports.\n * @public\n */\n\nmodule.exports = escapeHtml;\n\n/**\n * Escape special characters in the given string of html.\n *\n * @param {string} string The string to escape for inserting into HTML\n * @return {string}\n * @public\n */\n\nfunction escapeHtml(string) {\n var str = '' + string;\n var match = matchHtmlRegExp.exec(str);\n\n if (!match) {\n return str;\n }\n\n var escape;\n var html = '';\n var index = 0;\n var lastIndex = 0;\n\n for (index = match.index; index < str.length; index++) {\n switch (str.charCodeAt(index)) {\n case 34: // \"\n escape = '"';\n break;\n case 38: // &\n escape = '&';\n break;\n case 39: // '\n escape = ''';\n break;\n case 60: // <\n escape = '<';\n break;\n case 62: // >\n escape = '>';\n break;\n default:\n continue;\n }\n\n if (lastIndex !== index) {\n html += str.substring(lastIndex, index);\n }\n\n lastIndex = index + 1;\n html += escape;\n }\n\n return lastIndex !== index\n ? html + str.substring(lastIndex, index)\n : html;\n}\n", "/*!\n * clipboard.js v2.0.11\n * https://clipboardjs.com/\n *\n * Licensed MIT \u00A9 Zeno Rocha\n */\n(function webpackUniversalModuleDefinition(root, factory) {\n\tif(typeof exports === 'object' && typeof module === 'object')\n\t\tmodule.exports = factory();\n\telse if(typeof define === 'function' && define.amd)\n\t\tdefine([], factory);\n\telse if(typeof exports === 'object')\n\t\texports[\"ClipboardJS\"] = factory();\n\telse\n\t\troot[\"ClipboardJS\"] = factory();\n})(this, function() {\nreturn /******/ (function() { // webpackBootstrap\n/******/ \tvar __webpack_modules__ = ({\n\n/***/ 686:\n/***/ (function(__unused_webpack_module, __webpack_exports__, __webpack_require__) {\n\n\"use strict\";\n\n// EXPORTS\n__webpack_require__.d(__webpack_exports__, {\n \"default\": function() { return /* binding */ clipboard; }\n});\n\n// EXTERNAL MODULE: ./node_modules/tiny-emitter/index.js\nvar tiny_emitter = __webpack_require__(279);\nvar tiny_emitter_default = /*#__PURE__*/__webpack_require__.n(tiny_emitter);\n// EXTERNAL MODULE: ./node_modules/good-listener/src/listen.js\nvar listen = __webpack_require__(370);\nvar listen_default = /*#__PURE__*/__webpack_require__.n(listen);\n// EXTERNAL MODULE: ./node_modules/select/src/select.js\nvar src_select = __webpack_require__(817);\nvar select_default = /*#__PURE__*/__webpack_require__.n(src_select);\n;// CONCATENATED MODULE: ./src/common/command.js\n/**\n * Executes a given operation type.\n * @param {String} type\n * @return {Boolean}\n */\nfunction command(type) {\n try {\n return document.execCommand(type);\n } catch (err) {\n return false;\n }\n}\n;// CONCATENATED MODULE: ./src/actions/cut.js\n\n\n/**\n * Cut action wrapper.\n * @param {String|HTMLElement} target\n * @return {String}\n */\n\nvar ClipboardActionCut = function ClipboardActionCut(target) {\n var selectedText = select_default()(target);\n command('cut');\n return selectedText;\n};\n\n/* harmony default export */ var actions_cut = (ClipboardActionCut);\n;// CONCATENATED MODULE: ./src/common/create-fake-element.js\n/**\n * Creates a fake textarea element with a value.\n * @param {String} value\n * @return {HTMLElement}\n */\nfunction createFakeElement(value) {\n var isRTL = document.documentElement.getAttribute('dir') === 'rtl';\n var fakeElement = document.createElement('textarea'); // Prevent zooming on iOS\n\n fakeElement.style.fontSize = '12pt'; // Reset box model\n\n fakeElement.style.border = '0';\n fakeElement.style.padding = '0';\n fakeElement.style.margin = '0'; // Move element out of screen horizontally\n\n fakeElement.style.position = 'absolute';\n fakeElement.style[isRTL ? 'right' : 'left'] = '-9999px'; // Move element to the same position vertically\n\n var yPosition = window.pageYOffset || document.documentElement.scrollTop;\n fakeElement.style.top = \"\".concat(yPosition, \"px\");\n fakeElement.setAttribute('readonly', '');\n fakeElement.value = value;\n return fakeElement;\n}\n;// CONCATENATED MODULE: ./src/actions/copy.js\n\n\n\n/**\n * Create fake copy action wrapper using a fake element.\n * @param {String} target\n * @param {Object} options\n * @return {String}\n */\n\nvar fakeCopyAction = function fakeCopyAction(value, options) {\n var fakeElement = createFakeElement(value);\n options.container.appendChild(fakeElement);\n var selectedText = select_default()(fakeElement);\n command('copy');\n fakeElement.remove();\n return selectedText;\n};\n/**\n * Copy action wrapper.\n * @param {String|HTMLElement} target\n * @param {Object} options\n * @return {String}\n */\n\n\nvar ClipboardActionCopy = function ClipboardActionCopy(target) {\n var options = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : {\n container: document.body\n };\n var selectedText = '';\n\n if (typeof target === 'string') {\n selectedText = fakeCopyAction(target, options);\n } else if (target instanceof HTMLInputElement && !['text', 'search', 'url', 'tel', 'password'].includes(target === null || target === void 0 ? void 0 : target.type)) {\n // If input type doesn't support `setSelectionRange`. Simulate it. https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/setSelectionRange\n selectedText = fakeCopyAction(target.value, options);\n } else {\n selectedText = select_default()(target);\n command('copy');\n }\n\n return selectedText;\n};\n\n/* harmony default export */ var actions_copy = (ClipboardActionCopy);\n;// CONCATENATED MODULE: ./src/actions/default.js\nfunction _typeof(obj) { \"@babel/helpers - typeof\"; if (typeof Symbol === \"function\" && typeof Symbol.iterator === \"symbol\") { _typeof = function _typeof(obj) { return typeof obj; }; } else { _typeof = function _typeof(obj) { return obj && typeof Symbol === \"function\" && obj.constructor === Symbol && obj !== Symbol.prototype ? \"symbol\" : typeof obj; }; } return _typeof(obj); }\n\n\n\n/**\n * Inner function which performs selection from either `text` or `target`\n * properties and then executes copy or cut operations.\n * @param {Object} options\n */\n\nvar ClipboardActionDefault = function ClipboardActionDefault() {\n var options = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : {};\n // Defines base properties passed from constructor.\n var _options$action = options.action,\n action = _options$action === void 0 ? 'copy' : _options$action,\n container = options.container,\n target = options.target,\n text = options.text; // Sets the `action` to be performed which can be either 'copy' or 'cut'.\n\n if (action !== 'copy' && action !== 'cut') {\n throw new Error('Invalid \"action\" value, use either \"copy\" or \"cut\"');\n } // Sets the `target` property using an element that will be have its content copied.\n\n\n if (target !== undefined) {\n if (target && _typeof(target) === 'object' && target.nodeType === 1) {\n if (action === 'copy' && target.hasAttribute('disabled')) {\n throw new Error('Invalid \"target\" attribute. Please use \"readonly\" instead of \"disabled\" attribute');\n }\n\n if (action === 'cut' && (target.hasAttribute('readonly') || target.hasAttribute('disabled'))) {\n throw new Error('Invalid \"target\" attribute. You can\\'t cut text from elements with \"readonly\" or \"disabled\" attributes');\n }\n } else {\n throw new Error('Invalid \"target\" value, use a valid Element');\n }\n } // Define selection strategy based on `text` property.\n\n\n if (text) {\n return actions_copy(text, {\n container: container\n });\n } // Defines which selection strategy based on `target` property.\n\n\n if (target) {\n return action === 'cut' ? actions_cut(target) : actions_copy(target, {\n container: container\n });\n }\n};\n\n/* harmony default export */ var actions_default = (ClipboardActionDefault);\n;// CONCATENATED MODULE: ./src/clipboard.js\nfunction clipboard_typeof(obj) { \"@babel/helpers - typeof\"; if (typeof Symbol === \"function\" && typeof Symbol.iterator === \"symbol\") { clipboard_typeof = function _typeof(obj) { return typeof obj; }; } else { clipboard_typeof = function _typeof(obj) { return obj && typeof Symbol === \"function\" && obj.constructor === Symbol && obj !== Symbol.prototype ? \"symbol\" : typeof obj; }; } return clipboard_typeof(obj); }\n\nfunction _classCallCheck(instance, Constructor) { if (!(instance instanceof Constructor)) { throw new TypeError(\"Cannot call a class as a function\"); } }\n\nfunction _defineProperties(target, props) { for (var i = 0; i < props.length; i++) { var descriptor = props[i]; descriptor.enumerable = descriptor.enumerable || false; descriptor.configurable = true; if (\"value\" in descriptor) descriptor.writable = true; Object.defineProperty(target, descriptor.key, descriptor); } }\n\nfunction _createClass(Constructor, protoProps, staticProps) { if (protoProps) _defineProperties(Constructor.prototype, protoProps); if (staticProps) _defineProperties(Constructor, staticProps); return Constructor; }\n\nfunction _inherits(subClass, superClass) { if (typeof superClass !== \"function\" && superClass !== null) { throw new TypeError(\"Super expression must either be null or a function\"); } subClass.prototype = Object.create(superClass && superClass.prototype, { constructor: { value: subClass, writable: true, configurable: true } }); if (superClass) _setPrototypeOf(subClass, superClass); }\n\nfunction _setPrototypeOf(o, p) { _setPrototypeOf = Object.setPrototypeOf || function _setPrototypeOf(o, p) { o.__proto__ = p; return o; }; return _setPrototypeOf(o, p); }\n\nfunction _createSuper(Derived) { var hasNativeReflectConstruct = _isNativeReflectConstruct(); return function _createSuperInternal() { var Super = _getPrototypeOf(Derived), result; if (hasNativeReflectConstruct) { var NewTarget = _getPrototypeOf(this).constructor; result = Reflect.construct(Super, arguments, NewTarget); } else { result = Super.apply(this, arguments); } return _possibleConstructorReturn(this, result); }; }\n\nfunction _possibleConstructorReturn(self, call) { if (call && (clipboard_typeof(call) === \"object\" || typeof call === \"function\")) { return call; } return _assertThisInitialized(self); }\n\nfunction _assertThisInitialized(self) { if (self === void 0) { throw new ReferenceError(\"this hasn't been initialised - super() hasn't been called\"); } return self; }\n\nfunction _isNativeReflectConstruct() { if (typeof Reflect === \"undefined\" || !Reflect.construct) return false; if (Reflect.construct.sham) return false; if (typeof Proxy === \"function\") return true; try { Date.prototype.toString.call(Reflect.construct(Date, [], function () {})); return true; } catch (e) { return false; } }\n\nfunction _getPrototypeOf(o) { _getPrototypeOf = Object.setPrototypeOf ? Object.getPrototypeOf : function _getPrototypeOf(o) { return o.__proto__ || Object.getPrototypeOf(o); }; return _getPrototypeOf(o); }\n\n\n\n\n\n\n/**\n * Helper function to retrieve attribute value.\n * @param {String} suffix\n * @param {Element} element\n */\n\nfunction getAttributeValue(suffix, element) {\n var attribute = \"data-clipboard-\".concat(suffix);\n\n if (!element.hasAttribute(attribute)) {\n return;\n }\n\n return element.getAttribute(attribute);\n}\n/**\n * Base class which takes one or more elements, adds event listeners to them,\n * and instantiates a new `ClipboardAction` on each click.\n */\n\n\nvar Clipboard = /*#__PURE__*/function (_Emitter) {\n _inherits(Clipboard, _Emitter);\n\n var _super = _createSuper(Clipboard);\n\n /**\n * @param {String|HTMLElement|HTMLCollection|NodeList} trigger\n * @param {Object} options\n */\n function Clipboard(trigger, options) {\n var _this;\n\n _classCallCheck(this, Clipboard);\n\n _this = _super.call(this);\n\n _this.resolveOptions(options);\n\n _this.listenClick(trigger);\n\n return _this;\n }\n /**\n * Defines if attributes would be resolved using internal setter functions\n * or custom functions that were passed in the constructor.\n * @param {Object} options\n */\n\n\n _createClass(Clipboard, [{\n key: \"resolveOptions\",\n value: function resolveOptions() {\n var options = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : {};\n this.action = typeof options.action === 'function' ? options.action : this.defaultAction;\n this.target = typeof options.target === 'function' ? options.target : this.defaultTarget;\n this.text = typeof options.text === 'function' ? options.text : this.defaultText;\n this.container = clipboard_typeof(options.container) === 'object' ? options.container : document.body;\n }\n /**\n * Adds a click event listener to the passed trigger.\n * @param {String|HTMLElement|HTMLCollection|NodeList} trigger\n */\n\n }, {\n key: \"listenClick\",\n value: function listenClick(trigger) {\n var _this2 = this;\n\n this.listener = listen_default()(trigger, 'click', function (e) {\n return _this2.onClick(e);\n });\n }\n /**\n * Defines a new `ClipboardAction` on each click event.\n * @param {Event} e\n */\n\n }, {\n key: \"onClick\",\n value: function onClick(e) {\n var trigger = e.delegateTarget || e.currentTarget;\n var action = this.action(trigger) || 'copy';\n var text = actions_default({\n action: action,\n container: this.container,\n target: this.target(trigger),\n text: this.text(trigger)\n }); // Fires an event based on the copy operation result.\n\n this.emit(text ? 'success' : 'error', {\n action: action,\n text: text,\n trigger: trigger,\n clearSelection: function clearSelection() {\n if (trigger) {\n trigger.focus();\n }\n\n window.getSelection().removeAllRanges();\n }\n });\n }\n /**\n * Default `action` lookup function.\n * @param {Element} trigger\n */\n\n }, {\n key: \"defaultAction\",\n value: function defaultAction(trigger) {\n return getAttributeValue('action', trigger);\n }\n /**\n * Default `target` lookup function.\n * @param {Element} trigger\n */\n\n }, {\n key: \"defaultTarget\",\n value: function defaultTarget(trigger) {\n var selector = getAttributeValue('target', trigger);\n\n if (selector) {\n return document.querySelector(selector);\n }\n }\n /**\n * Allow fire programmatically a copy action\n * @param {String|HTMLElement} target\n * @param {Object} options\n * @returns Text copied.\n */\n\n }, {\n key: \"defaultText\",\n\n /**\n * Default `text` lookup function.\n * @param {Element} trigger\n */\n value: function defaultText(trigger) {\n return getAttributeValue('text', trigger);\n }\n /**\n * Destroy lifecycle.\n */\n\n }, {\n key: \"destroy\",\n value: function destroy() {\n this.listener.destroy();\n }\n }], [{\n key: \"copy\",\n value: function copy(target) {\n var options = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : {\n container: document.body\n };\n return actions_copy(target, options);\n }\n /**\n * Allow fire programmatically a cut action\n * @param {String|HTMLElement} target\n * @returns Text cutted.\n */\n\n }, {\n key: \"cut\",\n value: function cut(target) {\n return actions_cut(target);\n }\n /**\n * Returns the support of the given action, or all actions if no action is\n * given.\n * @param {String} [action]\n */\n\n }, {\n key: \"isSupported\",\n value: function isSupported() {\n var action = arguments.length > 0 && arguments[0] !== undefined ? arguments[0] : ['copy', 'cut'];\n var actions = typeof action === 'string' ? [action] : action;\n var support = !!document.queryCommandSupported;\n actions.forEach(function (action) {\n support = support && !!document.queryCommandSupported(action);\n });\n return support;\n }\n }]);\n\n return Clipboard;\n}((tiny_emitter_default()));\n\n/* harmony default export */ var clipboard = (Clipboard);\n\n/***/ }),\n\n/***/ 828:\n/***/ (function(module) {\n\nvar DOCUMENT_NODE_TYPE = 9;\n\n/**\n * A polyfill for Element.matches()\n */\nif (typeof Element !== 'undefined' && !Element.prototype.matches) {\n var proto = Element.prototype;\n\n proto.matches = proto.matchesSelector ||\n proto.mozMatchesSelector ||\n proto.msMatchesSelector ||\n proto.oMatchesSelector ||\n proto.webkitMatchesSelector;\n}\n\n/**\n * Finds the closest parent that matches a selector.\n *\n * @param {Element} element\n * @param {String} selector\n * @return {Function}\n */\nfunction closest (element, selector) {\n while (element && element.nodeType !== DOCUMENT_NODE_TYPE) {\n if (typeof element.matches === 'function' &&\n element.matches(selector)) {\n return element;\n }\n element = element.parentNode;\n }\n}\n\nmodule.exports = closest;\n\n\n/***/ }),\n\n/***/ 438:\n/***/ (function(module, __unused_webpack_exports, __webpack_require__) {\n\nvar closest = __webpack_require__(828);\n\n/**\n * Delegates event to a selector.\n *\n * @param {Element} element\n * @param {String} selector\n * @param {String} type\n * @param {Function} callback\n * @param {Boolean} useCapture\n * @return {Object}\n */\nfunction _delegate(element, selector, type, callback, useCapture) {\n var listenerFn = listener.apply(this, arguments);\n\n element.addEventListener(type, listenerFn, useCapture);\n\n return {\n destroy: function() {\n element.removeEventListener(type, listenerFn, useCapture);\n }\n }\n}\n\n/**\n * Delegates event to a selector.\n *\n * @param {Element|String|Array} [elements]\n * @param {String} selector\n * @param {String} type\n * @param {Function} callback\n * @param {Boolean} useCapture\n * @return {Object}\n */\nfunction delegate(elements, selector, type, callback, useCapture) {\n // Handle the regular Element usage\n if (typeof elements.addEventListener === 'function') {\n return _delegate.apply(null, arguments);\n }\n\n // Handle Element-less usage, it defaults to global delegation\n if (typeof type === 'function') {\n // Use `document` as the first parameter, then apply arguments\n // This is a short way to .unshift `arguments` without running into deoptimizations\n return _delegate.bind(null, document).apply(null, arguments);\n }\n\n // Handle Selector-based usage\n if (typeof elements === 'string') {\n elements = document.querySelectorAll(elements);\n }\n\n // Handle Array-like based usage\n return Array.prototype.map.call(elements, function (element) {\n return _delegate(element, selector, type, callback, useCapture);\n });\n}\n\n/**\n * Finds closest match and invokes callback.\n *\n * @param {Element} element\n * @param {String} selector\n * @param {String} type\n * @param {Function} callback\n * @return {Function}\n */\nfunction listener(element, selector, type, callback) {\n return function(e) {\n e.delegateTarget = closest(e.target, selector);\n\n if (e.delegateTarget) {\n callback.call(element, e);\n }\n }\n}\n\nmodule.exports = delegate;\n\n\n/***/ }),\n\n/***/ 879:\n/***/ (function(__unused_webpack_module, exports) {\n\n/**\n * Check if argument is a HTML element.\n *\n * @param {Object} value\n * @return {Boolean}\n */\nexports.node = function(value) {\n return value !== undefined\n && value instanceof HTMLElement\n && value.nodeType === 1;\n};\n\n/**\n * Check if argument is a list of HTML elements.\n *\n * @param {Object} value\n * @return {Boolean}\n */\nexports.nodeList = function(value) {\n var type = Object.prototype.toString.call(value);\n\n return value !== undefined\n && (type === '[object NodeList]' || type === '[object HTMLCollection]')\n && ('length' in value)\n && (value.length === 0 || exports.node(value[0]));\n};\n\n/**\n * Check if argument is a string.\n *\n * @param {Object} value\n * @return {Boolean}\n */\nexports.string = function(value) {\n return typeof value === 'string'\n || value instanceof String;\n};\n\n/**\n * Check if argument is a function.\n *\n * @param {Object} value\n * @return {Boolean}\n */\nexports.fn = function(value) {\n var type = Object.prototype.toString.call(value);\n\n return type === '[object Function]';\n};\n\n\n/***/ }),\n\n/***/ 370:\n/***/ (function(module, __unused_webpack_exports, __webpack_require__) {\n\nvar is = __webpack_require__(879);\nvar delegate = __webpack_require__(438);\n\n/**\n * Validates all params and calls the right\n * listener function based on its target type.\n *\n * @param {String|HTMLElement|HTMLCollection|NodeList} target\n * @param {String} type\n * @param {Function} callback\n * @return {Object}\n */\nfunction listen(target, type, callback) {\n if (!target && !type && !callback) {\n throw new Error('Missing required arguments');\n }\n\n if (!is.string(type)) {\n throw new TypeError('Second argument must be a String');\n }\n\n if (!is.fn(callback)) {\n throw new TypeError('Third argument must be a Function');\n }\n\n if (is.node(target)) {\n return listenNode(target, type, callback);\n }\n else if (is.nodeList(target)) {\n return listenNodeList(target, type, callback);\n }\n else if (is.string(target)) {\n return listenSelector(target, type, callback);\n }\n else {\n throw new TypeError('First argument must be a String, HTMLElement, HTMLCollection, or NodeList');\n }\n}\n\n/**\n * Adds an event listener to a HTML element\n * and returns a remove listener function.\n *\n * @param {HTMLElement} node\n * @param {String} type\n * @param {Function} callback\n * @return {Object}\n */\nfunction listenNode(node, type, callback) {\n node.addEventListener(type, callback);\n\n return {\n destroy: function() {\n node.removeEventListener(type, callback);\n }\n }\n}\n\n/**\n * Add an event listener to a list of HTML elements\n * and returns a remove listener function.\n *\n * @param {NodeList|HTMLCollection} nodeList\n * @param {String} type\n * @param {Function} callback\n * @return {Object}\n */\nfunction listenNodeList(nodeList, type, callback) {\n Array.prototype.forEach.call(nodeList, function(node) {\n node.addEventListener(type, callback);\n });\n\n return {\n destroy: function() {\n Array.prototype.forEach.call(nodeList, function(node) {\n node.removeEventListener(type, callback);\n });\n }\n }\n}\n\n/**\n * Add an event listener to a selector\n * and returns a remove listener function.\n *\n * @param {String} selector\n * @param {String} type\n * @param {Function} callback\n * @return {Object}\n */\nfunction listenSelector(selector, type, callback) {\n return delegate(document.body, selector, type, callback);\n}\n\nmodule.exports = listen;\n\n\n/***/ }),\n\n/***/ 817:\n/***/ (function(module) {\n\nfunction select(element) {\n var selectedText;\n\n if (element.nodeName === 'SELECT') {\n element.focus();\n\n selectedText = element.value;\n }\n else if (element.nodeName === 'INPUT' || element.nodeName === 'TEXTAREA') {\n var isReadOnly = element.hasAttribute('readonly');\n\n if (!isReadOnly) {\n element.setAttribute('readonly', '');\n }\n\n element.select();\n element.setSelectionRange(0, element.value.length);\n\n if (!isReadOnly) {\n element.removeAttribute('readonly');\n }\n\n selectedText = element.value;\n }\n else {\n if (element.hasAttribute('contenteditable')) {\n element.focus();\n }\n\n var selection = window.getSelection();\n var range = document.createRange();\n\n range.selectNodeContents(element);\n selection.removeAllRanges();\n selection.addRange(range);\n\n selectedText = selection.toString();\n }\n\n return selectedText;\n}\n\nmodule.exports = select;\n\n\n/***/ }),\n\n/***/ 279:\n/***/ (function(module) {\n\nfunction E () {\n // Keep this empty so it's easier to inherit from\n // (via https://github.com/lipsmack from https://github.com/scottcorgan/tiny-emitter/issues/3)\n}\n\nE.prototype = {\n on: function (name, callback, ctx) {\n var e = this.e || (this.e = {});\n\n (e[name] || (e[name] = [])).push({\n fn: callback,\n ctx: ctx\n });\n\n return this;\n },\n\n once: function (name, callback, ctx) {\n var self = this;\n function listener () {\n self.off(name, listener);\n callback.apply(ctx, arguments);\n };\n\n listener._ = callback\n return this.on(name, listener, ctx);\n },\n\n emit: function (name) {\n var data = [].slice.call(arguments, 1);\n var evtArr = ((this.e || (this.e = {}))[name] || []).slice();\n var i = 0;\n var len = evtArr.length;\n\n for (i; i < len; i++) {\n evtArr[i].fn.apply(evtArr[i].ctx, data);\n }\n\n return this;\n },\n\n off: function (name, callback) {\n var e = this.e || (this.e = {});\n var evts = e[name];\n var liveEvents = [];\n\n if (evts && callback) {\n for (var i = 0, len = evts.length; i < len; i++) {\n if (evts[i].fn !== callback && evts[i].fn._ !== callback)\n liveEvents.push(evts[i]);\n }\n }\n\n // Remove event from queue to prevent memory leak\n // Suggested by https://github.com/lazd\n // Ref: https://github.com/scottcorgan/tiny-emitter/commit/c6ebfaa9bc973b33d110a84a307742b7cf94c953#commitcomment-5024910\n\n (liveEvents.length)\n ? e[name] = liveEvents\n : delete e[name];\n\n return this;\n }\n};\n\nmodule.exports = E;\nmodule.exports.TinyEmitter = E;\n\n\n/***/ })\n\n/******/ \t});\n/************************************************************************/\n/******/ \t// The module cache\n/******/ \tvar __webpack_module_cache__ = {};\n/******/ \t\n/******/ \t// The require function\n/******/ \tfunction __webpack_require__(moduleId) {\n/******/ \t\t// Check if module is in cache\n/******/ \t\tif(__webpack_module_cache__[moduleId]) {\n/******/ \t\t\treturn __webpack_module_cache__[moduleId].exports;\n/******/ \t\t}\n/******/ \t\t// Create a new module (and put it into the cache)\n/******/ \t\tvar module = __webpack_module_cache__[moduleId] = {\n/******/ \t\t\t// no module.id needed\n/******/ \t\t\t// no module.loaded needed\n/******/ \t\t\texports: {}\n/******/ \t\t};\n/******/ \t\n/******/ \t\t// Execute the module function\n/******/ \t\t__webpack_modules__[moduleId](module, module.exports, __webpack_require__);\n/******/ \t\n/******/ \t\t// Return the exports of the module\n/******/ \t\treturn module.exports;\n/******/ \t}\n/******/ \t\n/************************************************************************/\n/******/ \t/* webpack/runtime/compat get default export */\n/******/ \t!function() {\n/******/ \t\t// getDefaultExport function for compatibility with non-harmony modules\n/******/ \t\t__webpack_require__.n = function(module) {\n/******/ \t\t\tvar getter = module && module.__esModule ?\n/******/ \t\t\t\tfunction() { return module['default']; } :\n/******/ \t\t\t\tfunction() { return module; };\n/******/ \t\t\t__webpack_require__.d(getter, { a: getter });\n/******/ \t\t\treturn getter;\n/******/ \t\t};\n/******/ \t}();\n/******/ \t\n/******/ \t/* webpack/runtime/define property getters */\n/******/ \t!function() {\n/******/ \t\t// define getter functions for harmony exports\n/******/ \t\t__webpack_require__.d = function(exports, definition) {\n/******/ \t\t\tfor(var key in definition) {\n/******/ \t\t\t\tif(__webpack_require__.o(definition, key) && !__webpack_require__.o(exports, key)) {\n/******/ \t\t\t\t\tObject.defineProperty(exports, key, { enumerable: true, get: definition[key] });\n/******/ \t\t\t\t}\n/******/ \t\t\t}\n/******/ \t\t};\n/******/ \t}();\n/******/ \t\n/******/ \t/* webpack/runtime/hasOwnProperty shorthand */\n/******/ \t!function() {\n/******/ \t\t__webpack_require__.o = function(obj, prop) { return Object.prototype.hasOwnProperty.call(obj, prop); }\n/******/ \t}();\n/******/ \t\n/************************************************************************/\n/******/ \t// module exports must be returned from runtime so entry inlining is disabled\n/******/ \t// startup\n/******/ \t// Load entry module and return exports\n/******/ \treturn __webpack_require__(686);\n/******/ })()\n.default;\n});", "/*\n * Copyright (c) 2016-2024 Martin Donath \n *\n * Permission is hereby granted, free of charge, to any person obtaining a copy\n * of this software and associated documentation files (the \"Software\"), to\n * deal in the Software without restriction, including without limitation the\n * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or\n * sell copies of the Software, and to permit persons to whom the Software is\n * furnished to do so, subject to the following conditions:\n *\n * The above copyright notice and this permission notice shall be included in\n * all copies or substantial portions of the Software.\n *\n * THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE\n * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS\n * IN THE SOFTWARE.\n */\n\nimport \"focus-visible\"\n\nimport {\n EMPTY,\n NEVER,\n Observable,\n Subject,\n defer,\n delay,\n filter,\n map,\n merge,\n mergeWith,\n shareReplay,\n switchMap\n} from \"rxjs\"\n\nimport { configuration, feature } from \"./_\"\nimport {\n at,\n getActiveElement,\n getOptionalElement,\n requestJSON,\n setLocation,\n setToggle,\n watchDocument,\n watchKeyboard,\n watchLocation,\n watchLocationTarget,\n watchMedia,\n watchPrint,\n watchScript,\n watchViewport\n} from \"./browser\"\nimport {\n getComponentElement,\n getComponentElements,\n mountAnnounce,\n mountBackToTop,\n mountConsent,\n mountContent,\n mountDialog,\n mountHeader,\n mountHeaderTitle,\n mountPalette,\n mountProgress,\n mountSearch,\n mountSearchHiglight,\n mountSidebar,\n mountSource,\n mountTableOfContents,\n mountTabs,\n watchHeader,\n watchMain\n} from \"./components\"\nimport {\n SearchIndex,\n setupClipboardJS,\n setupInstantNavigation,\n setupVersionSelector\n} from \"./integrations\"\nimport {\n patchEllipsis,\n patchIndeterminate,\n patchScrollfix,\n patchScrolllock\n} from \"./patches\"\nimport \"./polyfills\"\n\n/* ----------------------------------------------------------------------------\n * Functions - @todo refactor\n * ------------------------------------------------------------------------- */\n\n/**\n * Fetch search index\n *\n * @returns Search index observable\n */\nfunction fetchSearchIndex(): Observable {\n if (location.protocol === \"file:\") {\n return watchScript(\n `${new URL(\"search/search_index.js\", config.base)}`\n )\n .pipe(\n // @ts-ignore - @todo fix typings\n map(() => __index),\n shareReplay(1)\n )\n } else {\n return requestJSON(\n new URL(\"search/search_index.json\", config.base)\n )\n }\n}\n\n/* ----------------------------------------------------------------------------\n * Application\n * ------------------------------------------------------------------------- */\n\n/* Yay, JavaScript is available */\ndocument.documentElement.classList.remove(\"no-js\")\ndocument.documentElement.classList.add(\"js\")\n\n/* Set up navigation observables and subjects */\nconst document$ = watchDocument()\nconst location$ = watchLocation()\nconst target$ = watchLocationTarget(location$)\nconst keyboard$ = watchKeyboard()\n\n/* Set up media observables */\nconst viewport$ = watchViewport()\nconst tablet$ = watchMedia(\"(min-width: 960px)\")\nconst screen$ = watchMedia(\"(min-width: 1220px)\")\nconst print$ = watchPrint()\n\n/* Retrieve search index, if search is enabled */\nconst config = configuration()\nconst index$ = document.forms.namedItem(\"search\")\n ? fetchSearchIndex()\n : NEVER\n\n/* Set up Clipboard.js integration */\nconst alert$ = new Subject()\nsetupClipboardJS({ alert$ })\n\n/* Set up progress indicator */\nconst progress$ = new Subject()\n\n/* Set up instant navigation, if enabled */\nif (feature(\"navigation.instant\"))\n setupInstantNavigation({ location$, viewport$, progress$ })\n .subscribe(document$)\n\n/* Set up version selector */\nif (config.version?.provider === \"mike\")\n setupVersionSelector({ document$ })\n\n/* Always close drawer and search on navigation */\nmerge(location$, target$)\n .pipe(\n delay(125)\n )\n .subscribe(() => {\n setToggle(\"drawer\", false)\n setToggle(\"search\", false)\n })\n\n/* Set up global keyboard handlers */\nkeyboard$\n .pipe(\n filter(({ mode }) => mode === \"global\")\n )\n .subscribe(key => {\n switch (key.type) {\n\n /* Go to previous page */\n case \"p\":\n case \",\":\n const prev = getOptionalElement(\"link[rel=prev]\")\n if (typeof prev !== \"undefined\")\n setLocation(prev)\n break\n\n /* Go to next page */\n case \"n\":\n case \".\":\n const next = getOptionalElement(\"link[rel=next]\")\n if (typeof next !== \"undefined\")\n setLocation(next)\n break\n\n /* Expand navigation, see https://bit.ly/3ZjG5io */\n case \"Enter\":\n const active = getActiveElement()\n if (active instanceof HTMLLabelElement)\n active.click()\n }\n })\n\n/* Set up patches */\npatchEllipsis({ viewport$, document$ })\npatchIndeterminate({ document$, tablet$ })\npatchScrollfix({ document$ })\npatchScrolllock({ viewport$, tablet$ })\n\n/* Set up header and main area observable */\nconst header$ = watchHeader(getComponentElement(\"header\"), { viewport$ })\nconst main$ = document$\n .pipe(\n map(() => getComponentElement(\"main\")),\n switchMap(el => watchMain(el, { viewport$, header$ })),\n shareReplay(1)\n )\n\n/* Set up control component observables */\nconst control$ = merge(\n\n /* Consent */\n ...getComponentElements(\"consent\")\n .map(el => mountConsent(el, { target$ })),\n\n /* Dialog */\n ...getComponentElements(\"dialog\")\n .map(el => mountDialog(el, { alert$ })),\n\n /* Color palette */\n ...getComponentElements(\"palette\")\n .map(el => mountPalette(el)),\n\n /* Progress bar */\n ...getComponentElements(\"progress\")\n .map(el => mountProgress(el, { progress$ })),\n\n /* Search */\n ...getComponentElements(\"search\")\n .map(el => mountSearch(el, { index$, keyboard$ })),\n\n /* Repository information */\n ...getComponentElements(\"source\")\n .map(el => mountSource(el))\n)\n\n/* Set up content component observables */\nconst content$ = defer(() => merge(\n\n /* Announcement bar */\n ...getComponentElements(\"announce\")\n .map(el => mountAnnounce(el)),\n\n /* Content */\n ...getComponentElements(\"content\")\n .map(el => mountContent(el, { viewport$, target$, print$ })),\n\n /* Search highlighting */\n ...getComponentElements(\"content\")\n .map(el => feature(\"search.highlight\")\n ? mountSearchHiglight(el, { index$, location$ })\n : EMPTY\n ),\n\n /* Header */\n ...getComponentElements(\"header\")\n .map(el => mountHeader(el, { viewport$, header$, main$ })),\n\n /* Header title */\n ...getComponentElements(\"header-title\")\n .map(el => mountHeaderTitle(el, { viewport$, header$ })),\n\n /* Sidebar */\n ...getComponentElements(\"sidebar\")\n .map(el => el.getAttribute(\"data-md-type\") === \"navigation\"\n ? at(screen$, () => mountSidebar(el, { viewport$, header$, main$ }))\n : at(tablet$, () => mountSidebar(el, { viewport$, header$, main$ }))\n ),\n\n /* Navigation tabs */\n ...getComponentElements(\"tabs\")\n .map(el => mountTabs(el, { viewport$, header$ })),\n\n /* Table of contents */\n ...getComponentElements(\"toc\")\n .map(el => mountTableOfContents(el, {\n viewport$, header$, main$, target$\n })),\n\n /* Back-to-top button */\n ...getComponentElements(\"top\")\n .map(el => mountBackToTop(el, { viewport$, header$, main$, target$ }))\n))\n\n/* Set up component observables */\nconst component$ = document$\n .pipe(\n switchMap(() => content$),\n mergeWith(control$),\n shareReplay(1)\n )\n\n/* Subscribe to all components */\ncomponent$.subscribe()\n\n/* ----------------------------------------------------------------------------\n * Exports\n * ------------------------------------------------------------------------- */\n\nwindow.document$ = document$ /* Document observable */\nwindow.location$ = location$ /* Location subject */\nwindow.target$ = target$ /* Location target observable */\nwindow.keyboard$ = keyboard$ /* Keyboard observable */\nwindow.viewport$ = viewport$ /* Viewport observable */\nwindow.tablet$ = tablet$ /* Media tablet observable */\nwindow.screen$ = screen$ /* Media screen observable */\nwindow.print$ = print$ /* Media print observable */\nwindow.alert$ = alert$ /* Alert subject */\nwindow.progress$ = progress$ /* Progress indicator subject */\nwindow.component$ = component$ /* Component observable */\n", "/******************************************************************************\nCopyright (c) Microsoft Corporation.\n\nPermission to use, copy, modify, and/or distribute this software for any\npurpose with or without fee is hereby granted.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH\nREGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY\nAND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,\nINDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM\nLOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR\nOTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR\nPERFORMANCE OF THIS SOFTWARE.\n***************************************************************************** */\n/* global Reflect, Promise, SuppressedError, Symbol, Iterator */\n\nvar extendStatics = function(d, b) {\n extendStatics = Object.setPrototypeOf ||\n ({ __proto__: [] } instanceof Array && function (d, b) { d.__proto__ = b; }) ||\n function (d, b) { for (var p in b) if (Object.prototype.hasOwnProperty.call(b, p)) d[p] = b[p]; };\n return extendStatics(d, b);\n};\n\nexport function __extends(d, b) {\n if (typeof b !== \"function\" && b !== null)\n throw new TypeError(\"Class extends value \" + String(b) + \" is not a constructor or null\");\n extendStatics(d, b);\n function __() { this.constructor = d; }\n d.prototype = b === null ? Object.create(b) : (__.prototype = b.prototype, new __());\n}\n\nexport var __assign = function() {\n __assign = Object.assign || function __assign(t) {\n for (var s, i = 1, n = arguments.length; i < n; i++) {\n s = arguments[i];\n for (var p in s) if (Object.prototype.hasOwnProperty.call(s, p)) t[p] = s[p];\n }\n return t;\n }\n return __assign.apply(this, arguments);\n}\n\nexport function __rest(s, e) {\n var t = {};\n for (var p in s) if (Object.prototype.hasOwnProperty.call(s, p) && e.indexOf(p) < 0)\n t[p] = s[p];\n if (s != null && typeof Object.getOwnPropertySymbols === \"function\")\n for (var i = 0, p = Object.getOwnPropertySymbols(s); i < p.length; i++) {\n if (e.indexOf(p[i]) < 0 && Object.prototype.propertyIsEnumerable.call(s, p[i]))\n t[p[i]] = s[p[i]];\n }\n return t;\n}\n\nexport function __decorate(decorators, target, key, desc) {\n var c = arguments.length, r = c < 3 ? target : desc === null ? desc = Object.getOwnPropertyDescriptor(target, key) : desc, d;\n if (typeof Reflect === \"object\" && typeof Reflect.decorate === \"function\") r = Reflect.decorate(decorators, target, key, desc);\n else for (var i = decorators.length - 1; i >= 0; i--) if (d = decorators[i]) r = (c < 3 ? d(r) : c > 3 ? d(target, key, r) : d(target, key)) || r;\n return c > 3 && r && Object.defineProperty(target, key, r), r;\n}\n\nexport function __param(paramIndex, decorator) {\n return function (target, key) { decorator(target, key, paramIndex); }\n}\n\nexport function __esDecorate(ctor, descriptorIn, decorators, contextIn, initializers, extraInitializers) {\n function accept(f) { if (f !== void 0 && typeof f !== \"function\") throw new TypeError(\"Function expected\"); return f; }\n var kind = contextIn.kind, key = kind === \"getter\" ? \"get\" : kind === \"setter\" ? \"set\" : \"value\";\n var target = !descriptorIn && ctor ? contextIn[\"static\"] ? ctor : ctor.prototype : null;\n var descriptor = descriptorIn || (target ? Object.getOwnPropertyDescriptor(target, contextIn.name) : {});\n var _, done = false;\n for (var i = decorators.length - 1; i >= 0; i--) {\n var context = {};\n for (var p in contextIn) context[p] = p === \"access\" ? {} : contextIn[p];\n for (var p in contextIn.access) context.access[p] = contextIn.access[p];\n context.addInitializer = function (f) { if (done) throw new TypeError(\"Cannot add initializers after decoration has completed\"); extraInitializers.push(accept(f || null)); };\n var result = (0, decorators[i])(kind === \"accessor\" ? { get: descriptor.get, set: descriptor.set } : descriptor[key], context);\n if (kind === \"accessor\") {\n if (result === void 0) continue;\n if (result === null || typeof result !== \"object\") throw new TypeError(\"Object expected\");\n if (_ = accept(result.get)) descriptor.get = _;\n if (_ = accept(result.set)) descriptor.set = _;\n if (_ = accept(result.init)) initializers.unshift(_);\n }\n else if (_ = accept(result)) {\n if (kind === \"field\") initializers.unshift(_);\n else descriptor[key] = _;\n }\n }\n if (target) Object.defineProperty(target, contextIn.name, descriptor);\n done = true;\n};\n\nexport function __runInitializers(thisArg, initializers, value) {\n var useValue = arguments.length > 2;\n for (var i = 0; i < initializers.length; i++) {\n value = useValue ? initializers[i].call(thisArg, value) : initializers[i].call(thisArg);\n }\n return useValue ? value : void 0;\n};\n\nexport function __propKey(x) {\n return typeof x === \"symbol\" ? x : \"\".concat(x);\n};\n\nexport function __setFunctionName(f, name, prefix) {\n if (typeof name === \"symbol\") name = name.description ? \"[\".concat(name.description, \"]\") : \"\";\n return Object.defineProperty(f, \"name\", { configurable: true, value: prefix ? \"\".concat(prefix, \" \", name) : name });\n};\n\nexport function __metadata(metadataKey, metadataValue) {\n if (typeof Reflect === \"object\" && typeof Reflect.metadata === \"function\") return Reflect.metadata(metadataKey, metadataValue);\n}\n\nexport function __awaiter(thisArg, _arguments, P, generator) {\n function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }\n return new (P || (P = Promise))(function (resolve, reject) {\n function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }\n function rejected(value) { try { step(generator[\"throw\"](value)); } catch (e) { reject(e); } }\n function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }\n step((generator = generator.apply(thisArg, _arguments || [])).next());\n });\n}\n\nexport function __generator(thisArg, body) {\n var _ = { label: 0, sent: function() { if (t[0] & 1) throw t[1]; return t[1]; }, trys: [], ops: [] }, f, y, t, g = Object.create((typeof Iterator === \"function\" ? Iterator : Object).prototype);\n return g.next = verb(0), g[\"throw\"] = verb(1), g[\"return\"] = verb(2), typeof Symbol === \"function\" && (g[Symbol.iterator] = function() { return this; }), g;\n function verb(n) { return function (v) { return step([n, v]); }; }\n function step(op) {\n if (f) throw new TypeError(\"Generator is already executing.\");\n while (g && (g = 0, op[0] && (_ = 0)), _) try {\n if (f = 1, y && (t = op[0] & 2 ? y[\"return\"] : op[0] ? y[\"throw\"] || ((t = y[\"return\"]) && t.call(y), 0) : y.next) && !(t = t.call(y, op[1])).done) return t;\n if (y = 0, t) op = [op[0] & 2, t.value];\n switch (op[0]) {\n case 0: case 1: t = op; break;\n case 4: _.label++; return { value: op[1], done: false };\n case 5: _.label++; y = op[1]; op = [0]; continue;\n case 7: op = _.ops.pop(); _.trys.pop(); continue;\n default:\n if (!(t = _.trys, t = t.length > 0 && t[t.length - 1]) && (op[0] === 6 || op[0] === 2)) { _ = 0; continue; }\n if (op[0] === 3 && (!t || (op[1] > t[0] && op[1] < t[3]))) { _.label = op[1]; break; }\n if (op[0] === 6 && _.label < t[1]) { _.label = t[1]; t = op; break; }\n if (t && _.label < t[2]) { _.label = t[2]; _.ops.push(op); break; }\n if (t[2]) _.ops.pop();\n _.trys.pop(); continue;\n }\n op = body.call(thisArg, _);\n } catch (e) { op = [6, e]; y = 0; } finally { f = t = 0; }\n if (op[0] & 5) throw op[1]; return { value: op[0] ? op[1] : void 0, done: true };\n }\n}\n\nexport var __createBinding = Object.create ? (function(o, m, k, k2) {\n if (k2 === undefined) k2 = k;\n var desc = Object.getOwnPropertyDescriptor(m, k);\n if (!desc || (\"get\" in desc ? !m.__esModule : desc.writable || desc.configurable)) {\n desc = { enumerable: true, get: function() { return m[k]; } };\n }\n Object.defineProperty(o, k2, desc);\n}) : (function(o, m, k, k2) {\n if (k2 === undefined) k2 = k;\n o[k2] = m[k];\n});\n\nexport function __exportStar(m, o) {\n for (var p in m) if (p !== \"default\" && !Object.prototype.hasOwnProperty.call(o, p)) __createBinding(o, m, p);\n}\n\nexport function __values(o) {\n var s = typeof Symbol === \"function\" && Symbol.iterator, m = s && o[s], i = 0;\n if (m) return m.call(o);\n if (o && typeof o.length === \"number\") return {\n next: function () {\n if (o && i >= o.length) o = void 0;\n return { value: o && o[i++], done: !o };\n }\n };\n throw new TypeError(s ? \"Object is not iterable.\" : \"Symbol.iterator is not defined.\");\n}\n\nexport function __read(o, n) {\n var m = typeof Symbol === \"function\" && o[Symbol.iterator];\n if (!m) return o;\n var i = m.call(o), r, ar = [], e;\n try {\n while ((n === void 0 || n-- > 0) && !(r = i.next()).done) ar.push(r.value);\n }\n catch (error) { e = { error: error }; }\n finally {\n try {\n if (r && !r.done && (m = i[\"return\"])) m.call(i);\n }\n finally { if (e) throw e.error; }\n }\n return ar;\n}\n\n/** @deprecated */\nexport function __spread() {\n for (var ar = [], i = 0; i < arguments.length; i++)\n ar = ar.concat(__read(arguments[i]));\n return ar;\n}\n\n/** @deprecated */\nexport function __spreadArrays() {\n for (var s = 0, i = 0, il = arguments.length; i < il; i++) s += arguments[i].length;\n for (var r = Array(s), k = 0, i = 0; i < il; i++)\n for (var a = arguments[i], j = 0, jl = a.length; j < jl; j++, k++)\n r[k] = a[j];\n return r;\n}\n\nexport function __spreadArray(to, from, pack) {\n if (pack || arguments.length === 2) for (var i = 0, l = from.length, ar; i < l; i++) {\n if (ar || !(i in from)) {\n if (!ar) ar = Array.prototype.slice.call(from, 0, i);\n ar[i] = from[i];\n }\n }\n return to.concat(ar || Array.prototype.slice.call(from));\n}\n\nexport function __await(v) {\n return this instanceof __await ? (this.v = v, this) : new __await(v);\n}\n\nexport function __asyncGenerator(thisArg, _arguments, generator) {\n if (!Symbol.asyncIterator) throw new TypeError(\"Symbol.asyncIterator is not defined.\");\n var g = generator.apply(thisArg, _arguments || []), i, q = [];\n return i = Object.create((typeof AsyncIterator === \"function\" ? AsyncIterator : Object).prototype), verb(\"next\"), verb(\"throw\"), verb(\"return\", awaitReturn), i[Symbol.asyncIterator] = function () { return this; }, i;\n function awaitReturn(f) { return function (v) { return Promise.resolve(v).then(f, reject); }; }\n function verb(n, f) { if (g[n]) { i[n] = function (v) { return new Promise(function (a, b) { q.push([n, v, a, b]) > 1 || resume(n, v); }); }; if (f) i[n] = f(i[n]); } }\n function resume(n, v) { try { step(g[n](v)); } catch (e) { settle(q[0][3], e); } }\n function step(r) { r.value instanceof __await ? Promise.resolve(r.value.v).then(fulfill, reject) : settle(q[0][2], r); }\n function fulfill(value) { resume(\"next\", value); }\n function reject(value) { resume(\"throw\", value); }\n function settle(f, v) { if (f(v), q.shift(), q.length) resume(q[0][0], q[0][1]); }\n}\n\nexport function __asyncDelegator(o) {\n var i, p;\n return i = {}, verb(\"next\"), verb(\"throw\", function (e) { throw e; }), verb(\"return\"), i[Symbol.iterator] = function () { return this; }, i;\n function verb(n, f) { i[n] = o[n] ? function (v) { return (p = !p) ? { value: __await(o[n](v)), done: false } : f ? f(v) : v; } : f; }\n}\n\nexport function __asyncValues(o) {\n if (!Symbol.asyncIterator) throw new TypeError(\"Symbol.asyncIterator is not defined.\");\n var m = o[Symbol.asyncIterator], i;\n return m ? m.call(o) : (o = typeof __values === \"function\" ? __values(o) : o[Symbol.iterator](), i = {}, verb(\"next\"), verb(\"throw\"), verb(\"return\"), i[Symbol.asyncIterator] = function () { return this; }, i);\n function verb(n) { i[n] = o[n] && function (v) { return new Promise(function (resolve, reject) { v = o[n](v), settle(resolve, reject, v.done, v.value); }); }; }\n function settle(resolve, reject, d, v) { Promise.resolve(v).then(function(v) { resolve({ value: v, done: d }); }, reject); }\n}\n\nexport function __makeTemplateObject(cooked, raw) {\n if (Object.defineProperty) { Object.defineProperty(cooked, \"raw\", { value: raw }); } else { cooked.raw = raw; }\n return cooked;\n};\n\nvar __setModuleDefault = Object.create ? (function(o, v) {\n Object.defineProperty(o, \"default\", { enumerable: true, value: v });\n}) : function(o, v) {\n o[\"default\"] = v;\n};\n\nexport function __importStar(mod) {\n if (mod && mod.__esModule) return mod;\n var result = {};\n if (mod != null) for (var k in mod) if (k !== \"default\" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);\n __setModuleDefault(result, mod);\n return result;\n}\n\nexport function __importDefault(mod) {\n return (mod && mod.__esModule) ? mod : { default: mod };\n}\n\nexport function __classPrivateFieldGet(receiver, state, kind, f) {\n if (kind === \"a\" && !f) throw new TypeError(\"Private accessor was defined without a getter\");\n if (typeof state === \"function\" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError(\"Cannot read private member from an object whose class did not declare it\");\n return kind === \"m\" ? f : kind === \"a\" ? f.call(receiver) : f ? f.value : state.get(receiver);\n}\n\nexport function __classPrivateFieldSet(receiver, state, value, kind, f) {\n if (kind === \"m\") throw new TypeError(\"Private method is not writable\");\n if (kind === \"a\" && !f) throw new TypeError(\"Private accessor was defined without a setter\");\n if (typeof state === \"function\" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError(\"Cannot write private member to an object whose class did not declare it\");\n return (kind === \"a\" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;\n}\n\nexport function __classPrivateFieldIn(state, receiver) {\n if (receiver === null || (typeof receiver !== \"object\" && typeof receiver !== \"function\")) throw new TypeError(\"Cannot use 'in' operator on non-object\");\n return typeof state === \"function\" ? receiver === state : state.has(receiver);\n}\n\nexport function __addDisposableResource(env, value, async) {\n if (value !== null && value !== void 0) {\n if (typeof value !== \"object\" && typeof value !== \"function\") throw new TypeError(\"Object expected.\");\n var dispose, inner;\n if (async) {\n if (!Symbol.asyncDispose) throw new TypeError(\"Symbol.asyncDispose is not defined.\");\n dispose = value[Symbol.asyncDispose];\n }\n if (dispose === void 0) {\n if (!Symbol.dispose) throw new TypeError(\"Symbol.dispose is not defined.\");\n dispose = value[Symbol.dispose];\n if (async) inner = dispose;\n }\n if (typeof dispose !== \"function\") throw new TypeError(\"Object not disposable.\");\n if (inner) dispose = function() { try { inner.call(this); } catch (e) { return Promise.reject(e); } };\n env.stack.push({ value: value, dispose: dispose, async: async });\n }\n else if (async) {\n env.stack.push({ async: true });\n }\n return value;\n}\n\nvar _SuppressedError = typeof SuppressedError === \"function\" ? SuppressedError : function (error, suppressed, message) {\n var e = new Error(message);\n return e.name = \"SuppressedError\", e.error = error, e.suppressed = suppressed, e;\n};\n\nexport function __disposeResources(env) {\n function fail(e) {\n env.error = env.hasError ? new _SuppressedError(e, env.error, \"An error was suppressed during disposal.\") : e;\n env.hasError = true;\n }\n var r, s = 0;\n function next() {\n while (r = env.stack.pop()) {\n try {\n if (!r.async && s === 1) return s = 0, env.stack.push(r), Promise.resolve().then(next);\n if (r.dispose) {\n var result = r.dispose.call(r.value);\n if (r.async) return s |= 2, Promise.resolve(result).then(next, function(e) { fail(e); return next(); });\n }\n else s |= 1;\n }\n catch (e) {\n fail(e);\n }\n }\n if (s === 1) return env.hasError ? Promise.reject(env.error) : Promise.resolve();\n if (env.hasError) throw env.error;\n }\n return next();\n}\n\nexport default {\n __extends,\n __assign,\n __rest,\n __decorate,\n __param,\n __metadata,\n __awaiter,\n __generator,\n __createBinding,\n __exportStar,\n __values,\n __read,\n __spread,\n __spreadArrays,\n __spreadArray,\n __await,\n __asyncGenerator,\n __asyncDelegator,\n __asyncValues,\n __makeTemplateObject,\n __importStar,\n __importDefault,\n __classPrivateFieldGet,\n __classPrivateFieldSet,\n __classPrivateFieldIn,\n __addDisposableResource,\n __disposeResources,\n};\n", "/**\n * Returns true if the object is a function.\n * @param value The value to check\n */\nexport function isFunction(value: any): value is (...args: any[]) => any {\n return typeof value === 'function';\n}\n", "/**\n * Used to create Error subclasses until the community moves away from ES5.\n *\n * This is because compiling from TypeScript down to ES5 has issues with subclassing Errors\n * as well as other built-in types: https://github.com/Microsoft/TypeScript/issues/12123\n *\n * @param createImpl A factory function to create the actual constructor implementation. The returned\n * function should be a named function that calls `_super` internally.\n */\nexport function createErrorClass(createImpl: (_super: any) => any): T {\n const _super = (instance: any) => {\n Error.call(instance);\n instance.stack = new Error().stack;\n };\n\n const ctorFunc = createImpl(_super);\n ctorFunc.prototype = Object.create(Error.prototype);\n ctorFunc.prototype.constructor = ctorFunc;\n return ctorFunc;\n}\n", "import { createErrorClass } from './createErrorClass';\n\nexport interface UnsubscriptionError extends Error {\n readonly errors: any[];\n}\n\nexport interface UnsubscriptionErrorCtor {\n /**\n * @deprecated Internal implementation detail. Do not construct error instances.\n * Cannot be tagged as internal: https://github.com/ReactiveX/rxjs/issues/6269\n */\n new (errors: any[]): UnsubscriptionError;\n}\n\n/**\n * An error thrown when one or more errors have occurred during the\n * `unsubscribe` of a {@link Subscription}.\n */\nexport const UnsubscriptionError: UnsubscriptionErrorCtor = createErrorClass(\n (_super) =>\n function UnsubscriptionErrorImpl(this: any, errors: (Error | string)[]) {\n _super(this);\n this.message = errors\n ? `${errors.length} errors occurred during unsubscription:\n${errors.map((err, i) => `${i + 1}) ${err.toString()}`).join('\\n ')}`\n : '';\n this.name = 'UnsubscriptionError';\n this.errors = errors;\n }\n);\n", "/**\n * Removes an item from an array, mutating it.\n * @param arr The array to remove the item from\n * @param item The item to remove\n */\nexport function arrRemove(arr: T[] | undefined | null, item: T) {\n if (arr) {\n const index = arr.indexOf(item);\n 0 <= index && arr.splice(index, 1);\n }\n}\n", "import { isFunction } from './util/isFunction';\nimport { UnsubscriptionError } from './util/UnsubscriptionError';\nimport { SubscriptionLike, TeardownLogic, Unsubscribable } from './types';\nimport { arrRemove } from './util/arrRemove';\n\n/**\n * Represents a disposable resource, such as the execution of an Observable. A\n * Subscription has one important method, `unsubscribe`, that takes no argument\n * and just disposes the resource held by the subscription.\n *\n * Additionally, subscriptions may be grouped together through the `add()`\n * method, which will attach a child Subscription to the current Subscription.\n * When a Subscription is unsubscribed, all its children (and its grandchildren)\n * will be unsubscribed as well.\n *\n * @class Subscription\n */\nexport class Subscription implements SubscriptionLike {\n /** @nocollapse */\n public static EMPTY = (() => {\n const empty = new Subscription();\n empty.closed = true;\n return empty;\n })();\n\n /**\n * A flag to indicate whether this Subscription has already been unsubscribed.\n */\n public closed = false;\n\n private _parentage: Subscription[] | Subscription | null = null;\n\n /**\n * The list of registered finalizers to execute upon unsubscription. Adding and removing from this\n * list occurs in the {@link #add} and {@link #remove} methods.\n */\n private _finalizers: Exclude[] | null = null;\n\n /**\n * @param initialTeardown A function executed first as part of the finalization\n * process that is kicked off when {@link #unsubscribe} is called.\n */\n constructor(private initialTeardown?: () => void) {}\n\n /**\n * Disposes the resources held by the subscription. May, for instance, cancel\n * an ongoing Observable execution or cancel any other type of work that\n * started when the Subscription was created.\n * @return {void}\n */\n unsubscribe(): void {\n let errors: any[] | undefined;\n\n if (!this.closed) {\n this.closed = true;\n\n // Remove this from it's parents.\n const { _parentage } = this;\n if (_parentage) {\n this._parentage = null;\n if (Array.isArray(_parentage)) {\n for (const parent of _parentage) {\n parent.remove(this);\n }\n } else {\n _parentage.remove(this);\n }\n }\n\n const { initialTeardown: initialFinalizer } = this;\n if (isFunction(initialFinalizer)) {\n try {\n initialFinalizer();\n } catch (e) {\n errors = e instanceof UnsubscriptionError ? e.errors : [e];\n }\n }\n\n const { _finalizers } = this;\n if (_finalizers) {\n this._finalizers = null;\n for (const finalizer of _finalizers) {\n try {\n execFinalizer(finalizer);\n } catch (err) {\n errors = errors ?? [];\n if (err instanceof UnsubscriptionError) {\n errors = [...errors, ...err.errors];\n } else {\n errors.push(err);\n }\n }\n }\n }\n\n if (errors) {\n throw new UnsubscriptionError(errors);\n }\n }\n }\n\n /**\n * Adds a finalizer to this subscription, so that finalization will be unsubscribed/called\n * when this subscription is unsubscribed. If this subscription is already {@link #closed},\n * because it has already been unsubscribed, then whatever finalizer is passed to it\n * will automatically be executed (unless the finalizer itself is also a closed subscription).\n *\n * Closed Subscriptions cannot be added as finalizers to any subscription. Adding a closed\n * subscription to a any subscription will result in no operation. (A noop).\n *\n * Adding a subscription to itself, or adding `null` or `undefined` will not perform any\n * operation at all. (A noop).\n *\n * `Subscription` instances that are added to this instance will automatically remove themselves\n * if they are unsubscribed. Functions and {@link Unsubscribable} objects that you wish to remove\n * will need to be removed manually with {@link #remove}\n *\n * @param teardown The finalization logic to add to this subscription.\n */\n add(teardown: TeardownLogic): void {\n // Only add the finalizer if it's not undefined\n // and don't add a subscription to itself.\n if (teardown && teardown !== this) {\n if (this.closed) {\n // If this subscription is already closed,\n // execute whatever finalizer is handed to it automatically.\n execFinalizer(teardown);\n } else {\n if (teardown instanceof Subscription) {\n // We don't add closed subscriptions, and we don't add the same subscription\n // twice. Subscription unsubscribe is idempotent.\n if (teardown.closed || teardown._hasParent(this)) {\n return;\n }\n teardown._addParent(this);\n }\n (this._finalizers = this._finalizers ?? []).push(teardown);\n }\n }\n }\n\n /**\n * Checks to see if a this subscription already has a particular parent.\n * This will signal that this subscription has already been added to the parent in question.\n * @param parent the parent to check for\n */\n private _hasParent(parent: Subscription) {\n const { _parentage } = this;\n return _parentage === parent || (Array.isArray(_parentage) && _parentage.includes(parent));\n }\n\n /**\n * Adds a parent to this subscription so it can be removed from the parent if it\n * unsubscribes on it's own.\n *\n * NOTE: THIS ASSUMES THAT {@link _hasParent} HAS ALREADY BEEN CHECKED.\n * @param parent The parent subscription to add\n */\n private _addParent(parent: Subscription) {\n const { _parentage } = this;\n this._parentage = Array.isArray(_parentage) ? (_parentage.push(parent), _parentage) : _parentage ? [_parentage, parent] : parent;\n }\n\n /**\n * Called on a child when it is removed via {@link #remove}.\n * @param parent The parent to remove\n */\n private _removeParent(parent: Subscription) {\n const { _parentage } = this;\n if (_parentage === parent) {\n this._parentage = null;\n } else if (Array.isArray(_parentage)) {\n arrRemove(_parentage, parent);\n }\n }\n\n /**\n * Removes a finalizer from this subscription that was previously added with the {@link #add} method.\n *\n * Note that `Subscription` instances, when unsubscribed, will automatically remove themselves\n * from every other `Subscription` they have been added to. This means that using the `remove` method\n * is not a common thing and should be used thoughtfully.\n *\n * If you add the same finalizer instance of a function or an unsubscribable object to a `Subscription` instance\n * more than once, you will need to call `remove` the same number of times to remove all instances.\n *\n * All finalizer instances are removed to free up memory upon unsubscription.\n *\n * @param teardown The finalizer to remove from this subscription\n */\n remove(teardown: Exclude): void {\n const { _finalizers } = this;\n _finalizers && arrRemove(_finalizers, teardown);\n\n if (teardown instanceof Subscription) {\n teardown._removeParent(this);\n }\n }\n}\n\nexport const EMPTY_SUBSCRIPTION = Subscription.EMPTY;\n\nexport function isSubscription(value: any): value is Subscription {\n return (\n value instanceof Subscription ||\n (value && 'closed' in value && isFunction(value.remove) && isFunction(value.add) && isFunction(value.unsubscribe))\n );\n}\n\nfunction execFinalizer(finalizer: Unsubscribable | (() => void)) {\n if (isFunction(finalizer)) {\n finalizer();\n } else {\n finalizer.unsubscribe();\n }\n}\n", "import { Subscriber } from './Subscriber';\nimport { ObservableNotification } from './types';\n\n/**\n * The {@link GlobalConfig} object for RxJS. It is used to configure things\n * like how to react on unhandled errors.\n */\nexport const config: GlobalConfig = {\n onUnhandledError: null,\n onStoppedNotification: null,\n Promise: undefined,\n useDeprecatedSynchronousErrorHandling: false,\n useDeprecatedNextContext: false,\n};\n\n/**\n * The global configuration object for RxJS, used to configure things\n * like how to react on unhandled errors. Accessible via {@link config}\n * object.\n */\nexport interface GlobalConfig {\n /**\n * A registration point for unhandled errors from RxJS. These are errors that\n * cannot were not handled by consuming code in the usual subscription path. For\n * example, if you have this configured, and you subscribe to an observable without\n * providing an error handler, errors from that subscription will end up here. This\n * will _always_ be called asynchronously on another job in the runtime. This is because\n * we do not want errors thrown in this user-configured handler to interfere with the\n * behavior of the library.\n */\n onUnhandledError: ((err: any) => void) | null;\n\n /**\n * A registration point for notifications that cannot be sent to subscribers because they\n * have completed, errored or have been explicitly unsubscribed. By default, next, complete\n * and error notifications sent to stopped subscribers are noops. However, sometimes callers\n * might want a different behavior. For example, with sources that attempt to report errors\n * to stopped subscribers, a caller can configure RxJS to throw an unhandled error instead.\n * This will _always_ be called asynchronously on another job in the runtime. This is because\n * we do not want errors thrown in this user-configured handler to interfere with the\n * behavior of the library.\n */\n onStoppedNotification: ((notification: ObservableNotification, subscriber: Subscriber) => void) | null;\n\n /**\n * The promise constructor used by default for {@link Observable#toPromise toPromise} and {@link Observable#forEach forEach}\n * methods.\n *\n * @deprecated As of version 8, RxJS will no longer support this sort of injection of a\n * Promise constructor. If you need a Promise implementation other than native promises,\n * please polyfill/patch Promise as you see appropriate. Will be removed in v8.\n */\n Promise?: PromiseConstructorLike;\n\n /**\n * If true, turns on synchronous error rethrowing, which is a deprecated behavior\n * in v6 and higher. This behavior enables bad patterns like wrapping a subscribe\n * call in a try/catch block. It also enables producer interference, a nasty bug\n * where a multicast can be broken for all observers by a downstream consumer with\n * an unhandled error. DO NOT USE THIS FLAG UNLESS IT'S NEEDED TO BUY TIME\n * FOR MIGRATION REASONS.\n *\n * @deprecated As of version 8, RxJS will no longer support synchronous throwing\n * of unhandled errors. All errors will be thrown on a separate call stack to prevent bad\n * behaviors described above. Will be removed in v8.\n */\n useDeprecatedSynchronousErrorHandling: boolean;\n\n /**\n * If true, enables an as-of-yet undocumented feature from v5: The ability to access\n * `unsubscribe()` via `this` context in `next` functions created in observers passed\n * to `subscribe`.\n *\n * This is being removed because the performance was severely problematic, and it could also cause\n * issues when types other than POJOs are passed to subscribe as subscribers, as they will likely have\n * their `this` context overwritten.\n *\n * @deprecated As of version 8, RxJS will no longer support altering the\n * context of next functions provided as part of an observer to Subscribe. Instead,\n * you will have access to a subscription or a signal or token that will allow you to do things like\n * unsubscribe and test closed status. Will be removed in v8.\n */\n useDeprecatedNextContext: boolean;\n}\n", "import type { TimerHandle } from './timerHandle';\ntype SetTimeoutFunction = (handler: () => void, timeout?: number, ...args: any[]) => TimerHandle;\ntype ClearTimeoutFunction = (handle: TimerHandle) => void;\n\ninterface TimeoutProvider {\n setTimeout: SetTimeoutFunction;\n clearTimeout: ClearTimeoutFunction;\n delegate:\n | {\n setTimeout: SetTimeoutFunction;\n clearTimeout: ClearTimeoutFunction;\n }\n | undefined;\n}\n\nexport const timeoutProvider: TimeoutProvider = {\n // When accessing the delegate, use the variable rather than `this` so that\n // the functions can be called without being bound to the provider.\n setTimeout(handler: () => void, timeout?: number, ...args) {\n const { delegate } = timeoutProvider;\n if (delegate?.setTimeout) {\n return delegate.setTimeout(handler, timeout, ...args);\n }\n return setTimeout(handler, timeout, ...args);\n },\n clearTimeout(handle) {\n const { delegate } = timeoutProvider;\n return (delegate?.clearTimeout || clearTimeout)(handle as any);\n },\n delegate: undefined,\n};\n", "import { config } from '../config';\nimport { timeoutProvider } from '../scheduler/timeoutProvider';\n\n/**\n * Handles an error on another job either with the user-configured {@link onUnhandledError},\n * or by throwing it on that new job so it can be picked up by `window.onerror`, `process.on('error')`, etc.\n *\n * This should be called whenever there is an error that is out-of-band with the subscription\n * or when an error hits a terminal boundary of the subscription and no error handler was provided.\n *\n * @param err the error to report\n */\nexport function reportUnhandledError(err: any) {\n timeoutProvider.setTimeout(() => {\n const { onUnhandledError } = config;\n if (onUnhandledError) {\n // Execute the user-configured error handler.\n onUnhandledError(err);\n } else {\n // Throw so it is picked up by the runtime's uncaught error mechanism.\n throw err;\n }\n });\n}\n", "/* tslint:disable:no-empty */\nexport function noop() { }\n", "import { CompleteNotification, NextNotification, ErrorNotification } from './types';\n\n/**\n * A completion object optimized for memory use and created to be the\n * same \"shape\" as other notifications in v8.\n * @internal\n */\nexport const COMPLETE_NOTIFICATION = (() => createNotification('C', undefined, undefined) as CompleteNotification)();\n\n/**\n * Internal use only. Creates an optimized error notification that is the same \"shape\"\n * as other notifications.\n * @internal\n */\nexport function errorNotification(error: any): ErrorNotification {\n return createNotification('E', undefined, error) as any;\n}\n\n/**\n * Internal use only. Creates an optimized next notification that is the same \"shape\"\n * as other notifications.\n * @internal\n */\nexport function nextNotification(value: T) {\n return createNotification('N', value, undefined) as NextNotification;\n}\n\n/**\n * Ensures that all notifications created internally have the same \"shape\" in v8.\n *\n * TODO: This is only exported to support a crazy legacy test in `groupBy`.\n * @internal\n */\nexport function createNotification(kind: 'N' | 'E' | 'C', value: any, error: any) {\n return {\n kind,\n value,\n error,\n };\n}\n", "import { config } from '../config';\n\nlet context: { errorThrown: boolean; error: any } | null = null;\n\n/**\n * Handles dealing with errors for super-gross mode. Creates a context, in which\n * any synchronously thrown errors will be passed to {@link captureError}. Which\n * will record the error such that it will be rethrown after the call back is complete.\n * TODO: Remove in v8\n * @param cb An immediately executed function.\n */\nexport function errorContext(cb: () => void) {\n if (config.useDeprecatedSynchronousErrorHandling) {\n const isRoot = !context;\n if (isRoot) {\n context = { errorThrown: false, error: null };\n }\n cb();\n if (isRoot) {\n const { errorThrown, error } = context!;\n context = null;\n if (errorThrown) {\n throw error;\n }\n }\n } else {\n // This is the general non-deprecated path for everyone that\n // isn't crazy enough to use super-gross mode (useDeprecatedSynchronousErrorHandling)\n cb();\n }\n}\n\n/**\n * Captures errors only in super-gross mode.\n * @param err the error to capture\n */\nexport function captureError(err: any) {\n if (config.useDeprecatedSynchronousErrorHandling && context) {\n context.errorThrown = true;\n context.error = err;\n }\n}\n", "import { isFunction } from './util/isFunction';\nimport { Observer, ObservableNotification } from './types';\nimport { isSubscription, Subscription } from './Subscription';\nimport { config } from './config';\nimport { reportUnhandledError } from './util/reportUnhandledError';\nimport { noop } from './util/noop';\nimport { nextNotification, errorNotification, COMPLETE_NOTIFICATION } from './NotificationFactories';\nimport { timeoutProvider } from './scheduler/timeoutProvider';\nimport { captureError } from './util/errorContext';\n\n/**\n * Implements the {@link Observer} interface and extends the\n * {@link Subscription} class. While the {@link Observer} is the public API for\n * consuming the values of an {@link Observable}, all Observers get converted to\n * a Subscriber, in order to provide Subscription-like capabilities such as\n * `unsubscribe`. Subscriber is a common type in RxJS, and crucial for\n * implementing operators, but it is rarely used as a public API.\n *\n * @class Subscriber\n */\nexport class Subscriber extends Subscription implements Observer {\n /**\n * A static factory for a Subscriber, given a (potentially partial) definition\n * of an Observer.\n * @param next The `next` callback of an Observer.\n * @param error The `error` callback of an\n * Observer.\n * @param complete The `complete` callback of an\n * Observer.\n * @return A Subscriber wrapping the (partially defined)\n * Observer represented by the given arguments.\n * @nocollapse\n * @deprecated Do not use. Will be removed in v8. There is no replacement for this\n * method, and there is no reason to be creating instances of `Subscriber` directly.\n * If you have a specific use case, please file an issue.\n */\n static create(next?: (x?: T) => void, error?: (e?: any) => void, complete?: () => void): Subscriber {\n return new SafeSubscriber(next, error, complete);\n }\n\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n protected isStopped: boolean = false;\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n protected destination: Subscriber | Observer; // this `any` is the escape hatch to erase extra type param (e.g. R)\n\n /**\n * @deprecated Internal implementation detail, do not use directly. Will be made internal in v8.\n * There is no reason to directly create an instance of Subscriber. This type is exported for typings reasons.\n */\n constructor(destination?: Subscriber | Observer) {\n super();\n if (destination) {\n this.destination = destination;\n // Automatically chain subscriptions together here.\n // if destination is a Subscription, then it is a Subscriber.\n if (isSubscription(destination)) {\n destination.add(this);\n }\n } else {\n this.destination = EMPTY_OBSERVER;\n }\n }\n\n /**\n * The {@link Observer} callback to receive notifications of type `next` from\n * the Observable, with a value. The Observable may call this method 0 or more\n * times.\n * @param {T} [value] The `next` value.\n * @return {void}\n */\n next(value?: T): void {\n if (this.isStopped) {\n handleStoppedNotification(nextNotification(value), this);\n } else {\n this._next(value!);\n }\n }\n\n /**\n * The {@link Observer} callback to receive notifications of type `error` from\n * the Observable, with an attached `Error`. Notifies the Observer that\n * the Observable has experienced an error condition.\n * @param {any} [err] The `error` exception.\n * @return {void}\n */\n error(err?: any): void {\n if (this.isStopped) {\n handleStoppedNotification(errorNotification(err), this);\n } else {\n this.isStopped = true;\n this._error(err);\n }\n }\n\n /**\n * The {@link Observer} callback to receive a valueless notification of type\n * `complete` from the Observable. Notifies the Observer that the Observable\n * has finished sending push-based notifications.\n * @return {void}\n */\n complete(): void {\n if (this.isStopped) {\n handleStoppedNotification(COMPLETE_NOTIFICATION, this);\n } else {\n this.isStopped = true;\n this._complete();\n }\n }\n\n unsubscribe(): void {\n if (!this.closed) {\n this.isStopped = true;\n super.unsubscribe();\n this.destination = null!;\n }\n }\n\n protected _next(value: T): void {\n this.destination.next(value);\n }\n\n protected _error(err: any): void {\n try {\n this.destination.error(err);\n } finally {\n this.unsubscribe();\n }\n }\n\n protected _complete(): void {\n try {\n this.destination.complete();\n } finally {\n this.unsubscribe();\n }\n }\n}\n\n/**\n * This bind is captured here because we want to be able to have\n * compatibility with monoid libraries that tend to use a method named\n * `bind`. In particular, a library called Monio requires this.\n */\nconst _bind = Function.prototype.bind;\n\nfunction bind any>(fn: Fn, thisArg: any): Fn {\n return _bind.call(fn, thisArg);\n}\n\n/**\n * Internal optimization only, DO NOT EXPOSE.\n * @internal\n */\nclass ConsumerObserver implements Observer {\n constructor(private partialObserver: Partial>) {}\n\n next(value: T): void {\n const { partialObserver } = this;\n if (partialObserver.next) {\n try {\n partialObserver.next(value);\n } catch (error) {\n handleUnhandledError(error);\n }\n }\n }\n\n error(err: any): void {\n const { partialObserver } = this;\n if (partialObserver.error) {\n try {\n partialObserver.error(err);\n } catch (error) {\n handleUnhandledError(error);\n }\n } else {\n handleUnhandledError(err);\n }\n }\n\n complete(): void {\n const { partialObserver } = this;\n if (partialObserver.complete) {\n try {\n partialObserver.complete();\n } catch (error) {\n handleUnhandledError(error);\n }\n }\n }\n}\n\nexport class SafeSubscriber extends Subscriber {\n constructor(\n observerOrNext?: Partial> | ((value: T) => void) | null,\n error?: ((e?: any) => void) | null,\n complete?: (() => void) | null\n ) {\n super();\n\n let partialObserver: Partial>;\n if (isFunction(observerOrNext) || !observerOrNext) {\n // The first argument is a function, not an observer. The next\n // two arguments *could* be observers, or they could be empty.\n partialObserver = {\n next: (observerOrNext ?? undefined) as (((value: T) => void) | undefined),\n error: error ?? undefined,\n complete: complete ?? undefined,\n };\n } else {\n // The first argument is a partial observer.\n let context: any;\n if (this && config.useDeprecatedNextContext) {\n // This is a deprecated path that made `this.unsubscribe()` available in\n // next handler functions passed to subscribe. This only exists behind a flag\n // now, as it is *very* slow.\n context = Object.create(observerOrNext);\n context.unsubscribe = () => this.unsubscribe();\n partialObserver = {\n next: observerOrNext.next && bind(observerOrNext.next, context),\n error: observerOrNext.error && bind(observerOrNext.error, context),\n complete: observerOrNext.complete && bind(observerOrNext.complete, context),\n };\n } else {\n // The \"normal\" path. Just use the partial observer directly.\n partialObserver = observerOrNext;\n }\n }\n\n // Wrap the partial observer to ensure it's a full observer, and\n // make sure proper error handling is accounted for.\n this.destination = new ConsumerObserver(partialObserver);\n }\n}\n\nfunction handleUnhandledError(error: any) {\n if (config.useDeprecatedSynchronousErrorHandling) {\n captureError(error);\n } else {\n // Ideal path, we report this as an unhandled error,\n // which is thrown on a new call stack.\n reportUnhandledError(error);\n }\n}\n\n/**\n * An error handler used when no error handler was supplied\n * to the SafeSubscriber -- meaning no error handler was supplied\n * do the `subscribe` call on our observable.\n * @param err The error to handle\n */\nfunction defaultErrorHandler(err: any) {\n throw err;\n}\n\n/**\n * A handler for notifications that cannot be sent to a stopped subscriber.\n * @param notification The notification being sent\n * @param subscriber The stopped subscriber\n */\nfunction handleStoppedNotification(notification: ObservableNotification, subscriber: Subscriber) {\n const { onStoppedNotification } = config;\n onStoppedNotification && timeoutProvider.setTimeout(() => onStoppedNotification(notification, subscriber));\n}\n\n/**\n * The observer used as a stub for subscriptions where the user did not\n * pass any arguments to `subscribe`. Comes with the default error handling\n * behavior.\n */\nexport const EMPTY_OBSERVER: Readonly> & { closed: true } = {\n closed: true,\n next: noop,\n error: defaultErrorHandler,\n complete: noop,\n};\n", "/**\n * Symbol.observable or a string \"@@observable\". Used for interop\n *\n * @deprecated We will no longer be exporting this symbol in upcoming versions of RxJS.\n * Instead polyfill and use Symbol.observable directly *or* use https://www.npmjs.com/package/symbol-observable\n */\nexport const observable: string | symbol = (() => (typeof Symbol === 'function' && Symbol.observable) || '@@observable')();\n", "/**\n * This function takes one parameter and just returns it. Simply put,\n * this is like `(x: T): T => x`.\n *\n * ## Examples\n *\n * This is useful in some cases when using things like `mergeMap`\n *\n * ```ts\n * import { interval, take, map, range, mergeMap, identity } from 'rxjs';\n *\n * const source$ = interval(1000).pipe(take(5));\n *\n * const result$ = source$.pipe(\n * map(i => range(i)),\n * mergeMap(identity) // same as mergeMap(x => x)\n * );\n *\n * result$.subscribe({\n * next: console.log\n * });\n * ```\n *\n * Or when you want to selectively apply an operator\n *\n * ```ts\n * import { interval, take, identity } from 'rxjs';\n *\n * const shouldLimit = () => Math.random() < 0.5;\n *\n * const source$ = interval(1000);\n *\n * const result$ = source$.pipe(shouldLimit() ? take(5) : identity);\n *\n * result$.subscribe({\n * next: console.log\n * });\n * ```\n *\n * @param x Any value that is returned by this function\n * @returns The value passed as the first parameter to this function\n */\nexport function identity(x: T): T {\n return x;\n}\n", "import { identity } from './identity';\nimport { UnaryFunction } from '../types';\n\nexport function pipe(): typeof identity;\nexport function pipe(fn1: UnaryFunction): UnaryFunction;\nexport function pipe(fn1: UnaryFunction, fn2: UnaryFunction): UnaryFunction;\nexport function pipe(fn1: UnaryFunction, fn2: UnaryFunction, fn3: UnaryFunction): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction,\n fn6: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction,\n fn6: UnaryFunction,\n fn7: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction,\n fn6: UnaryFunction,\n fn7: UnaryFunction,\n fn8: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction,\n fn6: UnaryFunction,\n fn7: UnaryFunction,\n fn8: UnaryFunction,\n fn9: UnaryFunction\n): UnaryFunction;\nexport function pipe(\n fn1: UnaryFunction,\n fn2: UnaryFunction,\n fn3: UnaryFunction,\n fn4: UnaryFunction,\n fn5: UnaryFunction,\n fn6: UnaryFunction,\n fn7: UnaryFunction,\n fn8: UnaryFunction,\n fn9: UnaryFunction,\n ...fns: UnaryFunction[]\n): UnaryFunction;\n\n/**\n * pipe() can be called on one or more functions, each of which can take one argument (\"UnaryFunction\")\n * and uses it to return a value.\n * It returns a function that takes one argument, passes it to the first UnaryFunction, and then\n * passes the result to the next one, passes that result to the next one, and so on. \n */\nexport function pipe(...fns: Array>): UnaryFunction {\n return pipeFromArray(fns);\n}\n\n/** @internal */\nexport function pipeFromArray(fns: Array>): UnaryFunction {\n if (fns.length === 0) {\n return identity as UnaryFunction;\n }\n\n if (fns.length === 1) {\n return fns[0];\n }\n\n return function piped(input: T): R {\n return fns.reduce((prev: any, fn: UnaryFunction) => fn(prev), input as any);\n };\n}\n", "import { Operator } from './Operator';\nimport { SafeSubscriber, Subscriber } from './Subscriber';\nimport { isSubscription, Subscription } from './Subscription';\nimport { TeardownLogic, OperatorFunction, Subscribable, Observer } from './types';\nimport { observable as Symbol_observable } from './symbol/observable';\nimport { pipeFromArray } from './util/pipe';\nimport { config } from './config';\nimport { isFunction } from './util/isFunction';\nimport { errorContext } from './util/errorContext';\n\n/**\n * A representation of any set of values over any amount of time. This is the most basic building block\n * of RxJS.\n *\n * @class Observable\n */\nexport class Observable implements Subscribable {\n /**\n * @deprecated Internal implementation detail, do not use directly. Will be made internal in v8.\n */\n source: Observable | undefined;\n\n /**\n * @deprecated Internal implementation detail, do not use directly. Will be made internal in v8.\n */\n operator: Operator | undefined;\n\n /**\n * @constructor\n * @param {Function} subscribe the function that is called when the Observable is\n * initially subscribed to. This function is given a Subscriber, to which new values\n * can be `next`ed, or an `error` method can be called to raise an error, or\n * `complete` can be called to notify of a successful completion.\n */\n constructor(subscribe?: (this: Observable, subscriber: Subscriber) => TeardownLogic) {\n if (subscribe) {\n this._subscribe = subscribe;\n }\n }\n\n // HACK: Since TypeScript inherits static properties too, we have to\n // fight against TypeScript here so Subject can have a different static create signature\n /**\n * Creates a new Observable by calling the Observable constructor\n * @owner Observable\n * @method create\n * @param {Function} subscribe? the subscriber function to be passed to the Observable constructor\n * @return {Observable} a new observable\n * @nocollapse\n * @deprecated Use `new Observable()` instead. Will be removed in v8.\n */\n static create: (...args: any[]) => any = (subscribe?: (subscriber: Subscriber) => TeardownLogic) => {\n return new Observable(subscribe);\n };\n\n /**\n * Creates a new Observable, with this Observable instance as the source, and the passed\n * operator defined as the new observable's operator.\n * @method lift\n * @param operator the operator defining the operation to take on the observable\n * @return a new observable with the Operator applied\n * @deprecated Internal implementation detail, do not use directly. Will be made internal in v8.\n * If you have implemented an operator using `lift`, it is recommended that you create an\n * operator by simply returning `new Observable()` directly. See \"Creating new operators from\n * scratch\" section here: https://rxjs.dev/guide/operators\n */\n lift(operator?: Operator): Observable {\n const observable = new Observable();\n observable.source = this;\n observable.operator = operator;\n return observable;\n }\n\n subscribe(observerOrNext?: Partial> | ((value: T) => void)): Subscription;\n /** @deprecated Instead of passing separate callback arguments, use an observer argument. Signatures taking separate callback arguments will be removed in v8. Details: https://rxjs.dev/deprecations/subscribe-arguments */\n subscribe(next?: ((value: T) => void) | null, error?: ((error: any) => void) | null, complete?: (() => void) | null): Subscription;\n /**\n * Invokes an execution of an Observable and registers Observer handlers for notifications it will emit.\n *\n * Use it when you have all these Observables, but still nothing is happening.\n *\n * `subscribe` is not a regular operator, but a method that calls Observable's internal `subscribe` function. It\n * might be for example a function that you passed to Observable's constructor, but most of the time it is\n * a library implementation, which defines what will be emitted by an Observable, and when it be will emitted. This means\n * that calling `subscribe` is actually the moment when Observable starts its work, not when it is created, as it is often\n * the thought.\n *\n * Apart from starting the execution of an Observable, this method allows you to listen for values\n * that an Observable emits, as well as for when it completes or errors. You can achieve this in two\n * of the following ways.\n *\n * The first way is creating an object that implements {@link Observer} interface. It should have methods\n * defined by that interface, but note that it should be just a regular JavaScript object, which you can create\n * yourself in any way you want (ES6 class, classic function constructor, object literal etc.). In particular, do\n * not attempt to use any RxJS implementation details to create Observers - you don't need them. Remember also\n * that your object does not have to implement all methods. If you find yourself creating a method that doesn't\n * do anything, you can simply omit it. Note however, if the `error` method is not provided and an error happens,\n * it will be thrown asynchronously. Errors thrown asynchronously cannot be caught using `try`/`catch`. Instead,\n * use the {@link onUnhandledError} configuration option or use a runtime handler (like `window.onerror` or\n * `process.on('error)`) to be notified of unhandled errors. Because of this, it's recommended that you provide\n * an `error` method to avoid missing thrown errors.\n *\n * The second way is to give up on Observer object altogether and simply provide callback functions in place of its methods.\n * This means you can provide three functions as arguments to `subscribe`, where the first function is equivalent\n * of a `next` method, the second of an `error` method and the third of a `complete` method. Just as in case of an Observer,\n * if you do not need to listen for something, you can omit a function by passing `undefined` or `null`,\n * since `subscribe` recognizes these functions by where they were placed in function call. When it comes\n * to the `error` function, as with an Observer, if not provided, errors emitted by an Observable will be thrown asynchronously.\n *\n * You can, however, subscribe with no parameters at all. This may be the case where you're not interested in terminal events\n * and you also handled emissions internally by using operators (e.g. using `tap`).\n *\n * Whichever style of calling `subscribe` you use, in both cases it returns a Subscription object.\n * This object allows you to call `unsubscribe` on it, which in turn will stop the work that an Observable does and will clean\n * up all resources that an Observable used. Note that cancelling a subscription will not call `complete` callback\n * provided to `subscribe` function, which is reserved for a regular completion signal that comes from an Observable.\n *\n * Remember that callbacks provided to `subscribe` are not guaranteed to be called asynchronously.\n * It is an Observable itself that decides when these functions will be called. For example {@link of}\n * by default emits all its values synchronously. Always check documentation for how given Observable\n * will behave when subscribed and if its default behavior can be modified with a `scheduler`.\n *\n * #### Examples\n *\n * Subscribe with an {@link guide/observer Observer}\n *\n * ```ts\n * import { of } from 'rxjs';\n *\n * const sumObserver = {\n * sum: 0,\n * next(value) {\n * console.log('Adding: ' + value);\n * this.sum = this.sum + value;\n * },\n * error() {\n * // We actually could just remove this method,\n * // since we do not really care about errors right now.\n * },\n * complete() {\n * console.log('Sum equals: ' + this.sum);\n * }\n * };\n *\n * of(1, 2, 3) // Synchronously emits 1, 2, 3 and then completes.\n * .subscribe(sumObserver);\n *\n * // Logs:\n * // 'Adding: 1'\n * // 'Adding: 2'\n * // 'Adding: 3'\n * // 'Sum equals: 6'\n * ```\n *\n * Subscribe with functions ({@link deprecations/subscribe-arguments deprecated})\n *\n * ```ts\n * import { of } from 'rxjs'\n *\n * let sum = 0;\n *\n * of(1, 2, 3).subscribe(\n * value => {\n * console.log('Adding: ' + value);\n * sum = sum + value;\n * },\n * undefined,\n * () => console.log('Sum equals: ' + sum)\n * );\n *\n * // Logs:\n * // 'Adding: 1'\n * // 'Adding: 2'\n * // 'Adding: 3'\n * // 'Sum equals: 6'\n * ```\n *\n * Cancel a subscription\n *\n * ```ts\n * import { interval } from 'rxjs';\n *\n * const subscription = interval(1000).subscribe({\n * next(num) {\n * console.log(num)\n * },\n * complete() {\n * // Will not be called, even when cancelling subscription.\n * console.log('completed!');\n * }\n * });\n *\n * setTimeout(() => {\n * subscription.unsubscribe();\n * console.log('unsubscribed!');\n * }, 2500);\n *\n * // Logs:\n * // 0 after 1s\n * // 1 after 2s\n * // 'unsubscribed!' after 2.5s\n * ```\n *\n * @param {Observer|Function} observerOrNext (optional) Either an observer with methods to be called,\n * or the first of three possible handlers, which is the handler for each value emitted from the subscribed\n * Observable.\n * @param {Function} error (optional) A handler for a terminal event resulting from an error. If no error handler is provided,\n * the error will be thrown asynchronously as unhandled.\n * @param {Function} complete (optional) A handler for a terminal event resulting from successful completion.\n * @return {Subscription} a subscription reference to the registered handlers\n * @method subscribe\n */\n subscribe(\n observerOrNext?: Partial> | ((value: T) => void) | null,\n error?: ((error: any) => void) | null,\n complete?: (() => void) | null\n ): Subscription {\n const subscriber = isSubscriber(observerOrNext) ? observerOrNext : new SafeSubscriber(observerOrNext, error, complete);\n\n errorContext(() => {\n const { operator, source } = this;\n subscriber.add(\n operator\n ? // We're dealing with a subscription in the\n // operator chain to one of our lifted operators.\n operator.call(subscriber, source)\n : source\n ? // If `source` has a value, but `operator` does not, something that\n // had intimate knowledge of our API, like our `Subject`, must have\n // set it. We're going to just call `_subscribe` directly.\n this._subscribe(subscriber)\n : // In all other cases, we're likely wrapping a user-provided initializer\n // function, so we need to catch errors and handle them appropriately.\n this._trySubscribe(subscriber)\n );\n });\n\n return subscriber;\n }\n\n /** @internal */\n protected _trySubscribe(sink: Subscriber): TeardownLogic {\n try {\n return this._subscribe(sink);\n } catch (err) {\n // We don't need to return anything in this case,\n // because it's just going to try to `add()` to a subscription\n // above.\n sink.error(err);\n }\n }\n\n /**\n * Used as a NON-CANCELLABLE means of subscribing to an observable, for use with\n * APIs that expect promises, like `async/await`. You cannot unsubscribe from this.\n *\n * **WARNING**: Only use this with observables you *know* will complete. If the source\n * observable does not complete, you will end up with a promise that is hung up, and\n * potentially all of the state of an async function hanging out in memory. To avoid\n * this situation, look into adding something like {@link timeout}, {@link take},\n * {@link takeWhile}, or {@link takeUntil} amongst others.\n *\n * #### Example\n *\n * ```ts\n * import { interval, take } from 'rxjs';\n *\n * const source$ = interval(1000).pipe(take(4));\n *\n * async function getTotal() {\n * let total = 0;\n *\n * await source$.forEach(value => {\n * total += value;\n * console.log('observable -> ' + value);\n * });\n *\n * return total;\n * }\n *\n * getTotal().then(\n * total => console.log('Total: ' + total)\n * );\n *\n * // Expected:\n * // 'observable -> 0'\n * // 'observable -> 1'\n * // 'observable -> 2'\n * // 'observable -> 3'\n * // 'Total: 6'\n * ```\n *\n * @param next a handler for each value emitted by the observable\n * @return a promise that either resolves on observable completion or\n * rejects with the handled error\n */\n forEach(next: (value: T) => void): Promise;\n\n /**\n * @param next a handler for each value emitted by the observable\n * @param promiseCtor a constructor function used to instantiate the Promise\n * @return a promise that either resolves on observable completion or\n * rejects with the handled error\n * @deprecated Passing a Promise constructor will no longer be available\n * in upcoming versions of RxJS. This is because it adds weight to the library, for very\n * little benefit. If you need this functionality, it is recommended that you either\n * polyfill Promise, or you create an adapter to convert the returned native promise\n * to whatever promise implementation you wanted. Will be removed in v8.\n */\n forEach(next: (value: T) => void, promiseCtor: PromiseConstructorLike): Promise;\n\n forEach(next: (value: T) => void, promiseCtor?: PromiseConstructorLike): Promise {\n promiseCtor = getPromiseCtor(promiseCtor);\n\n return new promiseCtor((resolve, reject) => {\n const subscriber = new SafeSubscriber({\n next: (value) => {\n try {\n next(value);\n } catch (err) {\n reject(err);\n subscriber.unsubscribe();\n }\n },\n error: reject,\n complete: resolve,\n });\n this.subscribe(subscriber);\n }) as Promise;\n }\n\n /** @internal */\n protected _subscribe(subscriber: Subscriber): TeardownLogic {\n return this.source?.subscribe(subscriber);\n }\n\n /**\n * An interop point defined by the es7-observable spec https://github.com/zenparsing/es-observable\n * @method Symbol.observable\n * @return {Observable} this instance of the observable\n */\n [Symbol_observable]() {\n return this;\n }\n\n /* tslint:disable:max-line-length */\n pipe(): Observable;\n pipe(op1: OperatorFunction): Observable;\n pipe(op1: OperatorFunction, op2: OperatorFunction): Observable;\n pipe(op1: OperatorFunction, op2: OperatorFunction, op3: OperatorFunction): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction,\n op6: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction,\n op6: OperatorFunction,\n op7: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction,\n op6: OperatorFunction,\n op7: OperatorFunction,\n op8: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction,\n op6: OperatorFunction,\n op7: OperatorFunction,\n op8: OperatorFunction,\n op9: OperatorFunction\n ): Observable;\n pipe(\n op1: OperatorFunction,\n op2: OperatorFunction,\n op3: OperatorFunction,\n op4: OperatorFunction,\n op5: OperatorFunction,\n op6: OperatorFunction,\n op7: OperatorFunction,\n op8: OperatorFunction,\n op9: OperatorFunction,\n ...operations: OperatorFunction[]\n ): Observable;\n /* tslint:enable:max-line-length */\n\n /**\n * Used to stitch together functional operators into a chain.\n * @method pipe\n * @return {Observable} the Observable result of all of the operators having\n * been called in the order they were passed in.\n *\n * ## Example\n *\n * ```ts\n * import { interval, filter, map, scan } from 'rxjs';\n *\n * interval(1000)\n * .pipe(\n * filter(x => x % 2 === 0),\n * map(x => x + x),\n * scan((acc, x) => acc + x)\n * )\n * .subscribe(x => console.log(x));\n * ```\n */\n pipe(...operations: OperatorFunction[]): Observable {\n return pipeFromArray(operations)(this);\n }\n\n /* tslint:disable:max-line-length */\n /** @deprecated Replaced with {@link firstValueFrom} and {@link lastValueFrom}. Will be removed in v8. Details: https://rxjs.dev/deprecations/to-promise */\n toPromise(): Promise;\n /** @deprecated Replaced with {@link firstValueFrom} and {@link lastValueFrom}. Will be removed in v8. Details: https://rxjs.dev/deprecations/to-promise */\n toPromise(PromiseCtor: typeof Promise): Promise;\n /** @deprecated Replaced with {@link firstValueFrom} and {@link lastValueFrom}. Will be removed in v8. Details: https://rxjs.dev/deprecations/to-promise */\n toPromise(PromiseCtor: PromiseConstructorLike): Promise;\n /* tslint:enable:max-line-length */\n\n /**\n * Subscribe to this Observable and get a Promise resolving on\n * `complete` with the last emission (if any).\n *\n * **WARNING**: Only use this with observables you *know* will complete. If the source\n * observable does not complete, you will end up with a promise that is hung up, and\n * potentially all of the state of an async function hanging out in memory. To avoid\n * this situation, look into adding something like {@link timeout}, {@link take},\n * {@link takeWhile}, or {@link takeUntil} amongst others.\n *\n * @method toPromise\n * @param [promiseCtor] a constructor function used to instantiate\n * the Promise\n * @return A Promise that resolves with the last value emit, or\n * rejects on an error. If there were no emissions, Promise\n * resolves with undefined.\n * @deprecated Replaced with {@link firstValueFrom} and {@link lastValueFrom}. Will be removed in v8. Details: https://rxjs.dev/deprecations/to-promise\n */\n toPromise(promiseCtor?: PromiseConstructorLike): Promise {\n promiseCtor = getPromiseCtor(promiseCtor);\n\n return new promiseCtor((resolve, reject) => {\n let value: T | undefined;\n this.subscribe(\n (x: T) => (value = x),\n (err: any) => reject(err),\n () => resolve(value)\n );\n }) as Promise;\n }\n}\n\n/**\n * Decides between a passed promise constructor from consuming code,\n * A default configured promise constructor, and the native promise\n * constructor and returns it. If nothing can be found, it will throw\n * an error.\n * @param promiseCtor The optional promise constructor to passed by consuming code\n */\nfunction getPromiseCtor(promiseCtor: PromiseConstructorLike | undefined) {\n return promiseCtor ?? config.Promise ?? Promise;\n}\n\nfunction isObserver(value: any): value is Observer {\n return value && isFunction(value.next) && isFunction(value.error) && isFunction(value.complete);\n}\n\nfunction isSubscriber(value: any): value is Subscriber {\n return (value && value instanceof Subscriber) || (isObserver(value) && isSubscription(value));\n}\n", "import { Observable } from '../Observable';\nimport { Subscriber } from '../Subscriber';\nimport { OperatorFunction } from '../types';\nimport { isFunction } from './isFunction';\n\n/**\n * Used to determine if an object is an Observable with a lift function.\n */\nexport function hasLift(source: any): source is { lift: InstanceType['lift'] } {\n return isFunction(source?.lift);\n}\n\n/**\n * Creates an `OperatorFunction`. Used to define operators throughout the library in a concise way.\n * @param init The logic to connect the liftedSource to the subscriber at the moment of subscription.\n */\nexport function operate(\n init: (liftedSource: Observable, subscriber: Subscriber) => (() => void) | void\n): OperatorFunction {\n return (source: Observable) => {\n if (hasLift(source)) {\n return source.lift(function (this: Subscriber, liftedSource: Observable) {\n try {\n return init(liftedSource, this);\n } catch (err) {\n this.error(err);\n }\n });\n }\n throw new TypeError('Unable to lift unknown Observable type');\n };\n}\n", "import { Subscriber } from '../Subscriber';\n\n/**\n * Creates an instance of an `OperatorSubscriber`.\n * @param destination The downstream subscriber.\n * @param onNext Handles next values, only called if this subscriber is not stopped or closed. Any\n * error that occurs in this function is caught and sent to the `error` method of this subscriber.\n * @param onError Handles errors from the subscription, any errors that occur in this handler are caught\n * and send to the `destination` error handler.\n * @param onComplete Handles completion notification from the subscription. Any errors that occur in\n * this handler are sent to the `destination` error handler.\n * @param onFinalize Additional teardown logic here. This will only be called on teardown if the\n * subscriber itself is not already closed. This is called after all other teardown logic is executed.\n */\nexport function createOperatorSubscriber(\n destination: Subscriber,\n onNext?: (value: T) => void,\n onComplete?: () => void,\n onError?: (err: any) => void,\n onFinalize?: () => void\n): Subscriber {\n return new OperatorSubscriber(destination, onNext, onComplete, onError, onFinalize);\n}\n\n/**\n * A generic helper for allowing operators to be created with a Subscriber and\n * use closures to capture necessary state from the operator function itself.\n */\nexport class OperatorSubscriber extends Subscriber {\n /**\n * Creates an instance of an `OperatorSubscriber`.\n * @param destination The downstream subscriber.\n * @param onNext Handles next values, only called if this subscriber is not stopped or closed. Any\n * error that occurs in this function is caught and sent to the `error` method of this subscriber.\n * @param onError Handles errors from the subscription, any errors that occur in this handler are caught\n * and send to the `destination` error handler.\n * @param onComplete Handles completion notification from the subscription. Any errors that occur in\n * this handler are sent to the `destination` error handler.\n * @param onFinalize Additional finalization logic here. This will only be called on finalization if the\n * subscriber itself is not already closed. This is called after all other finalization logic is executed.\n * @param shouldUnsubscribe An optional check to see if an unsubscribe call should truly unsubscribe.\n * NOTE: This currently **ONLY** exists to support the strange behavior of {@link groupBy}, where unsubscription\n * to the resulting observable does not actually disconnect from the source if there are active subscriptions\n * to any grouped observable. (DO NOT EXPOSE OR USE EXTERNALLY!!!)\n */\n constructor(\n destination: Subscriber,\n onNext?: (value: T) => void,\n onComplete?: () => void,\n onError?: (err: any) => void,\n private onFinalize?: () => void,\n private shouldUnsubscribe?: () => boolean\n ) {\n // It's important - for performance reasons - that all of this class's\n // members are initialized and that they are always initialized in the same\n // order. This will ensure that all OperatorSubscriber instances have the\n // same hidden class in V8. This, in turn, will help keep the number of\n // hidden classes involved in property accesses within the base class as\n // low as possible. If the number of hidden classes involved exceeds four,\n // the property accesses will become megamorphic and performance penalties\n // will be incurred - i.e. inline caches won't be used.\n //\n // The reasons for ensuring all instances have the same hidden class are\n // further discussed in this blog post from Benedikt Meurer:\n // https://benediktmeurer.de/2018/03/23/impact-of-polymorphism-on-component-based-frameworks-like-react/\n super(destination);\n this._next = onNext\n ? function (this: OperatorSubscriber, value: T) {\n try {\n onNext(value);\n } catch (err) {\n destination.error(err);\n }\n }\n : super._next;\n this._error = onError\n ? function (this: OperatorSubscriber, err: any) {\n try {\n onError(err);\n } catch (err) {\n // Send any errors that occur down stream.\n destination.error(err);\n } finally {\n // Ensure finalization.\n this.unsubscribe();\n }\n }\n : super._error;\n this._complete = onComplete\n ? function (this: OperatorSubscriber) {\n try {\n onComplete();\n } catch (err) {\n // Send any errors that occur down stream.\n destination.error(err);\n } finally {\n // Ensure finalization.\n this.unsubscribe();\n }\n }\n : super._complete;\n }\n\n unsubscribe() {\n if (!this.shouldUnsubscribe || this.shouldUnsubscribe()) {\n const { closed } = this;\n super.unsubscribe();\n // Execute additional teardown if we have any and we didn't already do so.\n !closed && this.onFinalize?.();\n }\n }\n}\n", "import { Subscription } from '../Subscription';\n\ninterface AnimationFrameProvider {\n schedule(callback: FrameRequestCallback): Subscription;\n requestAnimationFrame: typeof requestAnimationFrame;\n cancelAnimationFrame: typeof cancelAnimationFrame;\n delegate:\n | {\n requestAnimationFrame: typeof requestAnimationFrame;\n cancelAnimationFrame: typeof cancelAnimationFrame;\n }\n | undefined;\n}\n\nexport const animationFrameProvider: AnimationFrameProvider = {\n // When accessing the delegate, use the variable rather than `this` so that\n // the functions can be called without being bound to the provider.\n schedule(callback) {\n let request = requestAnimationFrame;\n let cancel: typeof cancelAnimationFrame | undefined = cancelAnimationFrame;\n const { delegate } = animationFrameProvider;\n if (delegate) {\n request = delegate.requestAnimationFrame;\n cancel = delegate.cancelAnimationFrame;\n }\n const handle = request((timestamp) => {\n // Clear the cancel function. The request has been fulfilled, so\n // attempting to cancel the request upon unsubscription would be\n // pointless.\n cancel = undefined;\n callback(timestamp);\n });\n return new Subscription(() => cancel?.(handle));\n },\n requestAnimationFrame(...args) {\n const { delegate } = animationFrameProvider;\n return (delegate?.requestAnimationFrame || requestAnimationFrame)(...args);\n },\n cancelAnimationFrame(...args) {\n const { delegate } = animationFrameProvider;\n return (delegate?.cancelAnimationFrame || cancelAnimationFrame)(...args);\n },\n delegate: undefined,\n};\n", "import { createErrorClass } from './createErrorClass';\n\nexport interface ObjectUnsubscribedError extends Error {}\n\nexport interface ObjectUnsubscribedErrorCtor {\n /**\n * @deprecated Internal implementation detail. Do not construct error instances.\n * Cannot be tagged as internal: https://github.com/ReactiveX/rxjs/issues/6269\n */\n new (): ObjectUnsubscribedError;\n}\n\n/**\n * An error thrown when an action is invalid because the object has been\n * unsubscribed.\n *\n * @see {@link Subject}\n * @see {@link BehaviorSubject}\n *\n * @class ObjectUnsubscribedError\n */\nexport const ObjectUnsubscribedError: ObjectUnsubscribedErrorCtor = createErrorClass(\n (_super) =>\n function ObjectUnsubscribedErrorImpl(this: any) {\n _super(this);\n this.name = 'ObjectUnsubscribedError';\n this.message = 'object unsubscribed';\n }\n);\n", "import { Operator } from './Operator';\nimport { Observable } from './Observable';\nimport { Subscriber } from './Subscriber';\nimport { Subscription, EMPTY_SUBSCRIPTION } from './Subscription';\nimport { Observer, SubscriptionLike, TeardownLogic } from './types';\nimport { ObjectUnsubscribedError } from './util/ObjectUnsubscribedError';\nimport { arrRemove } from './util/arrRemove';\nimport { errorContext } from './util/errorContext';\n\n/**\n * A Subject is a special type of Observable that allows values to be\n * multicasted to many Observers. Subjects are like EventEmitters.\n *\n * Every Subject is an Observable and an Observer. You can subscribe to a\n * Subject, and you can call next to feed values as well as error and complete.\n */\nexport class Subject extends Observable implements SubscriptionLike {\n closed = false;\n\n private currentObservers: Observer[] | null = null;\n\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n observers: Observer[] = [];\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n isStopped = false;\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n hasError = false;\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n thrownError: any = null;\n\n /**\n * Creates a \"subject\" by basically gluing an observer to an observable.\n *\n * @nocollapse\n * @deprecated Recommended you do not use. Will be removed at some point in the future. Plans for replacement still under discussion.\n */\n static create: (...args: any[]) => any = (destination: Observer, source: Observable): AnonymousSubject => {\n return new AnonymousSubject(destination, source);\n };\n\n constructor() {\n // NOTE: This must be here to obscure Observable's constructor.\n super();\n }\n\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n lift(operator: Operator): Observable {\n const subject = new AnonymousSubject(this, this);\n subject.operator = operator as any;\n return subject as any;\n }\n\n /** @internal */\n protected _throwIfClosed() {\n if (this.closed) {\n throw new ObjectUnsubscribedError();\n }\n }\n\n next(value: T) {\n errorContext(() => {\n this._throwIfClosed();\n if (!this.isStopped) {\n if (!this.currentObservers) {\n this.currentObservers = Array.from(this.observers);\n }\n for (const observer of this.currentObservers) {\n observer.next(value);\n }\n }\n });\n }\n\n error(err: any) {\n errorContext(() => {\n this._throwIfClosed();\n if (!this.isStopped) {\n this.hasError = this.isStopped = true;\n this.thrownError = err;\n const { observers } = this;\n while (observers.length) {\n observers.shift()!.error(err);\n }\n }\n });\n }\n\n complete() {\n errorContext(() => {\n this._throwIfClosed();\n if (!this.isStopped) {\n this.isStopped = true;\n const { observers } = this;\n while (observers.length) {\n observers.shift()!.complete();\n }\n }\n });\n }\n\n unsubscribe() {\n this.isStopped = this.closed = true;\n this.observers = this.currentObservers = null!;\n }\n\n get observed() {\n return this.observers?.length > 0;\n }\n\n /** @internal */\n protected _trySubscribe(subscriber: Subscriber): TeardownLogic {\n this._throwIfClosed();\n return super._trySubscribe(subscriber);\n }\n\n /** @internal */\n protected _subscribe(subscriber: Subscriber): Subscription {\n this._throwIfClosed();\n this._checkFinalizedStatuses(subscriber);\n return this._innerSubscribe(subscriber);\n }\n\n /** @internal */\n protected _innerSubscribe(subscriber: Subscriber) {\n const { hasError, isStopped, observers } = this;\n if (hasError || isStopped) {\n return EMPTY_SUBSCRIPTION;\n }\n this.currentObservers = null;\n observers.push(subscriber);\n return new Subscription(() => {\n this.currentObservers = null;\n arrRemove(observers, subscriber);\n });\n }\n\n /** @internal */\n protected _checkFinalizedStatuses(subscriber: Subscriber) {\n const { hasError, thrownError, isStopped } = this;\n if (hasError) {\n subscriber.error(thrownError);\n } else if (isStopped) {\n subscriber.complete();\n }\n }\n\n /**\n * Creates a new Observable with this Subject as the source. You can do this\n * to create custom Observer-side logic of the Subject and conceal it from\n * code that uses the Observable.\n * @return {Observable} Observable that the Subject casts to\n */\n asObservable(): Observable {\n const observable: any = new Observable();\n observable.source = this;\n return observable;\n }\n}\n\n/**\n * @class AnonymousSubject\n */\nexport class AnonymousSubject extends Subject {\n constructor(\n /** @deprecated Internal implementation detail, do not use directly. Will be made internal in v8. */\n public destination?: Observer,\n source?: Observable\n ) {\n super();\n this.source = source;\n }\n\n next(value: T) {\n this.destination?.next?.(value);\n }\n\n error(err: any) {\n this.destination?.error?.(err);\n }\n\n complete() {\n this.destination?.complete?.();\n }\n\n /** @internal */\n protected _subscribe(subscriber: Subscriber): Subscription {\n return this.source?.subscribe(subscriber) ?? EMPTY_SUBSCRIPTION;\n }\n}\n", "import { Subject } from './Subject';\nimport { Subscriber } from './Subscriber';\nimport { Subscription } from './Subscription';\n\n/**\n * A variant of Subject that requires an initial value and emits its current\n * value whenever it is subscribed to.\n *\n * @class BehaviorSubject\n */\nexport class BehaviorSubject extends Subject {\n constructor(private _value: T) {\n super();\n }\n\n get value(): T {\n return this.getValue();\n }\n\n /** @internal */\n protected _subscribe(subscriber: Subscriber): Subscription {\n const subscription = super._subscribe(subscriber);\n !subscription.closed && subscriber.next(this._value);\n return subscription;\n }\n\n getValue(): T {\n const { hasError, thrownError, _value } = this;\n if (hasError) {\n throw thrownError;\n }\n this._throwIfClosed();\n return _value;\n }\n\n next(value: T): void {\n super.next((this._value = value));\n }\n}\n", "import { TimestampProvider } from '../types';\n\ninterface DateTimestampProvider extends TimestampProvider {\n delegate: TimestampProvider | undefined;\n}\n\nexport const dateTimestampProvider: DateTimestampProvider = {\n now() {\n // Use the variable rather than `this` so that the function can be called\n // without being bound to the provider.\n return (dateTimestampProvider.delegate || Date).now();\n },\n delegate: undefined,\n};\n", "import { Subject } from './Subject';\nimport { TimestampProvider } from './types';\nimport { Subscriber } from './Subscriber';\nimport { Subscription } from './Subscription';\nimport { dateTimestampProvider } from './scheduler/dateTimestampProvider';\n\n/**\n * A variant of {@link Subject} that \"replays\" old values to new subscribers by emitting them when they first subscribe.\n *\n * `ReplaySubject` has an internal buffer that will store a specified number of values that it has observed. Like `Subject`,\n * `ReplaySubject` \"observes\" values by having them passed to its `next` method. When it observes a value, it will store that\n * value for a time determined by the configuration of the `ReplaySubject`, as passed to its constructor.\n *\n * When a new subscriber subscribes to the `ReplaySubject` instance, it will synchronously emit all values in its buffer in\n * a First-In-First-Out (FIFO) manner. The `ReplaySubject` will also complete, if it has observed completion; and it will\n * error if it has observed an error.\n *\n * There are two main configuration items to be concerned with:\n *\n * 1. `bufferSize` - This will determine how many items are stored in the buffer, defaults to infinite.\n * 2. `windowTime` - The amount of time to hold a value in the buffer before removing it from the buffer.\n *\n * Both configurations may exist simultaneously. So if you would like to buffer a maximum of 3 values, as long as the values\n * are less than 2 seconds old, you could do so with a `new ReplaySubject(3, 2000)`.\n *\n * ### Differences with BehaviorSubject\n *\n * `BehaviorSubject` is similar to `new ReplaySubject(1)`, with a couple of exceptions:\n *\n * 1. `BehaviorSubject` comes \"primed\" with a single value upon construction.\n * 2. `ReplaySubject` will replay values, even after observing an error, where `BehaviorSubject` will not.\n *\n * @see {@link Subject}\n * @see {@link BehaviorSubject}\n * @see {@link shareReplay}\n */\nexport class ReplaySubject extends Subject {\n private _buffer: (T | number)[] = [];\n private _infiniteTimeWindow = true;\n\n /**\n * @param bufferSize The size of the buffer to replay on subscription\n * @param windowTime The amount of time the buffered items will stay buffered\n * @param timestampProvider An object with a `now()` method that provides the current timestamp. This is used to\n * calculate the amount of time something has been buffered.\n */\n constructor(\n private _bufferSize = Infinity,\n private _windowTime = Infinity,\n private _timestampProvider: TimestampProvider = dateTimestampProvider\n ) {\n super();\n this._infiniteTimeWindow = _windowTime === Infinity;\n this._bufferSize = Math.max(1, _bufferSize);\n this._windowTime = Math.max(1, _windowTime);\n }\n\n next(value: T): void {\n const { isStopped, _buffer, _infiniteTimeWindow, _timestampProvider, _windowTime } = this;\n if (!isStopped) {\n _buffer.push(value);\n !_infiniteTimeWindow && _buffer.push(_timestampProvider.now() + _windowTime);\n }\n this._trimBuffer();\n super.next(value);\n }\n\n /** @internal */\n protected _subscribe(subscriber: Subscriber): Subscription {\n this._throwIfClosed();\n this._trimBuffer();\n\n const subscription = this._innerSubscribe(subscriber);\n\n const { _infiniteTimeWindow, _buffer } = this;\n // We use a copy here, so reentrant code does not mutate our array while we're\n // emitting it to a new subscriber.\n const copy = _buffer.slice();\n for (let i = 0; i < copy.length && !subscriber.closed; i += _infiniteTimeWindow ? 1 : 2) {\n subscriber.next(copy[i] as T);\n }\n\n this._checkFinalizedStatuses(subscriber);\n\n return subscription;\n }\n\n private _trimBuffer() {\n const { _bufferSize, _timestampProvider, _buffer, _infiniteTimeWindow } = this;\n // If we don't have an infinite buffer size, and we're over the length,\n // use splice to truncate the old buffer values off. Note that we have to\n // double the size for instances where we're not using an infinite time window\n // because we're storing the values and the timestamps in the same array.\n const adjustedBufferSize = (_infiniteTimeWindow ? 1 : 2) * _bufferSize;\n _bufferSize < Infinity && adjustedBufferSize < _buffer.length && _buffer.splice(0, _buffer.length - adjustedBufferSize);\n\n // Now, if we're not in an infinite time window, remove all values where the time is\n // older than what is allowed.\n if (!_infiniteTimeWindow) {\n const now = _timestampProvider.now();\n let last = 0;\n // Search the array for the first timestamp that isn't expired and\n // truncate the buffer up to that point.\n for (let i = 1; i < _buffer.length && (_buffer[i] as number) <= now; i += 2) {\n last = i;\n }\n last && _buffer.splice(0, last + 1);\n }\n }\n}\n", "import { Scheduler } from '../Scheduler';\nimport { Subscription } from '../Subscription';\nimport { SchedulerAction } from '../types';\n\n/**\n * A unit of work to be executed in a `scheduler`. An action is typically\n * created from within a {@link SchedulerLike} and an RxJS user does not need to concern\n * themselves about creating and manipulating an Action.\n *\n * ```ts\n * class Action extends Subscription {\n * new (scheduler: Scheduler, work: (state?: T) => void);\n * schedule(state?: T, delay: number = 0): Subscription;\n * }\n * ```\n *\n * @class Action\n */\nexport class Action extends Subscription {\n constructor(scheduler: Scheduler, work: (this: SchedulerAction, state?: T) => void) {\n super();\n }\n /**\n * Schedules this action on its parent {@link SchedulerLike} for execution. May be passed\n * some context object, `state`. May happen at some point in the future,\n * according to the `delay` parameter, if specified.\n * @param {T} [state] Some contextual data that the `work` function uses when\n * called by the Scheduler.\n * @param {number} [delay] Time to wait before executing the work, where the\n * time unit is implicit and defined by the Scheduler.\n * @return {void}\n */\n public schedule(state?: T, delay: number = 0): Subscription {\n return this;\n }\n}\n", "import type { TimerHandle } from './timerHandle';\ntype SetIntervalFunction = (handler: () => void, timeout?: number, ...args: any[]) => TimerHandle;\ntype ClearIntervalFunction = (handle: TimerHandle) => void;\n\ninterface IntervalProvider {\n setInterval: SetIntervalFunction;\n clearInterval: ClearIntervalFunction;\n delegate:\n | {\n setInterval: SetIntervalFunction;\n clearInterval: ClearIntervalFunction;\n }\n | undefined;\n}\n\nexport const intervalProvider: IntervalProvider = {\n // When accessing the delegate, use the variable rather than `this` so that\n // the functions can be called without being bound to the provider.\n setInterval(handler: () => void, timeout?: number, ...args) {\n const { delegate } = intervalProvider;\n if (delegate?.setInterval) {\n return delegate.setInterval(handler, timeout, ...args);\n }\n return setInterval(handler, timeout, ...args);\n },\n clearInterval(handle) {\n const { delegate } = intervalProvider;\n return (delegate?.clearInterval || clearInterval)(handle as any);\n },\n delegate: undefined,\n};\n", "import { Action } from './Action';\nimport { SchedulerAction } from '../types';\nimport { Subscription } from '../Subscription';\nimport { AsyncScheduler } from './AsyncScheduler';\nimport { intervalProvider } from './intervalProvider';\nimport { arrRemove } from '../util/arrRemove';\nimport { TimerHandle } from './timerHandle';\n\nexport class AsyncAction extends Action {\n public id: TimerHandle | undefined;\n public state?: T;\n // @ts-ignore: Property has no initializer and is not definitely assigned\n public delay: number;\n protected pending: boolean = false;\n\n constructor(protected scheduler: AsyncScheduler, protected work: (this: SchedulerAction, state?: T) => void) {\n super(scheduler, work);\n }\n\n public schedule(state?: T, delay: number = 0): Subscription {\n if (this.closed) {\n return this;\n }\n\n // Always replace the current state with the new state.\n this.state = state;\n\n const id = this.id;\n const scheduler = this.scheduler;\n\n //\n // Important implementation note:\n //\n // Actions only execute once by default, unless rescheduled from within the\n // scheduled callback. This allows us to implement single and repeat\n // actions via the same code path, without adding API surface area, as well\n // as mimic traditional recursion but across asynchronous boundaries.\n //\n // However, JS runtimes and timers distinguish between intervals achieved by\n // serial `setTimeout` calls vs. a single `setInterval` call. An interval of\n // serial `setTimeout` calls can be individually delayed, which delays\n // scheduling the next `setTimeout`, and so on. `setInterval` attempts to\n // guarantee the interval callback will be invoked more precisely to the\n // interval period, regardless of load.\n //\n // Therefore, we use `setInterval` to schedule single and repeat actions.\n // If the action reschedules itself with the same delay, the interval is not\n // canceled. If the action doesn't reschedule, or reschedules with a\n // different delay, the interval will be canceled after scheduled callback\n // execution.\n //\n if (id != null) {\n this.id = this.recycleAsyncId(scheduler, id, delay);\n }\n\n // Set the pending flag indicating that this action has been scheduled, or\n // has recursively rescheduled itself.\n this.pending = true;\n\n this.delay = delay;\n // If this action has already an async Id, don't request a new one.\n this.id = this.id ?? this.requestAsyncId(scheduler, this.id, delay);\n\n return this;\n }\n\n protected requestAsyncId(scheduler: AsyncScheduler, _id?: TimerHandle, delay: number = 0): TimerHandle {\n return intervalProvider.setInterval(scheduler.flush.bind(scheduler, this), delay);\n }\n\n protected recycleAsyncId(_scheduler: AsyncScheduler, id?: TimerHandle, delay: number | null = 0): TimerHandle | undefined {\n // If this action is rescheduled with the same delay time, don't clear the interval id.\n if (delay != null && this.delay === delay && this.pending === false) {\n return id;\n }\n // Otherwise, if the action's delay time is different from the current delay,\n // or the action has been rescheduled before it's executed, clear the interval id\n if (id != null) {\n intervalProvider.clearInterval(id);\n }\n\n return undefined;\n }\n\n /**\n * Immediately executes this action and the `work` it contains.\n * @return {any}\n */\n public execute(state: T, delay: number): any {\n if (this.closed) {\n return new Error('executing a cancelled action');\n }\n\n this.pending = false;\n const error = this._execute(state, delay);\n if (error) {\n return error;\n } else if (this.pending === false && this.id != null) {\n // Dequeue if the action didn't reschedule itself. Don't call\n // unsubscribe(), because the action could reschedule later.\n // For example:\n // ```\n // scheduler.schedule(function doWork(counter) {\n // /* ... I'm a busy worker bee ... */\n // var originalAction = this;\n // /* wait 100ms before rescheduling the action */\n // setTimeout(function () {\n // originalAction.schedule(counter + 1);\n // }, 100);\n // }, 1000);\n // ```\n this.id = this.recycleAsyncId(this.scheduler, this.id, null);\n }\n }\n\n protected _execute(state: T, _delay: number): any {\n let errored: boolean = false;\n let errorValue: any;\n try {\n this.work(state);\n } catch (e) {\n errored = true;\n // HACK: Since code elsewhere is relying on the \"truthiness\" of the\n // return here, we can't have it return \"\" or 0 or false.\n // TODO: Clean this up when we refactor schedulers mid-version-8 or so.\n errorValue = e ? e : new Error('Scheduled action threw falsy error');\n }\n if (errored) {\n this.unsubscribe();\n return errorValue;\n }\n }\n\n unsubscribe() {\n if (!this.closed) {\n const { id, scheduler } = this;\n const { actions } = scheduler;\n\n this.work = this.state = this.scheduler = null!;\n this.pending = false;\n\n arrRemove(actions, this);\n if (id != null) {\n this.id = this.recycleAsyncId(scheduler, id, null);\n }\n\n this.delay = null!;\n super.unsubscribe();\n }\n }\n}\n", "import { Action } from './scheduler/Action';\nimport { Subscription } from './Subscription';\nimport { SchedulerLike, SchedulerAction } from './types';\nimport { dateTimestampProvider } from './scheduler/dateTimestampProvider';\n\n/**\n * An execution context and a data structure to order tasks and schedule their\n * execution. Provides a notion of (potentially virtual) time, through the\n * `now()` getter method.\n *\n * Each unit of work in a Scheduler is called an `Action`.\n *\n * ```ts\n * class Scheduler {\n * now(): number;\n * schedule(work, delay?, state?): Subscription;\n * }\n * ```\n *\n * @class Scheduler\n * @deprecated Scheduler is an internal implementation detail of RxJS, and\n * should not be used directly. Rather, create your own class and implement\n * {@link SchedulerLike}. Will be made internal in v8.\n */\nexport class Scheduler implements SchedulerLike {\n public static now: () => number = dateTimestampProvider.now;\n\n constructor(private schedulerActionCtor: typeof Action, now: () => number = Scheduler.now) {\n this.now = now;\n }\n\n /**\n * A getter method that returns a number representing the current time\n * (at the time this function was called) according to the scheduler's own\n * internal clock.\n * @return {number} A number that represents the current time. May or may not\n * have a relation to wall-clock time. May or may not refer to a time unit\n * (e.g. milliseconds).\n */\n public now: () => number;\n\n /**\n * Schedules a function, `work`, for execution. May happen at some point in\n * the future, according to the `delay` parameter, if specified. May be passed\n * some context object, `state`, which will be passed to the `work` function.\n *\n * The given arguments will be processed an stored as an Action object in a\n * queue of actions.\n *\n * @param {function(state: ?T): ?Subscription} work A function representing a\n * task, or some unit of work to be executed by the Scheduler.\n * @param {number} [delay] Time to wait before executing the work, where the\n * time unit is implicit and defined by the Scheduler itself.\n * @param {T} [state] Some contextual data that the `work` function uses when\n * called by the Scheduler.\n * @return {Subscription} A subscription in order to be able to unsubscribe\n * the scheduled work.\n */\n public schedule(work: (this: SchedulerAction, state?: T) => void, delay: number = 0, state?: T): Subscription {\n return new this.schedulerActionCtor(this, work).schedule(state, delay);\n }\n}\n", "import { Scheduler } from '../Scheduler';\nimport { Action } from './Action';\nimport { AsyncAction } from './AsyncAction';\nimport { TimerHandle } from './timerHandle';\n\nexport class AsyncScheduler extends Scheduler {\n public actions: Array> = [];\n /**\n * A flag to indicate whether the Scheduler is currently executing a batch of\n * queued actions.\n * @type {boolean}\n * @internal\n */\n public _active: boolean = false;\n /**\n * An internal ID used to track the latest asynchronous task such as those\n * coming from `setTimeout`, `setInterval`, `requestAnimationFrame`, and\n * others.\n * @type {any}\n * @internal\n */\n public _scheduled: TimerHandle | undefined;\n\n constructor(SchedulerAction: typeof Action, now: () => number = Scheduler.now) {\n super(SchedulerAction, now);\n }\n\n public flush(action: AsyncAction): void {\n const { actions } = this;\n\n if (this._active) {\n actions.push(action);\n return;\n }\n\n let error: any;\n this._active = true;\n\n do {\n if ((error = action.execute(action.state, action.delay))) {\n break;\n }\n } while ((action = actions.shift()!)); // exhaust the scheduler queue\n\n this._active = false;\n\n if (error) {\n while ((action = actions.shift()!)) {\n action.unsubscribe();\n }\n throw error;\n }\n }\n}\n", "import { AsyncAction } from './AsyncAction';\nimport { AsyncScheduler } from './AsyncScheduler';\n\n/**\n *\n * Async Scheduler\n *\n * Schedule task as if you used setTimeout(task, duration)\n *\n * `async` scheduler schedules tasks asynchronously, by putting them on the JavaScript\n * event loop queue. It is best used to delay tasks in time or to schedule tasks repeating\n * in intervals.\n *\n * If you just want to \"defer\" task, that is to perform it right after currently\n * executing synchronous code ends (commonly achieved by `setTimeout(deferredTask, 0)`),\n * better choice will be the {@link asapScheduler} scheduler.\n *\n * ## Examples\n * Use async scheduler to delay task\n * ```ts\n * import { asyncScheduler } from 'rxjs';\n *\n * const task = () => console.log('it works!');\n *\n * asyncScheduler.schedule(task, 2000);\n *\n * // After 2 seconds logs:\n * // \"it works!\"\n * ```\n *\n * Use async scheduler to repeat task in intervals\n * ```ts\n * import { asyncScheduler } from 'rxjs';\n *\n * function task(state) {\n * console.log(state);\n * this.schedule(state + 1, 1000); // `this` references currently executing Action,\n * // which we reschedule with new state and delay\n * }\n *\n * asyncScheduler.schedule(task, 3000, 0);\n *\n * // Logs:\n * // 0 after 3s\n * // 1 after 4s\n * // 2 after 5s\n * // 3 after 6s\n * ```\n */\n\nexport const asyncScheduler = new AsyncScheduler(AsyncAction);\n\n/**\n * @deprecated Renamed to {@link asyncScheduler}. Will be removed in v8.\n */\nexport const async = asyncScheduler;\n", "import { AsyncAction } from './AsyncAction';\nimport { Subscription } from '../Subscription';\nimport { QueueScheduler } from './QueueScheduler';\nimport { SchedulerAction } from '../types';\nimport { TimerHandle } from './timerHandle';\n\nexport class QueueAction extends AsyncAction {\n constructor(protected scheduler: QueueScheduler, protected work: (this: SchedulerAction, state?: T) => void) {\n super(scheduler, work);\n }\n\n public schedule(state?: T, delay: number = 0): Subscription {\n if (delay > 0) {\n return super.schedule(state, delay);\n }\n this.delay = delay;\n this.state = state;\n this.scheduler.flush(this);\n return this;\n }\n\n public execute(state: T, delay: number): any {\n return delay > 0 || this.closed ? super.execute(state, delay) : this._execute(state, delay);\n }\n\n protected requestAsyncId(scheduler: QueueScheduler, id?: TimerHandle, delay: number = 0): TimerHandle {\n // If delay exists and is greater than 0, or if the delay is null (the\n // action wasn't rescheduled) but was originally scheduled as an async\n // action, then recycle as an async action.\n\n if ((delay != null && delay > 0) || (delay == null && this.delay > 0)) {\n return super.requestAsyncId(scheduler, id, delay);\n }\n\n // Otherwise flush the scheduler starting with this action.\n scheduler.flush(this);\n\n // HACK: In the past, this was returning `void`. However, `void` isn't a valid\n // `TimerHandle`, and generally the return value here isn't really used. So the\n // compromise is to return `0` which is both \"falsy\" and a valid `TimerHandle`,\n // as opposed to refactoring every other instanceo of `requestAsyncId`.\n return 0;\n }\n}\n", "import { AsyncScheduler } from './AsyncScheduler';\n\nexport class QueueScheduler extends AsyncScheduler {\n}\n", "import { QueueAction } from './QueueAction';\nimport { QueueScheduler } from './QueueScheduler';\n\n/**\n *\n * Queue Scheduler\n *\n * Put every next task on a queue, instead of executing it immediately\n *\n * `queue` scheduler, when used with delay, behaves the same as {@link asyncScheduler} scheduler.\n *\n * When used without delay, it schedules given task synchronously - executes it right when\n * it is scheduled. However when called recursively, that is when inside the scheduled task,\n * another task is scheduled with queue scheduler, instead of executing immediately as well,\n * that task will be put on a queue and wait for current one to finish.\n *\n * This means that when you execute task with `queue` scheduler, you are sure it will end\n * before any other task scheduled with that scheduler will start.\n *\n * ## Examples\n * Schedule recursively first, then do something\n * ```ts\n * import { queueScheduler } from 'rxjs';\n *\n * queueScheduler.schedule(() => {\n * queueScheduler.schedule(() => console.log('second')); // will not happen now, but will be put on a queue\n *\n * console.log('first');\n * });\n *\n * // Logs:\n * // \"first\"\n * // \"second\"\n * ```\n *\n * Reschedule itself recursively\n * ```ts\n * import { queueScheduler } from 'rxjs';\n *\n * queueScheduler.schedule(function(state) {\n * if (state !== 0) {\n * console.log('before', state);\n * this.schedule(state - 1); // `this` references currently executing Action,\n * // which we reschedule with new state\n * console.log('after', state);\n * }\n * }, 0, 3);\n *\n * // In scheduler that runs recursively, you would expect:\n * // \"before\", 3\n * // \"before\", 2\n * // \"before\", 1\n * // \"after\", 1\n * // \"after\", 2\n * // \"after\", 3\n *\n * // But with queue it logs:\n * // \"before\", 3\n * // \"after\", 3\n * // \"before\", 2\n * // \"after\", 2\n * // \"before\", 1\n * // \"after\", 1\n * ```\n */\n\nexport const queueScheduler = new QueueScheduler(QueueAction);\n\n/**\n * @deprecated Renamed to {@link queueScheduler}. Will be removed in v8.\n */\nexport const queue = queueScheduler;\n", "import { AsyncAction } from './AsyncAction';\nimport { AnimationFrameScheduler } from './AnimationFrameScheduler';\nimport { SchedulerAction } from '../types';\nimport { animationFrameProvider } from './animationFrameProvider';\nimport { TimerHandle } from './timerHandle';\n\nexport class AnimationFrameAction extends AsyncAction {\n constructor(protected scheduler: AnimationFrameScheduler, protected work: (this: SchedulerAction, state?: T) => void) {\n super(scheduler, work);\n }\n\n protected requestAsyncId(scheduler: AnimationFrameScheduler, id?: TimerHandle, delay: number = 0): TimerHandle {\n // If delay is greater than 0, request as an async action.\n if (delay !== null && delay > 0) {\n return super.requestAsyncId(scheduler, id, delay);\n }\n // Push the action to the end of the scheduler queue.\n scheduler.actions.push(this);\n // If an animation frame has already been requested, don't request another\n // one. If an animation frame hasn't been requested yet, request one. Return\n // the current animation frame request id.\n return scheduler._scheduled || (scheduler._scheduled = animationFrameProvider.requestAnimationFrame(() => scheduler.flush(undefined)));\n }\n\n protected recycleAsyncId(scheduler: AnimationFrameScheduler, id?: TimerHandle, delay: number = 0): TimerHandle | undefined {\n // If delay exists and is greater than 0, or if the delay is null (the\n // action wasn't rescheduled) but was originally scheduled as an async\n // action, then recycle as an async action.\n if (delay != null ? delay > 0 : this.delay > 0) {\n return super.recycleAsyncId(scheduler, id, delay);\n }\n // If the scheduler queue has no remaining actions with the same async id,\n // cancel the requested animation frame and set the scheduled flag to\n // undefined so the next AnimationFrameAction will request its own.\n const { actions } = scheduler;\n if (id != null && actions[actions.length - 1]?.id !== id) {\n animationFrameProvider.cancelAnimationFrame(id as number);\n scheduler._scheduled = undefined;\n }\n // Return undefined so the action knows to request a new async id if it's rescheduled.\n return undefined;\n }\n}\n", "import { AsyncAction } from './AsyncAction';\nimport { AsyncScheduler } from './AsyncScheduler';\n\nexport class AnimationFrameScheduler extends AsyncScheduler {\n public flush(action?: AsyncAction): void {\n this._active = true;\n // The async id that effects a call to flush is stored in _scheduled.\n // Before executing an action, it's necessary to check the action's async\n // id to determine whether it's supposed to be executed in the current\n // flush.\n // Previous implementations of this method used a count to determine this,\n // but that was unsound, as actions that are unsubscribed - i.e. cancelled -\n // are removed from the actions array and that can shift actions that are\n // scheduled to be executed in a subsequent flush into positions at which\n // they are executed within the current flush.\n const flushId = this._scheduled;\n this._scheduled = undefined;\n\n const { actions } = this;\n let error: any;\n action = action || actions.shift()!;\n\n do {\n if ((error = action.execute(action.state, action.delay))) {\n break;\n }\n } while ((action = actions[0]) && action.id === flushId && actions.shift());\n\n this._active = false;\n\n if (error) {\n while ((action = actions[0]) && action.id === flushId && actions.shift()) {\n action.unsubscribe();\n }\n throw error;\n }\n }\n}\n", "import { AnimationFrameAction } from './AnimationFrameAction';\nimport { AnimationFrameScheduler } from './AnimationFrameScheduler';\n\n/**\n *\n * Animation Frame Scheduler\n *\n * Perform task when `window.requestAnimationFrame` would fire\n *\n * When `animationFrame` scheduler is used with delay, it will fall back to {@link asyncScheduler} scheduler\n * behaviour.\n *\n * Without delay, `animationFrame` scheduler can be used to create smooth browser animations.\n * It makes sure scheduled task will happen just before next browser content repaint,\n * thus performing animations as efficiently as possible.\n *\n * ## Example\n * Schedule div height animation\n * ```ts\n * // html:
\n * import { animationFrameScheduler } from 'rxjs';\n *\n * const div = document.querySelector('div');\n *\n * animationFrameScheduler.schedule(function(height) {\n * div.style.height = height + \"px\";\n *\n * this.schedule(height + 1); // `this` references currently executing Action,\n * // which we reschedule with new state\n * }, 0, 0);\n *\n * // You will see a div element growing in height\n * ```\n */\n\nexport const animationFrameScheduler = new AnimationFrameScheduler(AnimationFrameAction);\n\n/**\n * @deprecated Renamed to {@link animationFrameScheduler}. Will be removed in v8.\n */\nexport const animationFrame = animationFrameScheduler;\n", "import { Observable } from '../Observable';\nimport { SchedulerLike } from '../types';\n\n/**\n * A simple Observable that emits no items to the Observer and immediately\n * emits a complete notification.\n *\n * Just emits 'complete', and nothing else.\n *\n * ![](empty.png)\n *\n * A simple Observable that only emits the complete notification. It can be used\n * for composing with other Observables, such as in a {@link mergeMap}.\n *\n * ## Examples\n *\n * Log complete notification\n *\n * ```ts\n * import { EMPTY } from 'rxjs';\n *\n * EMPTY.subscribe({\n * next: () => console.log('Next'),\n * complete: () => console.log('Complete!')\n * });\n *\n * // Outputs\n * // Complete!\n * ```\n *\n * Emit the number 7, then complete\n *\n * ```ts\n * import { EMPTY, startWith } from 'rxjs';\n *\n * const result = EMPTY.pipe(startWith(7));\n * result.subscribe(x => console.log(x));\n *\n * // Outputs\n * // 7\n * ```\n *\n * Map and flatten only odd numbers to the sequence `'a'`, `'b'`, `'c'`\n *\n * ```ts\n * import { interval, mergeMap, of, EMPTY } from 'rxjs';\n *\n * const interval$ = interval(1000);\n * const result = interval$.pipe(\n * mergeMap(x => x % 2 === 1 ? of('a', 'b', 'c') : EMPTY),\n * );\n * result.subscribe(x => console.log(x));\n *\n * // Results in the following to the console:\n * // x is equal to the count on the interval, e.g. (0, 1, 2, 3, ...)\n * // x will occur every 1000ms\n * // if x % 2 is equal to 1, print a, b, c (each on its own)\n * // if x % 2 is not equal to 1, nothing will be output\n * ```\n *\n * @see {@link Observable}\n * @see {@link NEVER}\n * @see {@link of}\n * @see {@link throwError}\n */\nexport const EMPTY = new Observable((subscriber) => subscriber.complete());\n\n/**\n * @param scheduler A {@link SchedulerLike} to use for scheduling\n * the emission of the complete notification.\n * @deprecated Replaced with the {@link EMPTY} constant or {@link scheduled} (e.g. `scheduled([], scheduler)`). Will be removed in v8.\n */\nexport function empty(scheduler?: SchedulerLike) {\n return scheduler ? emptyScheduled(scheduler) : EMPTY;\n}\n\nfunction emptyScheduled(scheduler: SchedulerLike) {\n return new Observable((subscriber) => scheduler.schedule(() => subscriber.complete()));\n}\n", "import { SchedulerLike } from '../types';\nimport { isFunction } from './isFunction';\n\nexport function isScheduler(value: any): value is SchedulerLike {\n return value && isFunction(value.schedule);\n}\n", "import { SchedulerLike } from '../types';\nimport { isFunction } from './isFunction';\nimport { isScheduler } from './isScheduler';\n\nfunction last(arr: T[]): T | undefined {\n return arr[arr.length - 1];\n}\n\nexport function popResultSelector(args: any[]): ((...args: unknown[]) => unknown) | undefined {\n return isFunction(last(args)) ? args.pop() : undefined;\n}\n\nexport function popScheduler(args: any[]): SchedulerLike | undefined {\n return isScheduler(last(args)) ? args.pop() : undefined;\n}\n\nexport function popNumber(args: any[], defaultValue: number): number {\n return typeof last(args) === 'number' ? args.pop()! : defaultValue;\n}\n", "export const isArrayLike = ((x: any): x is ArrayLike => x && typeof x.length === 'number' && typeof x !== 'function');", "import { isFunction } from \"./isFunction\";\n\n/**\n * Tests to see if the object is \"thennable\".\n * @param value the object to test\n */\nexport function isPromise(value: any): value is PromiseLike {\n return isFunction(value?.then);\n}\n", "import { InteropObservable } from '../types';\nimport { observable as Symbol_observable } from '../symbol/observable';\nimport { isFunction } from './isFunction';\n\n/** Identifies an input as being Observable (but not necessary an Rx Observable) */\nexport function isInteropObservable(input: any): input is InteropObservable {\n return isFunction(input[Symbol_observable]);\n}\n", "import { isFunction } from './isFunction';\n\nexport function isAsyncIterable(obj: any): obj is AsyncIterable {\n return Symbol.asyncIterator && isFunction(obj?.[Symbol.asyncIterator]);\n}\n", "/**\n * Creates the TypeError to throw if an invalid object is passed to `from` or `scheduled`.\n * @param input The object that was passed.\n */\nexport function createInvalidObservableTypeError(input: any) {\n // TODO: We should create error codes that can be looked up, so this can be less verbose.\n return new TypeError(\n `You provided ${\n input !== null && typeof input === 'object' ? 'an invalid object' : `'${input}'`\n } where a stream was expected. You can provide an Observable, Promise, ReadableStream, Array, AsyncIterable, or Iterable.`\n );\n}\n", "export function getSymbolIterator(): symbol {\n if (typeof Symbol !== 'function' || !Symbol.iterator) {\n return '@@iterator' as any;\n }\n\n return Symbol.iterator;\n}\n\nexport const iterator = getSymbolIterator();\n", "import { iterator as Symbol_iterator } from '../symbol/iterator';\nimport { isFunction } from './isFunction';\n\n/** Identifies an input as being an Iterable */\nexport function isIterable(input: any): input is Iterable {\n return isFunction(input?.[Symbol_iterator]);\n}\n", "import { ReadableStreamLike } from '../types';\nimport { isFunction } from './isFunction';\n\nexport async function* readableStreamLikeToAsyncGenerator(readableStream: ReadableStreamLike): AsyncGenerator {\n const reader = readableStream.getReader();\n try {\n while (true) {\n const { value, done } = await reader.read();\n if (done) {\n return;\n }\n yield value!;\n }\n } finally {\n reader.releaseLock();\n }\n}\n\nexport function isReadableStreamLike(obj: any): obj is ReadableStreamLike {\n // We don't want to use instanceof checks because they would return\n // false for instances from another Realm, like an + +

MLOps with Red Hat OpenShift

+

Red Hat OpenShift includes key capabilities to enable machine learning operations (MLOps) in a consistent way across datacenters, public cloud computing, and edge computing.

+

By applying DevOps and GitOps principles, organizations automate and simplify the iterative process of integrating ML models into software development processes, production rollout, monitoring, retraining, and redeployment for continued prediction accuracy.

+
+

What is a ML lifecycle?

+

A multi-phase process to obtain the power of large volumes and a variety of data, abundant compute, and open source machine learning tools to build intelligent applications.

+

At a high level, there are four steps in the lifecycle:

+
    +
  1. Gather and prepare data to make sure the input data is complete, and of high quality
  2. +
  3. Develop model, including training, testing, and selection of the model with the highest prediction accuracy
  4. +
  5. Integrate models in application development process, and inferencing
  6. +
  7. Model monitoring and management, to measure business performance and address potential production data drift
  8. +
+

On this site, you will find recipes, patterns, demos for various AI/ML tools and applications used through those steps.

+

Why use containers and Kubernetes for your machine learning initiatives?

+

Containers and Kubernetes are key to accelerating the ML lifecycle as these technologies provide data scientists the much needed agility, flexibility, portability, and scalability to train, test, and deploy ML models.

+

Red Hat® OpenShift® is the industry's leading containers and Kubernetes hybrid cloud platform. It provides all these benefits, and through the integrated DevOps capabilities (e.g. OpenShift Pipelines, OpenShift GitOps, and Red Hat Quay) and integration with hardware accelerators, it enables better collaboration between data scientists and software developers, and accelerates the roll out of intelligent applications across hybrid cloud (data center, edge, and public clouds).

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/getting-started/why-this-site/index.html b/getting-started/why-this-site/index.html new file mode 100644 index 00000000..8d93dc6f --- /dev/null +++ b/getting-started/why-this-site/index.html @@ -0,0 +1,2774 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Why this site? - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Why this site?

+

As data scientists and engineers, it's easy to find detailed documentation on the tools and libraries we use. But what about end-to-end data pipeline solutions that involve multiple products? Unfortunately, those resources can be harder to come by. Open source communities often don't have the resources to create and maintain them. But don't worry, that's where this website comes in!

+

We've created a one-stop-shop for data practitioners to find recipes, reusable patterns, and actionable demos for building AI/ML solutions on OpenShift. And the best part? It's a community-driven resource site! So, feel free to ask questions, make feature requests, file issues, and even submit PRs to help us improve the content. Together, we can make data pipeline solutions easier to find and implement.

+

Happy Robot

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/google32e0ce0d0ebf499d.html b/google32e0ce0d0ebf499d.html new file mode 100644 index 00000000..1342c299 --- /dev/null +++ b/google32e0ce0d0ebf499d.html @@ -0,0 +1 @@ +google-site-verification: google32e0ce0d0ebf499d.html \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 00000000..39802a8a --- /dev/null +++ b/index.html @@ -0,0 +1,3027 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + AI on OpenShift - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + + + + + +
+
+
+
+ +
+
+ +

The one-stop shop for AI/ML,
Data Science and Data Engineering
on OpenShift!

+ + Get started + +
+
+
+
+ + + + + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ +
+
+ + + +
+ + + +
+ +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/index.md.bck b/index.md.bck new file mode 100644 index 00000000..93ba37b0 --- /dev/null +++ b/index.md.bck @@ -0,0 +1,10 @@ +--- +hide: + - navigation +--- + +Welcome to your one-stop shop for installation recipes, patterns, demos for various AI/ML tools and applications used in Data Science and Data Engineering projects running on OpenShift! + +More than often, Data Scientists and Data Engineers get wonderful documentation for a specific application, library or product. But they do not often get actionable documentation and sample code on how to compose end-to-end solutions from multiple products. + +The AI on OpenShift site aims at jumpstarting digital transformation productivity on the Red Hat OpenShift platform by combining practical, reusable patterns into use cases with end-to-end real world illustrations. diff --git a/odh-rhoai/accelerator-profiles/index.html b/odh-rhoai/accelerator-profiles/index.html new file mode 100644 index 00000000..fa8eba67 --- /dev/null +++ b/odh-rhoai/accelerator-profiles/index.html @@ -0,0 +1,2837 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Accelerator Profiles - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Accelerator Profiles

+ +

Accelerator Profiles

+

To effectively use accelerators in OpenShift AI, OpenShift AI Administrators need to create and manage associated accelerator profiles.

+

An accelerator profile is a custom resource definition (CRD) that defines specifications for this accelerator. It can be direclty managed via the OpenShift AI Dashboard under Settings → Accelerator profiles.

+

When working with GPU nodes in OpenShift AI, it is essential to set proper taints on those nodes. This prevents unwanted workloads from being scheduled on them when they don't have specific tolerations set. Those tolerations are configured in the accelerator profiles associated with each type of GPU, then applied to the workloads (Workbenches, Model servers,...) for which you have selected an accelerator profile.

+

The taints in the GPU Worker Nodes should be set like this:

+
  taints:
+    - effect: NoSchedule
+      key: nvidia.com/gpu
+      value: NVIDIA-A10G-SHARED
+
+

A corresponding Accelerator profile can then be created to allow workloads to run on this type of node (in this example, nodes having an A10G GPU). Workloads that use another accelerator profile (for another type of GPU for example) or that don't have any Accelerator profile set will not be scheduled on nodes tainted with NVIDIA-A10G-SHARED.

+

For a detailed guide on configuring and managing accelerator profiles in OpenShift AI, refer to our repository.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/cm-mlflow-enable.yaml b/odh-rhoai/cm-mlflow-enable.yaml new file mode 100644 index 00000000..4f178327 --- /dev/null +++ b/odh-rhoai/cm-mlflow-enable.yaml @@ -0,0 +1,7 @@ +kind: ConfigMap +apiVersion: v1 +metadata: + name: mlflow-enable + namespace: redhat-ods-applications +data: + validation_result: 'true' \ No newline at end of file diff --git a/odh-rhoai/configuration/index.html b/odh-rhoai/configuration/index.html new file mode 100644 index 00000000..63483456 --- /dev/null +++ b/odh-rhoai/configuration/index.html @@ -0,0 +1,3048 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Dashboard configuration - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+ +
+
+ + + +
+
+ + + + +

ODH and RHOAI Configuration

+

Standard configuration

+

As an administrator of ODH/RHOAI, you have access to different settings through the Settings menu on the dashboard:

+

Settings

+

Custom notebook images

+

This is where you can import other notebook images. You will find resources on available custom images and learn how to create your own in the Custom Notebooks section.

+

To import a new image, follow those steps.

+
    +
  • Click on import image.
  • +
+

Import

+
    +
  • Enter the full address of your container, set a name (this is what will appear in the launcher), and a description.
  • +
+

Import

+
    +
  • On the bottom part, add information regarding the software and the packages that are present in this image. This is purely informative.
  • +
+

Import

+
    +
  • Your image is now listed and enabled. You can hide it without removing it by simply disabling it.
  • +
+

Images

+
    +
  • It is now available in the launcher, as well as in the Data Science Projects.
  • +
+

Image Launcher

+

Cluster settings

+

In this panel, you can adjust:

+
    +
  • The default size of the volumes created for new users.
  • +
  • Whether you want to stop idle notebooks and, if so, after how much time.
  • +
+
+

Note

+

This feature currently looks at running Jupyter kernels, like a Python notebook. If you are only using a Terminal, or another IDE window like VSCode or RStudio from the custom images, this activity is not detected and your Pod can be stopped without notice after the set delay.

+
+
    +
  • Whether you allow usage data to be collected and reported.
  • +
  • Whether you want to add a toleration to the notebook pods to allow them to be scheduled on tainted nodes. That feature is really useful if you want to dedicate specific worker nodes to running notebooks. Tainting them will prevent other workloads from running on them. Of course, you have to add the toleration to the pods.
  • +
+

Cluster settings

+

User management

+

In this panel, you can edit who has access to RHOAI by defining the "Data Science user groups", and who has access to the Settings by defining the "Data Science administrator groups".

+

User management

+

Advanced configuration

+

Dashboard configuration

+

RHOAI or ODH main configuration is done through a Custom Resource (CR) of type odhdashboardconfigs.opendatahub.io.

+
    +
  • To get access to it, from your OpenShift console, navigate to Home->API Explorer, and filter for OdhDashboardConfig:
  • +
+

API explorer

+
    +
  • Click on OdhDashboardConfig and in the Instances tab, click on odh-dashboard-config:
  • +
+

Instance

+
    +
  • You can now view and edit the YAML file to modify the configuration:
  • +
+

Edit YAML

+

In the spec section, the following items are of interest:

+
    +
  • dashboardConfig: The different toggles will allow you to activate/deactivate certain features. For example, you may want to hide Model Serving for your users or prevent them from importing custom images.
  • +
  • notebookSizes: This is where you can fully customize the sizes of the notebooks. You can modify the resources and add or remove sizes from the default configuration as needed.
  • +
  • modelServerSizes: This setting operates on the same concept as the previous setting but for model servers.
  • +
  • notebookController: In this section you will find various settings related to the Workbenches and how they are launched.
  • +
  • If your GPUs are not correctly detected, the dropdown allowing you to select how many GPUs you want to use for a workbench will not be displayed. To force it, you can create/modify the parameter gpuSetting under notebookController. This will force the dropdown to appear, with the maximum being the number you set for the parameter. Example:
  • +
+
notebookController:
+    enabled: true
+    gpuSetting: '4'
+    ...
+
+

Adding a custom application

+

Let's say you have installed another application in your cluster and want to make it available through the dashboard. That's easy! A tile is, in fact, represented by a custom resource (CR) of type OdhApplication.

+

In this example, we will add a tile to access the MLFlow UI (see the MLFlow installation instructions to test it).

+
    +
  • The file mlflow-tile.yaml provides you with an example of how to create the tile.
  • +
  • Edit this file to set the route (the name of the Route CR) and routeNamespace parameters to where the UI is accessible. In this example, it is mlflow-server(route name) and mlflow (server). Apply this file to create the resource.
  • +
  • Wait 1-2 minutes for the change to take effect. Your tile is now available in the Explore view (bottom left):
  • +
+

Explore tile

+
    +
  • However, it is not yet enabled. To enable this tile, click on it in the Explorer view, then click the "Enable" button at the top of the description. You can also create a ConfigMap from the file cm-mlflow-enable.yaml.
  • +
  • Wait another 1-2 minutes, and your tile is now ready to use in the Enabled view:
  • +
+

Enabled tile

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/connect-vscode-to-rhoai-wb/index.html b/odh-rhoai/connect-vscode-to-rhoai-wb/index.html new file mode 100644 index 00000000..fc77a07e --- /dev/null +++ b/odh-rhoai/connect-vscode-to-rhoai-wb/index.html @@ -0,0 +1,2921 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Connect to RHOAI Workbench Kernel from VS Code - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Connect to RHOAI Workbench Kernel from local VS Code

+

Some users have expressed their desire to work directly on their local IDE and execute the Jupyter notebook(s) using the kernel on remote workbench running on RHOAI. Most IDEs provide connection to a remote Kernel as a standard feature. However, this standard feature does not work with RHOAI because of the way authentication to workbench is set up in RHOAI
+The standard feature of most IDEs to connect to remote kernel uses token based authentication. Workbench pods running on RHOAI contain an authentication mechanism that sits in front of the workbench container and handles the authentication of the user connecting to the workbench. This container uses Openshift Authentication mechanism and is not compatible with the standard connection feature of most IDEs.

+

Workaround: Connect to the remote kernel using Openshift port-forwarding

+

Use the following steps to connect your local VS Code to RHOAI Workbench kernel:

+ +
    +
  • +

    In your data science project in RHOAI, create a workbench that you would like to use as your remote kernel. If you want to use gpu accelerator, use the compatible workbench image (e.g. pytorch).

    +

    Create workbench

    +
  • +
  • +

    Open the workbench and copy the context path from the browser. You will need this later when connecting from VS Code.

    +

    Workbench context path

    +
  • +
  • +

    From terminal on your laptop/desktop login to openshift. + Login to openshift

    +
  • +
  • +

    Switch to your data science project + Switch project

    +
  • +
  • +

    Start port-forwarding to your workbench pod

    +
      +
    • List all the pods in your project. The pod running your workbench is named using the name of your workbench in RHOAI. e.g. mywb-0 if your workbench name is mywb.
    • +
    • Enable port-forwarding to your workbench pod. You need to forward to the port the pod is listening on. It is usually 8888 for RHOAI workbench. You can find this port from the service in your project with name same as your workbench. +Port-forward to wb pod
    • +
    +
  • +
  • +

    Open the Jupyter notebook in your VS Code + Jupyter Notebook

    +
  • +
  • +

    From the top right corner of the notebook, click on Select Kernel. + Select Kernel

    +
  • +
  • +

    From the options, select Existing Jupyter Server and then enter the url as follows: + localhost [:port] /context-path copied earlier that has the pattern /notebook/ds-project-name/workbench-name/lab. e.g. http://localhost:8888/notebook/rag-llm-demo/mywb/lab

    +
  • +
  • +

    A prompt saying
    +Connecting over HTTP without a token may be an insecure connection. Do you want to connect to a possibly insecure server?
    +is displayed. select Yes + Insecure connection prompt

    +
  • +
  • +

    Select the prompted Server display name or enter a new one. + Server display name

    +
  • +
  • +

    A list of available kernels is displayed. Choose Python 3.9. + Select Kernel

    +
  • +
  • +

    You should see the selected Kernel in the top right corner. + Remote kernel selected

    +
  • +
  • +

    The code inside of your notebook will now execute using the remote kernel on the RHOAI workbench pod.

    +
  • +
  • +

    If your workbench uses a Nvidia GPU, you can verify that it is being used in the execution of your notebook by adding a command !nvidia-smi. You should see output similar to the image below.

    +

    Nvidia GPU used

    +
  • +
+

Caveats

+
    +
  • Jupyter notebooks in your local VSCode environment will not be saved to the workbench.
  • +
  • If your notebook uses any files (models, inputdata etc.), they should be present on the workbench and their path should match the path specified in your notebook.
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/custom-notebooks/index.html b/odh-rhoai/custom-notebooks/index.html new file mode 100644 index 00000000..116cfce4 --- /dev/null +++ b/odh-rhoai/custom-notebooks/index.html @@ -0,0 +1,3097 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Custom notebooks - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Custom Notebooks

+

Custom notebook images are useful if you want to add libraries that you often use, or that you require at a specific version different than the one provided in the base images. It's also useful if you need to use OS packages or applications, which you cannot install on the fly in your running environment.

+

Image source and Pre-built images

+

In the opendatahub-io-contrib/workbench-images repository, you will find the source code as well as pre-built images for a lot of use cases. A few of the available images are:

+
    +
  • Base and CUDA-enabled images for different "lines" of OS: UBI8, UBI9, and Centos Stream 9.
  • +
  • Jupyter images enhanced with:
      +
    • specific libraries like OptaPy or Monai,
    • +
    • with integrated applications like Spark,
    • +
    • providing other IDEs like VSCode or RStudio
    • +
    +
  • +
  • VSCode
  • +
  • RStudio
  • +
+

All those images are constantly and automatically updated and rebuilt for the latest patch and fixes, and new releases are available regularly to provide new versions of the libraries or the applications.

+

Building your own images

+

In the repository above, you will find many examples from the source code to help you understand how to create your own image. Here are a few rules, tips and examples to help you.

+

Rules

+
    +
  • On OpenShift, every containers in a standard namespace (unless you modify security) run with a user with a random user id (uid), and the group id (gid) 0. Therefore, all the folders that you want to write in, and all the files you want to modify (temporarily) in your image must be accessible by this user. The best practice is to set the ownership at 1001:0 (user "default", group "0").
  • +
  • If you don't want/can't do that, another solution is to set permissions properly for any user, like 775.
  • +
  • When launching a notebook from Applications->Enabled, the "personal" volume of a user is mounted at /opt/app-root/src. This is not configurable, so make sure to build your images with this default location for the data that you want persisted.
  • +
+

How-tos

+

Install Python packages

+
    +
  • Start from a base image of your choice. Normally it's already running under user 1001, so no need to change it.
  • +
  • Copy your pipfile.lock or your requirements.txt
  • +
  • Install your packages
  • +
+

Example:

+
FROM BASE_IMAGE
+
+# Copying custom packages
+COPY Pipfile.lock ./
+
+# Install packages and cleanup
+# (all commands are chained to minimize layer size)
+RUN echo "Installing softwares and packages" && \
+    # Install Python packages \
+    micropipenv install && \
+    rm -f ./Pipfile.lock
+    # Fix permissions to support pip in OpenShift environments \
+    chmod -R g+w /opt/app-root/lib/python3.9/site-packages && \
+    fix-permissions /opt/app-root -P
+
+WORKDIR /opt/app-root/src
+
+ENTRYPOINT ["start-notebook.sh"]
+
+

In this example, the fix-permissions script (present in all standard images and custom images from the opendatahub-contrib repo) fixes any bad ownership or rights that may be present.

+

Install an OS package

+
    +
  • If you have to install OS packages and Python packages, it's better to start with the OS.
  • +
  • In your Containerfile/Dockerfile, switch to user 0, install your package(s), then switch back to user 1001. Example:
  • +
+
USER 0
+
+RUN INSTALL_PKGS="java-11-openjdk java-11-openjdk-devel" && \
+    yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \
+    yum -y clean all --enablerepo='*'
+
+USER 1001
+
+

Tips and tricks

+

Enabling CodeReady Builder (CRB) and EPEL

+

CRB and EPEL are repositories providing packages absent from a standard RHEL or UBI installation. They are useful and required to be able to install specific software (RStudio, I'm looking at you...).

+
    +
  • Enabling EPEL on UBI9-based images (on UBI9 images CRB is now enabled by default.):
  • +
+
RUN yum install -y https://download.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
+
+
    +
  • Enabling CRB and EPEL on Centos Stream 9-based images:
  • +
+
RUN yum install -y yum-utils && \
+    yum-config-manager --enable crb && \
+    yum install -y https://download.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
+
+

Minimizing image size

+

A container image uses a "layered" filesystem. Every time you have in your file a COPY or a RUN command, a new layer is created. Nothing is ever deleted: removing a file is simply "masking" it in the next layer. Therefore you must bee very careful when you create your Containerfile/Dockerfile.

+
    +
  • If you start from an image that is constantly updated, like ubi9/python-39 from the Red Hat Catalog, don't do a yum update. This will only fetch new metadata, update a few files that may not have any impact, and get you a bigger image.
  • +
  • Rebuilt your images often from scratch, but don't do a yum update on a previous version.
  • +
  • Group your RUN commands as much as you can, add && \ at the end of each line to chain your commands.
  • +
  • If you need to compile something for building an image, use the multi-stage builds approach. Build the library or application in an intermediate container image, then copy the result to your final image. Otherwise, all the build artefacts will persist in your image...
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/custom-runtime-triton/index.html b/odh-rhoai/custom-runtime-triton/index.html new file mode 100644 index 00000000..aaaf42c9 --- /dev/null +++ b/odh-rhoai/custom-runtime-triton/index.html @@ -0,0 +1,3189 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Custom Serving Runtime (Triton) - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Deploying and using a Custom Serving Runtime in ODH/RHOAI

+

Although these instructions were tested mostly using RHOAI (Red Hat OpenShift AI), they apply to ODH (Open Data Hub) as well.

+

Before you start

+

This document will guide you through the broad steps necessary to deploy a custom Serving Runtime in order to serve a model using the Triton Runtime (NVIDIA Triton Inference Server).

+

While RHOAI supports your ability to add your own runtime, it does not support the runtimes themselves. Therefore, it is up to you to configure, adjust and maintain your custom runtimes.

+

This document expects a bit of familiarity with RHOAI.

+

The sources used to create this document are mostly:

+ +

Adding the custom triton runtime

+
    +
  1. Log in to your OpenShift AI with a user who is part of the RHOAI admin group.
      +
    1. (by default, cluster-admins and dedicated admins are).
    2. +
    +
  2. +
  3. +

    Navigate to the Settings menu, then Serving Runtimes

    +

    Serving Runtime Settings

    +
  4. +
  5. +

    Click on the Add Serving Runtime button:

    +

    Add Serving Runtime

    +
  6. +
  7. +

    From the drop down menu, select **Multi-model serving platform. The option for REST will be selected automatically:

    +

    Select Multi-model serving

    +
  8. +
  9. +

    Click on Start from scratch and in the window that opens up, paste the following YAML: +

    # Copyright 2021 IBM Corporation
    +#
    +# Licensed under the Apache License, Version 2.0 (the "License");
    +# you may not use this file except in compliance with the License.
    +# You may obtain a copy of the License at
    +#
    +#     http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing, software
    +# distributed under the License is distributed on an "AS IS" BASIS,
    +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    +# See the License for the specific language governing permissions and
    +# limitations under the License.
    +apiVersion: serving.kserve.io/v1alpha1
    +# kind: ClusterServingRuntime     ## changed by EG
    +kind: ServingRuntime
    +metadata:
    +  name: triton-23.05-20230804
    +  labels:
    +    name: triton-23.05-20230804
    +  annotations:
    +    maxLoadingConcurrency: "2"
    +    openshift.io/display-name: "Triton runtime 23.05 - added on 20230804 - with /dev/shm"
    +spec:
    +  supportedModelFormats:
    +    - name: keras
    +      version: "2" # 2.6.0
    +      autoSelect: true
    +    - name: onnx
    +      version: "1" # 1.5.3
    +      autoSelect: true
    +    - name: pytorch
    +      version: "1" # 1.8.0a0+17f8c32
    +      autoSelect: true
    +    - name: tensorflow
    +      version: "1" # 1.15.4
    +      autoSelect: true
    +    - name: tensorflow
    +      version: "2" # 2.3.1
    +      autoSelect: true
    +    - name: tensorrt
    +      version: "7" # 7.2.1
    +      autoSelect: true
    +
    +  protocolVersions:
    +    - grpc-v2
    +  multiModel: true
    +
    +  grpcEndpoint: "port:8085"
    +  grpcDataEndpoint: "port:8001"
    +
    +  volumes:
    +    - name: shm
    +      emptyDir:
    +        medium: Memory
    +        sizeLimit: 2Gi
    +  containers:
    +    - name: triton
    +      # image: tritonserver-2:replace   ## changed by EG
    +      image: nvcr.io/nvidia/tritonserver:23.05-py3
    +      command: [/bin/sh]
    +      args:
    +        - -c
    +        - 'mkdir -p /models/_triton_models;
    +          chmod 777 /models/_triton_models;
    +          exec tritonserver
    +          "--model-repository=/models/_triton_models"
    +          "--model-control-mode=explicit"
    +          "--strict-model-config=false"
    +          "--strict-readiness=false"
    +          "--allow-http=true"
    +          "--allow-sagemaker=false"
    +          '
    +      volumeMounts:
    +        - name: shm
    +          mountPath: /dev/shm
    +      resources:
    +        requests:
    +          cpu: 500m
    +          memory: 1Gi
    +        limits:
    +          cpu: "5"
    +          memory: 1Gi
    +      livenessProbe:
    +        # the server is listening only on 127.0.0.1, so an httpGet probe sent
    +        # from the kublet running on the node cannot connect to the server
    +        # (not even with the Host header or host field)
    +        # exec a curl call to have the request originate from localhost in the
    +        # container
    +        exec:
    +          command:
    +            - curl
    +            - --fail
    +            - --silent
    +            - --show-error
    +            - --max-time
    +            - "9"
    +            - http://localhost:8000/v2/health/live
    +        initialDelaySeconds: 5
    +        periodSeconds: 30
    +        timeoutSeconds: 10
    +  builtInAdapter:
    +    serverType: triton
    +    runtimeManagementPort: 8001
    +    memBufferBytes: 134217728
    +    modelLoadingTimeoutMillis: 90000
    +

    +
  10. +
  11. You will likely want to update the name , as well as other parameters.
  12. +
  13. Click Add
  14. +
  15. +

    Confirm the new Runtime is in the list, and re-order the list as needed. + (the order chosen here is the order in which the users will see these choices)

    +

    Runtime List

    +
  16. +
+

Creating a project

+
    +
  • Create a new Data Science Project
  • +
  • In this example, the project is called fraud
  • +
+

Creating a model server

+
    +
  1. In your project, scroll down to the "Models and Model Servers" Section
  2. +
  3. +

    Click on Configure server

    +

    Configure Server

    +
  4. +
  5. +

    Fill out the details:

    +

    Server Details - 1

    +

    Server Details - 2

    +
  6. +
  7. +

    Click Configure

    +
  8. +
+

Deploying a model into it

+
    +
  1. If you don't have any model files handy, you can grab a copy of this file and upload it to your Object Storage of choice.
  2. +
  3. +

    Click on Deploy Model

    +

    Deploy a Model

    +
  4. +
  5. +

    Choose a model name and a framework:

    +

    Configure a Model

    +
  6. +
  7. +

    Then create a new data connection containing the details of where your model is stored in Object Storage:

    +

    Create a Data Connection

    +
  8. +
  9. +

    After a little while, you should see the following:

    +

    Deploy model

    +
  10. +
+

Validating the model

+
    +
  1. If you've used the model mentioned earlier in this document, you can run the following command from a Linux prompt: +
    function val-model {
    +    myhost="$1"
    +    echo "validating host $myhost"
    +    time curl -X POST -k "${myhost}" -d '{"inputs": [{ "name": "dense_input", "shape": [1, 7], "datatype": "FP32", "data": [57.87785658389723,0.3111400080477545,1.9459399775518593,1.0,1.0,0.0,0.0]}]}' | jq
    +}
    +
    +val-model "https://fraud-model-fraud.apps.mycluster.openshiftapps.com/v2/models/fraud-model/infer"
    +
  2. +
  3. Change the host to match the address for your model.
  4. +
  5. You should see an output similar to: +
    {
    +  "model_name": "fraud-model__isvc-c1529f9667",
    +  "model_version": "1",
    +  "outputs": [
    +    {
    +      "name": "dense_3",
    +      "datatype": "FP32",
    +      "shape": [
    +        1,
    +        1
    +      ],
    +      "data": [
    +        0.86280495
    +      ]
    +    }
    +  ]
    +}
    +
  6. +
+

Extra considerations for Disconnected environments.

+

The YAML included in this file makes a reference to the following Nvidia Triton Image: +nvcr.io/nvidia/tritonserver:23.05-py3

+

Ensure that this image is properly mirrored into the mirror registry.

+

Also, update the YAML definition as needed to point to the image address that matches the image registry.

+ +

Each of the activities performed via the user interface will create a Kubernetes Object inside your OpenShift Cluster.

+
    +
  • The addition of a new runtime creates a template in the redhat-ods-applications namespace.
  • +
  • Each model server is defined as a ServingRuntime
  • +
  • Each model is defined as an InferenceService
  • +
  • Each Data Connection is stored as a Secret
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/from-zero-to-workbench/using-cli/index.html b/odh-rhoai/from-zero-to-workbench/using-cli/index.html new file mode 100644 index 00000000..d90dab9d --- /dev/null +++ b/odh-rhoai/from-zero-to-workbench/using-cli/index.html @@ -0,0 +1,2781 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Using the CLI - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

From Zero To Workbench using the CLI

+

In this repo, you will find all the straightforward instructions to quickly deploy OpenShift AI only using a CLI or automation.

+

Detailed instructions, YAMLs, code, all is there to get you quickly started!

+
+

Note

+

This documentation was produced working off of mainly OpenShift v4.14 and OpenShift AI 2.7. The artifacts directory contains sample configuration files which are used throughout this repo to demonstrate various concepts. While an effort will be made to ensure that these artifacts stay up to date, there is a possibility that they will not always work as intended.

+

Suggestions and pull requests are welcome to maintain this content up to date!

+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/from-zero-to-workbench/using-developer-hub/index.html b/odh-rhoai/from-zero-to-workbench/using-developer-hub/index.html new file mode 100644 index 00000000..2e827abd --- /dev/null +++ b/odh-rhoai/from-zero-to-workbench/using-developer-hub/index.html @@ -0,0 +1,2781 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Using Developer Hub - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

OpenShift AI on Developer Hub

+

In this repo, you will find a Backstage Golden Path Template for a Red Hat OpenShift AI Data science project. +Detailed instructions, examples, all is there to get you quickly started!

+
+

Note

+

This documentation is a work in progress that provides recipes for some components only.

+

Suggestions and pull requests are welcome to maintain this content up to date and make it evolve!

+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/from-zero-to-workbench/using-ui/index.html b/odh-rhoai/from-zero-to-workbench/using-ui/index.html new file mode 100644 index 00000000..97002202 --- /dev/null +++ b/odh-rhoai/from-zero-to-workbench/using-ui/index.html @@ -0,0 +1,2781 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Using the UI - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

From Zero To Workbench using the GUI

+

In this repo, you will find all the straightforward instructions to quickly deploy OpenShift AI using mainly the OpenShift Console UI.

+

Detailed instructions, screenshots, examples, all is there to get you quickly started!

+
+

Note

+

This documentation was produced working off of mainly OpenShift v4.14 and OpenShift AI 2.7. The artifacts directory contains sample configuration files which are used throughout this repo to demonstrate various concepts. While an effort will be made to ensure that these artifacts stay up to date, there is a possibility that they will not always work as intended.

+

Suggestions and pull requests are welcome to maintain this content up to date!

+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/gitops/index.html b/odh-rhoai/gitops/index.html new file mode 100644 index 00000000..5253deb7 --- /dev/null +++ b/odh-rhoai/gitops/index.html @@ -0,0 +1,3956 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + GitOps (CRs, objects,...) - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Managing RHOAI with GitOps

+

GitOps is a common way to manage and deploy applications and resouces on Kubernetes clusters.

+

This page is intended to provide an overview of the different objects involved in manaing both the installation, administration, and usage of OpenShift AI components using GitOps. This page is by no means intended to be an exhaustive tutorial on each object and all of the features available in them.

+

When first implimenting features with GitOps it is highly recommended to deploy the resources manually using the Dashboard, then extract the resources created by the Dashboard and duplicate them in your gitops repo.

+

Installation

+

Operator Installation

+

The Red Hat OpenShift AI operator is installed and managed by OpenShift's Operator Lifecycle Manager (OLM) and follows common patterns that can be used to install many different operators.

+

The Red Hat OpenShift AI operator should be installed in the redhat-ods-operator namespace:

+
apiVersion: v1
+kind: Namespace
+metadata:
+  annotations:
+    openshift.io/display-name: "Red Hat OpenShift AI"
+  labels:
+    openshift.io/cluster-monitoring: 'true'
+  name: redhat-ods-operator
+
+

After creating the namespace, OLM requires you create an Operator Group to help manage any operators installed in that namespace:

+
apiVersion: operators.coreos.com/v1
+kind: OperatorGroup
+metadata:
+  name: redhat-ods-operator-group
+  namespace: redhat-ods-operator
+
+

Finally, a Subscription can be created to install the operator:

+
apiVersion: operators.coreos.com/v1alpha1
+kind: Subscription
+metadata:
+  name: rhods-operator
+  namespace: redhat-ods-operator
+spec:
+  channel: stable # <1>
+  installPlanApproval: Automatic # <2>
+  name: rhods-operator
+  source: redhat-operators
+  sourceNamespace: openshift-marketplace
+
+

Subscription Options:

+
    +
  1. Operator versions are managed with the channel in OLM. Users are able to select a channel that corresponds to the upgrade lifecycle they wish to follow and OLM will update versions as they are released on that channel. To learn more about the available channels and the release lifecycle, please refer to the official lifecycle documentation
  2. +
  3. Platform administrators also have an option to set how upgrades are applied for the operator with the installPlanApproval. If set to Automatic RHOAI is automatically updated to the latest version that is available on the selected channel. If set to Manual administrators will be required to approve all upgrades.
  4. +
+

Component Configuration

+

When the operator is installed it automatically creates a DSCInitialization object that sets up several default configurations. While it is not required, administrators can choose to manage the DSCinitialization object via GitOps.

+
apiVersion: dscinitialization.opendatahub.io/v1
+kind: DSCInitialization
+metadata:
+  name: default-dsci
+spec:
+  applicationsNamespace: redhat-ods-applications
+  monitoring:
+    managementState: Managed
+    namespace: redhat-ods-monitoring
+  serviceMesh:
+    auth:
+      audiences:
+        - 'https://kubernetes.default.svc'
+    controlPlane:
+      metricsCollection: Istio
+      name: data-science-smcp
+      namespace: istio-system
+    managementState: Managed # <1>
+  trustedCABundle:
+    customCABundle: ''
+    managementState: Managed # <2>
+
+

DSCInitialization Options:

+
    +
  1. KServe requires a ServiceMesh instance to be installed on the cluster. By default the Red Hat OpenShift AI operator will attempt to configure an instance if the ServiceMesh operator is installed. If your cluster already has ServiceMesh configured, you may choose to skip this option.
  2. +
  3. As part of the ServiceMesh configuration, the Red Hat OpenShift AI operator will configure a self-signed cert for any routes created by ServiceMesh.
  4. +
+

After the operator is installed, a DataScienceCluster object will need to be configured with the different components. Each component has a managementState option which can be set to Managed or Removed. Admins can choose which components are installed on the cluster.

+
kind: DataScienceCluster
+apiVersion: datasciencecluster.opendatahub.io/v1
+metadata:
+  name: default
+spec:
+  components:
+    codeflare:
+      managementState: Managed
+    kserve:
+      managementState: Managed
+      serving:
+        ingressGateway:
+          certificate:
+            type: SelfSigned
+        managementState: Managed
+        name: knative-serving
+    trustyai:
+      managementState: Removed
+    ray:
+      managementState: Managed
+    kueue:
+      managementState: Managed
+    workbenches:
+      managementState: Managed
+    dashboard:
+      managementState: Managed
+    modelmeshserving:
+      managementState: Managed
+    datasciencepipelines:
+      managementState: Managed
+
+

After the DataScienceCluster object is created, the operator will install and configure the different components on the cluster. Only one DataScienceCluster object can be created on a cluster.

+

Administration

+

Dashboard Configs

+

The Red Hat OpenShift AI Dashboard has many different configurable options through the UI that can be managed using the OdhDashboardConfig config object. A default OdhDashboardConfig is created when the Dashboard component is installed

+
apiVersion: opendatahub.io/v1alpha
+kind: OdhDashboardConfig
+metadata:
+  name: odh-dashboard-config
+  namespace: redhat-ods-applications
+  labels:
+    app.kubernetes.io/part-of: rhods-dashboard
+    app.opendatahub.io/rhods-dashboard: 'true'
+spec:
+  dashboardConfig:
+    enablement: true
+    disableDistributedWorkloads: false
+    disableProjects: false
+    disableBiasMetrics: false
+    disableSupport: false
+    disablePipelines: false
+    disableProjectSharing: false
+    disableModelServing: false
+    disableKServe: false
+    disableAcceleratorProfiles: false
+    disableCustomServingRuntimes: false
+    disableModelMesh: false
+    disableKServeAuth: false
+    disableISVBadges: false
+    disableInfo: false
+    disableClusterManager: false
+    disablePerformanceMetrics: false
+    disableBYONImageStream: false
+    disableModelRegistry: true
+    disableTracking: false
+  groupsConfig:
+    adminGroups: rhods-admins # <1>
+    allowedGroups: 'system:authenticated' # <2>
+  modelServerSizes: # <3>
+    - name: Small
+      resources:
+        limits:
+          cpu: '2'
+          memory: 8Gi
+        requests:
+          cpu: '1'
+          memory: 4Gi
+    - name: Medium
+      resources:
+        limits:
+          cpu: '8'
+          memory: 10Gi
+        requests:
+          cpu: '4'
+          memory: 8Gi
+    - name: Large
+      resources:
+        limits:
+          cpu: '10'
+          memory: 20Gi
+        requests:
+          cpu: '6'
+          memory: 16Gi
+  notebookController:
+    enabled: true
+    notebookNamespace: rhods-notebooks
+    pvcSize: 20Gi # <4>
+  notebookSizes: # <5>
+    - name: Small
+      resources:
+        limits:
+          cpu: '2'
+          memory: 8Gi
+        requests:
+          cpu: '1'
+          memory: 8Gi
+    - name: Medium
+      resources:
+        limits:
+          cpu: '6'
+          memory: 24Gi
+        requests:
+          cpu: '3'
+          memory: 24Gi
+    - name: Large
+      resources:
+        limits:
+          cpu: '14'
+          memory: 56Gi
+        requests:
+          cpu: '7'
+          memory: 56Gi
+    - name: X Large
+      resources:
+        limits:
+          cpu: '30'
+          memory: 120Gi
+        requests:
+          cpu: '15'
+          memory: 120Gi
+  templateDisablement: []
+  templateOrder:
+    - caikit-tgis-runtime
+    - kserve-ovms
+    - ovms
+    - tgis-grpc-runtime
+    - vllm-runtime
+
+

OdhDashboardConfig Options:

+
    +
  1. The Dashboard creates a group called rhods-admins by default which users can be added to be granted admin privileges through the Dashboard. Additionally, any user with the cluster-admin role are admins in the Dashboard by default. If you wish to change the group which is used to manage admin access, this option can be updated. It is important to note that this field only impacts a users ability to modify settings in the Dashboard, and will have no impact to a users ability to modify configurations through the Kubernetes objects such as this OdhDashboardConfig object.
  2. +
  3. By default any user that has access to the OpenShift cluster where Red Hat OpenShift AI is installed will have the ability to access the Dashboard. If you wish to restrict who has access to the Dashboard this option can be updated to another group. Like the admin group option, this option only impacts the users ability to access the Dashboard and does not restrict their ability to interact directly with the Kubernetes objects used to deploy AI resources.
  4. +
  5. When a user creates a new Model Server through the Dashboard they are presented with an option to choose a server size which will impact the resources available to the pod created for the Model Server. Administrators have the ability to configure the default options that are available to their users.
  6. +
  7. When creating a new Workbench, users are asked to create storage for their Workbench. The storage will default to the value set here and users will have the option to choose a different amount of storage if their use case requires more or less storage. Admins can choose another default storage size that is presented to users by configuring this option.
  8. +
  9. Like the Model Server size, users are presented with a drop down menu of options to select what size of Workbench they wish to create. Admins have the ability to customize the size options that are presented to users.
  10. +
+

Idle Notebook Culling

+

Admins have the ability to enable Idle Notebook Culling which will automatically stop any Notebooks/Workbenches that users haven't interacted with in a period of time by creating the following ConfigMap:

+
kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: notebook-controller-culler-config
+  namespace: redhat-ods-applications
+  labels:
+    opendatahub.io/dashboard: 'true'
+data:
+  CULL_IDLE_TIME: '240' # <1>
+  ENABLE_CULLING: 'true'
+  IDLENESS_CHECK_PERIOD: '1'
+
+

Idle Notebook Culling Options:

+
    +
  1. The CULL_IDLE_TIME looks for metrics from Jupyter to understand when the last time a user interacted with the Workbench and will shut the pod down if it has passed the time set here. The time is the number of minutes so 240 minutes or 4 hours.
  2. +
+

Accelerator Profiles

+

Accelerator Profiles allow admins to configure different types of GPU options that they can present to end users and automatically configure a toleration on Workbenches or Model Servers when they are selected. Admins can configure an Accelerator Profile with the AcceleratorProfile object:

+
apiVersion: dashboard.opendatahub.io/v1
+kind: AcceleratorProfile
+metadata:
+  name: nvidia-gpu
+  namespace: redhat-ods-applications
+spec:
+  displayName: nvidia-gpu
+  enabled: true
+  identifier: nvidia.com/gpu
+  tolerations:
+    - effect: NoSchedule
+      key: nvidia-gpu-only
+      operator: Exists
+      value: ''
+
+

Notebook Images

+

Red Hat OpenShift AI ships with several out of the box Notebook/Workbench Images but admins can create additional custom images that users can use to launch new Workbench instances. A Notebook Image is managed with an OpenShift ImageStream object with some required labels:

+
kind: ImageStream
+apiVersion: image.openshift.io/v1
+metadata:
+  annotations:
+    opendatahub.io/notebook-image-desc: A custom Jupyter Notebook built for my organization # <1>
+    opendatahub.io/notebook-image-name: My Custom Notebook # <2>
+  name: my-custom-notebook
+  namespace: redhat-ods-applications
+  labels: # <3>
+    app.kubernetes.io/created-by: byon
+    opendatahub.io/dashboard: 'true'
+    opendatahub.io/notebook-image: 'true'
+spec:
+  lookupPolicy:
+    local: true
+  tags:
+    - name: '1.0' # <4>
+      annotations:
+        opendatahub.io/notebook-python-dependencies: '[{"name":"PyTorch","version":"2.2"}]' # <5>
+        opendatahub.io/notebook-software: '[{"name":"Python","version":"v3.11"}]' # <6>
+        opendatahub.io/workbench-image-recommended: 'true' # <7>
+      from:
+        kind: DockerImage
+        name: 'quay.io/my-org/my-notebook:latest' # <8>
+      importPolicy:
+        importMode: Legacy
+      referencePolicy:
+        type: Source
+
+

Notebook Image Options:

+
    +
  1. A description for the purpose of the notebook image
  2. +
  3. The name that will be displayed to end users in the drop down menu when creating a Workbench
  4. +
  5. The notebook image requires several labels for them to appear in the Dashboard, including the app.kubernetes.io/created-by: byon label. While traditionally this label is utilized to trace where an object originated from, this label is required for the notebooks to be made available to end users.
  6. +
  7. Multiple image versions can be configured as part of the same Notebook and users have the ability to select which version of the image they wish to use. This is helpful if you release updated versions of the Notebook image and you wish to avoid breaking end user environments with package changes and allow them to upgrade as they wish.
  8. +
  9. When selecting a Notebook image users will be presented with some information about the notebook based on the information presented in this annotation. opendatahub.io/notebook-python-dependencies is most commonly used to present information about versions from the most important Python packages that are pre-installed in the Image.
  10. +
  11. Like the python dependencies annotation, the opendatahub.io/notebook-software annotation is used to present the end user with information about what software is installed in the Image. Most commonly this field is used to present information such as the Python version, Jupyter versions, or CUDA versions.
  12. +
  13. When multiple tags are created on the ImageStream, the opendatahub.io/workbench-image-recommended is used to control what version of the image is presented by default to end users. Only one tag should be set to true at any give time.
  14. +
  15. Notebook images are generally recommended to be stored in an Image Registry outside of the cluster and referenced in the ImageStream.
  16. +
+

While it is possible to build a Notebook Image on an OpenShift cluster and publish it directly to an ImageStream using a BuildConfig or a Tekton Pipeline, it can be challenging to get that image to be seen by the Red Hat OpenShift AI Dashboard. The Dashboard is only looks at images listed in the spec.tags section and images pushed directly to the internal image registry are recorded in the status.tags. As a work around, it is possible to "link" a tag pushed directly to the internal image registry to a tag that is visible by the Dashboard:

+
kind: ImageStream
+apiVersion: image.openshift.io/v1
+metadata:
+  annotations:
+    opendatahub.io/notebook-image-desc: A custom Jupyter Notebook built for my organization
+    opendatahub.io/notebook-image-name: My Custom Notebook
+  name: my-custom-notebook
+  namespace: redhat-ods-applications
+  labels:
+    app.kubernetes.io/created-by: byon
+    opendatahub.io/dashboard: 'true'
+    opendatahub.io/notebook-image: 'true'
+spec:
+  lookupPolicy:
+    local: false
+  tags:
+    - name: '1.0'
+      annotations:
+        opendatahub.io/notebook-python-dependencies: '[{"name":"PyTorch","version":"2.2"}]'
+        opendatahub.io/notebook-software: '[{"name":"Python","version":"v3.11"}]'
+        opendatahub.io/workbench-image-recommended: 'true'
+      from:
+        kind: DockerImage
+        name: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/my-custom-workbench:latest'
+      importPolicy:
+        importMode: Legacy
+      referencePolicy:
+        type: Source
+status:
+  dockerImageRepository: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/dsp-example'
+  tags:
+    - tag: latest
+
+

Serving Runtime Templates

+

Red Hat OpenShift AI ships with several out of the box Serving Runtime Templates such as OpenVino and vLLM, but admins have the ability to configure additional templates that allow users to deploy additional ServingRuntimes. A Serving Runtime template is an OpenShift Template object that wraps around a ServingRuntime object:

+
kind: Template
+apiVersion: template.openshift.io/v1
+metadata:
+  name: trition-serving-runtime
+  namespace: redhat-ods-applications
+  labels:
+    opendatahub.io/dashboard: 'true'
+  annotations:
+    opendatahub.io/apiProtocol: REST
+    opendatahub.io/modelServingSupport: '["multi"]'
+objects:
+  - apiVersion: serving.kserve.io/v1alpha1
+    kind: ServingRuntime
+    metadata:
+      name: triton-23.05
+      labels:
+        name: triton-23.05
+      annotations:
+        maxLoadingConcurrency: '2'
+        openshift.io/display-name: Triton runtime 23.05
+    spec:
+      supportedModelFormats:
+        - name: keras
+          version: '2'
+          autoSelect: true
+        - name: onnx
+          version: '1'
+          autoSelect: true
+        - name: pytorch
+          version: '1'
+          autoSelect: true
+        - name: tensorflow
+          version: '1'
+          autoSelect: true
+        - name: tensorflow
+          version: '2'
+          autoSelect: true
+        - name: tensorrt
+          version: '7'
+          autoSelect: true
+      protocolVersions:
+        - grpc-v2
+      multiModel: true
+      grpcEndpoint: 'port:8085'
+      grpcDataEndpoint: 'port:8001'
+      volumes:
+        - name: shm
+          emptyDir:
+            medium: Memory
+            sizeLimit: 2Gi
+      containers:
+        - name: triton
+          image: 'nvcr.io/nvidia/tritonserver:23.05-py3'
+          command:
+            - /bin/sh
+          args:
+            - '-c'
+            - 'mkdir -p /models/_triton_models; chmod 777 /models/_triton_models; exec tritonserver "--model-repository=/models/_triton_models" "--model-control-mode=explicit" "--strict-model-config=false" "--strict-readiness=false" "--allow-http=true" "--allow-sagemaker=false" '
+          volumeMounts:
+            - name: shm
+              mountPath: /dev/shm
+          resources:
+            requests:
+              cpu: 500m
+              memory: 1Gi
+            limits:
+              cpu: '5'
+              memory: 1Gi
+          livenessProbe:
+            exec:
+              command:
+                - curl
+                - '--fail'
+                - '--silent'
+                - '--show-error'
+                - '--max-time'
+                - '9'
+                - 'http://localhost:8000/v2/health/live'
+            initialDelaySeconds: 5
+            periodSeconds: 30
+            timeoutSeconds: 10
+      builtInAdapter:
+        serverType: triton
+        runtimeManagementPort: 8001
+        memBufferBytes: 134217728
+        modelLoadingTimeoutMillis: 90000
+
+

End User Resources

+

Data Science Projects

+

Data Science Projects are simply a normal OpenShift Project with an extra label to distinguish them from normal OpenShift projects by the Red Hat OpenShift AI Dashboard. Like OpenShift Projects it is recommended to create a namespace object and allow OpenShift to create the corresponding project object:

+
apiVersion: v1
+kind: Namespace
+metadata:
+  name: my-data-science-project
+  labels:
+    opendatahub.io/dashboard: "true"
+
+

Additionally, when a project going to be utilized by ModelMesh for Multi-model serving, there is an additional ModelMesh label that should be applied to the namespace:

+
apiVersion: v1
+kind: Namespace
+metadata:
+  name: my-multi-model-serving-project
+  labels:
+    opendatahub.io/dashboard: "true"
+    modelmesh-enabled: "true"
+
+

Workbenches

+

Workbench objects are managed using the Notebook custom resource. The Notebook object contains a fairly complex configuration, with many items that will be autogenerated, and required annotations to display correctly in the Dashboard. The Notebook object essentially acts as a wrapper around a normal pod definition and you will find many similarities to managing a pod with options such as the image, pvcs, secrets, etc.

+

It is highly recommended to thoroughly test any Notebook configurations configured with GitOps.

+
apiVersion: kubeflow.org/v1
+kind: Notebook
+metadata:
+  annotations:
+    notebooks.opendatahub.io/inject-oauth: 'true' # <1>
+    opendatahub.io/image-display-name: Minimal Python
+    notebooks.opendatahub.io/oauth-logout-url: 'https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com/projects/my-data-science-project?notebookLogout=my-workbench'
+    opendatahub.io/accelerator-name: ''
+    openshift.io/description: ''
+    openshift.io/display-name: my-workbench
+    notebooks.opendatahub.io/last-image-selection: 's2i-minimal-notebook:2024.1'
+    notebooks.kubeflow.org/last_activity_check_timestamp: '2024-07-30T20:43:25Z'
+    notebooks.opendatahub.io/last-size-selection: Small
+    opendatahub.io/username: 'kube:admin'
+    notebooks.kubeflow.org/last-activity: '2024-07-30T20:27:25Z'
+  name: my-workbench
+  namespace: my-data-science-project
+spec:
+  template:
+    spec:
+      affinity: {}
+      containers:
+        - resources: # <2>
+            limits:
+              cpu: '2'
+              memory: 8Gi
+            requests:
+              cpu: '1'
+              memory: 8Gi
+          readinessProbe:
+            failureThreshold: 3
+            httpGet:
+              path: /notebook/my-data-science-project/my-workbench/api
+              port: notebook-port
+              scheme: HTTP
+            initialDelaySeconds: 10
+            periodSeconds: 5
+            successThreshold: 1
+            timeoutSeconds: 1
+          name: my-workbench
+          livenessProbe:
+            failureThreshold: 3
+            httpGet:
+              path: /notebook/my-data-science-project/my-workbench/api
+              port: notebook-port
+              scheme: HTTP
+            initialDelaySeconds: 10
+            periodSeconds: 5
+            successThreshold: 1
+            timeoutSeconds: 1
+          env:
+            - name: NOTEBOOK_ARGS
+              value: |-
+                --ServerApp.port=8888
+                                  --ServerApp.token=''
+                                  --ServerApp.password=''
+                                  --ServerApp.base_url=/notebook/my-data-science-project/my-workbench
+                                  --ServerApp.quit_button=False
+                                  --ServerApp.tornado_settings={"user":"kube-3aadmin","hub_host":"https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com","hub_prefix":"/projects/my-data-science-project"}
+            - name: JUPYTER_IMAGE
+              value: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/s2i-minimal-notebook:2024.1'
+            - name: PIP_CERT
+              value: /etc/pki/tls/custom-certs/ca-bundle.crt
+            - name: REQUESTS_CA_BUNDLE
+              value: /etc/pki/tls/custom-certs/ca-bundle.crt
+            - name: SSL_CERT_FILE
+              value: /etc/pki/tls/custom-certs/ca-bundle.crt
+            - name: PIPELINES_SSL_SA_CERTS
+              value: /etc/pki/tls/custom-certs/ca-bundle.crt
+            - name: GIT_SSL_CAINFO
+              value: /etc/pki/tls/custom-certs/ca-bundle.crt
+          ports:
+            - containerPort: 8888
+              name: notebook-port
+              protocol: TCP
+          imagePullPolicy: Always
+          volumeMounts:
+            - mountPath: /opt/app-root/src
+              name: my-workbench
+            - mountPath: /dev/shm
+              name: shm
+            - mountPath: /etc/pki/tls/custom-certs/ca-bundle.crt
+              name: trusted-ca
+              readOnly: true
+              subPath: ca-bundle.crt
+          image: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/s2i-minimal-notebook:2024.1' # <3>
+          workingDir: /opt/app-root/src
+        - resources: # <4>
+            limits:
+              cpu: 100m
+              memory: 64Mi
+            requests:
+              cpu: 100m
+              memory: 64Mi
+          readinessProbe:
+            failureThreshold: 3
+            httpGet:
+              path: /oauth/healthz
+              port: oauth-proxy
+              scheme: HTTPS
+            initialDelaySeconds: 5
+            periodSeconds: 5
+            successThreshold: 1
+            timeoutSeconds: 1
+          name: oauth-proxy
+          livenessProbe:
+            failureThreshold: 3
+            httpGet:
+              path: /oauth/healthz
+              port: oauth-proxy
+              scheme: HTTPS
+            initialDelaySeconds: 30
+            periodSeconds: 5
+            successThreshold: 1
+            timeoutSeconds: 1
+          env:
+            - name: NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+          ports:
+            - containerPort: 8443
+              name: oauth-proxy
+              protocol: TCP
+          imagePullPolicy: Always
+          volumeMounts:
+            - mountPath: /etc/oauth/config
+              name: oauth-config
+            - mountPath: /etc/tls/private
+              name: tls-certificates
+          image: 'registry.redhat.io/openshift4/ose-oauth-proxy@sha256:4bef31eb993feb6f1096b51b4876c65a6fb1f4401fee97fa4f4542b6b7c9bc46'
+          args:
+            - '--provider=openshift'
+            - '--https-address=:8443'
+            - '--http-address='
+            - '--openshift-service-account=my-workbench'
+            - '--cookie-secret-file=/etc/oauth/config/cookie_secret'
+            - '--cookie-expire=24h0m0s'
+            - '--tls-cert=/etc/tls/private/tls.crt'
+            - '--tls-key=/etc/tls/private/tls.key'
+            - '--upstream=http://localhost:8888'
+            - '--upstream-ca=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt'
+            - '--email-domain=*'
+            - '--skip-provider-button'
+            - '--openshift-sar={"verb":"get","resource":"notebooks","resourceAPIGroup":"kubeflow.org","resourceName":"my-workbench","namespace":"$(NAMESPACE)"}'
+            - '--logout-url=https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com/projects/my-data-science-project?notebookLogout=my-workbench'
+      enableServiceLinks: false
+      serviceAccountName: my-workbench
+      volumes:
+        - name: my-workbench
+          persistentVolumeClaim:
+            claimName: my-workbench
+        - emptyDir:
+            medium: Memory
+          name: shm
+        - configMap:
+            items:
+              - key: ca-bundle.crt
+                path: ca-bundle.crt
+            name: workbench-trusted-ca-bundle
+            optional: true
+          name: trusted-ca
+        - name: oauth-config
+          secret:
+            defaultMode: 420
+            secretName: my-workbench-oauth-config
+        - name: tls-certificates
+          secret:
+            defaultMode: 420
+            secretName: my-workbench-tls
+
+
    +
  1. The Notebook object contains several different annotations that are used by OpenShift AI, but the inject-oauth annotation is one of the most important. There are several oauth based configurations that in the Notebook that will be automatically generated by this annotation, allowing you to exclude a large amount of notebook configuration from what is contained in your GitOps repo.
  2. +
  3. While selecting the resource size through the Dashboard you have more limited options for what sizes you can select, you can choose any size you wish for your notebook through the YAML. By selecting a non-standard size the Dashboard may report an "unknown" size however.
  4. +
  5. Just like the resources size, you can choose any number of images for the Notebook, including ones that are not available in the Dashboard. By selecting a non-standard notebook image the Dashboard may report issues however.
  6. +
  7. The oauth-proxy container is one such item that can be removed from the gitops based configuration when utilizing the inject-oauth annotation. Instead of including this section and some other oauth related configurations, you can simply rely on the annotation, and allow the Notebook controller to manage this portion of the object for you. This will help to prevent problems when upgrading RHOAI.
  8. +
+

Users have the ability to start and stop the Workbench to help conserve resources on the cluster. To stop a Notebook, the following annotation should be applied to the Notebook object:

+
metadata:
+  annotations:
+    kubeflow-resource-stopped: '2024-07-30T20:52:37Z'
+
+

Generally, you do not want to include this annotation in your GitOps configuration, as it will enforce the Notebook to be shutdown, not allowing users to start their Notebooks. The value of the annotation doesn't matter, but by default the Dashboard will apply a timestamp with the time the Notebook was shut down.

+

Data Science Connections

+

A Data Science Connection is a normal Kubernetes Secret object with several annotations that follow a specific format for the data.

+
kind: Secret
+apiVersion: v1
+type: Opaque
+metadata:
+  name: aws-connection-my-dataconnection # <1>
+  labels:
+    opendatahub.io/dashboard: 'true' # <2>
+    opendatahub.io/managed: 'true'
+  annotations:
+    opendatahub.io/connection-type: s3 # <3>
+    openshift.io/display-name: my-dataconnection # <4>
+data: # <5>
+  AWS_ACCESS_KEY_ID: dGVzdA==
+  AWS_DEFAULT_REGION: 'dGVzdA=='
+  AWS_S3_BUCKET: 'dGVzdA=='
+  AWS_S3_ENDPOINT: dGVzdA==
+  AWS_SECRET_ACCESS_KEY: dGVzdA==
+
+
    +
  1. When creating a data connection through the Dashboard, the name is automatically generated as aws-connection-<your-entered-name>. When generating the data connection from outside of the Dashboard, you do not need to follow this naming convention.
  2. +
  3. The opendatahub.io/dashboard: 'true' label is used to help determine what secrets to display in the Dashboard. This option must be set to true if you wish for it to be available in the UI.
  4. +
  5. At this point in time, the Dashboard only supports the S3 as a connection-type, but other types may be supported in the future.
  6. +
  7. The name of the data connection as it will appear in the Dashboard UI
  8. +
  9. Like all secrets, data connections data is stored in a base64 encoding. This data is not secure to be stored in this format and users should instead look into tools such as SealedSecrets or ExternalSecrets to manage secret data in a gitops workflow.
  10. +
+

Data Science Pipelines

+

When setting up a new project, a Data Science Pipeline instance needs to be created using the DataSciencePipelineApplication object. The DSPA will create the pipeline servers for the project and allow users to begin interacting with Data Science Pipelines.

+
apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1
+kind: DataSciencePipelinesApplication
+metadata:
+  name: dspa # <1>
+  namespace: my-data-science-project
+spec:
+  apiServer:
+    caBundleFileMountPath: ''
+    stripEOF: true
+    dbConfigConMaxLifetimeSec: 120
+    applyTektonCustomResource: true
+    caBundleFileName: ''
+    deploy: true
+    enableSamplePipeline: false
+    autoUpdatePipelineDefaultVersion: true
+    archiveLogs: false
+    terminateStatus: Cancelled
+    enableOauth: true
+    trackArtifacts: true
+    collectMetrics: true
+    injectDefaultScript: true
+  database:
+    disableHealthCheck: false
+    mariaDB:
+      deploy: true
+      pipelineDBName: mlpipeline
+      pvcSize: 10Gi
+      username: mlpipeline
+  dspVersion: v2
+  objectStorage:
+    disableHealthCheck: false
+    enableExternalRoute: false
+    externalStorage: # <2>
+      basePath: ''
+      bucket: pipelines
+      host: 'minio.ai-example-training.svc.cluster.local:9000'
+      port: ''
+      region: us-east-1
+      s3CredentialsSecret:
+        accessKey: AWS_SECRET_ACCESS_KEY
+        secretKey: AWS_ACCESS_KEY_ID
+        secretName: aws-connection-my-dataconnection
+      scheme: http
+  persistenceAgent:
+    deploy: true
+    numWorkers: 2
+  scheduledWorkflow:
+    cronScheduleTimezone: UTC
+    deploy: true
+
+
    +
  1. The Dashboard expects to look for an object called dspa and it is not recommended to deploy more than a single DataSciencePipelineApplication object in a single namespace.
  2. +
  3. The externalStorage is a critical configuration for setting up S3 backend storage for Data Science Pipelines. While using the dashboard you are required to configure the connection details. While you can import these details from a data connection, it will create a separate secret containing the s3 secrets instead of reusing the existing data connection secret.
  4. +
+

Once a Data Science Pipeline instance has been created, users may wish to configure and manage their pipelines via GitOps. It is important to note that Data Science Pipelines is not "gitops friendly". While working with Elyra or a kfp pipeline, users are required to manually upload a pipeline file to the Dashboard which does not generate a corresponding Kubernetes object. Additionally, when executing a pipeline run, uses may find a ArgoWorkflow object that is generated for the run, however this object can not be re-used in a gitops application to create a new pipeline run in Data Science Pipelines.

+

As a work around, one common pattern to "gitops-ify" a Data Science Pipeline while using kfp is to instead create a Tekton pipeline that either compiles the pipeline, and uses the kfp skd to upload the pipeline to Data Science Pipelines, or the kfp sdk can automatically trigger a new pipeline run directly from your pipeline code.

+

Model Serving

+

Model Serving in RHOAI has two different flavors, Single Model Serving (KServe) and Multi-Model Serving (ModelMesh). Both model server options utilize the same Kubernetes objects (ServingRuntime and InferenceService), but have different controllers managing them.

+

As mentioned in the Data Science Project section, in order to utilize ModelMesh, a modelmesh-enabled label must be applied to the namespace:

+
apiVersion: v1
+kind: Namespace
+metadata:
+  name: my-multi-model-serving-project
+  labels:
+    opendatahub.io/dashboard: "true"
+    modelmesh-enabled: "true"
+
+

When creating a model server through the Dashboard, users can select a "Serving Runtime Template" which will create a ServingRuntime instance in their namespace which can be managed via GitOps. The ServingRuntime helps to define different things such as the container definition, the supported model types, and available ports.

+
apiVersion: serving.kserve.io/v1alpha1
+kind: ServingRuntime
+metadata:
+  annotations: # <1>
+    enable-route: 'true'
+    opendatahub.io/accelerator-name: ''
+    opendatahub.io/apiProtocol: REST
+    opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]'
+    opendatahub.io/template-display-name: OpenVINO Model Server
+    opendatahub.io/template-name: ovms
+    openshift.io/display-name: multi-model-server
+  name: multi-model-server
+  labels:
+    opendatahub.io/dashboard: 'true'
+spec:
+  supportedModelFormats:
+    - autoSelect: true
+      name: openvino_ir
+      version: opset1
+    - autoSelect: true
+      name: onnx
+      version: '1'
+    - autoSelect: true
+      name: tensorflow
+      version: '2'
+  builtInAdapter:
+    env:
+      - name: OVMS_FORCE_TARGET_DEVICE
+        value: AUTO
+    memBufferBytes: 134217728
+    modelLoadingTimeoutMillis: 90000
+    runtimeManagementPort: 8888
+    serverType: ovms
+  multiModel: true
+  containers:
+    - args:
+        - '--port=8001'
+        - '--rest_port=8888'
+        - '--config_path=/models/model_config_list.json'
+        - '--file_system_poll_wait_seconds=0'
+        - '--grpc_bind_address=127.0.0.1'
+        - '--rest_bind_address=127.0.0.1'
+      image: 'quay.io/modh/openvino_model_server@sha256:5d04d405526ea4ce5b807d0cd199ccf7f71bab1228907c091e975efa770a4908'
+      name: ovms
+      resources:
+        limits:
+          cpu: '2'
+          memory: 8Gi
+        requests:
+          cpu: '1'
+          memory: 4Gi
+      volumeMounts:
+        - mountPath: /dev/shm
+          name: shm
+  protocolVersions:
+    - grpc-v1
+  grpcEndpoint: 'port:8085'
+  volumes:
+    - emptyDir:
+        medium: Memory
+        sizeLimit: 2Gi
+      name: shm
+  replicas: 1
+  tolerations: []
+  grpcDataEndpoint: 'port:8001'
+
+
    +
  1. While KServe and ModelMesh share the same object definition, they have some subtle differences, in particular the annotations that are available on them. enable-route is one annotation that is available on a ModelMesh ServingRuntime that is not available on a KServe based Model Server.
  2. +
+

The InferenceService is responsible for a definition of the model that will be deployed as well as which ServingRuntime it should use to deploy it.

+
apiVersion: serving.kserve.io/v1beta1
+kind: InferenceService
+metadata:
+  annotations:
+    openshift.io/display-name: fraud-detection-model
+    serving.kserve.io/deploymentMode: ModelMesh
+  name: fraud-detection-model
+  labels:
+    opendatahub.io/dashboard: 'true'
+spec:
+  predictor:
+    model:
+      modelFormat:
+        name: onnx
+        version: '1'
+      name: ''
+      resources: {}
+      runtime: multi-model-server  # <1>
+      storage:
+        key: aws-connection-multi-model
+        path: models/fraud-detection-model/frauddetectionmodel.onnx
+
+
    +
  1. The runtime must match the name of the ServingRuntime object that you wish to utilize to deploy the model.
  2. +
+

One major difference between ModelMesh and KServe is which object is responsible for creating and managing the pod where the model is deployed.

+

With KServe, the ServingRuntime acts as a "pod template" and each InferenceService creates it's own pod to deploy a model. A ServingRuntime can be used by multiple InferenceServices and each InferenceService will create a separate pod to deploy a model.

+

By contrast, a ServingRuntime creates a pod with ModelMesh, and the InferenceService simply tells the model server pod what models to load and from where. With ModelMesh a single ServingRuntime with multiple InferenceServices will create a single pod to load all of the models.

+

ArgoCD Health Checks

+

Out of the box, ArgoCD and OpenShift GitOps ship with a health check for a KServe InferenceService which is not compatible with a ModelMesh InferenceService. When attempting to deploy a ModelMesh based InferenceService, ArgoCD will report the object as degraded.

+

Custom health checks can be added to your ArgoCD instance that are compatible with both KServe and ModelMesh as well as other RHOAI objects to resolve this issue. The Red Hat AI Services Practice maintains several custom health checks that you can utilize in your own ArgoCD instance here.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/gpu-pruner/index.html b/odh-rhoai/gpu-pruner/index.html new file mode 100644 index 00000000..a9d7fbc2 --- /dev/null +++ b/odh-rhoai/gpu-pruner/index.html @@ -0,0 +1,2779 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + GPU pruner - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

GPU Pruner

+

In this repo, you will find the source code and usage instructions for the GPU Pruner.

+

In certain environments it is very easy for cluster users to request GPUs and then (either accidentally or not accidentally) not consume GPU resources. We needed a method to proactively identify this type of use, and scale down workloads that are idle from the GPU hardware perspective, compared to the default for Notebook resources which is web activity. It is totally possible for a user to consume a GPU from a pod PoV but never actually run a workload on it!

+

The gpu-pruner is a non-destructive idle culler that works with Red Hat OpenShift AI/Kubeflow provided APIs (InferenceService and Notebook), as well as generic Deployment, ReplicaSet and StatefulSet.

+

The culler politely pauses workloads that appear idle by scaling them down to 0 replicas. Features may be added in the future for better notifications, but the idea is that a user can simply re-enable the workload when they are ready to test/demo again.

+

It works is by querying cluster NVIDIA DCGM metrics and looking at a window of GPU utilization per pod. A scaling decision is made by looking up the pods metadata, and using owner-references to figure out the owning resource.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/img-triton/ServingRuntimes.png b/odh-rhoai/img-triton/ServingRuntimes.png new file mode 100644 index 00000000..0c250fed Binary files /dev/null and b/odh-rhoai/img-triton/ServingRuntimes.png differ diff --git a/odh-rhoai/img-triton/add.serving.runtime.png b/odh-rhoai/img-triton/add.serving.runtime.png new file mode 100644 index 00000000..1b311e6b Binary files /dev/null and b/odh-rhoai/img-triton/add.serving.runtime.png differ diff --git a/odh-rhoai/img-triton/card.fraud.detection.onnx b/odh-rhoai/img-triton/card.fraud.detection.onnx new file mode 100644 index 00000000..6082eff9 Binary files /dev/null and b/odh-rhoai/img-triton/card.fraud.detection.onnx differ diff --git a/odh-rhoai/img-triton/configure.server.png b/odh-rhoai/img-triton/configure.server.png new file mode 100644 index 00000000..c5b2d08b Binary files /dev/null and b/odh-rhoai/img-triton/configure.server.png differ diff --git a/odh-rhoai/img-triton/data.connection.png b/odh-rhoai/img-triton/data.connection.png new file mode 100644 index 00000000..a53c7f06 Binary files /dev/null and b/odh-rhoai/img-triton/data.connection.png differ diff --git a/odh-rhoai/img-triton/deploy.model.png b/odh-rhoai/img-triton/deploy.model.png new file mode 100644 index 00000000..6d14b27f Binary files /dev/null and b/odh-rhoai/img-triton/deploy.model.png differ diff --git a/odh-rhoai/img-triton/model.deployed.png b/odh-rhoai/img-triton/model.deployed.png new file mode 100644 index 00000000..4be89449 Binary files /dev/null and b/odh-rhoai/img-triton/model.deployed.png differ diff --git a/odh-rhoai/img-triton/model.name.framework.png b/odh-rhoai/img-triton/model.name.framework.png new file mode 100644 index 00000000..d65361cd Binary files /dev/null and b/odh-rhoai/img-triton/model.name.framework.png differ diff --git a/odh-rhoai/img-triton/multi.model.serving.png b/odh-rhoai/img-triton/multi.model.serving.png new file mode 100644 index 00000000..366b3604 Binary files /dev/null and b/odh-rhoai/img-triton/multi.model.serving.png differ diff --git a/odh-rhoai/img-triton/runtimes.list.png b/odh-rhoai/img-triton/runtimes.list.png new file mode 100644 index 00000000..ace386e9 Binary files /dev/null and b/odh-rhoai/img-triton/runtimes.list.png differ diff --git a/odh-rhoai/img-triton/server.details.01.png b/odh-rhoai/img-triton/server.details.01.png new file mode 100644 index 00000000..2be34a7e Binary files /dev/null and b/odh-rhoai/img-triton/server.details.01.png differ diff --git a/odh-rhoai/img-triton/server.details.02.png b/odh-rhoai/img-triton/server.details.02.png new file mode 100644 index 00000000..61cf9964 Binary files /dev/null and b/odh-rhoai/img-triton/server.details.02.png differ diff --git a/odh-rhoai/img/administration-namespaces.png b/odh-rhoai/img/administration-namespaces.png new file mode 100644 index 00000000..99656833 Binary files /dev/null and b/odh-rhoai/img/administration-namespaces.png differ diff --git a/odh-rhoai/img/api-explorer.png b/odh-rhoai/img/api-explorer.png new file mode 100644 index 00000000..ce181352 Binary files /dev/null and b/odh-rhoai/img/api-explorer.png differ diff --git a/odh-rhoai/img/api_notebook.png b/odh-rhoai/img/api_notebook.png new file mode 100644 index 00000000..b9c03f48 Binary files /dev/null and b/odh-rhoai/img/api_notebook.png differ diff --git a/odh-rhoai/img/cluster-policy.png b/odh-rhoai/img/cluster-policy.png new file mode 100644 index 00000000..b20e119d Binary files /dev/null and b/odh-rhoai/img/cluster-policy.png differ diff --git a/odh-rhoai/img/cluster-settings.png b/odh-rhoai/img/cluster-settings.png new file mode 100644 index 00000000..e3d1fd59 Binary files /dev/null and b/odh-rhoai/img/cluster-settings.png differ diff --git a/odh-rhoai/img/connect-over-http.png b/odh-rhoai/img/connect-over-http.png new file mode 100644 index 00000000..72b08280 Binary files /dev/null and b/odh-rhoai/img/connect-over-http.png differ diff --git a/odh-rhoai/img/custom-image-launcher.png b/odh-rhoai/img/custom-image-launcher.png new file mode 100644 index 00000000..49afeb94 Binary files /dev/null and b/odh-rhoai/img/custom-image-launcher.png differ diff --git a/odh-rhoai/img/custom-images-list.png b/odh-rhoai/img/custom-images-list.png new file mode 100644 index 00000000..e5c2f29b Binary files /dev/null and b/odh-rhoai/img/custom-images-list.png differ diff --git a/odh-rhoai/img/dashboard1.png b/odh-rhoai/img/dashboard1.png new file mode 100644 index 00000000..70600764 Binary files /dev/null and b/odh-rhoai/img/dashboard1.png differ diff --git a/odh-rhoai/img/dashboard2.png b/odh-rhoai/img/dashboard2.png new file mode 100644 index 00000000..7a6e21a7 Binary files /dev/null and b/odh-rhoai/img/dashboard2.png differ diff --git a/odh-rhoai/img/dashboard3.png b/odh-rhoai/img/dashboard3.png new file mode 100644 index 00000000..22ebc3d6 Binary files /dev/null and b/odh-rhoai/img/dashboard3.png differ diff --git a/odh-rhoai/img/dashboard4.png b/odh-rhoai/img/dashboard4.png new file mode 100644 index 00000000..6292243b Binary files /dev/null and b/odh-rhoai/img/dashboard4.png differ diff --git a/odh-rhoai/img/edit-yaml.png b/odh-rhoai/img/edit-yaml.png new file mode 100644 index 00000000..fcfa279f Binary files /dev/null and b/odh-rhoai/img/edit-yaml.png differ diff --git a/odh-rhoai/img/enabled-tile.png b/odh-rhoai/img/enabled-tile.png new file mode 100644 index 00000000..c554bc70 Binary files /dev/null and b/odh-rhoai/img/enabled-tile.png differ diff --git a/odh-rhoai/img/explore-tile.png b/odh-rhoai/img/explore-tile.png new file mode 100644 index 00000000..6817b244 Binary files /dev/null and b/odh-rhoai/img/explore-tile.png differ diff --git a/odh-rhoai/img/import-1.png b/odh-rhoai/img/import-1.png new file mode 100644 index 00000000..bfedf63d Binary files /dev/null and b/odh-rhoai/img/import-1.png differ diff --git a/odh-rhoai/img/import-2.png b/odh-rhoai/img/import-2.png new file mode 100644 index 00000000..bcfe3974 Binary files /dev/null and b/odh-rhoai/img/import-2.png differ diff --git a/odh-rhoai/img/import.png b/odh-rhoai/img/import.png new file mode 100644 index 00000000..095bdc80 Binary files /dev/null and b/odh-rhoai/img/import.png differ diff --git a/odh-rhoai/img/instance.png b/odh-rhoai/img/instance.png new file mode 100644 index 00000000..5f89cc52 Binary files /dev/null and b/odh-rhoai/img/instance.png differ diff --git a/odh-rhoai/img/jupyter-nb.png b/odh-rhoai/img/jupyter-nb.png new file mode 100644 index 00000000..5307f0d0 Binary files /dev/null and b/odh-rhoai/img/jupyter-nb.png differ diff --git a/odh-rhoai/img/login-to-ocp.png b/odh-rhoai/img/login-to-ocp.png new file mode 100644 index 00000000..229b1d2c Binary files /dev/null and b/odh-rhoai/img/login-to-ocp.png differ diff --git a/odh-rhoai/img/multi-model-enabled.png b/odh-rhoai/img/multi-model-enabled.png new file mode 100644 index 00000000..4a44b9b3 Binary files /dev/null and b/odh-rhoai/img/multi-model-enabled.png differ diff --git a/odh-rhoai/img/name-connection.png b/odh-rhoai/img/name-connection.png new file mode 100644 index 00000000..76485745 Binary files /dev/null and b/odh-rhoai/img/name-connection.png differ diff --git a/odh-rhoai/img/notebook_instances.png b/odh-rhoai/img/notebook_instances.png new file mode 100644 index 00000000..f312d1ad Binary files /dev/null and b/odh-rhoai/img/notebook_instances.png differ diff --git a/odh-rhoai/img/nvidia-gpu-used.png b/odh-rhoai/img/nvidia-gpu-used.png new file mode 100644 index 00000000..b2f698ca Binary files /dev/null and b/odh-rhoai/img/nvidia-gpu-used.png differ diff --git a/odh-rhoai/img/odh-tec-preview.png b/odh-rhoai/img/odh-tec-preview.png new file mode 100644 index 00000000..ddf3a3f1 Binary files /dev/null and b/odh-rhoai/img/odh-tec-preview.png differ diff --git a/odh-rhoai/img/port-forwarding.png b/odh-rhoai/img/port-forwarding.png new file mode 100644 index 00000000..3d9116e8 Binary files /dev/null and b/odh-rhoai/img/port-forwarding.png differ diff --git a/odh-rhoai/img/remote-kernel-selected.png b/odh-rhoai/img/remote-kernel-selected.png new file mode 100644 index 00000000..198fdeff Binary files /dev/null and b/odh-rhoai/img/remote-kernel-selected.png differ diff --git a/odh-rhoai/img/select-kernel.png b/odh-rhoai/img/select-kernel.png new file mode 100644 index 00000000..ae1492b5 Binary files /dev/null and b/odh-rhoai/img/select-kernel.png differ diff --git a/odh-rhoai/img/select-remote-kernel.png b/odh-rhoai/img/select-remote-kernel.png new file mode 100644 index 00000000..ef8404d2 Binary files /dev/null and b/odh-rhoai/img/select-remote-kernel.png differ diff --git a/odh-rhoai/img/select_model_serving_type.png b/odh-rhoai/img/select_model_serving_type.png new file mode 100644 index 00000000..a51d1835 Binary files /dev/null and b/odh-rhoai/img/select_model_serving_type.png differ diff --git a/odh-rhoai/img/settings.png b/odh-rhoai/img/settings.png new file mode 100644 index 00000000..657b02c6 Binary files /dev/null and b/odh-rhoai/img/settings.png differ diff --git a/odh-rhoai/img/single-model-enabled.png b/odh-rhoai/img/single-model-enabled.png new file mode 100644 index 00000000..61eb8a4f Binary files /dev/null and b/odh-rhoai/img/single-model-enabled.png differ diff --git a/odh-rhoai/img/switch-to-project.png b/odh-rhoai/img/switch-to-project.png new file mode 100644 index 00000000..26fc82f0 Binary files /dev/null and b/odh-rhoai/img/switch-to-project.png differ diff --git a/odh-rhoai/img/update.rhods-users.png b/odh-rhoai/img/update.rhods-users.png new file mode 100644 index 00000000..50ba33e5 Binary files /dev/null and b/odh-rhoai/img/update.rhods-users.png differ diff --git a/odh-rhoai/img/user-management.png b/odh-rhoai/img/user-management.png new file mode 100644 index 00000000..fc1914fd Binary files /dev/null and b/odh-rhoai/img/user-management.png differ diff --git a/odh-rhoai/img/workbench-context-path.png b/odh-rhoai/img/workbench-context-path.png new file mode 100644 index 00000000..2670f400 Binary files /dev/null and b/odh-rhoai/img/workbench-context-path.png differ diff --git a/odh-rhoai/img/workbench.png b/odh-rhoai/img/workbench.png new file mode 100644 index 00000000..01ceaa19 Binary files /dev/null and b/odh-rhoai/img/workbench.png differ diff --git a/odh-rhoai/kserve-timeout/index.html b/odh-rhoai/kserve-timeout/index.html new file mode 100644 index 00000000..65088506 --- /dev/null +++ b/odh-rhoai/kserve-timeout/index.html @@ -0,0 +1,2786 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + KServe Timeout Issues - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

KServe Timeout Issues

+

When deploying large models or when relying on node autoscaling with KServe, KServe may timeout before a model has successfully deployed due to the default progress deadline of 10 minutes set by KNative Serving.

+

When a pod takes longer than 10 minutes to deploy that leverages KNative Serving, like KServe does, KNative Serving will automatically back the pod deployment off and mark it as failed. This can happen for a number of reasons including deploying large models that take longer than 10m minutes to pull from S3 or if you are leveraging node autoscaling to reduce the consumption of expensive GPU nodes.

+

To resolve this issue, KNative supports an annotion that can be added to a KServe ServingRuntime that can be updated to set a custom progress-deadline for your application:

+
apiVersion: serving.kserve.io/v1alpha1
+kind: ServingRuntime
+metadata:
+  name: my-serving-runtime
+spec:
+  annotations:
+    serving.knative.dev/progress-deadline: 30m
+
+

It is important to note that the annotation must be set at spec.annotations and not metadata.annotations. By setting it in spec.annotations the annotation will be copied to the KNative Service object that is created by your KServe InferenceService. The annotation on the Service will allow KNative to utilize the manually defined progress-deadline.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/kserve-uwm-dashboard-metrics/index.html b/odh-rhoai/kserve-uwm-dashboard-metrics/index.html new file mode 100644 index 00000000..073538e0 --- /dev/null +++ b/odh-rhoai/kserve-uwm-dashboard-metrics/index.html @@ -0,0 +1,3014 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + RHOAI Metrics Dashboard for Model Serving - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

RHOAI Metrics Dashboard for Single Serving Models

+

Enable RHOAI User Workload Metrics for Single Serving Models and deploy the Grafana Metrics Dashboard to monitor the performance of your Single Serving Models and the resources they consume.

+

Overview

+

Enabling RHOAI User Workload Metrics for Single Serving Models and deploying a Grafana Metrics Dashboard provides valuable insights into the performance and resource usage of your Single Model Serving instances.

+

By monitoring these metrics, you can identify bottlenecks, optimize resource allocation, and ensure efficient infrastructure utilization. This enables data-driven decisions to improve the overall performance and scalability of your AI applications.

+ +

Prerequisites

+ +

Installation

+

To enable RHOAI User Workload Metrics for Single Serving Models and deploy the Grafana Metrics Dashboard, perform the following steps:

+ +

Configure Monitoring for the Single Model Serving Platform

+

To configure monitoring for the Single Model Serving Platform, refer to the official documentation. The Single Model Serving Platform includes metrics for supported runtimes of the KServe component. KServe relies on the underlying model-serving runtimes to provide metrics and does not generate its own. The available metrics for a deployed model depend on its model-serving runtime.

+

Additionally, you can configure monitoring for OpenShift Service Mesh to understand dependencies and traffic flow between components in the mesh.

+

Once monitoring is configured for the Single Model Serving Platform, you can view the metrics in the OpenShift Web Console under the Observe Dashboards section.

+

Configure GPU Monitoring Dashboard

+

To configure the GPU Monitoring Dashboard, refer to the official documentation. The GPU Monitoring Dashboard provides a comprehensive view of GPU utilization, memory usage, and other metrics for your GPU nodes.

+

The GPU Operator exposes GPU telemetry for Prometheus using the NVIDIA DCGM Exporter. These metrics can be visualized in the OpenShift Web Console under the Observe Dashboards section, specifically in the NVIDIA DCGM Exporter Dashboard.

+
+

Note: This step is optional but very useful for monitoring the GPU resources consumed by your Single Serving Models. If you do not enable this step, the Grafana Dashboard will not display GPU metrics.

+
+

Install the RHOAI Metrics Grafana and Dashboards for Single Serving Models

+

To install the RHOAI Metrics Grafana Dashboards for Single Serving Models (for both vLLM and OpenVino), refer to the RHOAI UWM repository. The Grafana Dashboard provides a comprehensive view of the performance and resource utilization of Single Serving Models.

+
kubectl apply -k overlays/grafana-uwm-user-app
+
+

The RHOAI UWM Grafana Dashboard will deploy a Grafana instance with pre-configured dashboards for monitoring the performance of your Single Serving Models using the Grafana Operator.

+

The following dashboards are currently available:

+
    +
  • vLLM Dashboard Model Metrics: Provides Model metrics for vLLM Single Serving Models.
  • +
+

vLLM Dashboard 1

+

vLLM Dashboard 2

+
    +
  • vLLM Dashboard Service Performance: Provides Service Performance for vLLM Single Serving Models.
  • +
+

vLLM Dashboard 3

+
    +
  • OpenVino Dashboard: Provides metrics for OpenVino Single Serving Models.
  • +
+

vLLM Dashboard 4

+

Conclusion

+

By turning on RHOAI User Workload Metrics and setting up the Grafana Dashboard, you can easily track how your Single Serving Models are doing and what resources they're using. It helps you find problems, tweak resource use, and make better choices to keep your AI apps running smoothly.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/Makefile b/odh-rhoai/kueue-preemption/Makefile new file mode 100644 index 00000000..6ec8a8e7 --- /dev/null +++ b/odh-rhoai/kueue-preemption/Makefile @@ -0,0 +1,27 @@ +BASE:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST)))) +SHELL=/bin/sh + +.PHONY: teardown-kueue-preemption setup-kueue-preemption + +teardown-kueue-preemption: + + -oc delete -f $(BASE)/team-a-ray-cluster-prod.yaml -f $(BASE)/team-b-ray-cluster-dev.yaml + -oc delete -f $(BASE)/team-a-cq.yaml -f $(BASE)/team-b-cq.yaml -f $(BASE)/shared-cq.yaml + -oc delete -f $(BASE)/team-a-local-queue.yaml -f $(BASE)/team-b-local-queue.yaml + -oc delete -f $(BASE)/default-flavor.yaml -f $(BASE)/gpu-flavor.yaml + -oc delete -f $(BASE)/team-a-rb.yaml -f $(BASE)/team-b-rb.yaml + -oc delete -f $(BASE)/team-a-ns.yaml -f $(BASE)/team-b-ns.yaml + + @echo "Deleting all clusterqueues" + oc delete clusterqueue --all --all-namespaces + + @echo "Deleting all resourceflavors" + oc delete resourceflavor --all --all-namespaces + +setup-kueue-preemption: + + oc create -f $(BASE)/team-a-ns.yaml -f $(BASE)/team-b-ns.yaml + oc create -f $(BASE)/team-a-rb.yaml -f $(BASE)/team-b-rb.yaml + oc create -f $(BASE)/default-flavor.yaml -f $(BASE)/gpu-flavor.yaml + oc create -f $(BASE)/team-a-cq.yaml -f $(BASE)/team-b-cq.yaml -f $(BASE)/shared-cq.yaml + oc create -f $(BASE)/team-a-local-queue.yaml -f $(BASE)/team-b-local-queue.yaml \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/clean-kueue.sh b/odh-rhoai/kueue-preemption/clean-kueue.sh new file mode 100644 index 00000000..a2eb8363 --- /dev/null +++ b/odh-rhoai/kueue-preemption/clean-kueue.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +echo "Deleting all rayclusters" +oc delete raycluster --all --all-namespaces > /dev/null + +echo "Deleting all localqueue" +oc delete localqueue --all --all-namespaces > /dev/null + +echo "Deleting all clusterqueues" +oc delete clusterqueue --all --all-namespaces > /dev/null + +echo "Deleting all resourceflavors" +oc delete resourceflavor --all --all-namespaces > /dev/null \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/default-flavor.yaml b/odh-rhoai/kueue-preemption/default-flavor.yaml new file mode 100644 index 00000000..288ff401 --- /dev/null +++ b/odh-rhoai/kueue-preemption/default-flavor.yaml @@ -0,0 +1,4 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: ResourceFlavor +metadata: + name: default-flavor \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/gpu-flavor.yaml b/odh-rhoai/kueue-preemption/gpu-flavor.yaml new file mode 100644 index 00000000..59c104ed --- /dev/null +++ b/odh-rhoai/kueue-preemption/gpu-flavor.yaml @@ -0,0 +1,11 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: ResourceFlavor +metadata: + name: gpu-flavor +spec: + nodeLabels: + nvidia.com/gpu.present: "true" + tolerations: + - key: nvidia.com/gpu + operator: Exists + effect: NoSchedule \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/readme/index.html b/odh-rhoai/kueue-preemption/readme/index.html new file mode 100644 index 00000000..8d06b31c --- /dev/null +++ b/odh-rhoai/kueue-preemption/readme/index.html @@ -0,0 +1,3097 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Kueue preemption - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Introduction

+

In this repo, we will demostrate how to use quota allocation with Kueue with preemption.

+

Overview

+

In this example, there are 2 teams that work in their own namespace:

+
    +
  1. Team A and B belongs to the same cohort
  2. +
  3. Both teams share a quota
  4. +
  5. Team A has access to GPU while team B does not
  6. +
  7. Team A has higher priority and can prempt others
  8. +
+

Kueue Configuration

+

There are 2 ResourceFlavor that manages the CPU/Memory and GPU resources. The GPU ResourceFlavor tolerates nodes that have been tainted.

+

Both teams have their invididual cluster queue that is associated with their respective namespace.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameCPUMemory (GB)GPU
Team A cq004
Team B cq000
Shared cq10640
+

A local queue is defined in their namespace to associate the cluster queue. E.g.

+
apiVersion: kueue.x-k8s.io/v1beta1
+kind: LocalQueue
+metadata:
+  name: local-queue
+  namespace: team-a
+spec:
+  clusterQueue: team-a-cq
+
+

When a Ray cluster is defined, it is submitted to the local queue with the associated priority.

+
apiVersion: ray.io/v1
+kind: RayCluster
+metadata:
+  labels:  
+    kueue.x-k8s.io/queue-name: local-queue
+    kueue.x-k8s.io/priority-class: dev-priority
+
+

Ray cluster configuration

+
+

The shared quota is only up to 10 CPU for both teams.

+
+ + + + + + + + + + + + + + + + + + + + + + + +
NameCPUMemory (GB)GPU
Team A10244
Team B6160
+

Premption

+

Team A cluster queue has preemption defined that can borrowWithinCohort of a lower priority which Team B belongs to.

+
apiVersion: kueue.x-k8s.io/v1beta1
+kind: ClusterQueue
+metadata:
+  name: team-a-cq
+spec:
+  preemption:
+    reclaimWithinCohort: Any
+    borrowWithinCohort:
+      policy: LowerPriority
+      maxPriorityThreshold: 100
+    withinClusterQueue: Never
+
+

Team A will preempt team B because it has insufficient resources to run.

+

Setting Up the Demo

+
    +
  1. +

    Install OpenShift AI Operator

    +
  2. +
  3. +

    Ensure there is at least 1 worker node that has a 4 GPUs. On AWS, this can be a p3.8xlarge instance.

    +
  4. +
  5. +

    Taint the GPU node +

      oc adm taint nodes <gpu-node> nvidia.com/gpu=Exists:NoSchedule
    +

    +
  6. +
  7. +

    Git clone the repo

    +
    git clone https://github.com/opendatahub-io-contrib/ai-on-openshift
    +cd ai-on-openshift/docs/odh-rhoai/kueue-preemption
    +
    +
  8. +
  9. +

    Run the makefile target to setup the example. This will setup 2 namespaces: team-a and team-b.

    +
    make setup-kueue-examples
    +
    +
  10. +
+

To teardown the example, you can use: +

make teardown-kueue-preemption
+

+
+

Warning

+

The setup script will delete all clusterqueues and resourceflavors in the cluster.

+
+

Running the example

+
    +
  1. +

    Create a ray cluster for team B. Wait for the cluster to be running. +

    oc create -f team-b-ray-cluster-dev.yaml
    +

    +
    $ oc get rayclusters -A
    +NAMESPACE   NAME             DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE
    +team-b      raycluster-dev   2                 2                   6      16G      0      ready    70s
    +
    +$ oc get po -n team-b
    +NAME                                           READY   STATUS    RESTARTS   AGE
    +raycluster-dev-head-zwfd8                      2/2     Running   0          45s
    +raycluster-dev-worker-small-group-test-4c85h   1/1     Running   0          43s
    +raycluster-dev-worker-small-group-test-5k9j5   1/1     Running   0          43s
    +
    +
  2. +
  3. +

    Create a Ray cluster for team A. +

    oc create -f team-a-ray-cluster-prod.yaml
    +

    +
  4. +
  5. +

    Observe team B cluster is suspended and team A cluster is running because of preemption. This may take a few seconds to happen.

    +
    $ oc get rayclusters -A
    +NAMESPACE   NAME              DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS      AGE
    +team-a      raycluster-prod   2                 2                   10     24G      4      ready       75s
    +team-b      raycluster-dev    2                                     6      16G      0      suspended   3m46s
    +
    +
  6. +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/shared-cq.yaml b/odh-rhoai/kueue-preemption/shared-cq.yaml new file mode 100644 index 00000000..a0b52661 --- /dev/null +++ b/odh-rhoai/kueue-preemption/shared-cq.yaml @@ -0,0 +1,24 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: ClusterQueue +metadata: + name: "shared-cq" +spec: + preemption: + reclaimWithinCohort: Any + borrowWithinCohort: + policy: LowerPriority + maxPriorityThreshold: 100 + withinClusterQueue: Never + namespaceSelector: {} # match all. + cohort: "team-ab" + resourceGroups: + - coveredResources: + - cpu + - memory + flavors: + - name: "default-flavor" + resources: + - name: "cpu" + nominalQuota: 10 + - name: "memory" + nominalQuota: 64Gi \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-a-cq.yaml b/odh-rhoai/kueue-preemption/team-a-cq.yaml new file mode 100644 index 00000000..ce691260 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-a-cq.yaml @@ -0,0 +1,35 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: ClusterQueue +metadata: + name: team-a-cq +spec: + preemption: + reclaimWithinCohort: Any + borrowWithinCohort: + policy: LowerPriority + maxPriorityThreshold: 100 + withinClusterQueue: Never + namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: team-a + queueingStrategy: BestEffortFIFO + cohort: team-ab + resourceGroups: + - coveredResources: + - cpu + - memory + flavors: + - name: default-flavor + resources: + - name: cpu + nominalQuota: 0 + - name: memory + nominalQuota: 0 + - coveredResources: + - nvidia.com/gpu + flavors: + - name: gpu-flavor + resources: + - name: nvidia.com/gpu + nominalQuota: "4" + \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-a-local-queue.yaml b/odh-rhoai/kueue-preemption/team-a-local-queue.yaml new file mode 100644 index 00000000..34d8bdcb --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-a-local-queue.yaml @@ -0,0 +1,7 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: LocalQueue +metadata: + name: local-queue + namespace: team-a +spec: + clusterQueue: team-a-cq \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-a-ns.yaml b/odh-rhoai/kueue-preemption/team-a-ns.yaml new file mode 100644 index 00000000..0eca2bc4 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-a-ns.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + kubernetes.io/metadata.name: team-a + opendatahub.io/dashboard: "true" + name: team-a \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-a-ray-cluster-prod.yaml b/odh-rhoai/kueue-preemption/team-a-ray-cluster-prod.yaml new file mode 100644 index 00000000..69ecb40c --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-a-ray-cluster-prod.yaml @@ -0,0 +1,68 @@ +# Team A is using prod-priority and will prempt team A because shared-cq quota +# is CPU: 10 and mem: 64G which has no spare quota for team A (GPU:10, MEM:24G). +apiVersion: ray.io/v1 +kind: RayCluster +metadata: + labels: + kueue.x-k8s.io/queue-name: local-queue + kueue.x-k8s.io/priority-class: prod-priority + name: raycluster-prod + namespace: team-a +spec: + rayVersion: 2.7.0 + headGroupSpec: + enableIngress: false + rayStartParams: + block: "true" + dashboard-host: 0.0.0.0 + num-gpus: "0" + template: + metadata: {} + spec: + containers: + - env: + - name: NVIDIA_VISIBLE_DEVICES + value: "void" + image: quay.io/project-codeflare/ray:latest-py39-cu118 + imagePullPolicy: Always + lifecycle: + preStop: + exec: + command: + - /bin/sh + - -c + - ray stop + name: ray-head + resources: + limits: + cpu: "2" + memory: 8G + requests: + cpu: "2" + memory: 8G + suspend: false + workerGroupSpecs: + - groupName: small-group-test + maxReplicas: 2 + minReplicas: 2 + numOfHosts: 1 + rayStartParams: + block: "true" + num-gpus: "2" + replicas: 2 + scaleStrategy: {} + template: + spec: + containers: + - name: machine-learning + image: quay.io/project-codeflare/ray:latest-py39-cu118 + imagePullPolicy: Always + resources: + limits: + cpu: "4" + memory: 8G + nvidia.com/gpu: "2" + requests: + cpu: "4" + memory: 8G + nvidia.com/gpu: "2" \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-a-rb.yaml b/odh-rhoai/kueue-preemption/team-a-rb.yaml new file mode 100644 index 00000000..14e06386 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-a-rb.yaml @@ -0,0 +1,13 @@ +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: edit + namespace: team-a +subjects: + - kind: ServiceAccount + name: team-a + namespace: team-a +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: edit diff --git a/odh-rhoai/kueue-preemption/team-b-cq.yaml b/odh-rhoai/kueue-preemption/team-b-cq.yaml new file mode 100644 index 00000000..00708817 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-b-cq.yaml @@ -0,0 +1,29 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: ClusterQueue +metadata: + name: team-b-cq +spec: + namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: team-b + queueingStrategy: BestEffortFIFO + cohort: team-ab + resourceGroups: + - coveredResources: + - nvidia.com/gpu + flavors: + - name: gpu-flavor + resources: + - name: nvidia.com/gpu + nominalQuota: "0" + borrowingLimit: "0" + - coveredResources: + - cpu + - memory + flavors: + - name: default-flavor + resources: + - name: cpu + nominalQuota: 0 + - name: memory + nominalQuota: 0 \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-b-local-queue.yaml b/odh-rhoai/kueue-preemption/team-b-local-queue.yaml new file mode 100644 index 00000000..2ffd41d6 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-b-local-queue.yaml @@ -0,0 +1,7 @@ +apiVersion: kueue.x-k8s.io/v1beta1 +kind: LocalQueue +metadata: + name: local-queue + namespace: team-b +spec: + clusterQueue: team-b-cq \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-b-ns.yaml b/odh-rhoai/kueue-preemption/team-b-ns.yaml new file mode 100644 index 00000000..b57297ae --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-b-ns.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + kubernetes.io/metadata.name: team-b + opendatahub.io/dashboard: "true" + name: team-b \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-b-ray-cluster-dev.yaml b/odh-rhoai/kueue-preemption/team-b-ray-cluster-dev.yaml new file mode 100644 index 00000000..4f3af9b1 --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-b-ray-cluster-dev.yaml @@ -0,0 +1,70 @@ +# Team B is using dev-priority +# Total CPU: 6 +# Total Mem: 16G +apiVersion: ray.io/v1 +kind: RayCluster +metadata: + labels: + kueue.x-k8s.io/queue-name: local-queue + kueue.x-k8s.io/priority-class: dev-priority + name: raycluster-dev + namespace: team-b +spec: + rayVersion: 2.7.0 + headGroupSpec: + enableIngress: false + rayStartParams: + block: "true" + dashboard-host: 0.0.0.0 + num-gpus: "0" + template: + metadata: {} + spec: + containers: + - env: + - name: NVIDIA_VISIBLE_DEVICES + value: "void" + image: quay.io/project-codeflare/ray:latest-py39-cu118 + imagePullPolicy: Always + lifecycle: + preStop: + exec: + command: + - /bin/sh + - -c + - ray stop + name: ray-head + resources: + limits: + cpu: "2" + memory: 8G + requests: + cpu: "2" + memory: 8G + suspend: false + workerGroupSpecs: + - groupName: small-group-test + maxReplicas: 2 + minReplicas: 2 + numOfHosts: 1 + rayStartParams: + block: "true" + num-gpus: "0" + replicas: 2 + scaleStrategy: {} + template: + spec: + containers: + - env: + - name: NVIDIA_VISIBLE_DEVICES + value: "void" + name: machine-learning + image: quay.io/project-codeflare/ray:latest-py39-cu118 + imagePullPolicy: Always + resources: + limits: + cpu: "2" + memory: 4G + requests: + cpu: "2" + memory: 4G \ No newline at end of file diff --git a/odh-rhoai/kueue-preemption/team-b-rb.yaml b/odh-rhoai/kueue-preemption/team-b-rb.yaml new file mode 100644 index 00000000..0c1f619d --- /dev/null +++ b/odh-rhoai/kueue-preemption/team-b-rb.yaml @@ -0,0 +1,13 @@ +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: edit + namespace: team-b +subjects: + - kind: ServiceAccount + name: team-b + namespace: team-b +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: edit diff --git a/odh-rhoai/kueue-preemption/workloadpriority.yml b/odh-rhoai/kueue-preemption/workloadpriority.yml new file mode 100644 index 00000000..da9791a0 --- /dev/null +++ b/odh-rhoai/kueue-preemption/workloadpriority.yml @@ -0,0 +1,14 @@ +--- +apiVersion: kueue.x-k8s.io/v1beta1 +kind: WorkloadPriorityClass +metadata: + name: prod-priority +value: 1000 +description: "Priority class for prod jobs" +--- +apiVersion: kueue.x-k8s.io/v1beta1 +kind: WorkloadPriorityClass +metadata: + name: dev-priority +value: 100 +description: "Priority class for development jobs" \ No newline at end of file diff --git a/odh-rhoai/mlflow-tile.yaml b/odh-rhoai/mlflow-tile.yaml new file mode 100644 index 00000000..1c4ac302 --- /dev/null +++ b/odh-rhoai/mlflow-tile.yaml @@ -0,0 +1,40 @@ +apiVersion: dashboard.opendatahub.io/v1 +kind: OdhApplication +metadata: + annotations: + opendatahub.io/categories: 'Data management,Data preprocessing,Model training' + name: mlflow + namespace: redhat-ods-applications + labels: + app: odh-dashboard + app.kubernetes.io/part-of: odh-dashboard +spec: + enable: + validationConfigMap: mlflow-enable + img: >- + + + + + getStartedLink: 'https://mlflow.org/docs/latest/quickstart.html' + route: mlflow-server + routeNamespace: mlflow + displayName: MLflow + kfdefApplications: [] + support: third party support + csvName: '' + provider: MLflow + docsLink: 'https://mlflow.org/docs/latest/index.html' + quickStart: '' + getStartedMarkDown: >- + # MLFlow + + MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It tackles four primary functions: + + - Tracking experiments to record and compare parameters and results (MLflow Tracking). + - Packaging ML code in a reusable, reproducible form in order to share with other data scientists or transfer to production (MLflow Projects). + - Managing and deploying models from a variety of ML libraries to a variety of model serving and inference platforms (MLflow Models). + - Providing a central model store to collaboratively manage the full lifecycle of an MLflow Model, including model versioning, stage transitions, and annotations (MLflow Model Registry). + description: >- + MLflow is an open source platform for managing the end-to-end machine learning lifecycle. + category: Self-managed diff --git a/odh-rhoai/model-serving-type-modification/index.html b/odh-rhoai/model-serving-type-modification/index.html new file mode 100644 index 00000000..cf140a72 --- /dev/null +++ b/odh-rhoai/model-serving-type-modification/index.html @@ -0,0 +1,2944 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Model serving type modification - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Model Serving Type Modification

+

Background

+

In each Project, you can only have one type of Model Serving: either Single-model Serving, or Multi-model Serving.

+

This is due to the fact that the engines responsible of model serving share the same definitions (CRDs) for them. If both were enabled simultaneously in the same project, it would be impossible to know which one should handle a model serving definition.

+

So when you serve the first model in a project, you need to choose which type of model serving you want to use. It then becomes "fixed" and all further model serving will be of the chose type.

+

Changing the Serving Type

+

How this works?

+

The behavior is controlled by a simple label in the Namespace definition of your project: modelmesh-enabled.

+
    +
  • +

    When this label does not exist, you get the initial choice panel on the Dashboard:

    +

    select_model_serving_type.png

    +
  • +
  • +

    When this label is set to true, so like this:

    +
      labels:
    +    modelmesh-enabled: 'true'
    +
    +

    Then Multi-model Serving (ModelMesh) is enabled:

    +

    multi-model-enabled.png

    +
  • +
  • +

    When this label is set to false (or whatever value other than true to be honest, you can try 😉), like this:

    +
      labels:
    +    modelmesh-enabled: 'false'
    +
    +

    Then Single-model Serving (ModelMesh) is enabled:

    +

    single-model-enabled.png

    +
  • +
+

Changing the type

+

So what to do if you have remorse at some point and want to change the serving type? The recipe is pretty simple with what we now know.

+
+

Important

+

Prerequisite: you need to be OpenShift Cluster admin, or find a friend who is, or have enough rights to modify the Namespace object from your project.

+
+
    +
  • Remove all existing served models. As the type of model serving and engine will change, the definitions won't be compatible in most cases and cause lots of trouble. If you don't use the Dashboard, remove all instances of InferenceService and ServingRuntime from your project/namespace.
  • +
  • +

    In the OpenShift Console, open the YAML definition of your Namespace (not the Project, that you can't modify!).
    + You will find it under the Administrator perspective, under Administration->Namespaces:

    +

    administration-namespaces.png

    +
  • +
  • +

    In the YAML, modify, delete or add the label modelmesh-enabled according to the rules explained above.

    +
  • +
  • The change will be immediately reflected in the Dashboard.
  • +
  • You can now deploy a new model with the Serving type you chose.
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/nvidia-gpus/index.html b/odh-rhoai/nvidia-gpus/index.html new file mode 100644 index 00000000..d3257788 --- /dev/null +++ b/odh-rhoai/nvidia-gpus/index.html @@ -0,0 +1,3440 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + NVIDIA GPUs - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Working with GPUs

+

Using NVIDIA GPUs on OpenShift

+

How does this work?

+

NVIDIA GPUs can be easily installed on OpenShift. Basically it involves installing two different operators.

+

The Node Feature Discovery operator will "discover" your cards from a hardware perspective and appropriately label the relevant nodes with this information.

+

Then the NVIDIA GPU operator will install the necessary drivers and tooling to those nodes. It will also integrate into Kubernetes so that when a Pod requires GPU resources it will be scheduled on the right node, and make sure that the containers are "injected" with the right drivers, configurations and tools to properly use the GPU.

+

So from a user perspective, the only thing you have to worry about is asking for GPU resources when defining your pods, with something like:

+
spec:
+  containers:
+  - name: app
+    image: ...
+    resources:
+      requests:
+        memory: "64Mi"
+        cpu: "250m"
+        nvidia.com/gpu: 2
+      limits:
+        memory: "128Mi"
+        cpu: "500m"
+
+

But don't worry, OpenShift AI and Open Data Hub take care of this part for you when you launch notebooks, workbenches, model servers, or pipeline runtimes!

+

Installation

+

Here is the documentation you can follow:

+ +

Advanced configuration

+

Working with taints

+

In many cases, you will want to restrict access to GPUs, or be able to provide choice between different types of GPUs: simply stating "I want a GPU" is not enough. Also, if you want to make sure that only the Pods requiring GPUs end up on GPU-enabled nodes (and not other Pods that just end up being there at random because that's how Kubernetes works...), you're at the right place!

+

The only supported method at the moment to achieve this is to taint nodes, then apply tolerations on the Pods depending on where you want them scheduled. If you don't pay close attention though when applying taints on Nodes, you may end up with the NVIDIA drivers not installed on those nodes...

+

In this case you must:

+
    +
  • +

    Apply the taints you need to your Nodes or MachineSets, for example:

    +
    apiVersion: machine.openshift.io/v1beta1
    +kind: MachineSet
    +metadata:
    +  ...
    +spec:
    +  replicas: 1
    +  selector:
    +    ...
    +  template:
    +    ...
    +    spec:
    +      ...
    +      taints:
    +        - key: restrictedaccess
    +          value: "yes"
    +          effect: NoSchedule
    +
    +
  • +
  • +

    Apply the relevant toleration to the NVIDIA Operator.

    +
      +
    • +

      In the nvidia-gpu-operator namespace, get to the Installed Operator menu, open the NVIDIA GPU Operator settings, get to the ClusterPolicy tab, and edit the ClusterPolicy.

      +

      Cluster Policy

      +
    • +
    • +

      Edit the YAML, and add the toleration in the daemonset section:

      +
      apiVersion: nvidia.com/v1
      +kind: ClusterPolicy
      +metadata:
      +  ...
      +  name: gpu-cluster-policy
      +spec:
      +  vgpuDeviceManager: ...
      +  migManager: ...
      +  operator: ...
      +  dcgm: ...
      +  gfd: ...
      +  dcgmExporter: ...
      +  cdi: ...
      +  driver: ...
      +  devicePlugin: ...
      +  mig: ...
      +  sandboxDevicePlugin: ...
      +  validator: ...
      +  nodeStatusExporter: ...
      +  daemonsets:
      +    ...
      +    tolerations:
      +      - effect: NoSchedule
      +        key: restrictedaccess
      +        operator: Exists
      +  sandboxWorkloads: ...
      +  gds: ...
      +  vgpuManager: ...
      +  vfioManager: ...
      +  toolkit: ...
      +...
      +
      +
    • +
    +
  • +
+

That's it, the operator is now able to deploy all the NVIDIA tooling on the nodes, even if they have the restrictedaccess taint. Repeat the procedure for any other taint you want to apply to your nodes.

+
+

Note

+

The first taint that you want to apply on GPU nodes is nvidia.com/gpu. This is the standard taint for which the NVIDIA Operator has a built-in toleration, so no need to add it. Likewise, Notebooks, Workbenches or other components from ODH/RHOAI that request GPUs will already have this toleration in place. For other Pods you schedule yourself, or using Pipelines, you should make sure the toleration is also applied. Doing this will ensure that only Pods really requiring GPUs are scheduled on those nodes.

+

You can of course apply many different taints at the same time. You would simply have to apply the matching toleration on the NVIDIA GPU Operator, as well as on the Pods that need to run there.

+
+

Autoscaler and GPUs

+

As they are expensive, GPUs are good candidates to put behind an Autoscaler. But due to this there are some subtleties if you want everything to go smoothly.

+

Configuration

+
+

Warning

+

For the autoscaler to work properly with GPUs, you have to set a specific label to the MachineSet. It will help to Autoscaler figure out (in fact simulate) what it is allowed to do. This is especially true if you have different MachineSets that feature different types of GPUs.

+

As per the referenced article above, the type for gpus you set through the label cannot be nvidia.com/gpu (as you will sometimes find in the standard documentation), because it's not a valid label. Therefore, only for the autoscaling purpose, you should give the type a specific name with letters, numbers and dashes only, like Tesla-T4-SHARED in this example.

+
+
    +
  • +

    Edit the MachineSet configuration to add the label that the Autoscaler will expect:

    +
    apiVersion: machine.openshift.io/v1beta1
    +kind: MachineSet
    +...
    +spec:
    +  ...
    +  template:
    +    ...
    +    spec:
    +      metadata:
    +        labels:
    +          cluster-api/accelerator: Tesla-T4-SHARED
    +
    +
  • +
  • +

    Create your ClusterAutoscaler configuration (example):

    +
    apiVersion: autoscaling.openshift.io/v1
    +kind: ClusterAutoscaler
    +metadata:
    +  name: "default"
    +spec:
    +  logVerbosity: 4
    +  maxNodeProvisionTime: 15m
    +  podPriorityThreshold: -10
    +  resourceLimits:
    +    gpus:
    +      - type: Tesla-T4-SHARED
    +        min: 0
    +        max: 8
    +  scaleDown:
    +    enabled: true
    +    delayAfterAdd: 20m
    +    delayAfterDelete: 5m
    +    delayAfterFailure: 30s
    +    unneededTime: 5m
    +
    +
    +

    Note

    +

    The delayAfterAdd parameter has to be set higher than standard value as NVIDIA tooling can take a lot of time to deploy, 10-15mn.

    +
    +
  • +
  • +

    Create the MachineSet Autoscaler:

    +
    apiVersion: autoscaling.openshift.io/v1beta1
    +kind: MachineAutoscaler
    +metadata:
    +  name: machineset-name
    +  namespace: "openshift-machine-api"
    +spec:
    +  minReplicas: 1
    +  maxReplicas: 2
    +  scaleTargetRef:
    +    apiVersion: machine.openshift.io/v1beta1
    +    kind: MachineSet
    +    name: machineset-name
    +
    +
  • +
+

Scaling to zero

+

As GPUs are expensive resources, you may want to scale down your MachineSet to zero to save on resources. This will however require some more configuration than just setting the minimum size to zero...

+

First, some background to help you understand and enable you to solve issues that may arise. You can skip the whole explanation, but it's worth it, so please bear with me.

+

When you request resources that aren't available, the Autoscaler looks at all the MachineAutoscalers that are available, with their corresponding MachineSets. But how to know which one to use? Well, it will first simulate the provisioning of a Node from each MachineSet, and see if it would fit the request. Of course, if there is already at least one Node available from a given MachineSet, the simulation would be bypassed as the Autoscaler already knows what it will get. If there are different MachineSets that fit and to choose from, the default and only "Expander" available for now in OpenShift to make its decision is random. So it will simply picks one totally randomly.

+

That's all perfect and everything, but for GPUs, if you don't start the Node for real, we don't know what's in it! So that's where we have to help the Autoscaler with a small hint.

+
    +
  • +

    Set this annotation manually if it's not there. It will stick after the first scale up though, along with some other annotations the Autoscaler will add, thanks for its newly discovered knowledge.

    +
    apiVersion: machine.openshift.io/v1beta1
    +kind: MachineSet
    +metadata:
    +  annotations:
    +    machine.openshift.io/GPU: "1"
    +
    +
  • +
+

Now to the other issue that may happen if you are in an environment with multiple Availability Zones (AZ)...

+

Although when you define a MachineSet you can set the AZ and have all the Nodes spawned properly in it, the Autoscaler simulator is not that clever. So it will simply pick a Zone at random. If this is not the one where you want/need your Pod to run, this will be a problem...

+

For example, you may already have a Persistent Volume (PV) attached to you Notebook. If your storage does now support AZ-spanning (like AWS EBS volumes), your PV is bound to a specific AZ. If the Simulator creates a virtual Node in a different AZ, there will be a mismatch, your Pod would not be schedulable on this Node, and the Autoscaler will (wrongly) conclude that it cannot use this MachineSet for a scale up!

+

Here again, we have to give a hint to the Autoscaler to what the Node will look like in the end.

+
    +
  • +

    In you MachineSet, in the labels that will be added to the node, add information regarding the topology of the Node, as well as for the volumes that may be attached to it. For example:

    +
    apiVersion: machine.openshift.io/v1beta1
    +kind: MachineSet
    +metadata:
    +spec:
    +  template:
    +    spec:
    +      metadata:
    +        labels:
    +          ...
    +          topology.kubernetes.io/zone: us-east-2a
    +          topology.ebs.csi.aws.com/zone: us-east-2a
    +
    +
  • +
+

With this, the simulated Node will be at the right place, and the Autoscaler will consider the MachineSet valid for scale up!

+

Reference material:

+ +

GPU Partitioning / Sharing

+

There are also situations where the GPU(s) you have access to might be oversized for the task at hand, and having a single user or process lock-up and "hog" that GPU can be inefficient. +There are thankfully some partitioning strategies that can be brought to bear in order to deal with these situations. +Although there are multiple techniques, with various pros and cons, the net effect of these implementations is that what used to look like a single GPU will then look like multiple GPUs. +Obviously, there is no magic in the process, and the laws of physics still hold: there are trade-offs, and the multiple "partitioned" GPUs are not going to be faster or crunch more data than the real underlying physical GPU.

+

If this is a situation that you are facing, consult this repository for more detailed information and examples.

+

Time Slicing (GPU sharing)

+

Do you want to share GPUs between different Pods? Time Slicing is one of the solutions you can use!

+

The NVIDIA GPU Operator enables oversubscription of GPUs through a set of extended options for the NVIDIA Kubernetes Device Plugin. GPU time-slicing enables workloads that are scheduled on oversubscribed GPUs to interleave with one another.

+

This mechanism for enabling time-slicing of GPUs in Kubernetes enables a system administrator to define a set of replicas for a GPU, each of which can be handed out independently to a pod to run workloads on. Unlike Multi-Instance GPU (MIG), there is no memory or fault-isolation between replicas, but for some workloads this is better than not being able to share at all. Internally, GPU time-slicing is used to multiplex workloads from replicas of the same underlying GPU.

+ +
Configuration
+

This is a simple example on how to quickly setup Time Slicing on your OpenShift cluster. In this example, we have a MachineSet that can provide nodes with one T4 card each that we want to make "seen" as 4 different cards so that multiple Pods requiring GPUs can be launched, even if we only have one node of this type.

+
    +
  • +

    Create the ConfigMap that will define how we want to slice our GPU:

    +
    kind: ConfigMap
    +apiVersion: v1
    +metadata:
    +  name: time-slicing-config
    +  namespace: nvidia-gpu-operator
    +data:
    +  tesla-t4: |-
    +    version: v1
    +    sharing:
    +      timeSlicing:
    +        resources:
    +        - name: nvidia.com/gpu
    +          replicas: 4
    +
    +
    +

    NOTE + - The ConfigMap has to be called time-slicing-config and must be created in the nvidia-gpu-operator namespace. + - You can add many different resources with different configurations. You simply have to provide the corresponding Node label that has been applied by the operator, for example name: nvidia.com/mig-1g.5gb / replicas: 2 if you have a MIG configuration applied to a Node with a A100. + - You can modify the value of replicas to present less/more GPUs. Be warned though: all the Pods on this node will share the GPU memory, with no reservation. The more slices you create, the more risks of OOM errors (out of memory) you get if your Pods are hungry (or even only one!).

    +
    +
  • +
  • +

    Modify the ClusterPolicy called gpu-cluster-policy (accessible from the NVIDIA Operator view in the nvidia-gpu-operator namespace) to point to this configuration, and eventually add the default configuration (in case you nodes are not labelled correctly, see below)

    +
    apiVersion: nvidia.com/v1
    +kind: ClusterPolicy
    +metadata:
    +  ...
    +  name: gpu-cluster-policy
    +spec:
    +  ...
    +  devicePlugin:
    +    config:
    +      default: tesla-t4
    +      name: time-slicing-config
    +  ...
    +
    +
  • +
  • +

    Apply label to your MachineSet for the specific slicing configuration you want to use on it:

    +
    apiVersion: machine.openshift.io/v1beta1
    +kind: MachineSet
    +metadata:
    +spec:
    +  template:
    +    spec:
    +      metadata:
    +        labels:
    +          nvidia.com/device-plugin.config: tesla-t4
    +
    +
  • +
+

Multi-Instance GPU (MIG)

+

Multi-Instance GPU (MIG) enables a single physical GPU to be partitioned into several isolated instances, each with its own compute resources, memory, and performance profiles.

+

There are two types of MIG strategies: Single and Mixed. The single MIG strategy should be utilized when all GPUs on a node have MIG enabled, while the Mixed MIG strategy should be utilized when not all GPUs on a node have MIG enabled.

+
+

NOTE: MIG is only supported with the following NVIDIA GPU Types - A30, A100, A100X, A800, AX800, H100, H200, and H800.

+
+ +

Multi-Process Service (MPS)

+

Multi-Process Service (MPS) facilitates concurrent sharing of a single GPU among multiple CUDA applications.

+

MPS is an alternative, binary-compatible implementation of the CUDA Application Programming Interface (API). The MPS runtime architecture is designed to transparently enable co-operative multi-process CUDA applications.

+ +
+

NOTE: Despite the tests passing, MPS isn't working correctly on OpenShift currently, due to only one process per GPU can run at any time. RH and NVIDIA engineers are working to fix this issue as soon as possible.

+
+

Aggregating GPUs (Multi-GPU)

+

Some Large Language Models (LLMs), such as Llama-3-70B and Falcon 180B, can be too large to fit into the memory of a single GPU (vRAM). Or in some cases, GPUs that would be large-enough might be difficult to obtain. If you find yourself in such a situation, it is natural to wonder whether an aggregation of multiple, smaller GPUs can be used instead of one single large GPU.

+

Thankfully, the answer is essentially Yes. To address these challenges, we can use more advanced configurations to distribute the LLM workload across several GPUs. One option is leveraging tensor parallelism, where the LLM is split across several GPUs, with each GPU processing a portion of the model's tensors. This approach ensures efficient utilization of available resources (GPUs) across one or several workers.

+

Some Serving Runtimes, such as vLLM, support tensor parallelism, allowing for both single-worker and multi-worker configurations (the difference whether your GPUs are all in the same machine, or are spread across machines).

+

vLLM has been added as an Out-of-the-box serving runtime, starting with Red Hat OpenShift AI version 2.10 link to our RHOAI doc

+

For a detailed guide on implementing these solutions, refer to our repository.

+ + + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/odh-tools-and-extensions-companion/index.html b/odh-rhoai/odh-tools-and-extensions-companion/index.html new file mode 100644 index 00000000..3df807d6 --- /dev/null +++ b/odh-rhoai/odh-tools-and-extensions-companion/index.html @@ -0,0 +1,2919 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + ODH Tools and Extensions Companion - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

ODH Tools and Extensions Companion

+

In this repo, you will find the source code and usage instructions for the Open Data Hub Tools & Extensions Companion.

+

This application implements tools that can be useful to anyone working with Open Data Hub (ODH), Red Hat OpenShift AI (RHOAI), or even locally with Podman.

+

odh-tec-preview.png

+

Current Features

+

S3 Tools

+
    +
  • S3 Buckets Management: browsing, creation, deletion
  • +
  • S3 Objects Browser:
      +
    • Single file upload, Multiple files uploads, Downloads
    • +
    • File preview
    • +
    • Model import from HuggingFace
    • +
    +
  • +
+

GPU Tools

+
    +
  • VRAM Estimator: helps you calculate the VRAM and number of GPUs you need for inference and training.
  • +
+

Settings and validations

+
    +
  • S3 connection testing
  • +
  • HuggingFace authentication testing
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/openshift-group-management/index.html b/odh-rhoai/openshift-group-management/index.html new file mode 100644 index 00000000..1e02c3ca --- /dev/null +++ b/odh-rhoai/openshift-group-management/index.html @@ -0,0 +1,3000 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + OpenShift Group Management - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

OpenShift Group Management

+

In the Red Hat OpenShift Documentation, there are instructions on how to configure a specific list of RHOAI Administrators and RHOAI Users.

+

However, if the list of users keeps changing, the membership of the groupe called rhods-users will have to be updated frequently. By default, in OpenShift, only OpenShift admins can edit group membership. Being a RHOAI Admin does not confer you those admin privileges, and so, it would fall to the OpenShift admin to administer that list.

+

The instructions in this page will show how the OpenShift Admin can create these groups in such a way that any member of the group rhods-admins can edit the users listed in the group rhods-users. +These makes the RHOAI Admins more self-sufficient, without giving them unneeded access.

+

For expediency in the instructions, we are using the oc cli, but these can also be achieved using the OpenShift Web Console. We will assume that the user setting this up has admin privileges to the cluster.

+

Creating the groups

+

Here, we will create the groups mentioned above. Note that you can alter those names if you want, but will then need to have the same alterations throughout the instructions.

+
    +
  1. To create the groups: +
    oc adm groups new rhods-users
    +oc adm groups new rhods-admins
    +
  2. +
  3. The above may complain about the group(s) already existing.
  4. +
  5. To confirm both groups exist: +
    oc get groups | grep rhods
    +
  6. +
  7. That should return: +
    bash-4.4 ~ $ oc get groups | grep rhods
    +rhods-admins
    +rhods-users
    +
  8. +
  9. Both groups now exist
  10. +
+

Creating ClusterRole and ClusterRoleBinding

+
    +
  1. This will create a Cluster Role and a Cluster Role Binding: +
    oc apply -f - <<EOF
    +apiVersion: rbac.authorization.k8s.io/v1
    +kind: ClusterRole
    +metadata:
    +  name: update-rhods-users
    +rules:
    +  - apiGroups: ["user.openshift.io"]
    +    resources: ["groups"]
    +    resourceNames: ["rhods-users"]
    +    verbs: ["update", "patch", "get"]
    +---
    +kind: ClusterRoleBinding
    +apiVersion: rbac.authorization.k8s.io/v1
    +metadata:
    +  name: rhods-admin-can-update-rhods-users
    +subjects:
    +  - kind: Group
    +    apiGroup: rbac.authorization.k8s.io
    +    name: rhods-admins
    +roleRef:
    +  apiGroup: rbac.authorization.k8s.io
    +  kind: ClusterRole
    +  name: update-rhods-users
    +EOF
    +
  2. +
  3. To confirm they were both succesfully created, run: +
    oc get ClusterRole,ClusterRoleBinding  | grep 'update\-rhods'
    +
  4. +
  5. You should see: +
    bash-4.4 ~ $ oc get ClusterRole,ClusterRoleBinding  | grep 'update\-rhods'
    +clusterrole.rbac.authorization.k8s.io/update-rhods-users
    +clusterrolebinding.rbac.authorization.k8s.io/rhods-admin-can-update-rhods-users
    +
  6. +
  7. You are pretty much done. You now just need to validate things worked.
  8. +
+

Add some users as rhods-admins

+

To confirm this works, add a user to the rhods-admin group. In my example, I'll add user1

+

Capture the URL needed to edit the rhods-users group

+

Since people who are not cluster admin won't be able to browse the list of groups, capture the URL that allows to control the membership of rhods-users.

+

It should look similar to:

+

https://console-openshift-console.apps.<thecluster>/k8s/cluster/user.openshift.io~v1~Group/rhods-users

+

Ensure that rhods-admins are now able to edit rhods-users

+

Ask someone in the rhods-admins group to confirm that it works for them. (Remember to provide them with the URL to do so).

+

They should be able to do so and successfully save their changes, as shown below:

+

+

Adding kube:admin to rhods-admins

+

To add kube:admin user to the list of RHOAI Administrators, you will need to prefix it with b64:

+
apiVersion: user.openshift.io/v1
+kind: Group
+metadata:
+  name: rhods-admins
+users:
+- b64:kube:admin
+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/odh-rhoai/single-stack-serving-certificate/index.html b/odh-rhoai/single-stack-serving-certificate/index.html new file mode 100644 index 00000000..c5db30e6 --- /dev/null +++ b/odh-rhoai/single-stack-serving-certificate/index.html @@ -0,0 +1,2939 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Single stack serving certificate - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Use Existing OpenShift Certificate for Single Stack Serving

+

By default, the Single Stack Serving in Openshift AI uses a self-signed certificate generated at installation for the endpoints that are created when deploying a server. +This can be counter-intuitive because if you already have certificates configured on your OpenShift cluster, they will be used by default for other types of endpoints like Routes.

+

The installation procedure for the Single Stack Serving available here (section 4.vi) states that you can provide your own certificate instead of using a self-signed one.

+

This following procedure explains how to use the same certificate that you already have for your OpenShift cluster.

+

Procedure

+
    +
  • Configure OpenShift to use a valid certificate for accessing the Console and in general for any created Route (normally, this is something that was already done).
  • +
  • From the openshift-ingress namespace, copy the content of a Secret whose name includes "certs". For example rhods-internal-primary-cert-bundle-secret or ingress-certs-..... The content of the Secret (data) should contain two items, tls.cert and tls.key. They are the certificate and key that are used for all the OpenShift Routes.
  • +
  • +

    Cleanup the YAML to just keep the relevant content. It should look like this (the name of the secret will be different, it's normally tied to your cluster name):

    +
    kind: Secret
    +apiVersion: v1
    +metadata:
    +name: rhods-internal-primary-cert-bundle-secret
    +data:
    +tls.crt: >-
    +    LS0tLS1CRUd...
    +tls.key: >-
    +    LS0tLS1CRUd...
    +type: kubernetes.io/tls
    +
    +
  • +
  • +

    Apply this YAML into the istio-system namespace (basically, copy the Secret from one namespace to the other).

    +
  • +
  • +

    Following the Single Stack Serving installation procedure, in your DSC Configuration, refer to this secret for the kserve configuration (don’t forget to change the secretName for the one you just created):

    +
    kserve:
    +devFlags: {}
    +managementState: Managed
    +serving:
    +    ingressGateway:
    +    certificate:
    +        secretName: rhods-internal-primary-cert-bundle-secret
    +        type: Provided
    +    managementState: Managed
    +    name: knative-serving
    +
    +
  • +
+

Your Model Servers will now be deployed with the same certificate as you are using for OpenShift Routes. If this is a trusted certificate, your Endpoints will be accessible using SSL without having to ignore error messages or create special configurations.

+

Other Workarounds

+

If the above method does not work or you don't want or can't do any modification, you can try to bypass the certificate verification in your application or code. Depending on the library used, there are different solutions.

+

Using Langchain with OpenAI API compatible runtimes

+

The underlying library used for communication by the base OpenAI module of Langchain is httpx. You can directly specify httpx settings when you instantiate the llm object in Langchain. With the following settings on the last two lines of this example, any certificate error will be ignored:

+
import httpx
+# LLM definition
+llm = VLLMOpenAI(
+    openai_api_key="EMPTY",
+    openai_api_base= f"{inference_server_url}/v1",
+    model_name="/mnt/models/",
+    top_p=0.92,
+    temperature=0.01,
+    max_tokens=512,
+    presence_penalty=1.03,
+    streaming=True,
+    callbacks=[StreamingStdOutCallbackHandler()]
+    async_client=httpx.AsyncClient(verify=False),
+    http_client=httpx.Client(verify=False)
+)
+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/patterns/bucket-notifications/bucket-notifications/index.html b/patterns/bucket-notifications/bucket-notifications/index.html new file mode 100644 index 00000000..f1c52902 --- /dev/null +++ b/patterns/bucket-notifications/bucket-notifications/index.html @@ -0,0 +1,3396 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Bucket notifications - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Bucket Notifications

+

Description

+

The Rados Gateway (RGW) component of Ceph provides Object Storage through an S3-compatible API on all Ceph implementations: OpenShift Data Foundation and its upstream version Rook-Ceph, Red Hat Ceph Storage, Ceph,…​

+

Bucket notifications provide a mechanism for sending information from the RGW when certain events are happening on a bucket. Currently, notifications can be sent to: HTTP, AMQP0.9.1 and Kafka endpoints.

+

From a data engineering point of view, bucket notifications allow to create an event-driven architecture, where messages (instead of simply log entries) can be sent to various processing components or event buses whenever something is happening on the object storage: object creation, deletion, with many fine-grained settings available.

+

Use cases

+

Application taking actions on the objects

+

As part of an event-driven architecture, this pattern can be used to trigger an application to perform an action following the storage event. An example could be the automated processing of a new image that has just been uploaded to the object storage (analysis, resizing,…​). Paired with Serverless functions this becomes a pretty efficient architecture compared to having an application constantly monitoring or polling the storage, or to have to implement this triggering process in the application interacting with the storage. This loosely-coupled architecture also gives much more agility for updates, technology evolution,…​

+

External monitoring systems

+

The events sent by the RGW are simple messages containing all the metadata relevant to the event and the object. So it can be an excellent source of information for a monitoring system. For example if you want to keep a trace or send an alert whenever a specific type of file, or with a specific name, is uploaded or deleted from the storage.

+

Implementations examples

+

This pattern is implemented in the XRay pipeline demo

+

How does it work?

+

Characteristics

+
    +
  • Notifications are sent directly from the RGW on which the event happened to an external endpoint.
  • +
  • Pluggable endpoint architecture:
      +
    • HTTP/S
    • +
    • AMQP 0.9.1
    • +
    • Kafka
    • +
    • Knative
    • +
    +
  • +
+

Data Model

+
    +
  • Topics contain the definition of a specific endpoint in “push mode”
  • +
  • Notifications tie topics with buckets, and may also include filter definition on the events
  • +
+

Data model

+

Configuration

+

This configuration shows how to create a notification that will send a message (event) to a Kafka topic when a new object is created in a bucket.

+

Requirements

+
    +
  • Access to a Ceph/ODF/RHCS installation with the RGW deployed.
  • +
  • Endpoint address (URL) for the RGW.
  • +
  • Credentials to connect to the RGW:
      +
    • AWS_ACCESS_KEY_ID
    • +
    • AWS_SECRET_ACCESS_KEY
    • +
    +
  • +
+
+

Note

+

As Ceph implements an S3-Compatible API to access Object Storage, standard naming for variables or procedures used with S3 were retained to stay coherent with examples, demos or documentation related to S3. Therefore the AWS prefix in the previous variables.

+
+

Topic Creation

+

A topic is the definition of a specific endpoint. It must be created first.

+

Method 1: "RAW" configuration

+

As everything is done through the RGW API, you can query it directly. To be fair, this method is almost never used (unless there is no SDK or S3 tool for your environment) but gives a good understanding of the process.

+

Example for a Kafka Endpoint:

+
POST
+Action=CreateTopic
+&Name=my-topic
+&push-endpoint=kafka://my-kafka-broker.my-net:9999
+&Attributes.entry.1.key=verify-ssl
+&Attributes.entry.1.value=true
+&Attributes.entry.2.key=kafka-ack-level
+&Attributes.entry.2.value=broker
+&Attributes.entry.3.key=use-ssl
+&Attributes.entry.3.value=true
+&Attributes.entry.4.key=OpaqueData
+&Attributes.entry.4.value=https://s3-proxy.my-zone.my-net
+
+
+

Note

+

The authentication part is not detailed here as the mechanism is pretty convoluted, but it is directly implemented in most API development tools, like Postman.

+
+

The full reference for the REST API for bucket notifications is available here.

+

Method 2: Python + AWS SDK

+

As the creator of the S3 API, AWS is providing SDKs for the main languages to interact with it. Thanks to this compatibility, you can use those SDKs to interact with Ceph in the same way. For Python, the library to interact with AWS services is called boto3.

+

Example for a Kafka Endpoint:

+
import boto3
+sns = boto3.client('sns',
+                endpoint_url = endpoint_url,
+                aws_access_key_id = aws_access_key_id,
+                aws_secret_access_key= aws_secret_access_key,
+                region_name='default',
+                config=botocore.client.Config(signature_version = 's3'))
+
+attributes = {}
+attributes['push-endpoint'] = 'kafka://my-cluster-kafka-bootstrap:9092'
+attributes['kafka-ack-level'] = 'broker'
+
+topic_arn = sns.create_topic(Name=my-topic, Attributes=attributes)['TopicArn']
+
+

Notification Configuration

+

The notification configuration will "tie" a bucket with a topic.

+

Method 1: "RAW" configuration

+

As previously, you can directly query the RGW REST API. This is done through an XML formatted payload that is sent with a PUT command.

+

Example for a Kafka Endpoint:

+
PUT /my-bucket?notification HTTP/1.1
+
+<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
+    <TopicConfiguration>
+        <Id>my-notification</Id>
+        <Topic>my-topic</Topic>
+        <Event>s3:ObjectCreated:*</Event>
+        <Event>s3:ObjectRemoved:DeleteMarkerCreated</Event>
+    </TopicConfiguration>
+    <TopicConfiguration>
+...
+    </TopicConfiguration>
+</NotificationConfiguration>
+
+

Again, the full reference for the REST API for bucket notifications is available here.

+

Method 2: Python + AWS SDK

+

Example for a Kafka Endpoint:

+
import boto3
+s3 = boto3.client('s3',
+                endpoint_url = endpoint_url,
+                aws_access_key_id = aws_access_key_id,
+                aws_secret_access_key = aws_secret_access_key,
+                region_name = 'default',
+                config=botocore.client.Config(signature_version = 's3'))
+
+bucket_notifications_configuration = {
+            "TopicConfigurations": [
+                {
+                    "Id": 'my-id',
+                    "TopicArn": 'arn:aws:sns:s3a::my-topic',
+                    "Events": ["s3:ObjectCreated:*"]
+                }
+            ]
+        }
+
+s3.put_bucket_notification_configuration(Bucket = bucket_name,
+        NotificationConfiguration=bucket_notifications_configuration)
+
+

Filters

+

Although a notification is specific to a bucket (and you can have multiple configurations on one bucket), you may want that it does not apply to all the objects from this bucket. For example you want to send an event when an image is uploaded, but not do anything it’s another type of file. You can do this with filters! And not only on the filename, but also on the tags associated to it in its metadata.

+

Filter examples, on keys or tags:

+
<Filter>
+    <S3Key>
+        <FilterRule>
+         <Name>regex</Name>
+         <Value>([0-9a-zA-Z\._-]+.(png|gif|jp[e]?g)</Value>
+        </FilterRule>
+    </S3Key>
+    <S3Tags>
+        <FilterRule>
+            <Name>Project</Name><Value>Blue</Value>
+        </FilterRule>
+        <FilterRule>
+            <Name>Classification</Name><Value>Confidential</Value>
+        </FilterRule>
+    </S3Tags>
+</Filter>
+
+

Events

+

The notifications sent to the endpoints are called events, and they are structured like this:

+

Event example:

+
{"Records":[
+    {
+        "eventVersion":"2.1",
+        "eventSource":"ceph:s3",
+        "awsRegion":"us-east-1",
+        "eventTime":"2019-11-22T13:47:35.124724Z",
+        "eventName":"ObjectCreated:Put",
+        "userIdentity":{
+            "principalId":"tester"
+        },
+        "requestParameters":{
+            "sourceIPAddress":""
+        },
+        "responseElements":{
+            "x-amz-request-id":"503a4c37-85eb-47cd-8681-2817e80b4281.5330.903595",
+            "x-amz-id-2":"14d2-zone1-zonegroup1"
+        },
+        "s3":{
+            "s3SchemaVersion":"1.0",
+            "configurationId":"mynotif1",
+            "bucket":{
+                "name":"mybucket1",
+                "ownerIdentity":{
+                    "principalId":"tester"
+                },
+                "arn":"arn:aws:s3:us-east-1::mybucket1",
+                "id":"503a4c37-85eb-47cd-8681-2817e80b4281.5332.38"
+            },
+            "object":{
+                "key":"myimage1.jpg",
+                "size":"1024",
+                "eTag":"37b51d194a7513e45b56f6524f2d51f2",
+                "versionId":"",
+                "sequencer": "F7E6D75DC742D108",
+                "metadata":[],
+                "tags":[]
+            }
+        },
+        "eventId":"",
+        "opaqueData":"me@example.com"
+    }
+]}
+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/patterns/bucket-notifications/img/data-model.png b/patterns/bucket-notifications/img/data-model.png new file mode 100644 index 00000000..bf37bc7c Binary files /dev/null and b/patterns/bucket-notifications/img/data-model.png differ diff --git a/patterns/kafka/kafka-to-object-storage/deployment/secor.yaml b/patterns/kafka/kafka-to-object-storage/deployment/secor.yaml new file mode 100644 index 00000000..3d4de560 --- /dev/null +++ b/patterns/kafka/kafka-to-object-storage/deployment/secor.yaml @@ -0,0 +1,82 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: secor + name: secor + namespace: NAMESPACE +spec: + replicas: 1 + selector: + matchLabels: + app: secor + strategy: {} + template: + metadata: + labels: + app: secor + spec: + containers: + - image: quay.io/rh-data-services/secor:0.29-hdp2.9.2_latest + name: secor-0-29-hadoop-2-9-2 + imagePullPolicy: Always + env: + - name: ZOOKEEPER_PATH + value: "/" + - name: ZOOKEEPER_QUORUM + value: "zoo-entrance:2181" + - name: KAFKA_SEED_BROKER_HOST + value: "YOUR_KAFKA-kafka-brokers" + - name: KAFKA_SEED_BROKER_PORT + value: "9092" + - name: AWS_ACCESS_KEY + value: YOUR_KEY + - name: AWS_SECRET_KEY + value: YOUR_SECRET + - name: AWS_ENDPOINT + value: YOUR_ENDPOINT + - name: AWS_PATH_STYLE_ACCESS + value: "true" + - name: SECOR_S3_BUCKET + value: YOUR_BUCKET + - name: SECOR_GROUP + value: "raw_logs" + # - name: SECOR_S3_PATH + # value: "kafka-messages" + - name: KAFKA_OFFSETS_STORAGE + value: "zookeeper" + - name: SECOR_MAX_FILE_SECONDS + value: "10" + - name: SECOR_MAX_FILE_BYTES + value: "10000" + - name: SECOR_UPLOAD_MANAGER + value: "com.pinterest.secor.uploader.S3UploadManager" + - name: SECOR_MESSAGE_PARSER + # value: "com.pinterest.secor.parser.OffsetMessageParser" + value: "com.pinterest.secor.parser.JsonMessageParser" + - name: DEBUG + value: "True" + - name: SECOR_KAFKA_TOPIC_FILTER + value: "my_topic" + - name: SECOR_WRITER_FACTORY + value: "com.pinterest.secor.io.impl.JsonORCFileReaderWriterFactory" + - name: SECOR_COMPRESSION_CODEC + value: "" + - name: SECOR_FILE_EXTENSION + value: "" + - name: PARTITIONER_GRANULARITY_HOUR + value: "false" + - name: PARTITIONER_GRANULARITY_MINUTE + value: "false" + - name: KAFKA_USE_TIMESTAMP + value: "true" + - name: SECOR_FILE_WRITER_DELIMITER + value: "" + - name: SECOR_ORC_MESSAGE_SCHEMA + value: '' + volumeMounts: + - name: "local-mount" + mountPath: "/mnt/secor_data/message_logs/partition" + volumes: + - name: local-mount + emptyDir: {} \ No newline at end of file diff --git a/patterns/kafka/kafka-to-object-storage/deployment/zookeeper-entrance.yaml b/patterns/kafka/kafka-to-object-storage/deployment/zookeeper-entrance.yaml new file mode 100644 index 00000000..ca821d7b --- /dev/null +++ b/patterns/kafka/kafka-to-object-storage/deployment/zookeeper-entrance.yaml @@ -0,0 +1,110 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: zoo-entrance + namespace: NAMESPACE + labels: + app: zoo-entrance +spec: + replicas: 1 + selector: + matchLabels: + app: zoo-entrance + strategy: + type: Recreate + template: + metadata: + labels: + app: zoo-entrance + spec: + containers: + - name: zoo-entrance + image: "quay.io/rh-data-services/zoo-entrance:latest" + command: + - /opt/stunnel/stunnel_run.sh + ports: + - containerPort: 2181 + name: zoo + protocol: TCP + env: + - name: LOG_LEVEL + value: notice + - name: STRIMZI_ZOOKEEPER_CONNECT + value: "YOUR_KAFKA-zookeeper-client:2181" + imagePullPolicy: Always + livenessProbe: + exec: + command: + - /opt/stunnel/stunnel_healthcheck.sh + - "2181" + failureThreshold: 3 + initialDelaySeconds: 15 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - /opt/stunnel/stunnel_healthcheck.sh + - "2181" + failureThreshold: 3 + initialDelaySeconds: 15 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + volumeMounts: + - mountPath: /etc/cluster-operator-certs/ + name: cluster-operator-certs + - mountPath: /etc/cluster-ca-certs/ + name: cluster-ca-certs + restartPolicy: Always + terminationGracePeriodSeconds: 30 + volumes: + - name: cluster-operator-certs + secret: + defaultMode: 288 + secretName: YOUR_KAFKA-cluster-operator-certs + - name: cluster-ca-certs + secret: + defaultMode: 288 + secretName: YOUR_KAFKA-cluster-ca-cert +--- +apiVersion: v1 +kind: Service +metadata: + labels: + app: zoo-entrance + name: zoo-entrance + namespace: NAMESPACE +spec: + ports: + - name: zoo + port: 2181 + protocol: TCP + targetPort: 2181 + selector: + app: zoo-entrance + type: ClusterIP +--- +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + labels: + app: zoo-entrance + name: zoo-entrance + namespace: NAMESPACE +spec: + ingress: + - from: + - podSelector: + matchLabels: + app: zoo-entrance + ports: + - port: 2181 + protocol: TCP + podSelector: + matchLabels: + strimzi.io/name: YOUR_KAFKA-zookeeper + policyTypes: + - Ingress \ No newline at end of file diff --git a/patterns/kafka/kafka-to-object-storage/img/kafka-secor.png b/patterns/kafka/kafka-to-object-storage/img/kafka-secor.png new file mode 100644 index 00000000..78c3694c Binary files /dev/null and b/patterns/kafka/kafka-to-object-storage/img/kafka-secor.png differ diff --git a/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/index.html b/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/index.html new file mode 100644 index 00000000..20ac8f0e --- /dev/null +++ b/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/index.html @@ -0,0 +1,3107 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Kafka to object storage - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Kafka to Object Storage

+

Description

+

Kafka is a distributed event stream processing system which is great for storing hot relevant data. Based on the retention policy of the data, it can be used to store data for a long time. However, it is not suitable for storing data for a long time. This is where we need a mechanism to move data from Kafka to the object storage.

+

Use Cases

+

Long term retention of data

+

As Kafka is not really suited for long term retention of data, persisting it inside an object store will allow you to keep your data for further use, backup or archival purposes. Depending on the solution you use, you can also transform or format you data while storing it, which will ease further retrieval.

+

Move data to Central Data Lake

+

Production Kafka environment may not be the best place to run analytics or do model training. Transferring or copying the date to a central data lake will allow you to decouple those two aspects (production and analytics), bringing peace of mind and further capabilities to the data consumers.

+

Implementations examples

+

This pattern is implemented in the Smart City demo

+

Configuration Using Secor

+

This pattern implements the Secor Kafka Consumer. It can be used to consume kafka messages from a kafka topic and store that to S3 compatible Objet Buckets.

+

Secor is a service persisting Kafka logs to Amazon S3, Google Cloud Storage, Microsoft Azure Blob Storage and Openstack Swift. Its key features are: strong consistency, fault tolerance, load distribution, horizontal scalability, output partitioning, configurable upload policies, monitoring, customizability, event transformation.

+

Kafka with Secor

+

Prerequisites

+

Bucket

+

An S3-compatible bucket, with its access key and secret key.

+

ZooKeeper Entrance

+

Secor needs to connect directly to Zookeeper to keep track of some data. If you have a secured installation of Zookeeper, like when you deploy Kafka using Strimzi or AMQStreams, you need to deploy a ZooKeeper Entrance. This is a special proxy to Zookeeper that will allow this direct connection.

+
+

Note

+

The deployment file is based on a Strimzi or AMQ Streams deployment of Kafka. If you configuration is different you may have to adapt some of the parameters.

+
+

Deployment:

+
    +
  • In the file deployment/zookeeper-entrance.yaml, replace:
      +
    • the occurrences of 'NAMESPACE' by the namespace where the Kafka cluster is.
    • +
    • the occurrences of 'YOUR_KAFKA' by the name of your Kafka cluster.
    • +
    • the parameters YOUR_KEY, YOUR_SECRET, YOUR_ENDPOINT, YOUR_BUCKET with the values corresponding to the bucket where you want to store the data.
    • +
    +
  • +
  • Apply the modified file to deploy ZooKeeper Entrance.
  • +
+

Deployment

+

Secor

+
    +
  • In the file deployment/secor.yaml, replace:
      +
    • the occurrences of 'NAMESPACE' by the namespace where the Kafka cluster is.
    • +
    • the occurrences of 'YOUR_KAFKA' by the name of your Kafka cluster.
    • +
    • adjust all the other Secor parameters or add others depending on the processing you want to do with the data: output format, aggregation,... Full instructions are available here.
    • +
    +
  • +
  • Apply the modified file to deploy Secor.
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/patterns/kafka/kafka-to-serverless/deployment/01_knative_serving_eventing_kafka_setup.yaml b/patterns/kafka/kafka-to-serverless/deployment/01_knative_serving_eventing_kafka_setup.yaml new file mode 100644 index 00000000..dd8a3694 --- /dev/null +++ b/patterns/kafka/kafka-to-serverless/deployment/01_knative_serving_eventing_kafka_setup.yaml @@ -0,0 +1,20 @@ +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeServing +metadata: + name: knative-serving + namespace: knative-serving +--- +apiVersion: operator.knative.dev/v1alpha1 +kind: KnativeEventing +metadata: + name: knative-eventing + namespace: knative-eventing +--- +apiVersion: operator.serverless.openshift.io/v1alpha1 +kind: KnativeKafka +metadata: + name: knative-kafka + namespace: knative-eventing +spec: + source: + enabled: true diff --git a/patterns/kafka/kafka-to-serverless/deployment/02_knative_service.yaml b/patterns/kafka/kafka-to-serverless/deployment/02_knative_service.yaml new file mode 100644 index 00000000..69deb793 --- /dev/null +++ b/patterns/kafka/kafka-to-serverless/deployment/02_knative_service.yaml @@ -0,0 +1,10 @@ +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: greeter + namespace: YOUR_NAMESPACE +spec: + template: + spec: + containers: + - image: quay.io/rhdevelopers/knative-tutorial-greeter:quarkus diff --git a/patterns/kafka/kafka-to-serverless/deployment/03_knative_kafka_source.yaml b/patterns/kafka/kafka-to-serverless/deployment/03_knative_kafka_source.yaml new file mode 100644 index 00000000..f8025981 --- /dev/null +++ b/patterns/kafka/kafka-to-serverless/deployment/03_knative_kafka_source.yaml @@ -0,0 +1,15 @@ +apiVersion: sources.knative.dev/v1beta1 +kind: KafkaSource +metadata: + name: kafka-source +spec: + consumerGroup: "knative-consumer-group" + bootstrapServers: + - YOUR_BOOTSTRAP.YOUR_NAMESPACE.svc:9092 + topics: + - example_topic + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: greeter diff --git a/patterns/kafka/kafka-to-serverless/img/eda.png b/patterns/kafka/kafka-to-serverless/img/eda.png new file mode 100644 index 00000000..a74b04fa Binary files /dev/null and b/patterns/kafka/kafka-to-serverless/img/eda.png differ diff --git a/patterns/kafka/kafka-to-serverless/kafka-to-serverless/index.html b/patterns/kafka/kafka-to-serverless/kafka-to-serverless/index.html new file mode 100644 index 00000000..e7a0fc31 --- /dev/null +++ b/patterns/kafka/kafka-to-serverless/kafka-to-serverless/index.html @@ -0,0 +1,3302 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Kafka to serverless - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Kafka to Serverless

+

Description

+

This pattern describes how to use AMQ Streams (Kafka) as an event source to OpenShift Serverless (Knative). You will learn how to implement Knative Eventing that can trigger a Knative Serving function when a messaged is posted to a Kafka Topic (Event).

+

Knative & OpenShift Serverless

+

Knative is an open source project that helps to deploy and manage modern serverless workloads on Kubernetes. Red Hat OpenShift Serverless is an enterprise-grade serverless offering based on knative that provides developers with a complete set of tools to build, deploy, and manage serverless applications on OpenShift Container Platform

+

Knative consists of 3 primary components:

+
    +
  • Build - A flexible approach to building source code into containers.
  • +
  • Serving - Enables rapid deployment and automatic scaling of containers through a request-driven model for serving workloads based on demand.
  • +
  • Eventing - An infrastructure for consuming and producing events to stimulate applications. Applications can be triggered by a variety of sources, such as events from your own applications, cloud services from multiple providers, Software-as-a-Service (SaaS) systems, and Red Hat AMQ streams.
  • +
+

EDA (Event Driven Architecture)

+

Event-Driven Architecture (EDA) is a way of designing applications and services to respond to real-time information based on the sending and receiving of information about individual events. EDA uses events to trigger and communicate between decoupled services and is common in modern applications built with microservices.

+

EDA

+

Use Cases

+
    +
  • Develop an event-driven architecture with serverless applications.
  • +
  • Serverless Business logic processing that is capable of automated scale-up and scale-down to zero.
  • +
+

Implementations examples

+

This pattern is implemented in the XRay Pipeline Demo

+

Deployment example

+

Requirements

+
    +
  • Red Hat OpenShift Container Platform
  • +
  • Red Hat AMQ Streams or Strimzi: the operator should be installed and a Kafka cluster must be created
  • +
  • Red Hat OpenShift Serverless: the operator must be installed
  • +
+

Part 1: Set up KNative

+

Once Red Hat OpenShift Serverless operator has been installed, we can create KnativeServing, KnativeEventing and KnativeKafka instances.

+

Step 1: Create required Knative instances

+ +
oc create -f 01_knative_serving_eventing_kafka_setup.yaml
+
+
+

Note

+

Those instances can also be deployed through the OpenShift Console if you prefer to use a UI. In this case, follow the Serverless deployment instructions (this section and the following ones).

+
+

Step 2: Verify Knative Instances

+
oc get po -n knative-serving
+oc get po -n knative-eventing
+
+
    +
  • Pod with prefix kafka-controller-manager represents Knative Kafka Event Source.
  • +
+

Part 2: Knative Serving

+

Knative Serving is your serverless business logic that you would like to execute based on the event generated by Kafka.

+

For example purpose we are using a simple greeter service here. Depending on your use case you will replace that with your own business logic.

+

Step 1: Create Knative Serving

+
    +
  • From the deployment folder, in the YAML file 02_knative_service.yaml, replace the placeholder YOUR_NAMESPACE with your namespace, and apply the file to create knative serving.
  • +
+
oc create -f 02_knative_service.yaml
+
+

Step 2: Verify Knative Serving

+
oc get serving
+
+

Part 3: Knative Eventing

+

Knative Eventing enables developers to use an event-driven architecture with serverless applications. An event-driven architecture is based on the concept of decoupled relationships between event producers that create events, and event sinks, or consumers, that receive them.

+

Step 1: Kafka topic

+
    +
  • Create a Kafka topic where the events will be sent. In this example, the topic will be example_topic.
  • +
+

Step 2: Create Knative Eventing

+
    +
  • To create a Knative Eventing, we need to create a Kafka Event Source. Before you apply the following YAML file, 03_knative_kafka_source.yaml, please make sure to edit namespace and bootstrapServers to match your Kafka cluster. Also make sure to use the correct Knative Service (serving) that you have created in the previous step (greeter in this example).
  • +
+
oc create -f 03_knative_kafka_source.yaml
+
+

Step 3: Verify Knative Eventing

+
oc get kafkasource
+
+

At this point, as soon as new messages are received in Kafka topic example_topic, Knative Eventing will trigger the Knative Service greeter to execute the business logic, allowing you to have event-driven serverless application running on OpenShift Container Platform.

+

Part 4: Testing

+
    +
  • Optional: to view the logs of Knative Serving you can install stern to them from the CLI, or use the OpenShift Web Console.
  • +
+
oc get ksvc
+stern --selector=serving.knative.dev/service=greeter -c user-container
+
+
    +
  • Launch a temporary Kafka CLI (kafkacat) in a new terminal
  • +
+
oc run kafkacat -i -t --image debezium/tooling --restart=Never
+
+
    +
  • From the kafkacat container shell, generate kafka messages in the topic example_topic of your Kafka cluster. Here we are generating Kafka messages with CloudEvents (CE) specification.
  • +
+
for i in {1..50} ; do sleep 10 ; \
+echo '{"message":"Hello Red Hat"}' | kafkacat -P -b core-kafka-kafka-bootstrap -t example_topic \
+  -H "content-type=application/json" \
+  -H "ce-id=CE-001" \
+  -H "ce-source=/kafkamessage"\
+  -H "ce-type=dev.knative.kafka.event" \
+  -H "ce-specversion=1.0" \
+  -H "ce-time=2018-04-05T03:56:24Z"
+done ;
+
+

The above command will generate 50 Kafka messages every 10 seconds. Knative Eventing will pick up the messages and invoke the greeter Knative service, that you can verify from the logs of Knative Serving.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/patterns/starproxy/img/starproxy-diagram.png b/patterns/starproxy/img/starproxy-diagram.png new file mode 100644 index 00000000..0c2895dd Binary files /dev/null and b/patterns/starproxy/img/starproxy-diagram.png differ diff --git a/patterns/starproxy/starproxy/index.html b/patterns/starproxy/starproxy/index.html new file mode 100644 index 00000000..4863eb54 --- /dev/null +++ b/patterns/starproxy/starproxy/index.html @@ -0,0 +1,2884 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Starburst/Trino proxy - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Starburst/Trino Proxy

+

What it is

+

Starproxy is a fully HTTP compliant proxy that is designed to sit between clients and a Trino/Starburst cluster. The motivation for developing a solution like this is laid out in some prior art below:

+ +

The most attractive items to us are probably:

+
    +
  • Enabling host based security
  • +
  • Detecting "bad" queries and blocking/deprioritizing them with custom rules
  • +
  • Load balancing across regions
  • +
+

How it works

+

First and foremost, starproxy is an http proxy implemented in rust using a combination of axum/hyper.

+

Diagram

+
    +
  1. +

    Parse the query AST, then check a variety of rules:

    +
      +
    • inbound CIDR rule checking
    • +
    • checking for predicates in queries
    • +
    • identifying select * queries with no limit, among other rules
    • +
    +
  2. +
  3. +

    If rules are violated they can be associated with actions, like tagging the query as low priority. This is done by modifying the request headers and injecting special tags. +Rules can also outright block requests by returning error status codes to the client directly.

    +
  4. +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/predictive-ai/what-is-predictive-ai/index.html b/predictive-ai/what-is-predictive-ai/index.html new file mode 100644 index 00000000..a5b44bd4 --- /dev/null +++ b/predictive-ai/what-is-predictive-ai/index.html @@ -0,0 +1,2772 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + What is Predictive AI? - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

What is Predictive AI?

+

Predictive AI generally has the following characteristics:

+
    +
  • A model generally created based on specific or narrow set of data.
  • +
  • Aims to make predictions on the likehood of an outcome, based on certain conditions and inputs.
  • +
  • e.g. "how likely is this person to default on their loan in the next 5 years, given what we know about them?"
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/search/search_index.json b/search/search_index.json new file mode 100644 index 00000000..631668cb --- /dev/null +++ b/search/search_index.json @@ -0,0 +1 @@ +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"demos/codellama-continue/codellama-continue/","title":"AI Assistant in Code Server","text":"

Who has never dreamed of having a know-it-all assistant that can help you writing boilerplate code, explain a function you got from a colleague but that remains obscure to you, or quickly parse through tons of error logs to fetch the one relevant line that can help you debug you application?

With the advent of AI and LLM, such services have become pretty common (Copilot, CodeWhisperer,...). However, you may wonder what happens with all the code you are sending to those services? Or more than often, you're simply not allowed to use them...

In this article, we will show you how to integrate a code assistant in Code Server, leveraging the CodeLlama model and the Continue plugin (an open source assistant for VSCode similar to Copilot).

All those solutions are open source, you fully control how they are deployed and what they do, and your code will never leave your environment!

"},{"location":"demos/codellama-continue/codellama-continue/#requirements","title":"Requirements","text":"
  • A model that has been trained for Code Generation. Many are available. The Big Code Leaderboard will give you information about the latest and greatest models. In this demo, we are going to use CodeLlama-7b-Instruct-hf because it's good enough for a demo and fits in our available GPU. For large real-world production workloads, you would of course need to consider a bigger version of this model (13B, 70B), or another model.
  • You must serve your model and have its inference endpoint. Refer to the LLM Serving section to learn how to deploy your model on RHOAI/ODH.
  • A Code Server workbench image available in your RHOAI or ODH environment. Starting at RHOAI 2.8, this image is available out of the box. Prior to that, you would have to import it as a custom image, like this one.
  • As Code Server is fully open source, it does not include the Microsoft Marketplace. So you must download the Continue extension file from the Open VSX Registry. Click on the Download button and select the Linux x64 version. You will get a file named Continue.continue-0.9.91@linux-x64.vsix (or whatever version you download).

Note

There is a repository containing the files that can be used to deploy some prerequisites for this demo. You can find it here.

"},{"location":"demos/codellama-continue/codellama-continue/#installation","title":"Installation","text":"
  • Create and Start your Workbench based on Code Server:

  • Upload the extension file:

  • Once uploaded, Code server will try to open the file and complain about its size, just close the tab:1

  • From the Extension menu on the sidebar, click on the three dots on the top right and select Install from VSIX:

  • From the menu that opens, select the file you uploaded:

  • Wait a few seconds... The installation completes and you're greeted with a welcome message:

  • You can close all the tabs or follow the tutorial.

"},{"location":"demos/codellama-continue/codellama-continue/#configuration","title":"Configuration","text":"

By default, Continue is configured with different providers and models to test. But of course we want to add our own inference endpoint to use the model we have deployed.

  • In the Sidebar, click on the new icon that was added at the bottom, the one from Continue (it may trigger an error about Headers not being defined, you can ignore it for now). At the bottom right of the Continue panel, click on the Gear icon to open the configuration file.

  • In the models section, add the following configuration (replace your inference endpoint with the right value, and eventually the name of your model):

    {\n    \"title\": \"CodeLlama-7b\",\n    \"model\": \"codellama/CodeLlama-7b-Instruct-hf\",\n    \"apiBase\": \"https://your-inference-endpoint/v1/\",\n    \"completionOptions\": {\n    \"temperature\": 0.1,\n    \"topK\": 1,\n    \"topP\": 1,\n    \"presencePenalty\": 0,\n    \"frequencyPenalty\": 0\n    },\n    \"provider\": \"openai\",\n    \"apiKey\": \"none\"\n}\n
  • You can also remove the other pre-defined models if you don't want to use them. You should end up with something like this:

  • In the tabAutocompleteModel section, add the following configuration (replace your inference endpoint with the right value, and eventually the name of your model), and add/modify the options to your liking (see documentation for all possible values):

    \"tabAutocompleteModel\": {\n    \"title\": \"CodeLlama-7b\",\n    \"model\": \"codellama/CodeLlama-7b-Instruct-hf\",\n    \"apiBase\": \"https://your-inference-endpoint/v1/\",\n    \"completionOptions\": {\n    \"temperature\": 0.1,\n    \"topK\": 1,\n    \"topP\": 1,\n    \"presencePenalty\": 0,\n    \"frequencyPenalty\": 0\n    },\n    \"provider\": \"openai\",\n    \"apiKey\": \"none\"\n},\n\"tabAutocompleteOptions\": {\n    \"useCopyBuffer\": false,\n    \"maxPromptTokens\": 1024,\n    \"prefixPercentage\": 0.5\n},\n
  • You can also disable telemetry by setting the parameter to false. You should end up with something like this:

  • Once the configuration is finished (file is auto-saved), you should see the model name available in the drop-down at the bottom of the Continue extension pane:

Continue is now ready to use!

"},{"location":"demos/codellama-continue/codellama-continue/#usage","title":"Usage","text":"

The best way to learn how to use it is to read the short documentation and experiment.

Here is a small example of what you can do: Fast Edit (ask for some code generation), then Tab completion (let the assistant suggest the next piece of code), then Explain or do something on the side:

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/","title":"Credit Card Fraud Detection Demo using MLFlow and Red Hat OpenShift AI","text":"

Info

The full source and instructions for this demo are available in this repo

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#demo-description-architecture","title":"Demo Description & Architecture","text":"

The goal of this demo is to demonstrate how RHOAI and MLFlow can be used together to build an end-to-end MLOps platform where we can:

  • Build and train models in RHOAI
  • Track and store those models with MLFlow
  • Serve a model stored in MLFlow using RHOAI Model Serving (or MLFlow serving)
  • Deploy a model application in OpenShift that runs sends data to the served model and displays the prediction

The architecture looks like this:

Description of each component:

  • Data Set: The data set contains the data used for training and evaluating the model we will build in this demo.
  • RHOAI Notebook: We will build and train the model using a Jupyter Notebook running in RHOAI.
  • MLFlow Experiment tracking: We use MLFlow to track the parameters and metrics (such as accuracy, loss, etc) of a model training run. These runs can be grouped under different \"experiments\", making it easy to keep track of the runs.
  • MLFlow Model registry: As we track the experiment we also store the trained model through MLFlow so we can easily version it and assign a stage to it (for example Staging, Production, Archive).
  • S3 (ODF): This is where the models are stored and what the MLFlow model registry interfaces with. We use ODF (OpenShift Data Foundation) according to the MLFlow guide, but it can be replaced with another solution.
  • RHOAI Model Serving: We recommend using RHOAI Model Serving for serving the model. It's based on ModelMesh and allows us to easily send requests to an endpoint for getting predictions.
  • Application interface: This is the interface used to run predictions with the model. In our case, we will build a visual interface (interactive app) using Gradio and let it load the model from the MLFlow model registry.

The model we will build is a Credit Card Fraud Detection model, which predicts if a credit card usage is fraudulent or not depending on a few parameters such as: distance from home and last transaction, purchase price compared to median, if it's from a retailer that already has been purchased from before, if the PIN number is used and if it's an online order or not.

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#deploying-the-demo","title":"Deploying the demo","text":""},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#pre-requisites","title":"Pre-requisites","text":"
  • Have Red Hat OpenShift AI (RHOAI) running in a cluster

Note

Note: You can use Open Data Hub instead of RHOAI, but some instructions and screenshots may not apply

  • Have MLFlow running in a cluster
"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#11-mlflow-route-through-the-visual-interface","title":"1.1: MLFlow Route through the visual interface","text":"

Start by finding your route to MLFlow. You will need it to send any data to MLFlow.

  • Go to the OpenShift Console as a Developer
  • Select your mlflow project
  • Press Topology
  • Press the mlflow-server circle
    • While you are at it, you can also press the little \"Open URL\" button in the top right corner of the circle to open up the MLFlow UI in a new tab - we will need it later.
  • Go to the Resources tab
  • Press mlflow-server under Services
  • Look at the Hostname and mlflow-server Port.

Note

This route and port only work internally in the cluster.

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#12-get-the-mlflow-route-using-command-line","title":"1.2: Get the MLFlow Route using command-line","text":"

Alternatively, you can use the OC command to get the hostname through: oc get svc mlflow-server -n mlflow -o go-template --template='{{.metadata.name}}.{{.metadata.namespace}}.svc.cluster.local{{println}}'

The port you will find with: oc get svc mlflow-server -n mlflow -o yaml

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#2-create-a-rhoai-workbench","title":"2: Create a RHOAI workbench","text":"

Start by opening up RHOAI by clicking on the 9 square symbol in the top menu and choosing \"Red Hat OpenShift AI\".

Then create a new Data Science project (see image), this is where we will build and train our model. This will also create a namespace in OpenShift which is where we will be running our application after the model is done. I'm calling my project 'Credit Card Fraud', feel free to call yours something different but be aware that some things further down in the demo may change.

After the project has been created, create a workbench where we can run Jupyter. There are a few important settings here that we need to set:

  • Name: Credit Fraud Model
  • Notebook Image: Standard Data Science
  • Deployment Size: Small
  • Environment Variable: Add a new one that's a Config Map -> Key/value and enter
    • Key: MLFLOW_ROUTE
    • Value: http://<route-to-mlflow>:<port>, replacing <route-to-mlflow> and <port> with the route and port that we found in step one. In my case it is http://mlflow-server.mlflow.svc.cluster.local:8080.
  • Cluster Storage: Create new persistent storage - I call it \"Credit Fraud Storage\" and set the size to 20GB.

Press Create Workbench and wait for it to start - status should say \"Running\" and you should be able to press the Open link.

Open the workbench and login if needed.

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#3-train-the-model","title":"3: Train the model","text":"

When inside the workbench (Jupyter), we are going to clone a GitHub repository which contains everything we need to train (and run) our model. You can clone the GitHub repository by pressing the GitHub button in the left side menu (see image), then select \"Clone a Repository\" and enter this GitHub URL: https://github.com/red-hat-data-services/credit-fraud-detection-demo

Open up the folder that was added (credit-fraud-detection-demo). It contains:

  • Data for training and evaluating the model.
  • A notebook (model.ipynb) inside the model folder with a Deep Neural Network model we will train.
  • An application (model_application.py) inside the application folder that will fetch the trained model from MLFlow and run a prediction on it whenever it gets any user input.

The model.ipynb is what we are going to use for building and training the model, so open that up and take a look inside, there is documentation outlining what each cell does. What is particularly interesting for this demo are the last two cells.

The second to last cell contains the code for setting up MLFlow tracking:

mlflow.set_tracking_uri(MLFLOW_ROUTE)\nmlflow.set_experiment(\"DNN-credit-card-fraud\")\nmlflow.tensorflow.autolog(registered_model_name=\"DNN-credit-card-fraud\")\n

mlflow.set_tracking_uri(MLFLOW_ROUTE) just points to where we should send our MLFlow data. mlflow.set_experiment(\"DNN-credit-card-fraud\") tells MLFlow that we want to create an experiment, and what we are going to call it. In this case I call it \"DNN-credit-card-fraud\" as we are building a Deep Neural Network. mlflow.tensorflow.autolog(registered_model_name=\"DNN-credit-card-fraud\") enables autologging of a bunch of variables (such as accuracy, loss, etc) so we don't manually have to track them. It also automatically uploads the model to MLFlow after the training completes. Here we name the model the same as the experiment.

Then in the last cell we have our training code:

with mlflow.start_run():\n    epochs = 2\n    history = model.fit(X_train, y_train, epochs=epochs, \\\n                        validation_data=(scaler.transform(X_val),y_val), \\\n                        verbose = True, class_weight = class_weights)\n\n    y_pred_temp = model.predict(scaler.transform(X_test))\n\n    threshold = 0.995\n\n    y_pred = np.where(y_pred_temp > threshold, 1,0)\n    c_matrix = confusion_matrix(y_test,y_pred)\n    ax = sns.heatmap(c_matrix, annot=True, cbar=False, cmap='Blues')\n    ax.set_xlabel(\"Prediction\")\n    ax.set_ylabel(\"Actual\")\n    ax.set_title('Confusion Matrix')\n    plt.show()\n\n    t_n, f_p, f_n, t_p = c_matrix.ravel()\n    mlflow.log_metric(\"tn\", t_n)\n    mlflow.log_metric(\"fp\", f_p)\n    mlflow.log_metric(\"fn\", f_n)\n    mlflow.log_metric(\"tp\", t_p)\n\n    model_proto,_ = tf2onnx.convert.from_keras(model)\n    mlflow.onnx.log_model(model_proto, \"models\")\n

with mlflow.start_run(): is used to tell MLFlow that we are starting a run, and we wrap our training code with it to define exactly what code belongs to the \"run\". Most of the rest of the code in this cell is normal model training and evaluation code, but at the bottom we can see how we send some custom metrics to MLFlow through mlflow.log_metric and then convert the model to ONNX. This is because ONNX is one of the standard formats for RHOAI Model Serving which we will use later.

Now run all the cells in the notebook from top to bottom, either by clicking Shift-Enter on every cell, or by going to Run->Run All Cells in the very top menu. If everything is set up correctly it will train the model and push both the run and the model to MLFlow. The run is a record with metrics of how the run went, while the model is the actual tensorflow and ONNX model which we later will use for inference. You may see some warnings in the last cell related to MLFlow, as long as you see a final progressbar for the model being pushed to MLFlow you are fine:

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#4-view-the-model-in-mlflow","title":"4: View the model in MLFlow","text":"

Let's take a look at how it looks inside MLFlow now that we have trained the model. If you opened the MLFlow UI in a new tab in step 1.1, then just swap over to that tab, otherwise follow these steps:

  • Go to the OpenShift Console
  • Make sure you are in Developer view in the left menu
  • Go to Topology in the left menu
  • At the top left, change your project to \"mlflow\" (or whatever you called it when installing the MLFlow operator in pre-requisites)
  • Press the \"Open URL\" icon in the top right of the MLFlow circle in the topology map

When inside the MLFlow interface you should see your new experiment in the left menu. Click on it to see all the runs under that experiment name, there should only be a single run from the model we just trained. You can now click on the row in the Created column to get more information about the run and how to use the model from MLFlow.

We will need the Full Path of the model in the next section when we are going to serve it, so keep this open.

"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#5-serve-the-model","title":"5: Serve the model","text":"

Note

You can either serve the model using RHOAI Model Serving or use the model straight from MLFlow. We will here show how you serve it with RHOAI Model Serving as that scales better for large applications and load. At the bottom of this section we'll go through how it would look like to use MLFlow instead.

To start, go to your RHOAI Project and click \"Add data connection\". This data connection connects us to a storage we can load our models from.

Here we need to fill out a few details. These are all assuming that you set up MLFlow according to this guide and have it connected to ODF. If that's not the case then enter the relevant details for your use case.

  • Name: mlflow-connection
  • AWS_ACCESS_KEY_ID: Run oc get secrets mlflow-server -n mlflow -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d' in your command prompt, in my case it's nB0z01i0PwD9PMSISQ2W
  • AWS_SECRET_ACCESS_KEY: Run oc get secrets mlflow-server -n mlflow -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d' in your command prompt, in my case it's FLgEJmGQm5CdRQRnXc8jVFcc+QDpM1lcrGpiPBzI.

Note

In my case the cluster and storage has already been shut down, don't share this in normal cases.

  • AWS_S3_ENDPOINT: Run oc get configmap mlflow-server -n mlflow -o yaml | grep BUCKET_HOS in your command prompt, in my case it's http://s3.openshift-storage.svc
  • AWS_DEFAULT_REGION: Where the cluster is being ran
  • AWS_S3_BUCKET: Run oc get obc -n mlflow -o yaml | grep bucketName in your command prompt, in my case it's mlflow-server-576a6525-cc5b-46cb-95f3-62c3986846df

Then press \"Add data connection\". Here's an example of how it can look like:

Then we will configure a model server, which will serve our models.

Just check the 'Make deployed available via an external route' checkbox and then press \"Configure\" at the bottom.

Finally, we will deploy the model, to do that, press the \"Deploy model\" button which is in the same place that \"Configure Model\" was before. We need to fill out a few settings here:

  • Name: credit card fraud
  • Model framework: onnx-1 - Since we saved the model as ONNX in the model training section
  • Model location:
    • Name: mlflow-connection
    • Folder path: This is the full path we can see in the MLFlow interface from the end of the previous section. In my case it's 1/b86481027f9b4b568c9efa3adc01929f/artifacts/models/. Beware that we only need the last part, which looks something like: 1/..../artifacts/models/

Press Deploy and wait for it to complete. It will show a green checkmark when done. You can see the status here:

Click on \"Internal Service\" in the same row to see the endpoints, we will need those when we deploy the model application.

[Optional] MLFlow Serving:

This section is optional

This section explains how to use MLFlow Serving instead of RHOAI Model Serving. We recommend using RHOAI Model Serving as it scales better. However, if you quickly want to get a model up and running for testing, this would be an easy way.

To use MLFlow serving, simply deploy an application which loads the model straight from MLFlow. You can find the model application code for using MLFlow serving in the \"application_mlflow_serving\" folder in the GitHub repository you cloned in step 3.

If you look inside model_application_mlflow_serve.py you are going to see a few particularly important lines of code:

# Get a few environment variables. These are so we can:\n# - get data from MLFlow\n# - Set server name and port for Gradio\nMLFLOW_ROUTE = os.getenv(\"MLFLOW_ROUTE\")\n...\n\n# Connect to MLFlow using the route.\nmlflow.set_tracking_uri(MLFLOW_ROUTE)\n\n# Specify what model and version we want to load, and then load it.\nmodel_name = \"DNN-credit-card-fraud\"\nmodel_version = 1\nmodel = mlflow.pyfunc.load_model(\n    model_uri=f\"models:/{model_name}/{model_version}\"\n)\n

Here is where we set up everything that's needed for loading the model from MLFlow. The environment variable MLFLOW_ROUTE is set in the Dockerfile. You can also see that we specifically load version 1 of the model called \"DNN-credit-card-fraud\" from MLFlow. This makes sense since we only ran the model once, but is easy to change if any other version or model should go into production

Follow the steps of the next section to see how to deploy an application, but when given the choice for \"Context dir\" and \"Environment variables (runtime only)\", use these settings instead:

  • Context dir: \"/model_application_mlflow_serve\"
  • Environment variables (runtime only) fields:
    • Name: MLFLOW_ROUTE
    • Value: The MLFlow route from step one (http://mlflow-server.mlflow.svc.cluster.local:8080 for example)
"},{"location":"demos/credit-card-fraud-detection-mlflow/credit-card-fraud/#6-deploy-the-model-application","title":"6: Deploy the model application","text":"

The model application is a visual interface for interacting with the model. You can use it to send data to the model and get a prediction of whether a transaction is fraudulent or not. You can find the model application code in the \"application\" folder in the GitHub repository you cloned in step 3.

If you look inside it model_application.py, you will see two particularly important lines of code:

# Get a few environment variables. These are so we:\n# - Know what endpoint we should request\n# - Set server name and port for Gradio\nURL = os.getenv(\"INFERENCE_ENDPOINT\") <----------\n...\n\n    response = requests.post(URL, json=payload, headers=headers)  <----------\n

This is what we use to send a request to our RHOAI Model Server with some data we want it to run a prediction on.

We are going to deploy the application with OpenShift by pointing to the GitHub repository. It will pull down the folder, automatically build a container image based on the Dockerfile, and publish it.

To do this, go to the OpenShift Console and make sure you are in Developer view and have selected the credit-card-fraud project. Then press \"+Add\" in the left menu and select Import from Git.

In the \"Git Repo URL\" enter: https://github.com/red-hat-data-services/credit-fraud-detection-demo (this is the same repository we pulled into RHOAI earlier). Then press \"Show advanced Git options\" and set \"Context dir\" to \"/application\". Finally, at the very bottom, click the blue \"Deployment\" link:

Set these values in the Environment variables (runtime only) fields:

  • Name: INFERENCE_ENDPOINT
  • Value: In the RHOAI projects interface (from the previous section), copy the \"restURL\" and add /v2/models/credit-card-fraud/infer to the end if it's not already there. For example: http://modelmesh-serving.credit-card-fraud:8008/v2/models/credit-card-fraud/infer

Your full settings page should look something like this:

Press Create to start deploying the application.

You should now see three objects in your topology map, one for the Workbench we created earlier, one for the model serving, and one for the application we just added. When the circle of your deployment turns dark blue it means that it has finished deploying.

If you want more details on how the deployment is going, you can press the circle and look at Resources in the right menu that opens up. There you can see how the build is going and what's happening to the pod. The application will be ready when the build is complete and the pod is \"Running\".

When the application has been deployed you can press the \"Open URL\" button to open up the interface in a new tab.

Congratulations, you now have an application running your AI model!

Try entering a few values and see if it predicts it as a credit fraud or not. You can select one of the examples at the bottom of the application page.

"},{"location":"demos/financial-fraud-detection/financial-fraud-detection/","title":"Financial Fraud Detection","text":"

Info

The full source and instructions for this demo are available on this repo

This demo shows how to use OpenShift AI to train and test a relatively simplistic fraud detection model. In exploring this content, you will become familiar with the OpenShift AI offering and common workflows to use with it.

"},{"location":"demos/llama2-finetune/llama2-finetune/","title":"Fine-Tune Llama 2 Models with Ray and DeepSpeed","text":"

In this repo, you will find an example demonstrating how to fine-tune LLMs with Ray on OpenShift AI, using HF Transformers, Accelerate, PEFT (LoRA), and DeepSpeed, for Llama 2 models.

It adapts the Fine-tuning Llama-2 series models with Deepspeed, Accelerate, and Ray Train TorchTrainer1 example from the Ray project, so it runs using the Distributed Workloads stack, on OpenShift AI.

Overview of the Ray Dashboard during the fine-tuning:

Example of a visualization comparing experiments with different combinations of context length and batch size:

  1. https://github.com/ray-project/ray/tree/master/doc/source/templates/04_finetuning_llms_with_deepspeed \u21a9

"},{"location":"demos/llm-chat-doc/llm-chat-doc/","title":"Chat with your Documentation","text":"

If you want to learn more about LLMs and how to serve them, please read the LLM Serving documentation first.

"},{"location":"demos/llm-chat-doc/llm-chat-doc/#rag-chatbot-full-walkthrough","title":"RAG Chatbot Full Walkthrough","text":"

Although the available code is normally pretty well documented, especially the notebooks, giving a full overview will surely help you understand how all of the different elements fit together.

For this walkthrough we will be using this application, which is a RAG-based Chatbot that will use a Milvus vector store, vLLM for LLM serving, Langchain as the \"glue\" between those components, and Gradio as the UI engine.

"},{"location":"demos/llm-chat-doc/llm-chat-doc/#requirements","title":"Requirements","text":"
  • An OpenShift cluster with RHOAI or ODH deployed.
  • A node with a GPU card. For the model we will use, 24GB memory on the GPU (VRAM) is necessary. If you have less than that you can either use quantization when loading the model, use an already quantized model (results may vary as they are not all compatible with the model server), or choose another compatible smaller model.
"},{"location":"demos/llm-chat-doc/llm-chat-doc/#model-serving","title":"Model Serving","text":"

Deploy vLLM Model Serving instance in the OpenAI compatible API mode, either:

  • as a custom server runtime in ODH/RHOAI.
  • as a standalone server in OpenShift.

In both cases, make sure you deploy the model mistralai/Mistral-7B-Instruct-v0.2.

"},{"location":"demos/llm-chat-doc/llm-chat-doc/#vector-store","title":"Vector Store","text":""},{"location":"demos/llm-chat-doc/llm-chat-doc/#milvus-deployment","title":"Milvus deployment","text":"

For our RAG we will need a Vector Database to store the Embeddings of the different documents. In this example we are using Milvus.

Deployment instructions specific to OpenShift are available here.

After you follow those instructions you should have a Milvus instance ready to be populated with documents.

"},{"location":"demos/llm-chat-doc/llm-chat-doc/#document-ingestion","title":"Document ingestion","text":"

In this notebook you will find detailed instructions on how to ingest different types of documents: PDFs first, then Web pages.

The examples are based on RHOAI documentation, but of course we encourage you to use your own documentation. After all that's the purpose of all of this!

This other notebook will allow you to execute simple queries against your Vector Store to make sure it works alright.

Note

Those notebooks are using the NomicAI Embeddings to create and query the collection. If you want to use the default embeddings from Langchain, other notebooks are available. They have the same name, just without the -nomic at the end.

"},{"location":"demos/llm-chat-doc/llm-chat-doc/#testing","title":"Testing","text":"

Now let's put all of this together!

This notebook will be used to create a RAG solution leveraging the LLM and the Vector Store we just populated. Don't forget to enter the relevant information about your Model Server (the Inference URL and model name), and about your Vector store (connection information and collection name) on the third cell.

You can also adjust other parameters as you see fit.

  • It will first initialize a connection to the vector database (embeddings are necessary for the Retriever to \"understand\" what is stored in the database):
model_kwargs = {'trust_remote_code': True}\nembeddings = HuggingFaceEmbeddings(\n    model_name=\"nomic-ai/nomic-embed-text-v1\",\n    model_kwargs=model_kwargs,\n    show_progress=False\n)\n
  • A prompt template is then defined. You can see that we will give it specific instructions on how the model must answer. This is necessary if you want to keep it focused on its task and not say anything that may not be appropriate (on top of getting you fired!). The format of this prompt is originally the one used for Llama2, but Mistral uses the same one. You may have to adapt this format if you use another model.
template=\"\"\"<s>[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant named HatBot answering questions.\nYou will be given a question you need to answer, and a context to provide you with information. You must answer the question based as much as possible on this context.\nAlways answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<</SYS>>\n\nContext: \n{context}\n\nQuestion: {question} [/INST]\n\"\"\"\n
  • Now we will define the LLM connection itself. As you can see there are many parameters you can define that will modify how the model will answer. Details on those parameters are available here.
llm =  VLLMOpenAI(\n    openai_api_key=\"EMPTY\",\n    openai_api_base=INFERENCE_SERVER_URL,\n    model_name=MODEL_NAME,\n    max_tokens=MAX_TOKENS,\n    top_p=TOP_P,\n    temperature=TEMPERATURE,\n    presence_penalty=PRESENCE_PENALTY,\n    streaming=True,\n    verbose=False,\n    callbacks=[StreamingStdOutCallbackHandler()]\n)\n
  • And finally we can tie it all together with a specific chain, RetrievalQA:
qa_chain = RetrievalQA.from_chain_type(\n        llm,\n        retriever=store.as_retriever(\n            search_type=\"similarity\",\n            search_kwargs={\"k\": 4}\n            ),\n        chain_type_kwargs={\"prompt\": QA_CHAIN_PROMPT},\n        return_source_documents=True\n        )\n
  • That's it! We can now use this chain to send queries. The retriever will look for relevant documents in the Vector Store, their content will be injected automatically in the prompt, and the LLM will try to create a valid answer based on its own knowledge and this content:
question = \"How can I create a Data Science Project?\"\nresult = qa_chain.invoke({\"query\": question})\n
  • The last cell in the notebook will simply filter for duplicates in the sources that were returned in the result, and display them:
def remove_duplicates(input_list):\n    unique_list = []\n    for item in input_list:\n        if item.metadata['source'] not in unique_list:\n            unique_list.append(item.metadata['source'])\n    return unique_list\n\nresults = remove_duplicates(result['source_documents'])\n\nfor s in results:\n    print(s)\n
"},{"location":"demos/llm-chat-doc/llm-chat-doc/#application","title":"Application","text":"

Notebooks are great and everything, but it's not what you want to show to your users. I hope...

So instead, here is a simple UI that implements mostly the same code we used in the notebooks.

The deployment is already explained in the repo and pretty straightforward as the application will only \"consume\" the same Vector Store and LLM Serving we have used in the notebooks. However I will point out some specificities:

  • This implementation allows you to have different collections in Milvus you can query from. This is fully configurable, you can create as many collections as you want and add them to the application.
  • The code is more complicated than the notebooks as it allows for multiple users to use the application simultaneously. They can all use a different collection, ask questions at the same time, they stay fully isolated. The limitation is the memory you have.
  • Most (if not all) parameters are configurable. They are all described in the README file.

Some info on the code itself (app.py):

  • load_dotenv, along with the env.example file (once renamed .env) will allow you to develop locally.
  • As normally your Milvus instance won't be exposed externally to OpenShift, if you want to develop locally you may want to open a tunnel to it with oc port-forward Service/milvus-service 19530:19530 (replace with the name of the Milvus Service along with the ports if you change them). You can use the same technique for the LLM endpoint if you have not exposed it as a route.
  • The class QueueCallback was necessary because the vLLMOpenAI library used to query the model does not return an iterator in the format Langchain expects it (at the time of this writing). Instead, this implementation of the Callback functions for the LLM puts the new tokens in a Queue (L43) that is then retrieved from continuously (L65), with the content being yielded for display. This is a little bit convoluted, but the whole stack is still in full development, so sometimes you have to be creative...
  • The default Milvus Retriever (same for almost all vector databases in Langchain) does not allow to filter on the score. This means that whatever your query, some documents will always be fetched and passed into the context. This is an unwanted behavior if the query has no relation to the knowledge base you are using. So I created a custom Retriever Class, in the file milvus_retriever_with_score_threshold.py that allows to filter the documents according to score. NOTE: this a similarity search with a cosine score, so the lesser, the better. The threshold calculation is \"no more than...\".
  • Gradio configuration is pretty straightforward trough the ChatInterface component, only hiding some buttons, adding an avatar image for the bot,... The only notable thing is the use of a State variable for the selected collection so that a switch from one collection to the other is not applied to all users (this is an early mistake I made \ud83d\ude0a) .

Here is what you RAG-based Chatbot should look like:

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/","title":"From Podman AI Lab to OpenShift AI","text":"

Overview

A common statement from many people today is \"I want the ability to chat with my documents\". They might be aware that Large Language Models (LLMs) provide them the ability to do that, but the implementation details are unknown as well as the possible risks. These unknowns make it difficult to understand where to start.

Thankfully, there is a path forward with the newly released Podman AI Lab extension. Podman AI Lab allows you to pull down models and test them out locally to see how they perform and which one(s) will work best for your use case(s). The chatbot recipe within Podman AI Lab makes integrating LLMs with applications as easy as the click of a button.

Podman AI Lab is an excellent place to evaluate and test models, but you'll eventually want to see how this will actually be deployed in your enterprise. For that, we can use OpenShift and OpenShift AI along with the Elasticsearch vector database to create a Retrieval Augmented Generation (RAG) chatbot.

This article will walk you through how to go from a chatbot recipe in the Podman AI Lab extension to a RAG chatbot deployed on OpenShift and OpenShift AI.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#high-level-architecture","title":"High Level Architecture","text":"
  1. An LLM is downloaded through Podman AI Lab.

  2. A chatbot recipe is started in Podman AI Lab with the downloaded model.

  3. The chatbot recipe code from Podman AI Lab is updated in VS Code with LangChain to connect to the Elasticsearch vector database and OpenShift AI model serving inference endpoint.

  4. An ingestion notebook is run in OpenShift AI to add data to the Elasticsearch vector database.

  5. The LLM we downloaded from Podman AI Lab is deployed to OpenShift AI on a custom serving runtime.

  6. The updated chatbot with LangChain is built as a container and deployed to OpenShift.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#requirements","title":"Requirements","text":"

It is expected that you have admin access to an OpenShift 4.12+ cluster. The following code was tested with an OpenShift 4.15 cluster and OpenShift AI 2.9.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#podman-ai-lab","title":"Podman AI Lab","text":""},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#install-podman-desktop-and-podman-ai-lab-extension","title":"Install Podman Desktop and Podman AI Lab extension","text":"

Follow the installation instructions for Podman Desktop and the Podman AI Lab extension in the below Red Hat Developer article. The article also gives a great overview of the features in Podman AI Lab. Podman AI Lab - Getting Started

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#download-the-model","title":"Download the model","text":"

We will be downloading and using the TheBloke/Mistral-7B-Instruct-v0.2-GGUF. This model is quantised (smaller) version of the full Mistral-7B-Instruct-v0.2. The smaller model will allow us to run inferencing on CPUs if GPUs are not an option.

  1. Go to the AI Lab extension and select Catalog under Models.

  2. If you haven't already, download the TheBloke/Mistral-7B-Instruct-v0.2-GGUF model.

  3. The model is around 4GB so it might take some time.

  4. Podman AI Lab allows you to get started quickly with downloaded models through Services, Playgrounds, and the Recipes Catalog.

    1. Services - The Services section allows you to create a model service endpoint for models you've downloaded. Client code is provided (cURL by default) in multiple formats to get you started quickly with sending in requests to the model service endpoint.

    2. Playgrounds - The Playgrounds area allows you to define system prompts and experiment with different settings like temperature, max tokens, and top-p.

    3. Recipes Catalog - The Recipes Catalog contains demo applications for Natural Language Processing (NLP), Computer Vision, and Audio. We'll be using the ChatBot recipe demo code in this example.

  5. Create the Chatbot - Make sure to select TheBloke/Mistral-7B-Instruct-v0.2-GGUF as your model and then click Start AI App button.

  6. After the chatbot has started open it up to test it out.

  7. At the bottom of the AI App Details section you'll see a Open in VSCode button. Clicking on that will open all of the code that is running your chatbot. Later we'll modify that code to connect langchain, TheBloke/Mistral-7B-Instruct-v0.2-GGUF model, and the Elasticsearch Vector Database.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#deploying-openshift-ai","title":"Deploying OpenShift AI","text":"

Optional: If you already have an OpenShift AI instance with a Data Science Cluster you can skip this section.

Follow the product documentation to install OpenShift AI.

OR

OpenShift AI automated install
  1. Clone podman-ai-lab-to-rhoai

  2. Login to your OpenShift cluster in a terminal with the API token. You can get your API token from the OpenShift web console.

    oc login --token=<YOUR_OPENSHIFT_API_TOKEN> --server=https://<YOUR_OPENSHIFT_API_URL>:6443\n

  3. We'll first deploy the OpenShift AI operator.

    oc apply -k ./components/openshift-ai/operator/overlays/fast\n

  4. Now we'll create a Data Science Cluster. Make sure the operator is fully deployed before creating the Data Science Cluster.

    watch oc get pods -n redhat-ods-operator\n

    Once the pod has a Running status and is ready you can run the below command.

    oc apply -k ./components/openshift-ai/instance/overlays/fast\n

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#deploy-elasticsearch-vector-db","title":"Deploy Elasticsearch Vector DB","text":"

Optional: If you already have an Elasticsearch instance you can skip this section.

Follow the product documentation to install Elasticsearch

OR

Elasticsearch automated install
  1. Clone podman-ai-lab-to-rhoai

  2. We'll now deploy the Elasticsearch operator. This will be our vector database.

    oc apply -k ./components/elasticsearch/base/\n
  3. Now we can create an Elasticsearch cluster instance. Make sure the Elasticsearch operator pod is in a running state and ready.

    watch oc get pods -n elastic-vectordb\n

    oc apply -f ./components/elasticsearch/cluster/instance.yaml\n
"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#ingest-data-into-the-elasticsearch-vector-database","title":"Ingest data into the Elasticsearch Vector Database","text":"

Now that the Elasticsearch operator has been deployed and an instance created, we can ingest some data and query the vector database.

  1. Go to your OpenShift AI Dashboard. You can get the URL either from the below oc command or the OpenShift web console.

    oc get routes -n redhat-ods-applications\n
    OR

    Select the redhat-ods-applications project, the Networking -> Routes, and then open the URL under Location.

  2. Login to the dashboard and select Data Science Projects and click the Create Data Science Project button.

  3. Name the project podman-ai-lab-rag-project and click the Create button.

  4. We'll now create a workbench where we can upload a Jupyter notebook to ingest data into the Elasticsearch vector DB. We'll then test it out by querying for some data. Click the Create a workbench button.

  5. Name the workbench elastic-vectordb-workbench, select a Standard Data Science notebook, and select a Medium size.

    You'll also want to set two environment variables so we can connect to Elasticsearch from the notebook.

    • CONNECTION_STRING - Copy the CLUSTER-IP and PORT from this oc command

      oc get service elasticsearch-sample-es-http -n elastic-vectordb\n

      Add the CONNECTION_STRING key/value as a ConfigMap environment variable.

    • PASSWORD - Create a secret environment variable with the Elasticsearch secret value.

      oc get secret elasticsearch-sample-es-elastic-user -n elastic-vectordb -o jsonpath=\"{.data['elastic']}\" | base64 -d > elastic_pass.txt\n

      Add the PASSWORD key/value as a Secret environment variable. The password is in the elastic_pass.txt file that was created by the above oc command.

    NOTE: You can delete the elastic_pass.txt file that you got the password from after you add it to the environment variable.

    Click on the Create Workbench button. Your workbench should start in a few minutes.

  6. Open your workbench after it has started and login.

    Note: If you have insufficient resources to start a medium container size then stop the workbench and change the workbench to start as a small container size.

  7. Upload or import the ./notebooks/Langchain-ElasticSearchVector-Ingest.ipynb notebook to your workbench.

  8. Run the first 3 cells (Shift + Enter in each cell to run). Make sure you can connect to your Elasticsearch cluster.

  9. Continue to run through each cell while reading through what is occurring in each one. The Create the index and ingest the documents cell is where all of the websites and pdfs are stored with embeddings into the Elasticsearch vector database.

    This cell will take a while to run. Good time for a coffee break.

  10. After all of the data is stored into our vector database we can directly query it. Run the last 2 cells to make sure the data was stored successfully and we get back the results we expect.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#deploy-s3-storage-minio","title":"Deploy s3 Storage (Minio)","text":"

Optional: If you already have s3 compatible storage you can skip to step 2 to create the bucket.

OpenShift AI model serving has a dependency on s3 storage. We'll deploy Minio for this tutorial, but any s3 compatible storage should work. For an enterprise s3 storage solution consider OpenShift Data Foundation.

Follow the Minio Installation if you don't have s3 compatible storage.

  1. Login to the Minio UI. You can find the route in either the web console or from the oc cli in your terminal. Login with minio/minio123. Minio contains 2 routes, an API route and UI route. Make sure you use the UI route.

  2. Create a bucket named models and click the Create Bucket button.

  3. Go to Object Browser, select the models bucket you just created, and click the Create new path button. Name the folder path mistral7b and click the Create button.

  4. Upload the Mistral7b model to the folder path you just created. You can find out where the model was downloaded if you go back to Podman AI Lab and click the Open Model Folder icon.

    In Minio click on the Upload File button and select the model file under the hf.TheBloke.mistral-7b-instruct-v0.2.Q4_K_M directory.

  5. If the model is uploaded successfully you should see the below screen.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#create-custom-model-serving-runtime","title":"Create Custom Model Serving Runtime","text":"

Follow the product documentation to install the single-model serving platform.

OR

Single-model serving platform automated install
  1. Clone podman-ai-lab-to-rhoai

  2. We first need to enable the single serving runtime before we can add our custom serving runtime.

    1. Run the following oc command to deploy Service Mesh.
      oc apply -k ./components/openshift-servicemesh/operator/overlays/stable\n
    2. Run the following oc command to deploy Serverless.

      oc apply -k ./components/openshift-serverless/operator/overlays/stable\n

    3. Wait until the Service Mesh and Serverless operators have installed successfully.

      watch oc get csv -n openshift-operators\n

    4. We'll be using the single stack serving in OpenShift AI so we'll want use a trusted certificate instead of a self signed one. This will allow our chatbot to access the model inference endpoint.

      Run the below oc commands

      1. Get the name of the ingress cert we will need to copy. Select a secret that has cert in the name.

        oc get secrets -n openshift-ingress | grep cert\n

      2. Copy the full name of the secret you chose and replace the name in the below oc command. Make sure you're in the top level directory of this project and run the below command.

        oc extract secret/<CERT_SECRET_FROM_ABOVE> -n openshift-ingress --to=ingress-certs --confirm\n

        You should now have a ingress-certs directory with a tls.crt and tls.key file.

      3. We'll now update the secret that will be used in our OpenShift AI data science cluster.

        cd ingress-certs\n\noc create secret generic knative-serving-cert -n istio-system --from-file=. --dry-run=client -o yaml | oc apply -f -\n\ncd ..\n

        NOTE: You can delete the ingress-certs folder after you have created the knative-serving-cert secret.

      4. To avoid possible SSL errors with the connection to the Minio bucket when creating your model server in the following steps, add a custom CA bundle to default-dsci:

        oc get secret -n openshift-ingress-operator router-ca -o jsonpath='{.data.tls\\.crt}' | base64 -d > openshift-ca-bundle.pem\n\noc get configmap -n openshift-config openshift-service-ca.crt -o jsonpath='{.data.service-ca\\.crt}' >> openshift-ca-bundle.pem\n\nCA_BUNDLE_FILE=./openshift-ca-bundle.pem\n\noc patch dscinitialization default-dsci --type='json' -p='[{\"op\":\"replace\",\"path\":\"/spec/trustedCABundle/customCABundle\",\"value\":\"'\"$(awk '{printf \"%s\\\\n\", $0}' $CA_BUNDLE_FILE)\"'\"}]'\n

        NOTE: You can delete the openshift-ca-bundle.pem file after you have patched your dscinitialization, or you can add it to your trusted CA sources if it's necessary.

    5. Run the following oc commands to enable the Single Model Serving runtime for OpenShift AI.

      oc apply -k ./components/model-server/components-serving\n

    6. It will take around 5 to 10 minutes for the changes to be applied. Single-model serving should be ready when Service Mesh and Serverless have the below instances created. Open the OpenShift web console and go to Operators -> Installed Operators.

  3. Go to the OpenShift AI dashboard and expand Settings and select Serving Runtimes. You should now see that Single-model serving enabled at the top of the page.

    NOTE: You might need to refresh the page and it could take a few minutes for the changes to be applied.

NOTE: Make sure your single-model serving platform is using a trusted certificate. If it is not or you're unsure see section D in the Single-model serving platform automated install above.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#add-a-custom-serving-runtime","title":"Add a Custom Serving Runtime","text":"

We'll now add a custom serving runtime so we can deploy the GGUF version of model.

NOTE: We will continue to use the GGUF version of the model to be able to deploy this model without the need for a hardware accelerator (e.g. GPU). OpenShift AI contains a scalable model serving platform to accommodate deploying multiple full sized LLMs.

  1. Click on the Add serving runtime button.

  2. Select Single-model serving platform for the runtime and select REST for the API protocol. Upload the ./components/custom-model-serving-runtime/llamacpp-runtime-custom.yaml file as the serving runtime. Click the Create button.

    NOTE: I've included a pre-built image that is public. You can build your own image with the Containerfile under ./components/ucstom-model-serving-runtime if you would rather pull from your own repository.

  3. If the serving runtime was added was succesfully you should now see it in the list of serving runtimes available.

"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#deploy-model","title":"Deploy Model","text":"
  1. Go to your podman-ai-lab-rag-project and select Models. You should see two model serving type options. Click on the Deploy model under the Single-model serving platform.

  2. Fill in the following values and click the Deploy button at the bottom of the form.

    • Model name = mistral7b
    • Serving runtime = LlamaCPP
    • Model framework = any
    • Model server size = Medium
    • Select New data connection
    • Name = models
    • Access key = minio
    • Secret key = minio123
    • Endpoint = Your Minio API URL
    • Region = us-east-1
    • Bucket = models
    • Path = mistral7b

  3. If your model deploys successfully you should see the following page.

  4. Test your model to make sure you can send in a request and get a response. You can use the client code that is provided by the model service in Podman AI Lab.

    Make sure to update the URL in the cURL command to the Inference endpoint on OpenShift AI.

    curl --location 'https://YOUR-OPENSHIFT-AI-INFERENCE-ENDPOINT/v1/chat/completions' --header 'Content-Type: application/json' --data '{\n  \"messages\": [\n    {\n      \"content\": \"You are a helpful assistant.\",\n      \"role\": \"system\"\n    },\n    {\n      \"content\": \"How large is the capital of France?\",\n      \"role\": \"user\"\n    }\n  ]\n}'\n

    Your response should be similar to the following

    {\"id\":\"chatcmpl-c76974b1-4709-41a5-87cf-1951e10886fe\",\"object\":\"chat.completion\",\"created\":1717616440,\"model\":\"/mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf\",\"choices\":[{\"index\":0,\n\"message\":{\"content\":\" The size of a city's area, including its metropolitan area, can vary greatly, and when referring to the \n\\\"capital\\\" of a country like France, people usually mean the city itself rather than its total metropolitan area. Paris, the capital \ncity of France, covers an urban area of approximately 105 square \nkilometers (40.5 square miles) within its administrative limits.\n\\n\\nHowever, if you are asking about the total area of the Paris \nMetropolitana region, which includes suburban areas and their \ncombined population, it is much larger at around 13,022 square \nkilometers (5,028 square miles). This encompasses more than just the city of Paris.\",\n\"role\":\"assistant\"},\"logprobs\":null,\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":32,\"completion\n

    If you face any SSL errors when running the previous command, try to add the max_tokens limit to 100, which will get you safely under the 60s timeout limit of the Knative queue-proxy service (PS. The use of jq is optional):

      curl -k --location 'https://YOUR-OPENSHIFT-AI-INFERENCE-ENDPOINT/v1/chat/completions' --header 'Content-Type: application/json' --data '{\n    \"messages\": [\n      {\n        \"content\": \"You are a helpful assistant.\",\n        \"role\": \"system\"\n      },\n      {\n        \"content\": \"How large is the capital of France?\",\n        \"role\": \"user\"\n      }\n    ],\n    \"max_tokens\": 100\n  }' | jq .\n    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                             Dload  Upload   Total   Spent    Left  Speed\n  100  1011  100   793  100   218     14      3  0:01:12  0:00:55  0:00:17   174\n  {\n    \"id\": \"chatcmpl-687c22c8-d0ba-4ea4-a012-d4b64069d7a2\",\n    \"object\": \"chat.completion\",\n    \"created\": 1727727459,\n    \"model\": \"/mnt/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf\",\n    \"choices\": [\n      {\n        \"index\": 0,\n        \"message\": {\n          \"content\": \" The size of a city's area, including its urban and rural parts, is typically measured in square kilometers or square miles. However, when referring to the size of a city's capital, people usually mean the size of its urban core or central business district rather than the entire metropolitan area. In this context, Paris, the capital city of France, has an urban area of approximately 105 square kilometers (40.5 square miles). However, if you meant\",\n          \"role\": \"assistant\"\n        },\n        \"logprobs\": null,\n        \"finish_reason\": \"length\"\n      }\n    ],\n    \"usage\": {\n      \"prompt_tokens\": 32,\n      \"completion_tokens\": 100,\n      \"total_tokens\": 132\n    }\n  }\n
"},{"location":"demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/#update-the-chat-recipe-application","title":"Update the Chat Recipe Application","text":"

We'll now update the chat recipe application that we created from Podman AI Lab to use Langchain to connect the model we just deployed on OpenShift AI and the Elasticsearch vector database.

  1. We'll start from the default chatbot recipe code accessible from Podman AI Lab.

    In Podman AI Lab, after clicking the button you should see the following.

    The only code we'll need to modify is under the app directory.

  2. Open the ./components/app/chatbot_ui.py file.

    • We'll first get some environment variables.

    Then we'll add in the Langchain code to give us our RAG functionality. Note the items highlighted in red. Specifically where the model_service or your OpenShift AI inference endpoint URL and the Elasticsearch setup. Finally, take note of how both of these are passed to Langchain (chain).

    • The last updates to the code are just to format the response so that the relevant documents will be included. Extra packages were also added to the ./components/app/requirements.txt file.
  3. You can build the Containerfile and push it to your own repository or you can use the one at quay.io/jhurlocker/elastic-vectordb-chat.

  4. If the chatbot app has SSL failures or timeouts similar to those mentioned in item 4 of the previous subtitle, add the max_tokens parameter to the chatbot_ui.py code in the ChatOpenAI connection part. Or if you want a built image with this parameter, you can use quay.io/alexonoliveira/elastic-vectordb-chat:latest.

  5. Update the ./components/app/deployment.yaml file with your values for the MODEL_ENDPOINT, ELASTIC_URL, and ELASTIC_PASS environment variables.

    NOTE: Make sure you include 'https://' and the port ':9200' in the ELASTIC_URL environment variable

  6. Create the project

    oc new-project elastic-vectordb-chat\n
  7. Apply the deployment.yaml you just updated to deploy the chatbot application.

    oc apply -f ./components/app/deployment.yaml\n
  8. Get the route to the chatbot application

    oc get route -n elastic-vectordb-chat\n

    Open the application in your browser

  9. Type in a message and press Enter. It might take awhile to respond if the model is deployed on a CPU.

  10. In the OpenShift web console you can check the model server logs under the podman-ai-lab-rag-project -> Workloads -> Pods (mistral7b-*) -> Logs. Note the log statements when a message is sent to the model inference endpoint.

  11. Congratulations! You've successfully taken a model and application from Podman AI Lab and created a RAG chatbot deployed on OpenShift and OpenShift AI.

Special thanks to the maintainers of the below repositories.

  • LLM On OpenShift The notebook to ingest data into Elasticsearch and the Langchain code added to the chatbot app.

  • AI Accelerator The code used to deploy the various components on OpenShift and OpenShift AI.

"},{"location":"demos/retail-object-detection/retail-object-detection/","title":"Object Detection in Retail","text":"

Info

The full source and instructions for this demo are available in this repo

In this demo, you can see how to build an intelligent application that gives a customer the ability to find merchandise discounts, for shirts, as they browse clothing in a department store.

You can download the related presentation.

"},{"location":"demos/smart-city/smart-city/","title":"Smart City, an Edge-to-Core Data Story","text":"

Info

The full source and instructions for this demo are available in this repo

In this demo, we show how to implement this scenario:

  • Using a trained ML model, licence plates are recognized at toll location.
  • Data (plate number, location, timestamp) is send from toll locations (edge) to the core using Kafka mirroring to handle communication issues and recovery.
  • Incoming data is screened real-time to trigger alerts for wanted vehicles (Amber Alert).
  • Data is aggregated and stored into object storage.
  • A central database contains other information coming from licence registry system: car model, color,\u2026\u200b
  • Data analysis leveraging Presto and Superset is done against stored data.

This demo is showcased in this video.

"},{"location":"demos/stable-diffusion/stable-diffusion/","title":"Text to Image using Stable Diffusion with DreamBooth","text":"

Stable Diffusion is a generative model that creates high-quality images by gradually denoising from random noise. DreamBooth fine-tuning customizes this model by training it on specific examples, allowing it to generate personalized images based on unique tokens and descriptive prompts.

Credit: DreamBooth

See the original DreamBooth project homepage for more details on what this fine-tuning method achieves.

"},{"location":"demos/stable-diffusion/stable-diffusion/#requirements","title":"Requirements","text":"
  • An OpenShift cluster with RHOAI or ODH deployed.
  • CSI driver capable of providing RWX volumes.
  • A node with a GPU card. The example has been tested with a AWS p3.8xlarge node.

Info

The full source and instructions for this demo are available on this repo and is based on this sample code.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/","title":"Telecom Customer Churn using Airflow and Red Hat OpenShift AI","text":"

Info

The full source and instructions for this demo are available in this repo

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#demo-description","title":"Demo description","text":"

The goal of this demo is to demonstrate how Red Hat OpenShift AI (RHOAI) and Airflow can be used together to build an easy-to-manage pipeline. To do that, we will show how to build and deploy an airflow pipeline, mainly with Elyra but also some tips if you want to build it manually. In the end, you will have a pipeline that:

  • Loads some data
  • Trains two different models
  • Evaluates which model is best
  • Saves that model to S3

Hint

You can expand on this demo by loading the pushed model into MLFlow, or automatically deploying it into some application, like in the Credit Card Fraud Demo

The models we build are used to predict customer churn for a Telecom company using structured data. The data contains fields such as: If they are a senior citizen, if they are a partner, their tenure, etc.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#deploying-the-demo","title":"Deploying the demo","text":""},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#pre-requisites","title":"Pre-requisites","text":"
  • Fork this git repository into a GitHub or GitLab repo (the demo shows steps for GitHub, but either works): https://github.com/red-hat-data-services/telecom-customer-churn-airflow
  • Have Airflow running in a cluster and point Airflow to the cloned git repository.
  • Have access to some S3 storage (this guide uses ODF with a bucket created in the namespace \"airflow\").
  • Have Red Hat OpenShift AI (RHOAI) running in a cluster. Make sure you have admin access in RHOAI, or know someone who does.

Note

Note: You can use Open Data Hub instead of RHOAI, but some instructions and screenshots may not apply

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#1-open-up-airflow","title":"1: Open up Airflow","text":"

You find the route to the Airflow console through this command: oc get route -n airflow

Enter it in the browser and you will see something like this:

Keep that open in a tab as we will come back to Airflow later on.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#2-add-elyra-as-a-custom-notebook-image","title":"2: Add Elyra as a Custom Notebook Image","text":"

It's possible to build pipelines by creating an Airflow DAG script in python. Another, arguably simpler, method is to use Elyra to visually build out the pipeline and then submit it to Airflow. Most of this demo is going to be revolving around using Elyra together with Airflow, but at the very end, there will be a bonus section for how to use Airflow independently.

To get access to Elyra, we will simply import it as a custom notebook image. Start by opening up RHOAI by clicking on the 9-square symbol in the top menu and choosing \"Red Hat OpenShift AI\".

Then go to Settings -> Notebook Images and press \"Import new image\". If you can't see Settings then you are lacking sufficient access. Ask your admin to add this image instead.

Under Repository enter: quay.io/eformat/elyra-base:0.2.1 and then name it something like Elyra.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#3-create-a-rhoai-workbench","title":"3: Create a RHOAI workbench","text":"

A workbench in RHOAI lets us spin up and down notebooks as needed and bundle them under Projects, which is a great way to get easy access to compute resources and keep track of your work. Start by creating a new Data Science project (see image). I'm calling my project 'Telecom Customer Churn', feel free to call yours something different but be aware that some things further down in the demo may change.

After the project has been created, create a workbench where we can run Jupyter. There are a few important settings here that we need to set:

  • Name: Customer Churn
  • Notebook Image: Elyra
  • Deployment Size: Small
  • Environment Variables: Secret -> AWS with your AWS details

Press Create Workbench and wait for it to start - status should say \"Running\" and you should be able to press the Open link.

Open the workbench and login if needed.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#4-load-a-git-repository","title":"4: Load a Git repository","text":"

When inside the workbench (Jupyter), we are going to clone a GitHub repository that contains everything we need to build our DAG. You can clone the GitHub repository by pressing the GitHub button in the left side menu (see image), then select \"Clone a Repository\" and enter your GitHub URL (Your forked version of this: https://github.com/red-hat-data-services/telecom-customer-churn-airflow)

The notebooks we will use are inside the include/notebooks folder, there should be 5 in total, 4 for building the pipeline and 1 for verifying that everything worked. They all run standard Python code, which is the beauty of Airflow combined with Elyra. There is no need to worry about additional syntax.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#5-configure-elyra-to-work-with-airflow","title":"5: Configure Elyra to work with Airflow","text":"

Before we can build and run any DAGs through Elyra, we first need to configure Elyra to talk with our Airflow instance. There will be two ways to configure this, either visually or through the terminal. Chose one for each section. If you want to do it through the terminal, then open the terminal like this:

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#51-create-a-runtime-image","title":"5.1 Create a Runtime Image","text":"

We will start by configuring a Runtime Image, this is the image we will use to run each node in our pipeline. Open Runtime Images on the left-hand side of the screen.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#511-create-the-runtime-image-visually","title":"5.1.1 Create the Runtime Image visually","text":"

Press the plus icon next to the Runtime Images title to start creating a new Runtime Image. There are only three fields we need to worry about here:

  • Display name: airflow-runner
  • Image Name: quay.io/eformat/airflow-runner:2.5.1
  • Image Pull Policy: Always

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#512-create-the-runtime-image-via-the-terminal","title":"5.1.2 Create the Runtime Image via the terminal","text":"

Execute this in the terminal:

mkdir -p ~/.local/share/jupyter/metadata/runtime-images/\ncat << EOF > ~/.local/share/jupyter/metadata/runtime-images/airflow-runner.json\n{\n  \"display_name\": \"airflow-runner\",\n  \"metadata\": {\n    \"tags\": [],\n    \"display_name\": \"airflow-runner\",\n    \"image_name\": \"quay.io/eformat/airflow-runner:2.5.1\",\n    \"pull_policy\": \"Always\"\n  },\n  \"schema_name\": \"runtime-image\"\n}\nEOF\n

Refresh and you should see airflow-runner appear in the Runtime Images.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#52-create-a-runtime","title":"5.2 Create a Runtime","text":"

Now we just need a Runtime configuration, which is what Elyra will use to save the DAG (in our Git repo), connect to Airflow and run the pipeline. Just like with the Runtime image, we can configure this visually or via the terminal.

Open Runtimes on the left-hand side of the screen.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#521-configure-the-runtime-visually","title":"5.2.1 Configure the Runtime visually","text":"

Press the plus icon next to the title, select \"New Apache Airflow runtime configuration\" and enter these fields:

General settings:

  • Display Name: airflow

Airflow settings:

  • Apache Airflow UI Endpoint: run oc get route -n airflow to get the route
  • Apache Airflow User Namespace: airflow

Github/GitLabs settings:

  • Git type: GITHUB or GITLAB, depending on where you stored the repository
  • GitHub or GitLab server API Endpoint: https://api.github.com or your GitLab endpoint
  • GitHub or GitLab DAG Repository: Your repository (red-hat-data-services/telecom-customer-churn-airflow in my case)
  • GitHub or GitLab DAG Repository Branch: Your branch (main in my case)
  • Personal Access Token: A personal access token for pushing to the repository

Cloud Object Storage settings: These completely depend on where and how you set up your S3 storage. If you created a bucket from ODF then it will look similar to this:

  • Cloud Object Storage Endpoint: http://s3.openshift-storage.svc
  • Cloud Object Storage Bucket Name: The name of your bucket (airflow-storage-729b10d1-f44d-451d-badb-fbd140418763 in my case)
  • Cloud Object Storage Authentication Type: KUBERNETES_SECRET
  • Cloud Object Storage Credentials Secret: The name of your secret containing the access and secret key is (in my case it was airflow-storage, which is the name I gave the Object Bucket Claim)
  • Cloud Object Storage Username: your AWS_ACCESS_KEY_ID
  • Cloud Object Storage Password: your AWS_SECRET_ACCESS_KEY
"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#522-configure-the-runtime-via-the-terminal","title":"5.2.2 Configure the Runtime via the terminal","text":"

If you prefer doing this through the terminal, then execute this in the terminal and replace any variables with their values (see the visual section for hints):

mkdir -p ~/.local/share/jupyter/metadata/runtimes\ncat << EOF >  ~/.local/share/jupyter/metadata/runtimes/airflow.json\n{\n  \"display_name\": \"airflow\",\n  \"metadata\": {\n     \"tags\": [],\n     \"display_name\": \"airflow\",\n     \"user_namespace\": \"airflow\",\n     \"git_type\": \"GITHUB\",\n     \"github_api_endpoint\": \"https://${GIT_SERVER}\",\n     \"api_endpoint\": \"${AIRFLOW_ROUTE}\",\n     \"github_repo\": \"${GIT_REPO}\",\n     \"github_branch\": \"main\",\n     \"github_repo_token\": \"${GIT_TOKEN}\",\n     \"cos_auth_type\": \"KUBERNETES_SECRET\",\n     \"cos_endpoint\": \"${STORAGE_ENDPOINT}\",\n     \"cos_bucket\": \"${STORAGE_BUCKET}\",\n     \"cos_secret\": \"airflow-storage\" - the name of your secret,\n     \"cos_username\": \"${AWS_ACCESS_KEY_ID}\",\n     \"cos_password\": \"${AWS_SECRET_ACCESS_KEY}\",\n     \"runtime_type\": \"APACHE_AIRFLOW\"\n  },\n  \"schema_name\": \"airflow\"\n}\nEOF\n

Refresh and you should see airflow appear in the Runtimes.

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#6-create-a-dag-with-elyra","title":"6. Create a DAG with Elyra","text":"

Now that we have a runtime and runtime image defined, we can build and run the pipeline. You can also find this pipeline in /dags/train_and_compare_models.pipeline if you prefer to just open an existing one.

To start creating a new pipeline, open up the launcher (click on the plus next to a notebook tab if you don't have it open), and press the \"Apache Airflow Pipeline Editor\".

Now drag the Notebooks in the correct order and connect them up with each other. You can find the Notebooks in /included/notebooks and the correct order is: process_data -> model_gradient_boost & model_randomforest -> compare_and_push. These are their functions:

  • process_data.ipynb: Downloads data from GitHub that we will use to train the models. Then processes it, splits it into training and testing partitions and finally pushes it to S3.
  • model_gradient_boost.ipynb: Fetches the processed data from S3 and uses it to train the model and evaluate it to get a test accuracy. Then pushes the model and the accompanying accuracy to S3.
  • model_randomforest.ipynb: Fetches the processed data from S3 and uses it to train the model and evaluate it to get a test accuracy. Then pushes the model and the accompanying accuracy to S3.
  • compare_and_push.ipynb: Downloads the models and their accuracies from S3, does a simple compare on which performs better, and pushes that model under the name \"best_model\" to S3.

After the notebooks are added, we need to go through each of them and change their Runtime Images to airflow-runner that we created earlier.

We also need to set some environment variables so that the airflow nodes get access to the bucket name and endpoint when running, without hard-coding it in the notebooks. These details are already added to the Airflow Runtime we set up before, but when running it only passes along the Kubernetes secret which contains AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Add these two environment variables (both should be the same as you entered in section 5.2):

  • Endpoint:
    • Name: AWS_S3_ENDPOINT
    • Value: http://s3.openshift-storage.svc (or similar endpoint address)
  • Bucket name:
    • Name: AWS_S3_BUCKET
    • Value: The name of your bucket (airflow-storage-729b10d1-f44d-451d-badb-fbd140418763 in my case)

Press Run to start the pipeline:

You can now go to the Airflow UI to see the progress. If you have closed the tab then refer to section 1.

In Airflow you will see a dag called train_and_compare_models with some numbers behind it. Click on it and go open the Graph tab.

If all are dark green that means that the run has completed successfully.

We can now also confirm that the trained model was saved in our bucket by going back to the RHOAI notebook and running the notebook test_airflow_success.ipynb. If all went well it should print the model, its type and its accuracy.

And that's how you can use Airflow together with RHOAI to create a pipeline!

"},{"location":"demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/#bonus-section-use-an-airflow-dag-file","title":"Bonus section: Use an Airflow DAG file","text":"

Instead of building a pipeline through notebooks in Elyra, we can of course build and use an Airflow DAG. You can develop individual methods (data processing, mode training, etc) in RHOAI notebooks and then pull them all together in a DAG python file. This is a more segmented way for a Data Scientist to work than with Elyra, but still very possible within OpenShift and provides some more flexibility.

I have created a simple test_dag.py just to show what it can look like. You can find it in the /dags folder. Then it's up to you what operators you want to run, which secrets you want to load, etc. For inspiration, you can open up the automatically created Elyra DAG we just ran. To do that, go into the DAG and press Code:

Some notes if you wish to manually build a similar DAG:

  • Make sure to add the environment variables
  • Don't hardcode secrets into the DAG, but rather reference a Kubernetes secret. For example:
secrets=[\n        Secret(\"env\", \"AWS_ACCESS_KEY_ID\", \"airflow-storage\", \"AWS_ACCESS_KEY_ID\"),\n        Secret(\n            \"env\", \"AWS_SECRET_ACCESS_KEY\", \"airflow-storage\", \"AWS_SECRET_ACCESS_KEY\"\n        ),\n    ]\n
  • The image that is being used for the KubernetesPodOperator is quay.io/eformat/airflow-runner:2.5.1
  • If you want to run notebooks manually, look at the Papermill Operator
"},{"location":"demos/water-pump-failure-prediction/water-pump-failure-prediction/","title":"Water Pump Failure Prediction","text":"

Info

The full source for this demo is available in this repo. Look in the workshop folder for the full instructions.

This demo shows how to do detection of anomalies in sensor data. This web app allows you to broadcast various sources of data in real time.

"},{"location":"demos/xray-pipeline/xray-pipeline/","title":"XRay Analysis Automated Pipeline","text":"

Info

The full source and instructions for this demo are available in this repo

In this demo, we implement an automated data pipeline for chest Xray analysis:

  • Ingest chest Xrays into an object store based on Ceph.
  • The Object store sends notifications to a Kafka topic.
  • A KNative Eventing Listener to the topic triggers a KNative Serving function.
  • An ML-trained model running in a container makes a risk of Pneumonia assessment for incoming images.
  • A Grafana dashboard displays the pipeline in real time, along with images incoming, processed and anonymized, as well as full metrics.

This pipeline is showcased in this video (slides are also here).

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/","title":"Model Training and Serving - YOLOv5","text":"

Info

The full source and instructions for this demo are available in these repos:

  • Model Training
  • Model Serving

In this tutorial, we're going to see how you can customize YOLOv5, an object detection model, to recognize specific objects in pictures, and how to deploy and use this model.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#yolo-and-yolov5","title":"YOLO and YOLOv5","text":"

YOLO (You Only Look Once) is a popular object detection and image segmentation model developed by Joseph Redmon and Ali Farhadi at the University of Washington. The first version of YOLO was released in 2015 and quickly gained popularity due to its high speed and accuracy.

YOLOv2 was released in 2016 and improved upon the original model by incorporating batch normalization, anchor boxes, and dimension clusters. YOLOv3 was released in 2018 and further improved the model's performance by using a more efficient backbone network, adding a feature pyramid, and making use of focal loss.

In 2020, YOLOv4 was released which introduced a number of innovations such as the use of Mosaic data augmentation, a new anchor-free detection head, and a new loss function.

In 2021, Ultralytics released YOLOv5, which further improved the model's performance and added new features such as support for panoptic segmentation and object tracking.

YOLO has been widely used in a variety of applications, including autonomous vehicles, security and surveillance, and medical imaging. It has also been used to win several competitions, such as the COCO Object Detection Challenge and the DOTA Object Detection Challenge.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#model-training","title":"Model training","text":"

YOLOv5 has already been trained to recognize some objects. Here we are going to use a technique called Transfer Learning to adjust YOLOv5 to recognize a custom set of images.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#transfer-learning","title":"Transfer Learning","text":"

Transfer learning is a machine learning technique in which a model trained on one task is repurposed or adapted to another related task. Instead of training a new model from scratch, transfer learning allows the use of a pre-trained model as a starting point, which can significantly reduce the amount of data and computing resources needed for training.

The idea behind transfer learning is that the knowledge gained by a model while solving one task can be applied to a new task, provided that the two tasks are similar in some way. By leveraging pre-trained models, transfer learning has become a powerful tool for solving a wide range of problems in various domains, including natural language processing, computer vision, and speech recognition.

Ultralytics have fully integrated the transfer learning process in YOLOv5, making it easy for us to do. Let's go!

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#environment-and-prerequisites","title":"Environment and prerequisites","text":"
  • This training should be done in a Data Science Project to be able to modify the Workbench configuration (see below the /dev/shm issue).
  • YOLOv5 is using PyTorch, so in RHOAI it's better to start with a notebook image already including this library, rather than having to install it afterwards.
  • PyTorch is internally using shared memory (/dev/shm) to exchange data between its internal worker processes. However, default container engine configurations limit this memory to the bare minimum, which can make the process exhaust this memory and crash. The solution is to manually increase this memory by mounting a specific volume with enough space at this emplacement. This problem will be fixed in an upcoming version. Meanwhile you can use this procedure.
  • Finally, a GPU is strongly recommended for this type of training.
"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#data-preparation","title":"Data Preparation","text":"

To train the model we will of course need some data. In this case a sufficient number of images for the various classes we want to recognize, along with their labels and the definitions of the bounding boxes for the object we want to detect.

In this example we will use images from Google's Open Images. We will work with 3 classes: Bicycle, Car and Traffic sign.

We have selected only a few classes in this example to speed up the process, but of course feel free to adapt and choose the ones you want.

For this first step:

  • If not already done, create your Data Science Project,
  • Create a Workbench of type PyTorch, with at least 8Gi of memory, 1 GPU and 20GB of storage.
  • Apply this procedure to increase shared memory.
  • Start the workbench.
  • Clone the repository https://github.com/rh-aiservices-bu/yolov5-transfer-learning, open the notebook 01-data_preparation.ipynb and follow the instructions.

Once you have completed to whole notebook the Dataset is ready for training!

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#training","title":"Training","text":"

In this example, we will do the training with the smallest base model available to save some time. Of course you can change this base model and adapt the various hyperparameters of the training to improve the result.

For this second step, from the same workbench environment, open the notebook 02-model_training.ipynb and follow the instructions.

Warning

The amount of memory you have assigned to your Workbench has a great impact on the batch size you will be able to work with, independently of the size of your GPU. For example, a batch size of 128 will barely fit into an 8Gi of memory Pod. The higher the better, until it breaks... Which you will find out soon anyway, after the first 1-2 epochs.

Note

During the training, you can launch and access Tensorboard by:

  • Opening a Terminal tab in Jupyter
  • Launch Tensorboard from this terminal with tensorboard --logdir yolov5/runs/train
  • Access Tensorboard in your browser using the same Route as your notebook, but replacing the .../lab/... part by .../proxy/6006/. Example: https://yolov5-yolo.apps.cluster-address/notebook/yolo/yolov5/proxy/6006/

Once you have completed to whole notebook you have a model that is able to recognize the three different classes on a given image.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#model-serving","title":"Model Serving","text":"

We are going to serve a YOLOv5 model using the ONNX format, a general purpose open format built to represent machine learning models. RHOAI Model Serving includes the OpenVino serving runtime that accepts two formats for models: OpenVino IR, its own format, and ONNX.

Note

Many files and code we are going to use, especially the ones from the utils and models folders, come directly from the YOLOv5 repository. They includes many utilities and functions needed for image pre-processing and post-processing. We kept only what is needed, rearranged in a way easier to follow within notebooks. YOLOv5 includes many different tools and CLI commands that are worth learning, so don't hesitate to have a look at it directly.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#environment-and-prerequisites_1","title":"Environment and prerequisites","text":"
  • YOLOv5 is using PyTorch, so in RHOAI it's better to start with a notebook image already including this library, rather than having to install it afterwards.
  • Although not necessary as in this example we won't use the model we trained in the previous section, the same environment can totally be reused.
"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#converting-a-yolov5-model-to-onnx","title":"Converting a YOLOv5 model to ONNX","text":"

YOLOv5 is based on PyTorch. So base YOLOv5 models, or the ones you retrain using this framework, will come in the form of a model.pt file. We will first need to convert it to the ONNX format.

  • From your workbench, clone the repository https://github.com/rh-aiservices-bu/yolov5-model-serving.
  • Open the notebook 01-yolov5_to_onnx.ipynb and follow the instructions.
  • The notebook will guide you through all the steps for the conversion. If you don't want to do it at this time, you can also find in this repo the original YOLOv5 \"nano\" model, yolov5n.pt, and its already converted ONNX version, yolov5n.onnx.

Once converted, you can save/upload your ONNX model to the storage you will use in your Data Connection on RHOAI. At the moment it has to be an S3-Compatible Object Storage, and the model must be in it own folder (not at the root of the bucket).

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#serving-the-model","title":"Serving the model","text":"

Here we can use the standard configuration path for RHOAI Model Serving:

  • Create a Data Connection to the storage where you saved your model. In this example we don't need to expose an external Route, but of course you can. In this case though, you won't be able to directly see the internal gRPC and REST endpoints in the RHOAI UI, you will have to get them from the Network->Services panel in the OpenShift Console.
  • Create a Model Server, then deploy the model using the ONNX format.

Note

You can find full detailed versions of this procedure in this Learning Path or in the RHOAI documentation.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#grpc-connection","title":"gRPC connection","text":"

With the gRPC interface of the model server, you have access to different Services. They are described, along with their format, in the grpc_predict_v2.proto file.

There are lots of important information in this file: how to query the service, how to format the data,... This is really important as the data format is not something you can \"invent\", and not exactly the same compared as the REST interface (!).

This proto file, which is a service description meant to be used with any programming language, has already been converted to usable Python modules defining objects and classes to be used to interact with the service: grpc_predict_v2_pb2.py and grpc_predict_v2_pb2_grpc.py. If you want to learn more about this, the conversion can be done using the protoc tool.

You can use the notebook 02-grpc.ipynb to connect to the interface and test some of the services. You will see that many \"possible\" services from ModelMesh are unfortunately simply not implemented with the OpenVino backend at the time of this writing. But at least ModelMetadata will give some information on the formats we have to use for inputs and outputs when doing the inference.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#consuming-the-model-over-grpc","title":"Consuming the model over gRPC","text":"

In the 03-remote_inference_grpc.ipynb notebook, you will find a full example on how to query the grpc endpoint to make an inference. It is backed by the file remote_infer_grpc.py, where most of the relevant code is:

  • Image preprocessing on L35: reads the image and transforms it in a proper numpy array
  • gRPC request content building on L44: transforms the array in the expected input shape (refer to model metadata obtained in the previous notebook), then flatten it as expected by ModelMesh.
  • gRPC calling on L58.
  • Response processing on L73: reshape the response from flat array to expected output shape (refer to model metadata obtained in the previous notebook), run NMS to remove overlapping boxes, draw the boxes from results.

The notebook gives the example for one image, as well as the processing of several ones from the images folder. This allows for a small benchmark on processing/inference time.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#consuming-the-model-over-rest","title":"Consuming the model over REST","text":"

In the 04-remote_inference_rest.ipynb notebook, you will find a full example on how to query the gRPC endpoint to make an inference. It is backed by the file remote_infer_rest.py, where most of the relevant code is:

  • Image preprocessing on L30: reads the image and transforms it in a proper numpy array
  • Payload building on L39: transforms the array in the expected input shape (refer to model metadata obtained in the previous notebook).
  • REST calling on L54.
  • Response processing on L60: reshape the response from flat array to expected output shape (refer to model metadata obtained in the previous notebook), run NMS to remove overlapping boxes, draw the boxes from results.

The notebook gives the example for one image, as well as the processing of several ones from the images folder. This allows for a small benchmark on processing/inference time.

"},{"location":"demos/yolov5-training-serving/yolov5-training-serving/#grpc-vs-rest","title":"gRPC vs REST","text":"

Here are a few elements to help you choose between the two available interfaces to query your model:

  • REST is easier to implement: it is a much better known protocol for most people, and involves a little bit less programming. There is no need to create a connection, instantiate objects,... So it's often easier to use.
  • If you want to query the model directly from outside OpenShift, you have to use REST which is the only one exposed. You can expose gRPC too, but it's kind of difficult right now.
  • gRPC is wwwwwaaaayyyyy much faster than REST. With the exact same model serving instance, as showed in the notebooks, inferences are about 30x faster. That is huge when you have score of images to process.
"},{"location":"generative-ai/llm-serving/","title":"LLM Serving","text":"

Info

All source files and examples used in this article are available on this repo!

LLMs (Large Language Models) are the subject of the day. And of course, you can definitely work with them on OpenShift with ODH or RHOAI, from creating a Chatbot, using them as simple APIs to summarize or translate texts, to deploying a full application that will allow you to quickly query your documentation or knowledge base in natural language.

You will find on this page instructions and examples on how to set up the different elements that are needed for those different use cases, as well as fully implemented and ready-to-use applications.

"},{"location":"generative-ai/llm-serving/#context-and-definitions","title":"Context and definitions","text":"

Many people are only beginning to discover those technologies. After all, it has been a little bit more than a year since the general public is aware of them, and many related technologies, tools or applications are only a few months, even weeks (and sometimes days!) old. So let's start with a few definitions of the different terms that will be used in this article.

  • LLM: A Large Language Model (LLM) is a sophisticated artificial intelligence system designed for natural language processing. It leverages deep learning techniques to understand and generate human-like text. LLMs use vast datasets to learn language patterns, enabling tasks like text generation, translation, summarization, and more. These models are versatile and can be fine-tuned for specific applications, like chatbots or content creation. LLMs have wide-ranging potential in various industries, from customer support and content generation to research and education, but their use also raises concerns about ethics, bias, and data privacy, necessitating responsible deployment and ongoing research.
  • Fine-tuning: Fine-tuning in the context of Large Language Models (LLMs) is a process of adapting a pre-trained, general-purpose model to perform specific tasks or cater to particular applications. It involves training the model on a narrower dataset related to the desired task, allowing it to specialize and improve performance. Fine-tuning customizes the LLM's capabilities for tasks like sentiment analysis, question answering, or chatbots. This process involves adjusting hyperparameters, data preprocessing, and possibly modifying the model architecture. Fine-tuning enables LLMs to be more effective and efficient in specific domains, extending their utility across various applications while preserving their initial language understanding capabilities.
  • RAG: RAG, or Retrieval-Augmented Generation, is a framework in natural language processing. It combines two key components: retrieval and generation. Retrieval involves selecting relevant information from a vast knowledge base, like the internet, and generation pertains to creating human-like text. RAG models employ a retriever to fetch context and facts related to a specific query or topic and a generator, often a language model, to produce coherent responses. This approach enhances the quality and relevance of generated text, making it useful for tasks like question answering, content summarization, and information synthesis, offering a versatile solution for leveraging external knowledge in AI-powered language understanding and production.
  • Embeddings: Embeddings refer to a technique in natural language processing and machine learning where words, phrases, or entities are represented as multi-dimensional vectors in a continuous vector space. These vectors capture semantic relationships and similarities between words based on their context and usage. Embeddings are created through unsupervised learning, often using models like Word2Vec or GloVe, which transform words into fixed-length numerical representations. These representations enable machines to better understand and process language, as similar words have closer vector representations, allowing algorithms to learn contextual associations. Embeddings are foundational in tasks like text classification, sentiment analysis, machine translation, and recommendation systems.
  • Vector Database: A vector database is a type of database designed to efficiently store and manage vector data, which represents information as multidimensional arrays or vectors. Unlike traditional relational databases, which organize data in structured tables, vector databases excel at handling unstructured or semi-structured data. They are well-suited for applications in data science, machine learning, and spatial data analysis, as they enable efficient storage, retrieval, and manipulation of high-dimensional data points. Vector databases play a crucial role in various fields, such as recommendation systems, image processing, natural language processing, and geospatial analysis, by facilitating complex mathematical operations on vector data for insights and decision-making.
  • Quantization: Model quantization is a technique in machine learning and deep learning aimed at reducing the computational and memory requirements of neural networks. It involves converting high-precision model parameters (usually 32-bit floating-point values) into lower precision formats (typically 8-bit integers or even binary values). This process helps in compressing the model, making it more lightweight and faster to execute on hardware with limited resources, such as edge devices or mobile phones. Quantization can result in some loss of model accuracy, but it's a trade-off that balances efficiency with performance, enabling the deployment of deep learning models in resource-constrained environments without significant sacrifices in functionality.

Fun fact: all those definitions were generated by an LLM...

Do you want to know more?

Here are a few worth reading articles:

  • Best article ever: A jargon-free explanation of how AI large language models work
  • Understanding LLama2 and its architecture
  • RAG vs Fine-Tuning, which is best?
"},{"location":"generative-ai/llm-serving/#llm-serving_1","title":"LLM Serving","text":"

LLM Serving is not a trivial task, at least in a production environment...

  • LLMs are usually huge (several GBs, tens of GBs...) and require GPU(s) with enough memory if you want decent accuracy and performance. Granted, you can run smaller models on home hardware with good results, but that's not the subject here. After all we are on OpenShift, so more in a large organization environment than in an enthusiastic programmer basement!
  • A served LLM will generally be used by multiple applications and users simultaneously. Since you can't just throw resources at it and scale your infrastructure easily because of the previous point, you want to optimize response time by for example batching queries, caching or buffering them,... Those are special operations that have to be handled specifically.
  • When you load an LLM, there are parameters you want to tweak at load time, so a \"generic\" model loader is not the best suited solution.
"},{"location":"generative-ai/llm-serving/#llm-serving-solutions","title":"LLM Serving solutions","text":"

Fortunately, we have different solutions to handle LLM Serving.

On this repo you will find:

  • Recipes to deploy different types of LLM Servers, either using the Single Stack Model Serving available in ODH and RHOAI, or as Standalone deployments.
  • Notebook examples on how to query those different servers are available.
"},{"location":"generative-ai/llm-serving/#serving-runtimes-for-single-stack-model-serving","title":"Serving Runtimes for Single Stack Model Serving","text":"

On top of the Caikit+TGIS or TGIS built-in runtimes, the following custom runtimes can be imported in the Single-Model Serving stack of Open Data Hub or OpenShift AI.

  • vLLM Serving Runtime
  • Hugging Face Text Generation Inference
"},{"location":"generative-ai/llm-serving/#standalone-inference-servers","title":"Standalone Inference Servers","text":"
  • vLLM: how to deploy vLLM, the \"Easy, fast, and cheap LLM serving for everyone\".
    • on GPU:
    • on CPU:
  • Hugging Face TGI: how to deploy the Text Generation Inference server from Hugging Face.
  • Caikit-TGIS-Serving: how to deploy the Caikit-TGIS-Serving stack, from OpenDataHub.
"},{"location":"generative-ai/llm-serving/#what-are-the-differences","title":"What are the differences?","text":"

You should read the documentation of those different model servers, as they present different characteristics:

  • Different acceleration features can be implemented from one solution to the other. Depending on the model you choose, some features may be required.
  • The endpoints can be accessed in REST or gRPC mode, or both, depending on the server.
  • APIs are different, with some solutions offering an OpenAI compatible, which simplifies the integration with some libraries like Langchain.
"},{"location":"generative-ai/llm-serving/#which-model-to-use","title":"Which model to use?","text":"

In this section we will assume that you want to work with a \"local\" open source model, and not consume a commercial one through an API, like OpenAI's ChatGPT or Anthropic's Claude.

There are literally hundreds of thousands of models available, almost all of them available on the Hugging Face site. If you don't know what this site is, you can think of it as what Quay or DockerHub are for containers: a big repository of models and datasets ready to download and use. Of course Hugging Face (the company) is also creating code, providing hosting capabilities,... but that's another story.

So which model to choose will depend on several factors:

  • Of course how good this model is. There are several benchmarks that have been published, as well as constantly updated rankings.
  • The dataset it was trained on. Was it curated or just raw data from anywhere, does it contain nsfw material,...? And of course what the license is (some datasets are provided for research only or non-commercial).
  • The license of the model itself. Some are fully open source, some claim to be... They may be free to use in most cases, but have some restrictions attached to them (looking at you Llama2...).
  • The size of the model. Unfortunately that may be the most restrictive point for your choice. The model simply must fit on the hardware you have at your disposal, or the amount of money you are willing to pay.

Currently, a good LLM with interesting performance for a relatively small size is Mistral-7B. Fully Open Source with an Apache 2.0 license, it will fit in an unquantized version on about 22GB of VRAM, which is perfect for an A10G card.

Embeddings are another type of model often associated with LLMs as they are used to convert documents into vectors. A database of those vectors can then be queried to find the documents related to a query you make. This is very useful for the RAG solution we are going to talk about later on. NomicAI recently released a really performant and fully open source embeddings model, nomic-embed-text-v1

"},{"location":"generative-ai/llm-serving/#llm-consumption","title":"LLM Consumption","text":"

Once served, consuming an LLM is pretty straightforward, as at the end of the day it's only an API call.

You can always query those models directly through a curl command or a simple request using Python. However, for easier consumption and integration with other tools, a few libraries/SDKs are available to streamline the process. They will allow you to easily connect to Vector Databases or Search Agents, chain multiple models, tweak parameters,... in a few lines of code. The main libraries at the time of this writing are Langchain, LlamaIndex and Haystack.

In the LLM on OpenShift examples section repo, you will find several notebooks and full UI examples that will show you how to use those libraries with different types of model servers to create your own Chatbot!

"},{"location":"generative-ai/what-is-generative-ai/","title":"What is Generative AI?","text":"

Generative AI generally has the following characteristics:

  • A model generally created based on a vast and broad set of data.
  • Aims to generate novel content or data
  • by mimicking patterns found in the training data
  • e.g. \"compose an email to let the customer know that their loan application has been denied\"
"},{"location":"getting-started/opendatahub/","title":"What is Open Data Hub?","text":"

Open Data Hub (ODH) is an open source project that provides open source AI tools for running large and distributed AI workloads on the OpenShift Container Platform. Currently, the Open Data Hub project provides open source tools for distributed AI and Machine Learning (ML) workflows, Jupyter Notebook development environment and monitoring. The Open Data Hub project roadmap offers a view on new tools and integration the project developers are planning to add.

Included in the Open Data Hub core deployment is several open source components, which can be individually enabled. They include:

  • Jupyter Notebooks
  • ODH Dashboard
  • Data Science Pipelines
  • Model Mesh Serving

Want to know more?

"},{"location":"getting-started/openshift-ai/","title":"OpenShift AI","text":""},{"location":"getting-started/openshift-ai/#what-is-red-hat-openshift-ai","title":"What is Red Hat OpenShift AI?","text":"

Red Hat OpenShift AI (RHOAI) builds on the capabilities of Red Hat OpenShift to provide a single, consistent, enterprise-ready hybrid AI and MLOps platform. It provides tools across the full lifecycle of AI/ML experiments and models including training, serving, monitoring, and managing AI/ML models and AI-enabled applications.

Note

Red Hat OpenShift AI was previously named Red Hat OpenShift Data Science. Some texts or screenshots may not yet reflect this change.

Documentation for Self-Managed RHOAI

Documentation for Managed RHOAI

Want to know more?

"},{"location":"getting-started/openshift/","title":"OpenShift and AI","text":""},{"location":"getting-started/openshift/#what-is-red-hat-openshift","title":"What is Red Hat OpenShift?","text":"

Red Hat OpenShift brings together tested and trusted services to reduce the friction of developing, modernizing, deploying, running, and managing applications. Built on Kubernetes, it delivers a consistent experience across public cloud, on-premise, hybrid cloud, or edge architecture. Choose a self-managed or fully managed solution. No matter how you run it, OpenShift helps teams focus on the work that matters.

Want to know more?

"},{"location":"getting-started/openshift/#why-ai-on-openshift","title":"Why AI on OpenShift?","text":"

AI/ML on OpenShift accelerates AI/ML workflows and the delivery of AI-powered intelligent application.

"},{"location":"getting-started/openshift/#mlops-with-red-hat-openshift","title":"MLOps with Red Hat OpenShift","text":"

Red Hat OpenShift includes key capabilities to enable machine learning operations (MLOps) in a consistent way across datacenters, public cloud computing, and edge computing.

By applying DevOps and GitOps principles, organizations automate and simplify the iterative process of integrating ML models into software development processes, production rollout, monitoring, retraining, and redeployment for continued prediction accuracy.

Learn more

"},{"location":"getting-started/openshift/#what-is-a-ml-lifecycle","title":"What is a ML lifecycle?","text":"

A multi-phase process to obtain the power of large volumes and a variety of data, abundant compute, and open source machine learning tools to build intelligent applications.

At a high level, there are four steps in the lifecycle:

  1. Gather and prepare data to make sure the input data is complete, and of high quality
  2. Develop model, including training, testing, and selection of the model with the highest prediction accuracy
  3. Integrate models in application development process, and inferencing
  4. Model monitoring and management, to measure business performance and address potential production data drift

On this site, you will find recipes, patterns, demos for various AI/ML tools and applications used through those steps.

"},{"location":"getting-started/openshift/#why-use-containers-and-kubernetes-for-your-machine-learning-initiatives","title":"Why use containers and Kubernetes for your machine learning initiatives?","text":"

Containers and Kubernetes are key to accelerating the ML lifecycle as these technologies provide data scientists the much needed agility, flexibility, portability, and scalability to train, test, and deploy ML models.

Red Hat\u00ae OpenShift\u00ae is the industry's leading containers and Kubernetes hybrid cloud platform. It provides all these benefits, and through the integrated DevOps capabilities (e.g. OpenShift Pipelines, OpenShift GitOps, and Red Hat Quay) and integration with hardware accelerators, it enables better collaboration between data scientists and software developers, and accelerates the roll out of intelligent applications across hybrid cloud (data center, edge, and public clouds).

"},{"location":"getting-started/why-this-site/","title":"Why this site?","text":"

As data scientists and engineers, it's easy to find detailed documentation on the tools and libraries we use. But what about end-to-end data pipeline solutions that involve multiple products? Unfortunately, those resources can be harder to come by. Open source communities often don't have the resources to create and maintain them. But don't worry, that's where this website comes in!

We've created a one-stop-shop for data practitioners to find recipes, reusable patterns, and actionable demos for building AI/ML solutions on OpenShift. And the best part? It's a community-driven resource site! So, feel free to ask questions, make feature requests, file issues, and even submit PRs to help us improve the content. Together, we can make data pipeline solutions easier to find and implement.

"},{"location":"odh-rhoai/accelerator-profiles/","title":"Accelerator Profiles","text":""},{"location":"odh-rhoai/accelerator-profiles/#accelerator-profiles","title":"Accelerator Profiles","text":"

To effectively use accelerators in OpenShift AI, OpenShift AI Administrators need to create and manage associated accelerator profiles.

An accelerator profile is a custom resource definition (CRD) that defines specifications for this accelerator. It can be direclty managed via the OpenShift AI Dashboard under Settings \u2192 Accelerator profiles.

When working with GPU nodes in OpenShift AI, it is essential to set proper taints on those nodes. This prevents unwanted workloads from being scheduled on them when they don't have specific tolerations set. Those tolerations are configured in the accelerator profiles associated with each type of GPU, then applied to the workloads (Workbenches, Model servers,...) for which you have selected an accelerator profile.

The taints in the GPU Worker Nodes should be set like this:

  taints:\n    - effect: NoSchedule\n      key: nvidia.com/gpu\n      value: NVIDIA-A10G-SHARED\n

A corresponding Accelerator profile can then be created to allow workloads to run on this type of node (in this example, nodes having an A10G GPU). Workloads that use another accelerator profile (for another type of GPU for example) or that don't have any Accelerator profile set will not be scheduled on nodes tainted with NVIDIA-A10G-SHARED.

For a detailed guide on configuring and managing accelerator profiles in OpenShift AI, refer to our repository.

"},{"location":"odh-rhoai/configuration/","title":"ODH and RHOAI Configuration","text":""},{"location":"odh-rhoai/configuration/#standard-configuration","title":"Standard configuration","text":"

As an administrator of ODH/RHOAI, you have access to different settings through the Settings menu on the dashboard:

"},{"location":"odh-rhoai/configuration/#custom-notebook-images","title":"Custom notebook images","text":"

This is where you can import other notebook images. You will find resources on available custom images and learn how to create your own in the Custom Notebooks section.

To import a new image, follow those steps.

  • Click on import image.

  • Enter the full address of your container, set a name (this is what will appear in the launcher), and a description.

  • On the bottom part, add information regarding the software and the packages that are present in this image. This is purely informative.

  • Your image is now listed and enabled. You can hide it without removing it by simply disabling it.

  • It is now available in the launcher, as well as in the Data Science Projects.

"},{"location":"odh-rhoai/configuration/#cluster-settings","title":"Cluster settings","text":"

In this panel, you can adjust:

  • The default size of the volumes created for new users.
  • Whether you want to stop idle notebooks and, if so, after how much time.

Note

This feature currently looks at running Jupyter kernels, like a Python notebook. If you are only using a Terminal, or another IDE window like VSCode or RStudio from the custom images, this activity is not detected and your Pod can be stopped without notice after the set delay.

  • Whether you allow usage data to be collected and reported.
  • Whether you want to add a toleration to the notebook pods to allow them to be scheduled on tainted nodes. That feature is really useful if you want to dedicate specific worker nodes to running notebooks. Tainting them will prevent other workloads from running on them. Of course, you have to add the toleration to the pods.

"},{"location":"odh-rhoai/configuration/#user-management","title":"User management","text":"

In this panel, you can edit who has access to RHOAI by defining the \"Data Science user groups\", and who has access to the Settings by defining the \"Data Science administrator groups\".

"},{"location":"odh-rhoai/configuration/#advanced-configuration","title":"Advanced configuration","text":""},{"location":"odh-rhoai/configuration/#dashboard-configuration","title":"Dashboard configuration","text":"

RHOAI or ODH main configuration is done through a Custom Resource (CR) of type odhdashboardconfigs.opendatahub.io.

  • To get access to it, from your OpenShift console, navigate to Home->API Explorer, and filter for OdhDashboardConfig:

  • Click on OdhDashboardConfig and in the Instances tab, click on odh-dashboard-config:

  • You can now view and edit the YAML file to modify the configuration:

In the spec section, the following items are of interest:

  • dashboardConfig: The different toggles will allow you to activate/deactivate certain features. For example, you may want to hide Model Serving for your users or prevent them from importing custom images.
  • notebookSizes: This is where you can fully customize the sizes of the notebooks. You can modify the resources and add or remove sizes from the default configuration as needed.
  • modelServerSizes: This setting operates on the same concept as the previous setting but for model servers.
  • notebookController: In this section you will find various settings related to the Workbenches and how they are launched.
  • If your GPUs are not correctly detected, the dropdown allowing you to select how many GPUs you want to use for a workbench will not be displayed. To force it, you can create/modify the parameter gpuSetting under notebookController. This will force the dropdown to appear, with the maximum being the number you set for the parameter. Example:
notebookController:\n    enabled: true\n    gpuSetting: '4'\n    ...\n
"},{"location":"odh-rhoai/configuration/#adding-a-custom-application","title":"Adding a custom application","text":"

Let's say you have installed another application in your cluster and want to make it available through the dashboard. That's easy! A tile is, in fact, represented by a custom resource (CR) of type OdhApplication.

In this example, we will add a tile to access the MLFlow UI (see the MLFlow installation instructions to test it).

  • The file mlflow-tile.yaml provides you with an example of how to create the tile.
  • Edit this file to set the route (the name of the Route CR) and routeNamespace parameters to where the UI is accessible. In this example, it is mlflow-server(route name) and mlflow (server). Apply this file to create the resource.
  • Wait 1-2 minutes for the change to take effect. Your tile is now available in the Explore view (bottom left):

  • However, it is not yet enabled. To enable this tile, click on it in the Explorer view, then click the \"Enable\" button at the top of the description. You can also create a ConfigMap from the file cm-mlflow-enable.yaml.
  • Wait another 1-2 minutes, and your tile is now ready to use in the Enabled view:

"},{"location":"odh-rhoai/connect-vscode-to-rhoai-wb/","title":"Connect to RHOAI Workbench Kernel from local VS Code","text":"

Some users have expressed their desire to work directly on their local IDE and execute the Jupyter notebook(s) using the kernel on remote workbench running on RHOAI. Most IDEs provide connection to a remote Kernel as a standard feature. However, this standard feature does not work with RHOAI because of the way authentication to workbench is set up in RHOAI The standard feature of most IDEs to connect to remote kernel uses token based authentication. Workbench pods running on RHOAI contain an authentication mechanism that sits in front of the workbench container and handles the authentication of the user connecting to the workbench. This container uses Openshift Authentication mechanism and is not compatible with the standard connection feature of most IDEs.

"},{"location":"odh-rhoai/connect-vscode-to-rhoai-wb/#workaround-connect-to-the-remote-kernel-using-openshift-port-forwarding","title":"Workaround: Connect to the remote kernel using Openshift port-forwarding","text":"

Use the following steps to connect your local VS Code to RHOAI Workbench kernel:

  • In your data science project in RHOAI, create a workbench that you would like to use as your remote kernel. If you want to use gpu accelerator, use the compatible workbench image (e.g. pytorch).

  • Open the workbench and copy the context path from the browser. You will need this later when connecting from VS Code.

  • From terminal on your laptop/desktop login to openshift.

  • Switch to your data science project

  • Start port-forwarding to your workbench pod

    • List all the pods in your project. The pod running your workbench is named using the name of your workbench in RHOAI. e.g. mywb-0 if your workbench name is mywb.
    • Enable port-forwarding to your workbench pod. You need to forward to the port the pod is listening on. It is usually 8888 for RHOAI workbench. You can find this port from the service in your project with name same as your workbench.
  • Open the Jupyter notebook in your VS Code

  • From the top right corner of the notebook, click on Select Kernel.

  • From the options, select Existing Jupyter Server and then enter the url as follows: localhost [:port] /context-path copied earlier that has the pattern /notebook/ds-project-name/workbench-name/lab. e.g. http://localhost:8888/notebook/rag-llm-demo/mywb/lab

  • A prompt saying Connecting over HTTP without a token may be an insecure connection. Do you want to connect to a possibly insecure server? is displayed. select Yes

  • Select the prompted Server display name or enter a new one.

  • A list of available kernels is displayed. Choose Python 3.9.

  • You should see the selected Kernel in the top right corner.

  • The code inside of your notebook will now execute using the remote kernel on the RHOAI workbench pod.

  • If your workbench uses a Nvidia GPU, you can verify that it is being used in the execution of your notebook by adding a command !nvidia-smi. You should see output similar to the image below.

"},{"location":"odh-rhoai/connect-vscode-to-rhoai-wb/#caveats","title":"Caveats","text":"
  • Jupyter notebooks in your local VSCode environment will not be saved to the workbench.
  • If your notebook uses any files (models, inputdata etc.), they should be present on the workbench and their path should match the path specified in your notebook.
"},{"location":"odh-rhoai/custom-notebooks/","title":"Custom Notebooks","text":"

Custom notebook images are useful if you want to add libraries that you often use, or that you require at a specific version different than the one provided in the base images. It's also useful if you need to use OS packages or applications, which you cannot install on the fly in your running environment.

"},{"location":"odh-rhoai/custom-notebooks/#image-source-and-pre-built-images","title":"Image source and Pre-built images","text":"

In the opendatahub-io-contrib/workbench-images repository, you will find the source code as well as pre-built images for a lot of use cases. A few of the available images are:

  • Base and CUDA-enabled images for different \"lines\" of OS: UBI8, UBI9, and Centos Stream 9.
  • Jupyter images enhanced with:
    • specific libraries like OptaPy or Monai,
    • with integrated applications like Spark,
    • providing other IDEs like VSCode or RStudio
  • VSCode
  • RStudio

All those images are constantly and automatically updated and rebuilt for the latest patch and fixes, and new releases are available regularly to provide new versions of the libraries or the applications.

"},{"location":"odh-rhoai/custom-notebooks/#building-your-own-images","title":"Building your own images","text":"

In the repository above, you will find many examples from the source code to help you understand how to create your own image. Here are a few rules, tips and examples to help you.

"},{"location":"odh-rhoai/custom-notebooks/#rules","title":"Rules","text":"
  • On OpenShift, every containers in a standard namespace (unless you modify security) run with a user with a random user id (uid), and the group id (gid) 0. Therefore, all the folders that you want to write in, and all the files you want to modify (temporarily) in your image must be accessible by this user. The best practice is to set the ownership at 1001:0 (user \"default\", group \"0\").
  • If you don't want/can't do that, another solution is to set permissions properly for any user, like 775.
  • When launching a notebook from Applications->Enabled, the \"personal\" volume of a user is mounted at /opt/app-root/src. This is not configurable, so make sure to build your images with this default location for the data that you want persisted.
"},{"location":"odh-rhoai/custom-notebooks/#how-tos","title":"How-tos","text":""},{"location":"odh-rhoai/custom-notebooks/#install-python-packages","title":"Install Python packages","text":"
  • Start from a base image of your choice. Normally it's already running under user 1001, so no need to change it.
  • Copy your pipfile.lock or your requirements.txt
  • Install your packages

Example:

FROM BASE_IMAGE\n\n# Copying custom packages\nCOPY Pipfile.lock ./\n\n# Install packages and cleanup\n# (all commands are chained to minimize layer size)\nRUN echo \"Installing softwares and packages\" && \\\n    # Install Python packages \\\n    micropipenv install && \\\n    rm -f ./Pipfile.lock\n    # Fix permissions to support pip in OpenShift environments \\\n    chmod -R g+w /opt/app-root/lib/python3.9/site-packages && \\\n    fix-permissions /opt/app-root -P\n\nWORKDIR /opt/app-root/src\n\nENTRYPOINT [\"start-notebook.sh\"]\n

In this example, the fix-permissions script (present in all standard images and custom images from the opendatahub-contrib repo) fixes any bad ownership or rights that may be present.

"},{"location":"odh-rhoai/custom-notebooks/#install-an-os-package","title":"Install an OS package","text":"
  • If you have to install OS packages and Python packages, it's better to start with the OS.
  • In your Containerfile/Dockerfile, switch to user 0, install your package(s), then switch back to user 1001. Example:
USER 0\n\nRUN INSTALL_PKGS=\"java-11-openjdk java-11-openjdk-devel\" && \\\n    yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \\\n    yum -y clean all --enablerepo='*'\n\nUSER 1001\n
"},{"location":"odh-rhoai/custom-notebooks/#tips-and-tricks","title":"Tips and tricks","text":""},{"location":"odh-rhoai/custom-notebooks/#enabling-codeready-builder-crb-and-epel","title":"Enabling CodeReady Builder (CRB) and EPEL","text":"

CRB and EPEL are repositories providing packages absent from a standard RHEL or UBI installation. They are useful and required to be able to install specific software (RStudio, I'm looking at you...).

  • Enabling EPEL on UBI9-based images (on UBI9 images CRB is now enabled by default.):
RUN yum install -y https://download.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm\n
  • Enabling CRB and EPEL on Centos Stream 9-based images:
RUN yum install -y yum-utils && \\\n    yum-config-manager --enable crb && \\\n    yum install -y https://download.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm\n
"},{"location":"odh-rhoai/custom-notebooks/#minimizing-image-size","title":"Minimizing image size","text":"

A container image uses a \"layered\" filesystem. Every time you have in your file a COPY or a RUN command, a new layer is created. Nothing is ever deleted: removing a file is simply \"masking\" it in the next layer. Therefore you must bee very careful when you create your Containerfile/Dockerfile.

  • If you start from an image that is constantly updated, like ubi9/python-39 from the Red Hat Catalog, don't do a yum update. This will only fetch new metadata, update a few files that may not have any impact, and get you a bigger image.
  • Rebuilt your images often from scratch, but don't do a yum update on a previous version.
  • Group your RUN commands as much as you can, add && \\ at the end of each line to chain your commands.
  • If you need to compile something for building an image, use the multi-stage builds approach. Build the library or application in an intermediate container image, then copy the result to your final image. Otherwise, all the build artefacts will persist in your image...
"},{"location":"odh-rhoai/custom-runtime-triton/","title":"Deploying and using a Custom Serving Runtime in ODH/RHOAI","text":"

Although these instructions were tested mostly using RHOAI (Red Hat OpenShift AI), they apply to ODH (Open Data Hub) as well.

"},{"location":"odh-rhoai/custom-runtime-triton/#before-you-start","title":"Before you start","text":"

This document will guide you through the broad steps necessary to deploy a custom Serving Runtime in order to serve a model using the Triton Runtime (NVIDIA Triton Inference Server).

While RHOAI supports your ability to add your own runtime, it does not support the runtimes themselves. Therefore, it is up to you to configure, adjust and maintain your custom runtimes.

This document expects a bit of familiarity with RHOAI.

The sources used to create this document are mostly:

  • https://github.com/kserve/modelmesh-serving/tree/main/config/runtimes
  • https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver
  • Official Red Hat OpenShift AI Documentation
"},{"location":"odh-rhoai/custom-runtime-triton/#adding-the-custom-triton-runtime","title":"Adding the custom triton runtime","text":"
  1. Log in to your OpenShift AI with a user who is part of the RHOAI admin group.
    1. (by default, cluster-admins and dedicated admins are).
  2. Navigate to the Settings menu, then Serving Runtimes

  3. Click on the Add Serving Runtime button:

  4. From the drop down menu, select **Multi-model serving platform. The option for REST will be selected automatically:

  5. Click on Start from scratch and in the window that opens up, paste the following YAML:

    # Copyright 2021 IBM Corporation\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\napiVersion: serving.kserve.io/v1alpha1\n# kind: ClusterServingRuntime     ## changed by EG\nkind: ServingRuntime\nmetadata:\n  name: triton-23.05-20230804\n  labels:\n    name: triton-23.05-20230804\n  annotations:\n    maxLoadingConcurrency: \"2\"\n    openshift.io/display-name: \"Triton runtime 23.05 - added on 20230804 - with /dev/shm\"\nspec:\n  supportedModelFormats:\n    - name: keras\n      version: \"2\" # 2.6.0\n      autoSelect: true\n    - name: onnx\n      version: \"1\" # 1.5.3\n      autoSelect: true\n    - name: pytorch\n      version: \"1\" # 1.8.0a0+17f8c32\n      autoSelect: true\n    - name: tensorflow\n      version: \"1\" # 1.15.4\n      autoSelect: true\n    - name: tensorflow\n      version: \"2\" # 2.3.1\n      autoSelect: true\n    - name: tensorrt\n      version: \"7\" # 7.2.1\n      autoSelect: true\n\n  protocolVersions:\n    - grpc-v2\n  multiModel: true\n\n  grpcEndpoint: \"port:8085\"\n  grpcDataEndpoint: \"port:8001\"\n\n  volumes:\n    - name: shm\n      emptyDir:\n        medium: Memory\n        sizeLimit: 2Gi\n  containers:\n    - name: triton\n      # image: tritonserver-2:replace   ## changed by EG\n      image: nvcr.io/nvidia/tritonserver:23.05-py3\n      command: [/bin/sh]\n      args:\n        - -c\n        - 'mkdir -p /models/_triton_models;\n          chmod 777 /models/_triton_models;\n          exec tritonserver\n          \"--model-repository=/models/_triton_models\"\n          \"--model-control-mode=explicit\"\n          \"--strict-model-config=false\"\n          \"--strict-readiness=false\"\n          \"--allow-http=true\"\n          \"--allow-sagemaker=false\"\n          '\n      volumeMounts:\n        - name: shm\n          mountPath: /dev/shm\n      resources:\n        requests:\n          cpu: 500m\n          memory: 1Gi\n        limits:\n          cpu: \"5\"\n          memory: 1Gi\n      livenessProbe:\n        # the server is listening only on 127.0.0.1, so an httpGet probe sent\n        # from the kublet running on the node cannot connect to the server\n        # (not even with the Host header or host field)\n        # exec a curl call to have the request originate from localhost in the\n        # container\n        exec:\n          command:\n            - curl\n            - --fail\n            - --silent\n            - --show-error\n            - --max-time\n            - \"9\"\n            - http://localhost:8000/v2/health/live\n        initialDelaySeconds: 5\n        periodSeconds: 30\n        timeoutSeconds: 10\n  builtInAdapter:\n    serverType: triton\n    runtimeManagementPort: 8001\n    memBufferBytes: 134217728\n    modelLoadingTimeoutMillis: 90000\n

  6. You will likely want to update the name , as well as other parameters.
  7. Click Add
  8. Confirm the new Runtime is in the list, and re-order the list as needed. (the order chosen here is the order in which the users will see these choices)

"},{"location":"odh-rhoai/custom-runtime-triton/#creating-a-project","title":"Creating a project","text":"
  • Create a new Data Science Project
  • In this example, the project is called fraud
"},{"location":"odh-rhoai/custom-runtime-triton/#creating-a-model-server","title":"Creating a model server","text":"
  1. In your project, scroll down to the \"Models and Model Servers\" Section
  2. Click on Configure server

  3. Fill out the details:

  4. Click Configure

"},{"location":"odh-rhoai/custom-runtime-triton/#deploying-a-model-into-it","title":"Deploying a model into it","text":"
  1. If you don't have any model files handy, you can grab a copy of this file and upload it to your Object Storage of choice.
  2. Click on Deploy Model

  3. Choose a model name and a framework:

  4. Then create a new data connection containing the details of where your model is stored in Object Storage:

  5. After a little while, you should see the following:

"},{"location":"odh-rhoai/custom-runtime-triton/#validating-the-model","title":"Validating the model","text":"
  1. If you've used the model mentioned earlier in this document, you can run the following command from a Linux prompt:
    function val-model {\n    myhost=\"$1\"\n    echo \"validating host $myhost\"\n    time curl -X POST -k \"${myhost}\" -d '{\"inputs\": [{ \"name\": \"dense_input\", \"shape\": [1, 7], \"datatype\": \"FP32\", \"data\": [57.87785658389723,0.3111400080477545,1.9459399775518593,1.0,1.0,0.0,0.0]}]}' | jq\n}\n\nval-model \"https://fraud-model-fraud.apps.mycluster.openshiftapps.com/v2/models/fraud-model/infer\"\n
  2. Change the host to match the address for your model.
  3. You should see an output similar to:
    {\n  \"model_name\": \"fraud-model__isvc-c1529f9667\",\n  \"model_version\": \"1\",\n  \"outputs\": [\n    {\n      \"name\": \"dense_3\",\n      \"datatype\": \"FP32\",\n      \"shape\": [\n        1,\n        1\n      ],\n      \"data\": [\n        0.86280495\n      ]\n    }\n  ]\n}\n
"},{"location":"odh-rhoai/custom-runtime-triton/#extra-considerations-for-disconnected-environments","title":"Extra considerations for Disconnected environments.","text":"

The YAML included in this file makes a reference to the following Nvidia Triton Image: nvcr.io/nvidia/tritonserver:23.05-py3

Ensure that this image is properly mirrored into the mirror registry.

Also, update the YAML definition as needed to point to the image address that matches the image registry.

"},{"location":"odh-rhoai/custom-runtime-triton/#gitops-related-information","title":"GitOps related information","text":"

Each of the activities performed via the user interface will create a Kubernetes Object inside your OpenShift Cluster.

  • The addition of a new runtime creates a template in the redhat-ods-applications namespace.
  • Each model server is defined as a ServingRuntime
  • Each model is defined as an InferenceService
  • Each Data Connection is stored as a Secret
"},{"location":"odh-rhoai/gitops/","title":"Managing RHOAI with GitOps","text":"

GitOps is a common way to manage and deploy applications and resouces on Kubernetes clusters.

This page is intended to provide an overview of the different objects involved in manaing both the installation, administration, and usage of OpenShift AI components using GitOps. This page is by no means intended to be an exhaustive tutorial on each object and all of the features available in them.

When first implimenting features with GitOps it is highly recommended to deploy the resources manually using the Dashboard, then extract the resources created by the Dashboard and duplicate them in your gitops repo.

"},{"location":"odh-rhoai/gitops/#installation","title":"Installation","text":""},{"location":"odh-rhoai/gitops/#operator-installation","title":"Operator Installation","text":"

The Red Hat OpenShift AI operator is installed and managed by OpenShift's Operator Lifecycle Manager (OLM) and follows common patterns that can be used to install many different operators.

The Red Hat OpenShift AI operator should be installed in the redhat-ods-operator namespace:

apiVersion: v1\nkind: Namespace\nmetadata:\n  annotations:\n    openshift.io/display-name: \"Red Hat OpenShift AI\"\n  labels:\n    openshift.io/cluster-monitoring: 'true'\n  name: redhat-ods-operator\n

After creating the namespace, OLM requires you create an Operator Group to help manage any operators installed in that namespace:

apiVersion: operators.coreos.com/v1\nkind: OperatorGroup\nmetadata:\n  name: redhat-ods-operator-group\n  namespace: redhat-ods-operator\n

Finally, a Subscription can be created to install the operator:

apiVersion: operators.coreos.com/v1alpha1\nkind: Subscription\nmetadata:\n  name: rhods-operator\n  namespace: redhat-ods-operator\nspec:\n  channel: stable # <1>\n  installPlanApproval: Automatic # <2>\n  name: rhods-operator\n  source: redhat-operators\n  sourceNamespace: openshift-marketplace\n

Subscription Options:

  1. Operator versions are managed with the channel in OLM. Users are able to select a channel that corresponds to the upgrade lifecycle they wish to follow and OLM will update versions as they are released on that channel. To learn more about the available channels and the release lifecycle, please refer to the official lifecycle documentation
  2. Platform administrators also have an option to set how upgrades are applied for the operator with the installPlanApproval. If set to Automatic RHOAI is automatically updated to the latest version that is available on the selected channel. If set to Manual administrators will be required to approve all upgrades.
"},{"location":"odh-rhoai/gitops/#component-configuration","title":"Component Configuration","text":"

When the operator is installed it automatically creates a DSCInitialization object that sets up several default configurations. While it is not required, administrators can choose to manage the DSCinitialization object via GitOps.

apiVersion: dscinitialization.opendatahub.io/v1\nkind: DSCInitialization\nmetadata:\n  name: default-dsci\nspec:\n  applicationsNamespace: redhat-ods-applications\n  monitoring:\n    managementState: Managed\n    namespace: redhat-ods-monitoring\n  serviceMesh:\n    auth:\n      audiences:\n        - 'https://kubernetes.default.svc'\n    controlPlane:\n      metricsCollection: Istio\n      name: data-science-smcp\n      namespace: istio-system\n    managementState: Managed # <1>\n  trustedCABundle:\n    customCABundle: ''\n    managementState: Managed # <2>\n

DSCInitialization Options:

  1. KServe requires a ServiceMesh instance to be installed on the cluster. By default the Red Hat OpenShift AI operator will attempt to configure an instance if the ServiceMesh operator is installed. If your cluster already has ServiceMesh configured, you may choose to skip this option.
  2. As part of the ServiceMesh configuration, the Red Hat OpenShift AI operator will configure a self-signed cert for any routes created by ServiceMesh.

After the operator is installed, a DataScienceCluster object will need to be configured with the different components. Each component has a managementState option which can be set to Managed or Removed. Admins can choose which components are installed on the cluster.

kind: DataScienceCluster\napiVersion: datasciencecluster.opendatahub.io/v1\nmetadata:\n  name: default\nspec:\n  components:\n    codeflare:\n      managementState: Managed\n    kserve:\n      managementState: Managed\n      serving:\n        ingressGateway:\n          certificate:\n            type: SelfSigned\n        managementState: Managed\n        name: knative-serving\n    trustyai:\n      managementState: Removed\n    ray:\n      managementState: Managed\n    kueue:\n      managementState: Managed\n    workbenches:\n      managementState: Managed\n    dashboard:\n      managementState: Managed\n    modelmeshserving:\n      managementState: Managed\n    datasciencepipelines:\n      managementState: Managed\n

After the DataScienceCluster object is created, the operator will install and configure the different components on the cluster. Only one DataScienceCluster object can be created on a cluster.

"},{"location":"odh-rhoai/gitops/#administration","title":"Administration","text":""},{"location":"odh-rhoai/gitops/#dashboard-configs","title":"Dashboard Configs","text":"

The Red Hat OpenShift AI Dashboard has many different configurable options through the UI that can be managed using the OdhDashboardConfig config object. A default OdhDashboardConfig is created when the Dashboard component is installed

apiVersion: opendatahub.io/v1alpha\nkind: OdhDashboardConfig\nmetadata:\n  name: odh-dashboard-config\n  namespace: redhat-ods-applications\n  labels:\n    app.kubernetes.io/part-of: rhods-dashboard\n    app.opendatahub.io/rhods-dashboard: 'true'\nspec:\n  dashboardConfig:\n    enablement: true\n    disableDistributedWorkloads: false\n    disableProjects: false\n    disableBiasMetrics: false\n    disableSupport: false\n    disablePipelines: false\n    disableProjectSharing: false\n    disableModelServing: false\n    disableKServe: false\n    disableAcceleratorProfiles: false\n    disableCustomServingRuntimes: false\n    disableModelMesh: false\n    disableKServeAuth: false\n    disableISVBadges: false\n    disableInfo: false\n    disableClusterManager: false\n    disablePerformanceMetrics: false\n    disableBYONImageStream: false\n    disableModelRegistry: true\n    disableTracking: false\n  groupsConfig:\n    adminGroups: rhods-admins # <1>\n    allowedGroups: 'system:authenticated' # <2>\n  modelServerSizes: # <3>\n    - name: Small\n      resources:\n        limits:\n          cpu: '2'\n          memory: 8Gi\n        requests:\n          cpu: '1'\n          memory: 4Gi\n    - name: Medium\n      resources:\n        limits:\n          cpu: '8'\n          memory: 10Gi\n        requests:\n          cpu: '4'\n          memory: 8Gi\n    - name: Large\n      resources:\n        limits:\n          cpu: '10'\n          memory: 20Gi\n        requests:\n          cpu: '6'\n          memory: 16Gi\n  notebookController:\n    enabled: true\n    notebookNamespace: rhods-notebooks\n    pvcSize: 20Gi # <4>\n  notebookSizes: # <5>\n    - name: Small\n      resources:\n        limits:\n          cpu: '2'\n          memory: 8Gi\n        requests:\n          cpu: '1'\n          memory: 8Gi\n    - name: Medium\n      resources:\n        limits:\n          cpu: '6'\n          memory: 24Gi\n        requests:\n          cpu: '3'\n          memory: 24Gi\n    - name: Large\n      resources:\n        limits:\n          cpu: '14'\n          memory: 56Gi\n        requests:\n          cpu: '7'\n          memory: 56Gi\n    - name: X Large\n      resources:\n        limits:\n          cpu: '30'\n          memory: 120Gi\n        requests:\n          cpu: '15'\n          memory: 120Gi\n  templateDisablement: []\n  templateOrder:\n    - caikit-tgis-runtime\n    - kserve-ovms\n    - ovms\n    - tgis-grpc-runtime\n    - vllm-runtime\n

OdhDashboardConfig Options:

  1. The Dashboard creates a group called rhods-admins by default which users can be added to be granted admin privileges through the Dashboard. Additionally, any user with the cluster-admin role are admins in the Dashboard by default. If you wish to change the group which is used to manage admin access, this option can be updated. It is important to note that this field only impacts a users ability to modify settings in the Dashboard, and will have no impact to a users ability to modify configurations through the Kubernetes objects such as this OdhDashboardConfig object.
  2. By default any user that has access to the OpenShift cluster where Red Hat OpenShift AI is installed will have the ability to access the Dashboard. If you wish to restrict who has access to the Dashboard this option can be updated to another group. Like the admin group option, this option only impacts the users ability to access the Dashboard and does not restrict their ability to interact directly with the Kubernetes objects used to deploy AI resources.
  3. When a user creates a new Model Server through the Dashboard they are presented with an option to choose a server size which will impact the resources available to the pod created for the Model Server. Administrators have the ability to configure the default options that are available to their users.
  4. When creating a new Workbench, users are asked to create storage for their Workbench. The storage will default to the value set here and users will have the option to choose a different amount of storage if their use case requires more or less storage. Admins can choose another default storage size that is presented to users by configuring this option.
  5. Like the Model Server size, users are presented with a drop down menu of options to select what size of Workbench they wish to create. Admins have the ability to customize the size options that are presented to users.
"},{"location":"odh-rhoai/gitops/#idle-notebook-culling","title":"Idle Notebook Culling","text":"

Admins have the ability to enable Idle Notebook Culling which will automatically stop any Notebooks/Workbenches that users haven't interacted with in a period of time by creating the following ConfigMap:

kind: ConfigMap\napiVersion: v1\nmetadata:\n  name: notebook-controller-culler-config\n  namespace: redhat-ods-applications\n  labels:\n    opendatahub.io/dashboard: 'true'\ndata:\n  CULL_IDLE_TIME: '240' # <1>\n  ENABLE_CULLING: 'true'\n  IDLENESS_CHECK_PERIOD: '1'\n

Idle Notebook Culling Options:

  1. The CULL_IDLE_TIME looks for metrics from Jupyter to understand when the last time a user interacted with the Workbench and will shut the pod down if it has passed the time set here. The time is the number of minutes so 240 minutes or 4 hours.
"},{"location":"odh-rhoai/gitops/#accelerator-profiles","title":"Accelerator Profiles","text":"

Accelerator Profiles allow admins to configure different types of GPU options that they can present to end users and automatically configure a toleration on Workbenches or Model Servers when they are selected. Admins can configure an Accelerator Profile with the AcceleratorProfile object:

apiVersion: dashboard.opendatahub.io/v1\nkind: AcceleratorProfile\nmetadata:\n  name: nvidia-gpu\n  namespace: redhat-ods-applications\nspec:\n  displayName: nvidia-gpu\n  enabled: true\n  identifier: nvidia.com/gpu\n  tolerations:\n    - effect: NoSchedule\n      key: nvidia-gpu-only\n      operator: Exists\n      value: ''\n
"},{"location":"odh-rhoai/gitops/#notebook-images","title":"Notebook Images","text":"

Red Hat OpenShift AI ships with several out of the box Notebook/Workbench Images but admins can create additional custom images that users can use to launch new Workbench instances. A Notebook Image is managed with an OpenShift ImageStream object with some required labels:

kind: ImageStream\napiVersion: image.openshift.io/v1\nmetadata:\n  annotations:\n    opendatahub.io/notebook-image-desc: A custom Jupyter Notebook built for my organization # <1>\n    opendatahub.io/notebook-image-name: My Custom Notebook # <2>\n  name: my-custom-notebook\n  namespace: redhat-ods-applications\n  labels: # <3>\n    app.kubernetes.io/created-by: byon\n    opendatahub.io/dashboard: 'true'\n    opendatahub.io/notebook-image: 'true'\nspec:\n  lookupPolicy:\n    local: true\n  tags:\n    - name: '1.0' # <4>\n      annotations:\n        opendatahub.io/notebook-python-dependencies: '[{\"name\":\"PyTorch\",\"version\":\"2.2\"}]' # <5>\n        opendatahub.io/notebook-software: '[{\"name\":\"Python\",\"version\":\"v3.11\"}]' # <6>\n        opendatahub.io/workbench-image-recommended: 'true' # <7>\n      from:\n        kind: DockerImage\n        name: 'quay.io/my-org/my-notebook:latest' # <8>\n      importPolicy:\n        importMode: Legacy\n      referencePolicy:\n        type: Source\n

Notebook Image Options:

  1. A description for the purpose of the notebook image
  2. The name that will be displayed to end users in the drop down menu when creating a Workbench
  3. The notebook image requires several labels for them to appear in the Dashboard, including the app.kubernetes.io/created-by: byon label. While traditionally this label is utilized to trace where an object originated from, this label is required for the notebooks to be made available to end users.
  4. Multiple image versions can be configured as part of the same Notebook and users have the ability to select which version of the image they wish to use. This is helpful if you release updated versions of the Notebook image and you wish to avoid breaking end user environments with package changes and allow them to upgrade as they wish.
  5. When selecting a Notebook image users will be presented with some information about the notebook based on the information presented in this annotation. opendatahub.io/notebook-python-dependencies is most commonly used to present information about versions from the most important Python packages that are pre-installed in the Image.
  6. Like the python dependencies annotation, the opendatahub.io/notebook-software annotation is used to present the end user with information about what software is installed in the Image. Most commonly this field is used to present information such as the Python version, Jupyter versions, or CUDA versions.
  7. When multiple tags are created on the ImageStream, the opendatahub.io/workbench-image-recommended is used to control what version of the image is presented by default to end users. Only one tag should be set to true at any give time.
  8. Notebook images are generally recommended to be stored in an Image Registry outside of the cluster and referenced in the ImageStream.

While it is possible to build a Notebook Image on an OpenShift cluster and publish it directly to an ImageStream using a BuildConfig or a Tekton Pipeline, it can be challenging to get that image to be seen by the Red Hat OpenShift AI Dashboard. The Dashboard is only looks at images listed in the spec.tags section and images pushed directly to the internal image registry are recorded in the status.tags. As a work around, it is possible to \"link\" a tag pushed directly to the internal image registry to a tag that is visible by the Dashboard:

kind: ImageStream\napiVersion: image.openshift.io/v1\nmetadata:\n  annotations:\n    opendatahub.io/notebook-image-desc: A custom Jupyter Notebook built for my organization\n    opendatahub.io/notebook-image-name: My Custom Notebook\n  name: my-custom-notebook\n  namespace: redhat-ods-applications\n  labels:\n    app.kubernetes.io/created-by: byon\n    opendatahub.io/dashboard: 'true'\n    opendatahub.io/notebook-image: 'true'\nspec:\n  lookupPolicy:\n    local: false\n  tags:\n    - name: '1.0'\n      annotations:\n        opendatahub.io/notebook-python-dependencies: '[{\"name\":\"PyTorch\",\"version\":\"2.2\"}]'\n        opendatahub.io/notebook-software: '[{\"name\":\"Python\",\"version\":\"v3.11\"}]'\n        opendatahub.io/workbench-image-recommended: 'true'\n      from:\n        kind: DockerImage\n        name: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/my-custom-workbench:latest'\n      importPolicy:\n        importMode: Legacy\n      referencePolicy:\n        type: Source\nstatus:\n  dockerImageRepository: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/dsp-example'\n  tags:\n    - tag: latest\n
"},{"location":"odh-rhoai/gitops/#serving-runtime-templates","title":"Serving Runtime Templates","text":"

Red Hat OpenShift AI ships with several out of the box Serving Runtime Templates such as OpenVino and vLLM, but admins have the ability to configure additional templates that allow users to deploy additional ServingRuntimes. A Serving Runtime template is an OpenShift Template object that wraps around a ServingRuntime object:

kind: Template\napiVersion: template.openshift.io/v1\nmetadata:\n  name: trition-serving-runtime\n  namespace: redhat-ods-applications\n  labels:\n    opendatahub.io/dashboard: 'true'\n  annotations:\n    opendatahub.io/apiProtocol: REST\n    opendatahub.io/modelServingSupport: '[\"multi\"]'\nobjects:\n  - apiVersion: serving.kserve.io/v1alpha1\n    kind: ServingRuntime\n    metadata:\n      name: triton-23.05\n      labels:\n        name: triton-23.05\n      annotations:\n        maxLoadingConcurrency: '2'\n        openshift.io/display-name: Triton runtime 23.05\n    spec:\n      supportedModelFormats:\n        - name: keras\n          version: '2'\n          autoSelect: true\n        - name: onnx\n          version: '1'\n          autoSelect: true\n        - name: pytorch\n          version: '1'\n          autoSelect: true\n        - name: tensorflow\n          version: '1'\n          autoSelect: true\n        - name: tensorflow\n          version: '2'\n          autoSelect: true\n        - name: tensorrt\n          version: '7'\n          autoSelect: true\n      protocolVersions:\n        - grpc-v2\n      multiModel: true\n      grpcEndpoint: 'port:8085'\n      grpcDataEndpoint: 'port:8001'\n      volumes:\n        - name: shm\n          emptyDir:\n            medium: Memory\n            sizeLimit: 2Gi\n      containers:\n        - name: triton\n          image: 'nvcr.io/nvidia/tritonserver:23.05-py3'\n          command:\n            - /bin/sh\n          args:\n            - '-c'\n            - 'mkdir -p /models/_triton_models; chmod 777 /models/_triton_models; exec tritonserver \"--model-repository=/models/_triton_models\" \"--model-control-mode=explicit\" \"--strict-model-config=false\" \"--strict-readiness=false\" \"--allow-http=true\" \"--allow-sagemaker=false\" '\n          volumeMounts:\n            - name: shm\n              mountPath: /dev/shm\n          resources:\n            requests:\n              cpu: 500m\n              memory: 1Gi\n            limits:\n              cpu: '5'\n              memory: 1Gi\n          livenessProbe:\n            exec:\n              command:\n                - curl\n                - '--fail'\n                - '--silent'\n                - '--show-error'\n                - '--max-time'\n                - '9'\n                - 'http://localhost:8000/v2/health/live'\n            initialDelaySeconds: 5\n            periodSeconds: 30\n            timeoutSeconds: 10\n      builtInAdapter:\n        serverType: triton\n        runtimeManagementPort: 8001\n        memBufferBytes: 134217728\n        modelLoadingTimeoutMillis: 90000\n
"},{"location":"odh-rhoai/gitops/#end-user-resources","title":"End User Resources","text":""},{"location":"odh-rhoai/gitops/#data-science-projects","title":"Data Science Projects","text":"

Data Science Projects are simply a normal OpenShift Project with an extra label to distinguish them from normal OpenShift projects by the Red Hat OpenShift AI Dashboard. Like OpenShift Projects it is recommended to create a namespace object and allow OpenShift to create the corresponding project object:

apiVersion: v1\nkind: Namespace\nmetadata:\n  name: my-data-science-project\n  labels:\n    opendatahub.io/dashboard: \"true\"\n

Additionally, when a project going to be utilized by ModelMesh for Multi-model serving, there is an additional ModelMesh label that should be applied to the namespace:

apiVersion: v1\nkind: Namespace\nmetadata:\n  name: my-multi-model-serving-project\n  labels:\n    opendatahub.io/dashboard: \"true\"\n    modelmesh-enabled: \"true\"\n
"},{"location":"odh-rhoai/gitops/#workbenches","title":"Workbenches","text":"

Workbench objects are managed using the Notebook custom resource. The Notebook object contains a fairly complex configuration, with many items that will be autogenerated, and required annotations to display correctly in the Dashboard. The Notebook object essentially acts as a wrapper around a normal pod definition and you will find many similarities to managing a pod with options such as the image, pvcs, secrets, etc.

It is highly recommended to thoroughly test any Notebook configurations configured with GitOps.

apiVersion: kubeflow.org/v1\nkind: Notebook\nmetadata:\n  annotations:\n    notebooks.opendatahub.io/inject-oauth: 'true' # <1>\n    opendatahub.io/image-display-name: Minimal Python\n    notebooks.opendatahub.io/oauth-logout-url: 'https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com/projects/my-data-science-project?notebookLogout=my-workbench'\n    opendatahub.io/accelerator-name: ''\n    openshift.io/description: ''\n    openshift.io/display-name: my-workbench\n    notebooks.opendatahub.io/last-image-selection: 's2i-minimal-notebook:2024.1'\n    notebooks.kubeflow.org/last_activity_check_timestamp: '2024-07-30T20:43:25Z'\n    notebooks.opendatahub.io/last-size-selection: Small\n    opendatahub.io/username: 'kube:admin'\n    notebooks.kubeflow.org/last-activity: '2024-07-30T20:27:25Z'\n  name: my-workbench\n  namespace: my-data-science-project\nspec:\n  template:\n    spec:\n      affinity: {}\n      containers:\n        - resources: # <2>\n            limits:\n              cpu: '2'\n              memory: 8Gi\n            requests:\n              cpu: '1'\n              memory: 8Gi\n          readinessProbe:\n            failureThreshold: 3\n            httpGet:\n              path: /notebook/my-data-science-project/my-workbench/api\n              port: notebook-port\n              scheme: HTTP\n            initialDelaySeconds: 10\n            periodSeconds: 5\n            successThreshold: 1\n            timeoutSeconds: 1\n          name: my-workbench\n          livenessProbe:\n            failureThreshold: 3\n            httpGet:\n              path: /notebook/my-data-science-project/my-workbench/api\n              port: notebook-port\n              scheme: HTTP\n            initialDelaySeconds: 10\n            periodSeconds: 5\n            successThreshold: 1\n            timeoutSeconds: 1\n          env:\n            - name: NOTEBOOK_ARGS\n              value: |-\n                --ServerApp.port=8888\n                                  --ServerApp.token=''\n                                  --ServerApp.password=''\n                                  --ServerApp.base_url=/notebook/my-data-science-project/my-workbench\n                                  --ServerApp.quit_button=False\n                                  --ServerApp.tornado_settings={\"user\":\"kube-3aadmin\",\"hub_host\":\"https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com\",\"hub_prefix\":\"/projects/my-data-science-project\"}\n            - name: JUPYTER_IMAGE\n              value: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/s2i-minimal-notebook:2024.1'\n            - name: PIP_CERT\n              value: /etc/pki/tls/custom-certs/ca-bundle.crt\n            - name: REQUESTS_CA_BUNDLE\n              value: /etc/pki/tls/custom-certs/ca-bundle.crt\n            - name: SSL_CERT_FILE\n              value: /etc/pki/tls/custom-certs/ca-bundle.crt\n            - name: PIPELINES_SSL_SA_CERTS\n              value: /etc/pki/tls/custom-certs/ca-bundle.crt\n            - name: GIT_SSL_CAINFO\n              value: /etc/pki/tls/custom-certs/ca-bundle.crt\n          ports:\n            - containerPort: 8888\n              name: notebook-port\n              protocol: TCP\n          imagePullPolicy: Always\n          volumeMounts:\n            - mountPath: /opt/app-root/src\n              name: my-workbench\n            - mountPath: /dev/shm\n              name: shm\n            - mountPath: /etc/pki/tls/custom-certs/ca-bundle.crt\n              name: trusted-ca\n              readOnly: true\n              subPath: ca-bundle.crt\n          image: 'image-registry.openshift-image-registry.svc:5000/redhat-ods-applications/s2i-minimal-notebook:2024.1' # <3>\n          workingDir: /opt/app-root/src\n        - resources: # <4>\n            limits:\n              cpu: 100m\n              memory: 64Mi\n            requests:\n              cpu: 100m\n              memory: 64Mi\n          readinessProbe:\n            failureThreshold: 3\n            httpGet:\n              path: /oauth/healthz\n              port: oauth-proxy\n              scheme: HTTPS\n            initialDelaySeconds: 5\n            periodSeconds: 5\n            successThreshold: 1\n            timeoutSeconds: 1\n          name: oauth-proxy\n          livenessProbe:\n            failureThreshold: 3\n            httpGet:\n              path: /oauth/healthz\n              port: oauth-proxy\n              scheme: HTTPS\n            initialDelaySeconds: 30\n            periodSeconds: 5\n            successThreshold: 1\n            timeoutSeconds: 1\n          env:\n            - name: NAMESPACE\n              valueFrom:\n                fieldRef:\n                  fieldPath: metadata.namespace\n          ports:\n            - containerPort: 8443\n              name: oauth-proxy\n              protocol: TCP\n          imagePullPolicy: Always\n          volumeMounts:\n            - mountPath: /etc/oauth/config\n              name: oauth-config\n            - mountPath: /etc/tls/private\n              name: tls-certificates\n          image: 'registry.redhat.io/openshift4/ose-oauth-proxy@sha256:4bef31eb993feb6f1096b51b4876c65a6fb1f4401fee97fa4f4542b6b7c9bc46'\n          args:\n            - '--provider=openshift'\n            - '--https-address=:8443'\n            - '--http-address='\n            - '--openshift-service-account=my-workbench'\n            - '--cookie-secret-file=/etc/oauth/config/cookie_secret'\n            - '--cookie-expire=24h0m0s'\n            - '--tls-cert=/etc/tls/private/tls.crt'\n            - '--tls-key=/etc/tls/private/tls.key'\n            - '--upstream=http://localhost:8888'\n            - '--upstream-ca=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt'\n            - '--email-domain=*'\n            - '--skip-provider-button'\n            - '--openshift-sar={\"verb\":\"get\",\"resource\":\"notebooks\",\"resourceAPIGroup\":\"kubeflow.org\",\"resourceName\":\"my-workbench\",\"namespace\":\"$(NAMESPACE)\"}'\n            - '--logout-url=https://rhods-dashboard-redhat-ods-applications.apps.my-cluster.com/projects/my-data-science-project?notebookLogout=my-workbench'\n      enableServiceLinks: false\n      serviceAccountName: my-workbench\n      volumes:\n        - name: my-workbench\n          persistentVolumeClaim:\n            claimName: my-workbench\n        - emptyDir:\n            medium: Memory\n          name: shm\n        - configMap:\n            items:\n              - key: ca-bundle.crt\n                path: ca-bundle.crt\n            name: workbench-trusted-ca-bundle\n            optional: true\n          name: trusted-ca\n        - name: oauth-config\n          secret:\n            defaultMode: 420\n            secretName: my-workbench-oauth-config\n        - name: tls-certificates\n          secret:\n            defaultMode: 420\n            secretName: my-workbench-tls\n
  1. The Notebook object contains several different annotations that are used by OpenShift AI, but the inject-oauth annotation is one of the most important. There are several oauth based configurations that in the Notebook that will be automatically generated by this annotation, allowing you to exclude a large amount of notebook configuration from what is contained in your GitOps repo.
  2. While selecting the resource size through the Dashboard you have more limited options for what sizes you can select, you can choose any size you wish for your notebook through the YAML. By selecting a non-standard size the Dashboard may report an \"unknown\" size however.
  3. Just like the resources size, you can choose any number of images for the Notebook, including ones that are not available in the Dashboard. By selecting a non-standard notebook image the Dashboard may report issues however.
  4. The oauth-proxy container is one such item that can be removed from the gitops based configuration when utilizing the inject-oauth annotation. Instead of including this section and some other oauth related configurations, you can simply rely on the annotation, and allow the Notebook controller to manage this portion of the object for you. This will help to prevent problems when upgrading RHOAI.

Users have the ability to start and stop the Workbench to help conserve resources on the cluster. To stop a Notebook, the following annotation should be applied to the Notebook object:

metadata:\n  annotations:\n    kubeflow-resource-stopped: '2024-07-30T20:52:37Z'\n

Generally, you do not want to include this annotation in your GitOps configuration, as it will enforce the Notebook to be shutdown, not allowing users to start their Notebooks. The value of the annotation doesn't matter, but by default the Dashboard will apply a timestamp with the time the Notebook was shut down.

"},{"location":"odh-rhoai/gitops/#data-science-connections","title":"Data Science Connections","text":"

A Data Science Connection is a normal Kubernetes Secret object with several annotations that follow a specific format for the data.

kind: Secret\napiVersion: v1\ntype: Opaque\nmetadata:\n  name: aws-connection-my-dataconnection # <1>\n  labels:\n    opendatahub.io/dashboard: 'true' # <2>\n    opendatahub.io/managed: 'true'\n  annotations:\n    opendatahub.io/connection-type: s3 # <3>\n    openshift.io/display-name: my-dataconnection # <4>\ndata: # <5>\n  AWS_ACCESS_KEY_ID: dGVzdA==\n  AWS_DEFAULT_REGION: 'dGVzdA=='\n  AWS_S3_BUCKET: 'dGVzdA=='\n  AWS_S3_ENDPOINT: dGVzdA==\n  AWS_SECRET_ACCESS_KEY: dGVzdA==\n
  1. When creating a data connection through the Dashboard, the name is automatically generated as aws-connection-<your-entered-name>. When generating the data connection from outside of the Dashboard, you do not need to follow this naming convention.
  2. The opendatahub.io/dashboard: 'true' label is used to help determine what secrets to display in the Dashboard. This option must be set to true if you wish for it to be available in the UI.
  3. At this point in time, the Dashboard only supports the S3 as a connection-type, but other types may be supported in the future.
  4. The name of the data connection as it will appear in the Dashboard UI
  5. Like all secrets, data connections data is stored in a base64 encoding. This data is not secure to be stored in this format and users should instead look into tools such as SealedSecrets or ExternalSecrets to manage secret data in a gitops workflow.
"},{"location":"odh-rhoai/gitops/#data-science-pipelines","title":"Data Science Pipelines","text":"

When setting up a new project, a Data Science Pipeline instance needs to be created using the DataSciencePipelineApplication object. The DSPA will create the pipeline servers for the project and allow users to begin interacting with Data Science Pipelines.

apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1\nkind: DataSciencePipelinesApplication\nmetadata:\n  name: dspa # <1>\n  namespace: my-data-science-project\nspec:\n  apiServer:\n    caBundleFileMountPath: ''\n    stripEOF: true\n    dbConfigConMaxLifetimeSec: 120\n    applyTektonCustomResource: true\n    caBundleFileName: ''\n    deploy: true\n    enableSamplePipeline: false\n    autoUpdatePipelineDefaultVersion: true\n    archiveLogs: false\n    terminateStatus: Cancelled\n    enableOauth: true\n    trackArtifacts: true\n    collectMetrics: true\n    injectDefaultScript: true\n  database:\n    disableHealthCheck: false\n    mariaDB:\n      deploy: true\n      pipelineDBName: mlpipeline\n      pvcSize: 10Gi\n      username: mlpipeline\n  dspVersion: v2\n  objectStorage:\n    disableHealthCheck: false\n    enableExternalRoute: false\n    externalStorage: # <2>\n      basePath: ''\n      bucket: pipelines\n      host: 'minio.ai-example-training.svc.cluster.local:9000'\n      port: ''\n      region: us-east-1\n      s3CredentialsSecret:\n        accessKey: AWS_SECRET_ACCESS_KEY\n        secretKey: AWS_ACCESS_KEY_ID\n        secretName: aws-connection-my-dataconnection\n      scheme: http\n  persistenceAgent:\n    deploy: true\n    numWorkers: 2\n  scheduledWorkflow:\n    cronScheduleTimezone: UTC\n    deploy: true\n
  1. The Dashboard expects to look for an object called dspa and it is not recommended to deploy more than a single DataSciencePipelineApplication object in a single namespace.
  2. The externalStorage is a critical configuration for setting up S3 backend storage for Data Science Pipelines. While using the dashboard you are required to configure the connection details. While you can import these details from a data connection, it will create a separate secret containing the s3 secrets instead of reusing the existing data connection secret.

Once a Data Science Pipeline instance has been created, users may wish to configure and manage their pipelines via GitOps. It is important to note that Data Science Pipelines is not \"gitops friendly\". While working with Elyra or a kfp pipeline, users are required to manually upload a pipeline file to the Dashboard which does not generate a corresponding Kubernetes object. Additionally, when executing a pipeline run, uses may find a ArgoWorkflow object that is generated for the run, however this object can not be re-used in a gitops application to create a new pipeline run in Data Science Pipelines.

As a work around, one common pattern to \"gitops-ify\" a Data Science Pipeline while using kfp is to instead create a Tekton pipeline that either compiles the pipeline, and uses the kfp skd to upload the pipeline to Data Science Pipelines, or the kfp sdk can automatically trigger a new pipeline run directly from your pipeline code.

"},{"location":"odh-rhoai/gitops/#model-serving","title":"Model Serving","text":"

Model Serving in RHOAI has two different flavors, Single Model Serving (KServe) and Multi-Model Serving (ModelMesh). Both model server options utilize the same Kubernetes objects (ServingRuntime and InferenceService), but have different controllers managing them.

As mentioned in the Data Science Project section, in order to utilize ModelMesh, a modelmesh-enabled label must be applied to the namespace:

apiVersion: v1\nkind: Namespace\nmetadata:\n  name: my-multi-model-serving-project\n  labels:\n    opendatahub.io/dashboard: \"true\"\n    modelmesh-enabled: \"true\"\n

When creating a model server through the Dashboard, users can select a \"Serving Runtime Template\" which will create a ServingRuntime instance in their namespace which can be managed via GitOps. The ServingRuntime helps to define different things such as the container definition, the supported model types, and available ports.

apiVersion: serving.kserve.io/v1alpha1\nkind: ServingRuntime\nmetadata:\n  annotations: # <1>\n    enable-route: 'true'\n    opendatahub.io/accelerator-name: ''\n    opendatahub.io/apiProtocol: REST\n    opendatahub.io/recommended-accelerators: '[\"nvidia.com/gpu\"]'\n    opendatahub.io/template-display-name: OpenVINO Model Server\n    opendatahub.io/template-name: ovms\n    openshift.io/display-name: multi-model-server\n  name: multi-model-server\n  labels:\n    opendatahub.io/dashboard: 'true'\nspec:\n  supportedModelFormats:\n    - autoSelect: true\n      name: openvino_ir\n      version: opset1\n    - autoSelect: true\n      name: onnx\n      version: '1'\n    - autoSelect: true\n      name: tensorflow\n      version: '2'\n  builtInAdapter:\n    env:\n      - name: OVMS_FORCE_TARGET_DEVICE\n        value: AUTO\n    memBufferBytes: 134217728\n    modelLoadingTimeoutMillis: 90000\n    runtimeManagementPort: 8888\n    serverType: ovms\n  multiModel: true\n  containers:\n    - args:\n        - '--port=8001'\n        - '--rest_port=8888'\n        - '--config_path=/models/model_config_list.json'\n        - '--file_system_poll_wait_seconds=0'\n        - '--grpc_bind_address=127.0.0.1'\n        - '--rest_bind_address=127.0.0.1'\n      image: 'quay.io/modh/openvino_model_server@sha256:5d04d405526ea4ce5b807d0cd199ccf7f71bab1228907c091e975efa770a4908'\n      name: ovms\n      resources:\n        limits:\n          cpu: '2'\n          memory: 8Gi\n        requests:\n          cpu: '1'\n          memory: 4Gi\n      volumeMounts:\n        - mountPath: /dev/shm\n          name: shm\n  protocolVersions:\n    - grpc-v1\n  grpcEndpoint: 'port:8085'\n  volumes:\n    - emptyDir:\n        medium: Memory\n        sizeLimit: 2Gi\n      name: shm\n  replicas: 1\n  tolerations: []\n  grpcDataEndpoint: 'port:8001'\n
  1. While KServe and ModelMesh share the same object definition, they have some subtle differences, in particular the annotations that are available on them. enable-route is one annotation that is available on a ModelMesh ServingRuntime that is not available on a KServe based Model Server.

The InferenceService is responsible for a definition of the model that will be deployed as well as which ServingRuntime it should use to deploy it.

apiVersion: serving.kserve.io/v1beta1\nkind: InferenceService\nmetadata:\n  annotations:\n    openshift.io/display-name: fraud-detection-model\n    serving.kserve.io/deploymentMode: ModelMesh\n  name: fraud-detection-model\n  labels:\n    opendatahub.io/dashboard: 'true'\nspec:\n  predictor:\n    model:\n      modelFormat:\n        name: onnx\n        version: '1'\n      name: ''\n      resources: {}\n      runtime: multi-model-server  # <1>\n      storage:\n        key: aws-connection-multi-model\n        path: models/fraud-detection-model/frauddetectionmodel.onnx\n
  1. The runtime must match the name of the ServingRuntime object that you wish to utilize to deploy the model.

One major difference between ModelMesh and KServe is which object is responsible for creating and managing the pod where the model is deployed.

With KServe, the ServingRuntime acts as a \"pod template\" and each InferenceService creates it's own pod to deploy a model. A ServingRuntime can be used by multiple InferenceServices and each InferenceService will create a separate pod to deploy a model.

By contrast, a ServingRuntime creates a pod with ModelMesh, and the InferenceService simply tells the model server pod what models to load and from where. With ModelMesh a single ServingRuntime with multiple InferenceServices will create a single pod to load all of the models.

"},{"location":"odh-rhoai/gitops/#argocd-health-checks","title":"ArgoCD Health Checks","text":"

Out of the box, ArgoCD and OpenShift GitOps ship with a health check for a KServe InferenceService which is not compatible with a ModelMesh InferenceService. When attempting to deploy a ModelMesh based InferenceService, ArgoCD will report the object as degraded.

Custom health checks can be added to your ArgoCD instance that are compatible with both KServe and ModelMesh as well as other RHOAI objects to resolve this issue. The Red Hat AI Services Practice maintains several custom health checks that you can utilize in your own ArgoCD instance here.

"},{"location":"odh-rhoai/gpu-pruner/","title":"GPU Pruner","text":"

In this repo, you will find the source code and usage instructions for the GPU Pruner.

In certain environments it is very easy for cluster users to request GPUs and then (either accidentally or not accidentally) not consume GPU resources. We needed a method to proactively identify this type of use, and scale down workloads that are idle from the GPU hardware perspective, compared to the default for Notebook resources which is web activity. It is totally possible for a user to consume a GPU from a pod PoV but never actually run a workload on it!

The gpu-pruner is a non-destructive idle culler that works with Red Hat OpenShift AI/Kubeflow provided APIs (InferenceService and Notebook), as well as generic Deployment, ReplicaSet and StatefulSet.

The culler politely pauses workloads that appear idle by scaling them down to 0 replicas. Features may be added in the future for better notifications, but the idea is that a user can simply re-enable the workload when they are ready to test/demo again.

It works is by querying cluster NVIDIA DCGM metrics and looking at a window of GPU utilization per pod. A scaling decision is made by looking up the pods metadata, and using owner-references to figure out the owning resource.

"},{"location":"odh-rhoai/kserve-timeout/","title":"KServe Timeout Issues","text":"

When deploying large models or when relying on node autoscaling with KServe, KServe may timeout before a model has successfully deployed due to the default progress deadline of 10 minutes set by KNative Serving.

When a pod takes longer than 10 minutes to deploy that leverages KNative Serving, like KServe does, KNative Serving will automatically back the pod deployment off and mark it as failed. This can happen for a number of reasons including deploying large models that take longer than 10m minutes to pull from S3 or if you are leveraging node autoscaling to reduce the consumption of expensive GPU nodes.

To resolve this issue, KNative supports an annotion that can be added to a KServe ServingRuntime that can be updated to set a custom progress-deadline for your application:

apiVersion: serving.kserve.io/v1alpha1\nkind: ServingRuntime\nmetadata:\n  name: my-serving-runtime\nspec:\n  annotations:\n    serving.knative.dev/progress-deadline: 30m\n

It is important to note that the annotation must be set at spec.annotations and not metadata.annotations. By setting it in spec.annotations the annotation will be copied to the KNative Service object that is created by your KServe InferenceService. The annotation on the Service will allow KNative to utilize the manually defined progress-deadline.

"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/","title":"RHOAI Metrics Dashboard for Single Serving Models","text":"

Enable RHOAI User Workload Metrics for Single Serving Models and deploy the Grafana Metrics Dashboard to monitor the performance of your Single Serving Models and the resources they consume.

"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#overview","title":"Overview","text":"

Enabling RHOAI User Workload Metrics for Single Serving Models and deploying a Grafana Metrics Dashboard provides valuable insights into the performance and resource usage of your Single Model Serving instances.

By monitoring these metrics, you can identify bottlenecks, optimize resource allocation, and ensure efficient infrastructure utilization. This enables data-driven decisions to improve the overall performance and scalability of your AI applications.

  • RHOAI Metrics Dashboard for Single Serving Models Code Repository
"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#prerequisites","title":"Prerequisites","text":"
  • OpenShift 4.10 or later
  • OpenShift AI 2.10+ installed
  • OpenShift AI KServe installed and configured
  • NVIDIA GPU Operator installed and configured
"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#installation","title":"Installation","text":"

To enable RHOAI User Workload Metrics for Single Serving Models and deploy the Grafana Metrics Dashboard, perform the following steps:

  • Configure Monitoring for the Single Model Serving Platform
  • Configure GPU Monitoring Dashboard
  • Install the RHOAI Metrics Grafana and Dashboards for Single Serving Models
"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#configure-monitoring-for-the-single-model-serving-platform","title":"Configure Monitoring for the Single Model Serving Platform","text":"

To configure monitoring for the Single Model Serving Platform, refer to the official documentation. The Single Model Serving Platform includes metrics for supported runtimes of the KServe component. KServe relies on the underlying model-serving runtimes to provide metrics and does not generate its own. The available metrics for a deployed model depend on its model-serving runtime.

Additionally, you can configure monitoring for OpenShift Service Mesh to understand dependencies and traffic flow between components in the mesh.

Once monitoring is configured for the Single Model Serving Platform, you can view the metrics in the OpenShift Web Console under the Observe Dashboards section.

"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#configure-gpu-monitoring-dashboard","title":"Configure GPU Monitoring Dashboard","text":"

To configure the GPU Monitoring Dashboard, refer to the official documentation. The GPU Monitoring Dashboard provides a comprehensive view of GPU utilization, memory usage, and other metrics for your GPU nodes.

The GPU Operator exposes GPU telemetry for Prometheus using the NVIDIA DCGM Exporter. These metrics can be visualized in the OpenShift Web Console under the Observe Dashboards section, specifically in the NVIDIA DCGM Exporter Dashboard.

Note: This step is optional but very useful for monitoring the GPU resources consumed by your Single Serving Models. If you do not enable this step, the Grafana Dashboard will not display GPU metrics.

"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#install-the-rhoai-metrics-grafana-and-dashboards-for-single-serving-models","title":"Install the RHOAI Metrics Grafana and Dashboards for Single Serving Models","text":"

To install the RHOAI Metrics Grafana Dashboards for Single Serving Models (for both vLLM and OpenVino), refer to the RHOAI UWM repository. The Grafana Dashboard provides a comprehensive view of the performance and resource utilization of Single Serving Models.

kubectl apply -k overlays/grafana-uwm-user-app\n

The RHOAI UWM Grafana Dashboard will deploy a Grafana instance with pre-configured dashboards for monitoring the performance of your Single Serving Models using the Grafana Operator.

The following dashboards are currently available:

  • vLLM Dashboard Model Metrics: Provides Model metrics for vLLM Single Serving Models.

  • vLLM Dashboard Service Performance: Provides Service Performance for vLLM Single Serving Models.

  • OpenVino Dashboard: Provides metrics for OpenVino Single Serving Models.

"},{"location":"odh-rhoai/kserve-uwm-dashboard-metrics/#conclusion","title":"Conclusion","text":"

By turning on RHOAI User Workload Metrics and setting up the Grafana Dashboard, you can easily track how your Single Serving Models are doing and what resources they're using. It helps you find problems, tweak resource use, and make better choices to keep your AI apps running smoothly.

"},{"location":"odh-rhoai/model-serving-type-modification/","title":"Model Serving Type Modification","text":""},{"location":"odh-rhoai/model-serving-type-modification/#background","title":"Background","text":"

In each Project, you can only have one type of Model Serving: either Single-model Serving, or Multi-model Serving.

This is due to the fact that the engines responsible of model serving share the same definitions (CRDs) for them. If both were enabled simultaneously in the same project, it would be impossible to know which one should handle a model serving definition.

So when you serve the first model in a project, you need to choose which type of model serving you want to use. It then becomes \"fixed\" and all further model serving will be of the chose type.

"},{"location":"odh-rhoai/model-serving-type-modification/#changing-the-serving-type","title":"Changing the Serving Type","text":""},{"location":"odh-rhoai/model-serving-type-modification/#how-this-works","title":"How this works?","text":"

The behavior is controlled by a simple label in the Namespace definition of your project: modelmesh-enabled.

  • When this label does not exist, you get the initial choice panel on the Dashboard:

  • When this label is set to true, so like this:

      labels:\n    modelmesh-enabled: 'true'\n

    Then Multi-model Serving (ModelMesh) is enabled:

  • When this label is set to false (or whatever value other than true to be honest, you can try \ud83d\ude09), like this:

      labels:\n    modelmesh-enabled: 'false'\n

    Then Single-model Serving (ModelMesh) is enabled:

"},{"location":"odh-rhoai/model-serving-type-modification/#changing-the-type","title":"Changing the type","text":"

So what to do if you have remorse at some point and want to change the serving type? The recipe is pretty simple with what we now know.

Important

Prerequisite: you need to be OpenShift Cluster admin, or find a friend who is, or have enough rights to modify the Namespace object from your project.

  • Remove all existing served models. As the type of model serving and engine will change, the definitions won't be compatible in most cases and cause lots of trouble. If you don't use the Dashboard, remove all instances of InferenceService and ServingRuntime from your project/namespace.
  • In the OpenShift Console, open the YAML definition of your Namespace (not the Project, that you can't modify!). You will find it under the Administrator perspective, under Administration->Namespaces:

  • In the YAML, modify, delete or add the label modelmesh-enabled according to the rules explained above.

  • The change will be immediately reflected in the Dashboard.
  • You can now deploy a new model with the Serving type you chose.
"},{"location":"odh-rhoai/nvidia-gpus/","title":"Working with GPUs","text":""},{"location":"odh-rhoai/nvidia-gpus/#using-nvidia-gpus-on-openshift","title":"Using NVIDIA GPUs on OpenShift","text":""},{"location":"odh-rhoai/nvidia-gpus/#how-does-this-work","title":"How does this work?","text":"

NVIDIA GPUs can be easily installed on OpenShift. Basically it involves installing two different operators.

The Node Feature Discovery operator will \"discover\" your cards from a hardware perspective and appropriately label the relevant nodes with this information.

Then the NVIDIA GPU operator will install the necessary drivers and tooling to those nodes. It will also integrate into Kubernetes so that when a Pod requires GPU resources it will be scheduled on the right node, and make sure that the containers are \"injected\" with the right drivers, configurations and tools to properly use the GPU.

So from a user perspective, the only thing you have to worry about is asking for GPU resources when defining your pods, with something like:

spec:\n  containers:\n  - name: app\n    image: ...\n    resources:\n      requests:\n        memory: \"64Mi\"\n        cpu: \"250m\"\n        nvidia.com/gpu: 2\n      limits:\n        memory: \"128Mi\"\n        cpu: \"500m\"\n

But don't worry, OpenShift AI and Open Data Hub take care of this part for you when you launch notebooks, workbenches, model servers, or pipeline runtimes!

"},{"location":"odh-rhoai/nvidia-gpus/#installation","title":"Installation","text":"

Here is the documentation you can follow:

  • OpenShift AI documentation
  • NVIDIA documentation (more detailed)
"},{"location":"odh-rhoai/nvidia-gpus/#advanced-configuration","title":"Advanced configuration","text":""},{"location":"odh-rhoai/nvidia-gpus/#working-with-taints","title":"Working with taints","text":"

In many cases, you will want to restrict access to GPUs, or be able to provide choice between different types of GPUs: simply stating \"I want a GPU\" is not enough. Also, if you want to make sure that only the Pods requiring GPUs end up on GPU-enabled nodes (and not other Pods that just end up being there at random because that's how Kubernetes works...), you're at the right place!

The only supported method at the moment to achieve this is to taint nodes, then apply tolerations on the Pods depending on where you want them scheduled. If you don't pay close attention though when applying taints on Nodes, you may end up with the NVIDIA drivers not installed on those nodes...

In this case you must:

  • Apply the taints you need to your Nodes or MachineSets, for example:

    apiVersion: machine.openshift.io/v1beta1\nkind: MachineSet\nmetadata:\n  ...\nspec:\n  replicas: 1\n  selector:\n    ...\n  template:\n    ...\n    spec:\n      ...\n      taints:\n        - key: restrictedaccess\n          value: \"yes\"\n          effect: NoSchedule\n
  • Apply the relevant toleration to the NVIDIA Operator.

    • In the nvidia-gpu-operator namespace, get to the Installed Operator menu, open the NVIDIA GPU Operator settings, get to the ClusterPolicy tab, and edit the ClusterPolicy.

    • Edit the YAML, and add the toleration in the daemonset section:

      apiVersion: nvidia.com/v1\nkind: ClusterPolicy\nmetadata:\n  ...\n  name: gpu-cluster-policy\nspec:\n  vgpuDeviceManager: ...\n  migManager: ...\n  operator: ...\n  dcgm: ...\n  gfd: ...\n  dcgmExporter: ...\n  cdi: ...\n  driver: ...\n  devicePlugin: ...\n  mig: ...\n  sandboxDevicePlugin: ...\n  validator: ...\n  nodeStatusExporter: ...\n  daemonsets:\n    ...\n    tolerations:\n      - effect: NoSchedule\n        key: restrictedaccess\n        operator: Exists\n  sandboxWorkloads: ...\n  gds: ...\n  vgpuManager: ...\n  vfioManager: ...\n  toolkit: ...\n...\n

That's it, the operator is now able to deploy all the NVIDIA tooling on the nodes, even if they have the restrictedaccess taint. Repeat the procedure for any other taint you want to apply to your nodes.

Note

The first taint that you want to apply on GPU nodes is nvidia.com/gpu. This is the standard taint for which the NVIDIA Operator has a built-in toleration, so no need to add it. Likewise, Notebooks, Workbenches or other components from ODH/RHOAI that request GPUs will already have this toleration in place. For other Pods you schedule yourself, or using Pipelines, you should make sure the toleration is also applied. Doing this will ensure that only Pods really requiring GPUs are scheduled on those nodes.

You can of course apply many different taints at the same time. You would simply have to apply the matching toleration on the NVIDIA GPU Operator, as well as on the Pods that need to run there.

"},{"location":"odh-rhoai/nvidia-gpus/#autoscaler-and-gpus","title":"Autoscaler and GPUs","text":"

As they are expensive, GPUs are good candidates to put behind an Autoscaler. But due to this there are some subtleties if you want everything to go smoothly.

"},{"location":"odh-rhoai/nvidia-gpus/#configuration","title":"Configuration","text":"

Warning

For the autoscaler to work properly with GPUs, you have to set a specific label to the MachineSet. It will help to Autoscaler figure out (in fact simulate) what it is allowed to do. This is especially true if you have different MachineSets that feature different types of GPUs.

As per the referenced article above, the type for gpus you set through the label cannot be nvidia.com/gpu (as you will sometimes find in the standard documentation), because it's not a valid label. Therefore, only for the autoscaling purpose, you should give the type a specific name with letters, numbers and dashes only, like Tesla-T4-SHARED in this example.

  • Edit the MachineSet configuration to add the label that the Autoscaler will expect:

    apiVersion: machine.openshift.io/v1beta1\nkind: MachineSet\n...\nspec:\n  ...\n  template:\n    ...\n    spec:\n      metadata:\n        labels:\n          cluster-api/accelerator: Tesla-T4-SHARED\n
  • Create your ClusterAutoscaler configuration (example):

    apiVersion: autoscaling.openshift.io/v1\nkind: ClusterAutoscaler\nmetadata:\n  name: \"default\"\nspec:\n  logVerbosity: 4\n  maxNodeProvisionTime: 15m\n  podPriorityThreshold: -10\n  resourceLimits:\n    gpus:\n      - type: Tesla-T4-SHARED\n        min: 0\n        max: 8\n  scaleDown:\n    enabled: true\n    delayAfterAdd: 20m\n    delayAfterDelete: 5m\n    delayAfterFailure: 30s\n    unneededTime: 5m\n

    Note

    The delayAfterAdd parameter has to be set higher than standard value as NVIDIA tooling can take a lot of time to deploy, 10-15mn.

  • Create the MachineSet Autoscaler:

    apiVersion: autoscaling.openshift.io/v1beta1\nkind: MachineAutoscaler\nmetadata:\n  name: machineset-name\n  namespace: \"openshift-machine-api\"\nspec:\n  minReplicas: 1\n  maxReplicas: 2\n  scaleTargetRef:\n    apiVersion: machine.openshift.io/v1beta1\n    kind: MachineSet\n    name: machineset-name\n
"},{"location":"odh-rhoai/nvidia-gpus/#scaling-to-zero","title":"Scaling to zero","text":"

As GPUs are expensive resources, you may want to scale down your MachineSet to zero to save on resources. This will however require some more configuration than just setting the minimum size to zero...

First, some background to help you understand and enable you to solve issues that may arise. You can skip the whole explanation, but it's worth it, so please bear with me.

When you request resources that aren't available, the Autoscaler looks at all the MachineAutoscalers that are available, with their corresponding MachineSets. But how to know which one to use? Well, it will first simulate the provisioning of a Node from each MachineSet, and see if it would fit the request. Of course, if there is already at least one Node available from a given MachineSet, the simulation would be bypassed as the Autoscaler already knows what it will get. If there are different MachineSets that fit and to choose from, the default and only \"Expander\" available for now in OpenShift to make its decision is random. So it will simply picks one totally randomly.

That's all perfect and everything, but for GPUs, if you don't start the Node for real, we don't know what's in it! So that's where we have to help the Autoscaler with a small hint.

  • Set this annotation manually if it's not there. It will stick after the first scale up though, along with some other annotations the Autoscaler will add, thanks for its newly discovered knowledge.

    apiVersion: machine.openshift.io/v1beta1\nkind: MachineSet\nmetadata:\n  annotations:\n    machine.openshift.io/GPU: \"1\"\n

Now to the other issue that may happen if you are in an environment with multiple Availability Zones (AZ)...

Although when you define a MachineSet you can set the AZ and have all the Nodes spawned properly in it, the Autoscaler simulator is not that clever. So it will simply pick a Zone at random. If this is not the one where you want/need your Pod to run, this will be a problem...

For example, you may already have a Persistent Volume (PV) attached to you Notebook. If your storage does now support AZ-spanning (like AWS EBS volumes), your PV is bound to a specific AZ. If the Simulator creates a virtual Node in a different AZ, there will be a mismatch, your Pod would not be schedulable on this Node, and the Autoscaler will (wrongly) conclude that it cannot use this MachineSet for a scale up!

Here again, we have to give a hint to the Autoscaler to what the Node will look like in the end.

  • In you MachineSet, in the labels that will be added to the node, add information regarding the topology of the Node, as well as for the volumes that may be attached to it. For example:

    apiVersion: machine.openshift.io/v1beta1\nkind: MachineSet\nmetadata:\nspec:\n  template:\n    spec:\n      metadata:\n        labels:\n          ...\n          topology.kubernetes.io/zone: us-east-2a\n          topology.ebs.csi.aws.com/zone: us-east-2a\n

With this, the simulated Node will be at the right place, and the Autoscaler will consider the MachineSet valid for scale up!

Reference material:

  • https://cloud.redhat.com/blog/autoscaling-nvidia-gpus-on-red-hat-openshift
  • https://access.redhat.com/solutions/6055181
  • https://bugzilla.redhat.com/show_bug.cgi?id=1943194
"},{"location":"odh-rhoai/nvidia-gpus/#gpu-partitioning-sharing","title":"GPU Partitioning / Sharing","text":"

There are also situations where the GPU(s) you have access to might be oversized for the task at hand, and having a single user or process lock-up and \"hog\" that GPU can be inefficient. There are thankfully some partitioning strategies that can be brought to bear in order to deal with these situations. Although there are multiple techniques, with various pros and cons, the net effect of these implementations is that what used to look like a single GPU will then look like multiple GPUs. Obviously, there is no magic in the process, and the laws of physics still hold: there are trade-offs, and the multiple \"partitioned\" GPUs are not going to be faster or crunch more data than the real underlying physical GPU.

If this is a situation that you are facing, consult this repository for more detailed information and examples.

"},{"location":"odh-rhoai/nvidia-gpus/#time-slicing-gpu-sharing","title":"Time Slicing (GPU sharing)","text":"

Do you want to share GPUs between different Pods? Time Slicing is one of the solutions you can use!

The NVIDIA GPU Operator enables oversubscription of GPUs through a set of extended options for the NVIDIA Kubernetes Device Plugin. GPU time-slicing enables workloads that are scheduled on oversubscribed GPUs to interleave with one another.

This mechanism for enabling time-slicing of GPUs in Kubernetes enables a system administrator to define a set of replicas for a GPU, each of which can be handed out independently to a pod to run workloads on. Unlike Multi-Instance GPU (MIG), there is no memory or fault-isolation between replicas, but for some workloads this is better than not being able to share at all. Internally, GPU time-slicing is used to multiplex workloads from replicas of the same underlying GPU.

  • Time Slicing Full reference
  • Time Slicing Example Repository
"},{"location":"odh-rhoai/nvidia-gpus/#configuration_1","title":"Configuration","text":"

This is a simple example on how to quickly setup Time Slicing on your OpenShift cluster. In this example, we have a MachineSet that can provide nodes with one T4 card each that we want to make \"seen\" as 4 different cards so that multiple Pods requiring GPUs can be launched, even if we only have one node of this type.

  • Create the ConfigMap that will define how we want to slice our GPU:

    kind: ConfigMap\napiVersion: v1\nmetadata:\n  name: time-slicing-config\n  namespace: nvidia-gpu-operator\ndata:\n  tesla-t4: |-\n    version: v1\n    sharing:\n      timeSlicing:\n        resources:\n        - name: nvidia.com/gpu\n          replicas: 4\n

    NOTE - The ConfigMap has to be called time-slicing-config and must be created in the nvidia-gpu-operator namespace. - You can add many different resources with different configurations. You simply have to provide the corresponding Node label that has been applied by the operator, for example name: nvidia.com/mig-1g.5gb / replicas: 2 if you have a MIG configuration applied to a Node with a A100. - You can modify the value of replicas to present less/more GPUs. Be warned though: all the Pods on this node will share the GPU memory, with no reservation. The more slices you create, the more risks of OOM errors (out of memory) you get if your Pods are hungry (or even only one!).

  • Modify the ClusterPolicy called gpu-cluster-policy (accessible from the NVIDIA Operator view in the nvidia-gpu-operator namespace) to point to this configuration, and eventually add the default configuration (in case you nodes are not labelled correctly, see below)

    apiVersion: nvidia.com/v1\nkind: ClusterPolicy\nmetadata:\n  ...\n  name: gpu-cluster-policy\nspec:\n  ...\n  devicePlugin:\n    config:\n      default: tesla-t4\n      name: time-slicing-config\n  ...\n
  • Apply label to your MachineSet for the specific slicing configuration you want to use on it:

    apiVersion: machine.openshift.io/v1beta1\nkind: MachineSet\nmetadata:\nspec:\n  template:\n    spec:\n      metadata:\n        labels:\n          nvidia.com/device-plugin.config: tesla-t4\n
"},{"location":"odh-rhoai/nvidia-gpus/#multi-instance-gpu-mig","title":"Multi-Instance GPU (MIG)","text":"

Multi-Instance GPU (MIG) enables a single physical GPU to be partitioned into several isolated instances, each with its own compute resources, memory, and performance profiles.

There are two types of MIG strategies: Single and Mixed. The single MIG strategy should be utilized when all GPUs on a node have MIG enabled, while the Mixed MIG strategy should be utilized when not all GPUs on a node have MIG enabled.

NOTE: MIG is only supported with the following NVIDIA GPU Types - A30, A100, A100X, A800, AX800, H100, H200, and H800.

  • MIG Full reference
  • MIG Single Example Repository
  • MIG Mixed Example Repository
"},{"location":"odh-rhoai/nvidia-gpus/#multi-process-service-mps","title":"Multi-Process Service (MPS)","text":"

Multi-Process Service (MPS) facilitates concurrent sharing of a single GPU among multiple CUDA applications.

MPS is an alternative, binary-compatible implementation of the CUDA Application Programming Interface (API). The MPS runtime architecture is designed to transparently enable co-operative multi-process CUDA applications.

  • MPS Full reference
  • MPS Example Repository

NOTE: Despite the tests passing, MPS isn't working correctly on OpenShift currently, due to only one process per GPU can run at any time. RH and NVIDIA engineers are working to fix this issue as soon as possible.

"},{"location":"odh-rhoai/nvidia-gpus/#aggregating-gpus-multi-gpu","title":"Aggregating GPUs (Multi-GPU)","text":"

Some Large Language Models (LLMs), such as Llama-3-70B and Falcon 180B, can be too large to fit into the memory of a single GPU (vRAM). Or in some cases, GPUs that would be large-enough might be difficult to obtain. If you find yourself in such a situation, it is natural to wonder whether an aggregation of multiple, smaller GPUs can be used instead of one single large GPU.

Thankfully, the answer is essentially Yes. To address these challenges, we can use more advanced configurations to distribute the LLM workload across several GPUs. One option is leveraging tensor parallelism, where the LLM is split across several GPUs, with each GPU processing a portion of the model's tensors. This approach ensures efficient utilization of available resources (GPUs) across one or several workers.

Some Serving Runtimes, such as vLLM, support tensor parallelism, allowing for both single-worker and multi-worker configurations (the difference whether your GPUs are all in the same machine, or are spread across machines).

vLLM has been added as an Out-of-the-box serving runtime, starting with Red Hat OpenShift AI version 2.10 link to our RHOAI doc

For a detailed guide on implementing these solutions, refer to our repository.

  • Single Worker Node - Multiple GPUs Example Repository
  • Multiple Worker Node - Multiple GPUs Example Repository
"},{"location":"odh-rhoai/odh-tools-and-extensions-companion/","title":"ODH Tools and Extensions Companion","text":"

In this repo, you will find the source code and usage instructions for the Open Data Hub Tools & Extensions Companion.

This application implements tools that can be useful to anyone working with Open Data Hub (ODH), Red Hat OpenShift AI (RHOAI), or even locally with Podman.

"},{"location":"odh-rhoai/odh-tools-and-extensions-companion/#current-features","title":"Current Features","text":""},{"location":"odh-rhoai/odh-tools-and-extensions-companion/#s3-tools","title":"S3 Tools","text":"
  • S3 Buckets Management: browsing, creation, deletion
  • S3 Objects Browser:
    • Single file upload, Multiple files uploads, Downloads
    • File preview
    • Model import from HuggingFace
"},{"location":"odh-rhoai/odh-tools-and-extensions-companion/#gpu-tools","title":"GPU Tools","text":"
  • VRAM Estimator: helps you calculate the VRAM and number of GPUs you need for inference and training.
"},{"location":"odh-rhoai/odh-tools-and-extensions-companion/#settings-and-validations","title":"Settings and validations","text":"
  • S3 connection testing
  • HuggingFace authentication testing
"},{"location":"odh-rhoai/openshift-group-management/","title":"OpenShift Group Management","text":"

In the Red Hat OpenShift Documentation, there are instructions on how to configure a specific list of RHOAI Administrators and RHOAI Users.

However, if the list of users keeps changing, the membership of the groupe called rhods-users will have to be updated frequently. By default, in OpenShift, only OpenShift admins can edit group membership. Being a RHOAI Admin does not confer you those admin privileges, and so, it would fall to the OpenShift admin to administer that list.

The instructions in this page will show how the OpenShift Admin can create these groups in such a way that any member of the group rhods-admins can edit the users listed in the group rhods-users. These makes the RHOAI Admins more self-sufficient, without giving them unneeded access.

For expediency in the instructions, we are using the oc cli, but these can also be achieved using the OpenShift Web Console. We will assume that the user setting this up has admin privileges to the cluster.

"},{"location":"odh-rhoai/openshift-group-management/#creating-the-groups","title":"Creating the groups","text":"

Here, we will create the groups mentioned above. Note that you can alter those names if you want, but will then need to have the same alterations throughout the instructions.

  1. To create the groups:
    oc adm groups new rhods-users\noc adm groups new rhods-admins\n
  2. The above may complain about the group(s) already existing.
  3. To confirm both groups exist:
    oc get groups | grep rhods\n
  4. That should return:
    bash-4.4 ~ $ oc get groups | grep rhods\nrhods-admins\nrhods-users\n
  5. Both groups now exist
"},{"location":"odh-rhoai/openshift-group-management/#creating-clusterrole-and-clusterrolebinding","title":"Creating ClusterRole and ClusterRoleBinding","text":"
  1. This will create a Cluster Role and a Cluster Role Binding:
    oc apply -f - <<EOF\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  name: update-rhods-users\nrules:\n  - apiGroups: [\"user.openshift.io\"]\n    resources: [\"groups\"]\n    resourceNames: [\"rhods-users\"]\n    verbs: [\"update\", \"patch\", \"get\"]\n---\nkind: ClusterRoleBinding\napiVersion: rbac.authorization.k8s.io/v1\nmetadata:\n  name: rhods-admin-can-update-rhods-users\nsubjects:\n  - kind: Group\n    apiGroup: rbac.authorization.k8s.io\n    name: rhods-admins\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: update-rhods-users\nEOF\n
  2. To confirm they were both succesfully created, run:
    oc get ClusterRole,ClusterRoleBinding  | grep 'update\\-rhods'\n
  3. You should see:
    bash-4.4 ~ $ oc get ClusterRole,ClusterRoleBinding  | grep 'update\\-rhods'\nclusterrole.rbac.authorization.k8s.io/update-rhods-users\nclusterrolebinding.rbac.authorization.k8s.io/rhods-admin-can-update-rhods-users\n
  4. You are pretty much done. You now just need to validate things worked.
"},{"location":"odh-rhoai/openshift-group-management/#add-some-users-as-rhods-admins","title":"Add some users as rhods-admins","text":"

To confirm this works, add a user to the rhods-admin group. In my example, I'll add user1

"},{"location":"odh-rhoai/openshift-group-management/#capture-the-url-needed-to-edit-the-rhods-users-group","title":"Capture the URL needed to edit the rhods-users group","text":"

Since people who are not cluster admin won't be able to browse the list of groups, capture the URL that allows to control the membership of rhods-users.

It should look similar to:

https://console-openshift-console.apps.<thecluster>/k8s/cluster/user.openshift.io~v1~Group/rhods-users

"},{"location":"odh-rhoai/openshift-group-management/#ensure-that-rhods-admins-are-now-able-to-edit-rhods-users","title":"Ensure that rhods-admins are now able to edit rhods-users","text":"

Ask someone in the rhods-admins group to confirm that it works for them. (Remember to provide them with the URL to do so).

They should be able to do so and successfully save their changes, as shown below:

"},{"location":"odh-rhoai/openshift-group-management/#adding-kubeadmin-to-rhods-admins","title":"Adding kube:admin to rhods-admins","text":"

To add kube:admin user to the list of RHOAI Administrators, you will need to prefix it with b64:

apiVersion: user.openshift.io/v1\nkind: Group\nmetadata:\n  name: rhods-admins\nusers:\n- b64:kube:admin\n
"},{"location":"odh-rhoai/single-stack-serving-certificate/","title":"Use Existing OpenShift Certificate for Single Stack Serving","text":"

By default, the Single Stack Serving in Openshift AI uses a self-signed certificate generated at installation for the endpoints that are created when deploying a server. This can be counter-intuitive because if you already have certificates configured on your OpenShift cluster, they will be used by default for other types of endpoints like Routes.

The installation procedure for the Single Stack Serving available here (section 4.vi) states that you can provide your own certificate instead of using a self-signed one.

This following procedure explains how to use the same certificate that you already have for your OpenShift cluster.

"},{"location":"odh-rhoai/single-stack-serving-certificate/#procedure","title":"Procedure","text":"
  • Configure OpenShift to use a valid certificate for accessing the Console and in general for any created Route (normally, this is something that was already done).
  • From the openshift-ingress namespace, copy the content of a Secret whose name includes \"certs\". For example rhods-internal-primary-cert-bundle-secret or ingress-certs-..... The content of the Secret (data) should contain two items, tls.cert and tls.key. They are the certificate and key that are used for all the OpenShift Routes.
  • Cleanup the YAML to just keep the relevant content. It should look like this (the name of the secret will be different, it's normally tied to your cluster name):

    kind: Secret\napiVersion: v1\nmetadata:\nname: rhods-internal-primary-cert-bundle-secret\ndata:\ntls.crt: >-\n    LS0tLS1CRUd...\ntls.key: >-\n    LS0tLS1CRUd...\ntype: kubernetes.io/tls\n
  • Apply this YAML into the istio-system namespace (basically, copy the Secret from one namespace to the other).

  • Following the Single Stack Serving installation procedure, in your DSC Configuration, refer to this secret for the kserve configuration (don\u2019t forget to change the secretName for the one you just created):

    kserve:\ndevFlags: {}\nmanagementState: Managed\nserving:\n    ingressGateway:\n    certificate:\n        secretName: rhods-internal-primary-cert-bundle-secret\n        type: Provided\n    managementState: Managed\n    name: knative-serving\n

Your Model Servers will now be deployed with the same certificate as you are using for OpenShift Routes. If this is a trusted certificate, your Endpoints will be accessible using SSL without having to ignore error messages or create special configurations.

"},{"location":"odh-rhoai/single-stack-serving-certificate/#other-workarounds","title":"Other Workarounds","text":"

If the above method does not work or you don't want or can't do any modification, you can try to bypass the certificate verification in your application or code. Depending on the library used, there are different solutions.

"},{"location":"odh-rhoai/single-stack-serving-certificate/#using-langchain-with-openai-api-compatible-runtimes","title":"Using Langchain with OpenAI API compatible runtimes","text":"

The underlying library used for communication by the base OpenAI module of Langchain is httpx. You can directly specify httpx settings when you instantiate the llm object in Langchain. With the following settings on the last two lines of this example, any certificate error will be ignored:

import httpx\n# LLM definition\nllm = VLLMOpenAI(\n    openai_api_key=\"EMPTY\",\n    openai_api_base= f\"{inference_server_url}/v1\",\n    model_name=\"/mnt/models/\",\n    top_p=0.92,\n    temperature=0.01,\n    max_tokens=512,\n    presence_penalty=1.03,\n    streaming=True,\n    callbacks=[StreamingStdOutCallbackHandler()]\n    async_client=httpx.AsyncClient(verify=False),\n    http_client=httpx.Client(verify=False)\n)\n
"},{"location":"odh-rhoai/from-zero-to-workbench/using-cli/","title":"From Zero To Workbench using the CLI","text":"

In this repo, you will find all the straightforward instructions to quickly deploy OpenShift AI only using a CLI or automation.

Detailed instructions, YAMLs, code, all is there to get you quickly started!

Note

This documentation was produced working off of mainly OpenShift v4.14 and OpenShift AI 2.7. The artifacts directory contains sample configuration files which are used throughout this repo to demonstrate various concepts. While an effort will be made to ensure that these artifacts stay up to date, there is a possibility that they will not always work as intended.

Suggestions and pull requests are welcome to maintain this content up to date!

"},{"location":"odh-rhoai/from-zero-to-workbench/using-developer-hub/","title":"OpenShift AI on Developer Hub","text":"

In this repo, you will find a Backstage Golden Path Template for a Red Hat OpenShift AI Data science project. Detailed instructions, examples, all is there to get you quickly started!

Note

This documentation is a work in progress that provides recipes for some components only.

Suggestions and pull requests are welcome to maintain this content up to date and make it evolve!

"},{"location":"odh-rhoai/from-zero-to-workbench/using-ui/","title":"From Zero To Workbench using the GUI","text":"

In this repo, you will find all the straightforward instructions to quickly deploy OpenShift AI using mainly the OpenShift Console UI.

Detailed instructions, screenshots, examples, all is there to get you quickly started!

Note

This documentation was produced working off of mainly OpenShift v4.14 and OpenShift AI 2.7. The artifacts directory contains sample configuration files which are used throughout this repo to demonstrate various concepts. While an effort will be made to ensure that these artifacts stay up to date, there is a possibility that they will not always work as intended.

Suggestions and pull requests are welcome to maintain this content up to date!

"},{"location":"odh-rhoai/kueue-preemption/readme/","title":"Introduction","text":"

In this repo, we will demostrate how to use quota allocation with Kueue with preemption.

"},{"location":"odh-rhoai/kueue-preemption/readme/#overview","title":"Overview","text":"

In this example, there are 2 teams that work in their own namespace:

  1. Team A and B belongs to the same cohort
  2. Both teams share a quota
  3. Team A has access to GPU while team B does not
  4. Team A has higher priority and can prempt others
"},{"location":"odh-rhoai/kueue-preemption/readme/#kueue-configuration","title":"Kueue Configuration","text":"

There are 2 ResourceFlavor that manages the CPU/Memory and GPU resources. The GPU ResourceFlavor tolerates nodes that have been tainted.

Both teams have their invididual cluster queue that is associated with their respective namespace.

Name CPU Memory (GB) GPU Team A cq 0 0 4 Team B cq 0 0 0 Shared cq 10 64 0

A local queue is defined in their namespace to associate the cluster queue. E.g.

apiVersion: kueue.x-k8s.io/v1beta1\nkind: LocalQueue\nmetadata:\n  name: local-queue\n  namespace: team-a\nspec:\n  clusterQueue: team-a-cq\n

When a Ray cluster is defined, it is submitted to the local queue with the associated priority.

apiVersion: ray.io/v1\nkind: RayCluster\nmetadata:\n  labels:  \n    kueue.x-k8s.io/queue-name: local-queue\n    kueue.x-k8s.io/priority-class: dev-priority\n
"},{"location":"odh-rhoai/kueue-preemption/readme/#ray-cluster-configuration","title":"Ray cluster configuration","text":"

The shared quota is only up to 10 CPU for both teams.

Name CPU Memory (GB) GPU Team A 10 24 4 Team B 6 16 0"},{"location":"odh-rhoai/kueue-preemption/readme/#premption","title":"Premption","text":"

Team A cluster queue has preemption defined that can borrowWithinCohort of a lower priority which Team B belongs to.

apiVersion: kueue.x-k8s.io/v1beta1\nkind: ClusterQueue\nmetadata:\n  name: team-a-cq\nspec:\n  preemption:\n    reclaimWithinCohort: Any\n    borrowWithinCohort:\n      policy: LowerPriority\n      maxPriorityThreshold: 100\n    withinClusterQueue: Never\n

Team A will preempt team B because it has insufficient resources to run.

"},{"location":"odh-rhoai/kueue-preemption/readme/#setting-up-the-demo","title":"Setting Up the Demo","text":"
  1. Install OpenShift AI Operator

  2. Ensure there is at least 1 worker node that has a 4 GPUs. On AWS, this can be a p3.8xlarge instance.

  3. Taint the GPU node

      oc adm taint nodes <gpu-node> nvidia.com/gpu=Exists:NoSchedule\n

  4. Git clone the repo

    git clone https://github.com/opendatahub-io-contrib/ai-on-openshift\ncd ai-on-openshift/docs/odh-rhoai/kueue-preemption\n
  5. Run the makefile target to setup the example. This will setup 2 namespaces: team-a and team-b.

    make setup-kueue-examples\n

To teardown the example, you can use:

make teardown-kueue-preemption\n

Warning

The setup script will delete all clusterqueues and resourceflavors in the cluster.

"},{"location":"odh-rhoai/kueue-preemption/readme/#running-the-example","title":"Running the example","text":"
  1. Create a ray cluster for team B. Wait for the cluster to be running.

    oc create -f team-b-ray-cluster-dev.yaml\n

    $ oc get rayclusters -A\nNAMESPACE   NAME             DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS   AGE\nteam-b      raycluster-dev   2                 2                   6      16G      0      ready    70s\n\n$ oc get po -n team-b\nNAME                                           READY   STATUS    RESTARTS   AGE\nraycluster-dev-head-zwfd8                      2/2     Running   0          45s\nraycluster-dev-worker-small-group-test-4c85h   1/1     Running   0          43s\nraycluster-dev-worker-small-group-test-5k9j5   1/1     Running   0          43s\n
  2. Create a Ray cluster for team A.

    oc create -f team-a-ray-cluster-prod.yaml\n

  3. Observe team B cluster is suspended and team A cluster is running because of preemption. This may take a few seconds to happen.

    $ oc get rayclusters -A\nNAMESPACE   NAME              DESIRED WORKERS   AVAILABLE WORKERS   CPUS   MEMORY   GPUS   STATUS      AGE\nteam-a      raycluster-prod   2                 2                   10     24G      4      ready       75s\nteam-b      raycluster-dev    2                                     6      16G      0      suspended   3m46s\n
"},{"location":"patterns/bucket-notifications/bucket-notifications/","title":"Bucket Notifications","text":""},{"location":"patterns/bucket-notifications/bucket-notifications/#description","title":"Description","text":"

The Rados Gateway (RGW) component of Ceph provides Object Storage through an S3-compatible API on all Ceph implementations: OpenShift Data Foundation and its upstream version Rook-Ceph, Red Hat Ceph Storage, Ceph,\u2026\u200b

Bucket notifications provide a mechanism for sending information from the RGW when certain events are happening on a bucket. Currently, notifications can be sent to: HTTP, AMQP0.9.1 and Kafka endpoints.

From a data engineering point of view, bucket notifications allow to create an event-driven architecture, where messages (instead of simply log entries) can be sent to various processing components or event buses whenever something is happening on the object storage: object creation, deletion, with many fine-grained settings available.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#use-cases","title":"Use cases","text":""},{"location":"patterns/bucket-notifications/bucket-notifications/#application-taking-actions-on-the-objects","title":"Application taking actions on the objects","text":"

As part of an event-driven architecture, this pattern can be used to trigger an application to perform an action following the storage event. An example could be the automated processing of a new image that has just been uploaded to the object storage (analysis, resizing,\u2026\u200b). Paired with Serverless functions this becomes a pretty efficient architecture compared to having an application constantly monitoring or polling the storage, or to have to implement this triggering process in the application interacting with the storage. This loosely-coupled architecture also gives much more agility for updates, technology evolution,\u2026\u200b

"},{"location":"patterns/bucket-notifications/bucket-notifications/#external-monitoring-systems","title":"External monitoring systems","text":"

The events sent by the RGW are simple messages containing all the metadata relevant to the event and the object. So it can be an excellent source of information for a monitoring system. For example if you want to keep a trace or send an alert whenever a specific type of file, or with a specific name, is uploaded or deleted from the storage.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#implementations-examples","title":"Implementations examples","text":"

This pattern is implemented in the XRay pipeline demo

"},{"location":"patterns/bucket-notifications/bucket-notifications/#how-does-it-work","title":"How does it work?","text":""},{"location":"patterns/bucket-notifications/bucket-notifications/#characteristics","title":"Characteristics","text":"
  • Notifications are sent directly from the RGW on which the event happened to an external endpoint.
  • Pluggable endpoint architecture:
    • HTTP/S
    • AMQP 0.9.1
    • Kafka
    • Knative
"},{"location":"patterns/bucket-notifications/bucket-notifications/#data-model","title":"Data Model","text":"
  • Topics contain the definition of a specific endpoint in \u201cpush mode\u201d
  • Notifications tie topics with buckets, and may also include filter definition on the events
"},{"location":"patterns/bucket-notifications/bucket-notifications/#configuration","title":"Configuration","text":"

This configuration shows how to create a notification that will send a message (event) to a Kafka topic when a new object is created in a bucket.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#requirements","title":"Requirements","text":"
  • Access to a Ceph/ODF/RHCS installation with the RGW deployed.
  • Endpoint address (URL) for the RGW.
  • Credentials to connect to the RGW:
    • AWS_ACCESS_KEY_ID
    • AWS_SECRET_ACCESS_KEY

Note

As Ceph implements an S3-Compatible API to access Object Storage, standard naming for variables or procedures used with S3 were retained to stay coherent with examples, demos or documentation related to S3. Therefore the AWS prefix in the previous variables.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#topic-creation","title":"Topic Creation","text":"

A topic is the definition of a specific endpoint. It must be created first.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#method-1-raw-configuration","title":"Method 1: \"RAW\" configuration","text":"

As everything is done through the RGW API, you can query it directly. To be fair, this method is almost never used (unless there is no SDK or S3 tool for your environment) but gives a good understanding of the process.

Example for a Kafka Endpoint:

POST\nAction=CreateTopic\n&Name=my-topic\n&push-endpoint=kafka://my-kafka-broker.my-net:9999\n&Attributes.entry.1.key=verify-ssl\n&Attributes.entry.1.value=true\n&Attributes.entry.2.key=kafka-ack-level\n&Attributes.entry.2.value=broker\n&Attributes.entry.3.key=use-ssl\n&Attributes.entry.3.value=true\n&Attributes.entry.4.key=OpaqueData\n&Attributes.entry.4.value=https://s3-proxy.my-zone.my-net\n

Note

The authentication part is not detailed here as the mechanism is pretty convoluted, but it is directly implemented in most API development tools, like Postman.

The full reference for the REST API for bucket notifications is available here.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#method-2-python-aws-sdk","title":"Method 2: Python + AWS SDK","text":"

As the creator of the S3 API, AWS is providing SDKs for the main languages to interact with it. Thanks to this compatibility, you can use those SDKs to interact with Ceph in the same way. For Python, the library to interact with AWS services is called boto3.

Example for a Kafka Endpoint:

import boto3\nsns = boto3.client('sns',\n                endpoint_url = endpoint_url,\n                aws_access_key_id = aws_access_key_id,\n                aws_secret_access_key= aws_secret_access_key,\n                region_name='default',\n                config=botocore.client.Config(signature_version = 's3'))\n\nattributes = {}\nattributes['push-endpoint'] = 'kafka://my-cluster-kafka-bootstrap:9092'\nattributes['kafka-ack-level'] = 'broker'\n\ntopic_arn = sns.create_topic(Name=my-topic, Attributes=attributes)['TopicArn']\n
"},{"location":"patterns/bucket-notifications/bucket-notifications/#notification-configuration","title":"Notification Configuration","text":"

The notification configuration will \"tie\" a bucket with a topic.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#method-1-raw-configuration_1","title":"Method 1: \"RAW\" configuration","text":"

As previously, you can directly query the RGW REST API. This is done through an XML formatted payload that is sent with a PUT command.

Example for a Kafka Endpoint:

PUT /my-bucket?notification HTTP/1.1\n\n<NotificationConfiguration xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\">\n    <TopicConfiguration>\n        <Id>my-notification</Id>\n        <Topic>my-topic</Topic>\n        <Event>s3:ObjectCreated:*</Event>\n        <Event>s3:ObjectRemoved:DeleteMarkerCreated</Event>\n    </TopicConfiguration>\n    <TopicConfiguration>\n...\n    </TopicConfiguration>\n</NotificationConfiguration>\n

Again, the full reference for the REST API for bucket notifications is available here.

"},{"location":"patterns/bucket-notifications/bucket-notifications/#method-2-python-aws-sdk_1","title":"Method 2: Python + AWS SDK","text":"

Example for a Kafka Endpoint:

import boto3\ns3 = boto3.client('s3',\n                endpoint_url = endpoint_url,\n                aws_access_key_id = aws_access_key_id,\n                aws_secret_access_key = aws_secret_access_key,\n                region_name = 'default',\n                config=botocore.client.Config(signature_version = 's3'))\n\nbucket_notifications_configuration = {\n            \"TopicConfigurations\": [\n                {\n                    \"Id\": 'my-id',\n                    \"TopicArn\": 'arn:aws:sns:s3a::my-topic',\n                    \"Events\": [\"s3:ObjectCreated:*\"]\n                }\n            ]\n        }\n\ns3.put_bucket_notification_configuration(Bucket = bucket_name,\n        NotificationConfiguration=bucket_notifications_configuration)\n
"},{"location":"patterns/bucket-notifications/bucket-notifications/#filters","title":"Filters","text":"

Although a notification is specific to a bucket (and you can have multiple configurations on one bucket), you may want that it does not apply to all the objects from this bucket. For example you want to send an event when an image is uploaded, but not do anything it\u2019s another type of file. You can do this with filters! And not only on the filename, but also on the tags associated to it in its metadata.

Filter examples, on keys or tags:

<Filter>\n    <S3Key>\n        <FilterRule>\n         <Name>regex</Name>\n         <Value>([0-9a-zA-Z\\._-]+.(png|gif|jp[e]?g)</Value>\n        </FilterRule>\n    </S3Key>\n    <S3Tags>\n        <FilterRule>\n            <Name>Project</Name><Value>Blue</Value>\n        </FilterRule>\n        <FilterRule>\n            <Name>Classification</Name><Value>Confidential</Value>\n        </FilterRule>\n    </S3Tags>\n</Filter>\n
"},{"location":"patterns/bucket-notifications/bucket-notifications/#events","title":"Events","text":"

The notifications sent to the endpoints are called events, and they are structured like this:

Event example:

{\"Records\":[\n    {\n        \"eventVersion\":\"2.1\",\n        \"eventSource\":\"ceph:s3\",\n        \"awsRegion\":\"us-east-1\",\n        \"eventTime\":\"2019-11-22T13:47:35.124724Z\",\n        \"eventName\":\"ObjectCreated:Put\",\n        \"userIdentity\":{\n            \"principalId\":\"tester\"\n        },\n        \"requestParameters\":{\n            \"sourceIPAddress\":\"\"\n        },\n        \"responseElements\":{\n            \"x-amz-request-id\":\"503a4c37-85eb-47cd-8681-2817e80b4281.5330.903595\",\n            \"x-amz-id-2\":\"14d2-zone1-zonegroup1\"\n        },\n        \"s3\":{\n            \"s3SchemaVersion\":\"1.0\",\n            \"configurationId\":\"mynotif1\",\n            \"bucket\":{\n                \"name\":\"mybucket1\",\n                \"ownerIdentity\":{\n                    \"principalId\":\"tester\"\n                },\n                \"arn\":\"arn:aws:s3:us-east-1::mybucket1\",\n                \"id\":\"503a4c37-85eb-47cd-8681-2817e80b4281.5332.38\"\n            },\n            \"object\":{\n                \"key\":\"myimage1.jpg\",\n                \"size\":\"1024\",\n                \"eTag\":\"37b51d194a7513e45b56f6524f2d51f2\",\n                \"versionId\":\"\",\n                \"sequencer\": \"F7E6D75DC742D108\",\n                \"metadata\":[],\n                \"tags\":[]\n            }\n        },\n        \"eventId\":\"\",\n        \"opaqueData\":\"me@example.com\"\n    }\n]}\n
"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/","title":"Kafka to Object Storage","text":""},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#description","title":"Description","text":"

Kafka is a distributed event stream processing system which is great for storing hot relevant data. Based on the retention policy of the data, it can be used to store data for a long time. However, it is not suitable for storing data for a long time. This is where we need a mechanism to move data from Kafka to the object storage.

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#use-cases","title":"Use Cases","text":""},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#long-term-retention-of-data","title":"Long term retention of data","text":"

As Kafka is not really suited for long term retention of data, persisting it inside an object store will allow you to keep your data for further use, backup or archival purposes. Depending on the solution you use, you can also transform or format you data while storing it, which will ease further retrieval.

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#move-data-to-central-data-lake","title":"Move data to Central Data Lake","text":"

Production Kafka environment may not be the best place to run analytics or do model training. Transferring or copying the date to a central data lake will allow you to decouple those two aspects (production and analytics), bringing peace of mind and further capabilities to the data consumers.

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#implementations-examples","title":"Implementations examples","text":"

This pattern is implemented in the Smart City demo

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#configuration-using-secor","title":"Configuration Using Secor","text":"

This pattern implements the Secor Kafka Consumer. It can be used to consume kafka messages from a kafka topic and store that to S3 compatible Objet Buckets.

Secor is a service persisting Kafka logs to Amazon S3, Google Cloud Storage, Microsoft Azure Blob Storage and Openstack Swift. Its key features are: strong consistency, fault tolerance, load distribution, horizontal scalability, output partitioning, configurable upload policies, monitoring, customizability, event transformation.

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#prerequisites","title":"Prerequisites","text":""},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#bucket","title":"Bucket","text":"

An S3-compatible bucket, with its access key and secret key.

"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#zookeeper-entrance","title":"ZooKeeper Entrance","text":"

Secor needs to connect directly to Zookeeper to keep track of some data. If you have a secured installation of Zookeeper, like when you deploy Kafka using Strimzi or AMQStreams, you need to deploy a ZooKeeper Entrance. This is a special proxy to Zookeeper that will allow this direct connection.

Note

The deployment file is based on a Strimzi or AMQ Streams deployment of Kafka. If you configuration is different you may have to adapt some of the parameters.

Deployment:

  • In the file deployment/zookeeper-entrance.yaml, replace:
    • the occurrences of 'NAMESPACE' by the namespace where the Kafka cluster is.
    • the occurrences of 'YOUR_KAFKA' by the name of your Kafka cluster.
    • the parameters YOUR_KEY, YOUR_SECRET, YOUR_ENDPOINT, YOUR_BUCKET with the values corresponding to the bucket where you want to store the data.
  • Apply the modified file to deploy ZooKeeper Entrance.
"},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#deployment","title":"Deployment","text":""},{"location":"patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/#secor","title":"Secor","text":"
  • In the file deployment/secor.yaml, replace:
    • the occurrences of 'NAMESPACE' by the namespace where the Kafka cluster is.
    • the occurrences of 'YOUR_KAFKA' by the name of your Kafka cluster.
    • adjust all the other Secor parameters or add others depending on the processing you want to do with the data: output format, aggregation,... Full instructions are available here.
  • Apply the modified file to deploy Secor.
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/","title":"Kafka to Serverless","text":""},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#description","title":"Description","text":"

This pattern describes how to use AMQ Streams (Kafka) as an event source to OpenShift Serverless (Knative). You will learn how to implement Knative Eventing that can trigger a Knative Serving function when a messaged is posted to a Kafka Topic (Event).

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#knative-openshift-serverless","title":"Knative & OpenShift Serverless","text":"

Knative is an open source project that helps to deploy and manage modern serverless workloads on Kubernetes. Red Hat OpenShift Serverless is an enterprise-grade serverless offering based on knative that provides developers with a complete set of tools to build, deploy, and manage serverless applications on OpenShift Container Platform

Knative consists of 3 primary components:

  • Build - A flexible approach to building source code into containers.
  • Serving - Enables rapid deployment and automatic scaling of containers through a request-driven model for serving workloads based on demand.
  • Eventing - An infrastructure for consuming and producing events to stimulate applications. Applications can be triggered by a variety of sources, such as events from your own applications, cloud services from multiple providers, Software-as-a-Service (SaaS) systems, and Red Hat AMQ streams.
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#eda-event-driven-architecture","title":"EDA (Event Driven Architecture)","text":"

Event-Driven Architecture (EDA) is a way of designing applications and services to respond to real-time information based on the sending and receiving of information about individual events. EDA uses events to trigger and communicate between decoupled services and is common in modern applications built with microservices.

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#use-cases","title":"Use Cases","text":"
  • Develop an event-driven architecture with serverless applications.
  • Serverless Business logic processing that is capable of automated scale-up and scale-down to zero.
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#implementations-examples","title":"Implementations examples","text":"

This pattern is implemented in the XRay Pipeline Demo

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#deployment-example","title":"Deployment example","text":""},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#requirements","title":"Requirements","text":"
  • Red Hat OpenShift Container Platform
  • Red Hat AMQ Streams or Strimzi: the operator should be installed and a Kafka cluster must be created
  • Red Hat OpenShift Serverless: the operator must be installed
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#part-1-set-up-knative","title":"Part 1: Set up KNative","text":"

Once Red Hat OpenShift Serverless operator has been installed, we can create KnativeServing, KnativeEventing and KnativeKafka instances.

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-1-create-required-knative-instances","title":"Step 1: Create required Knative instances","text":"
  • From the deployment folder, apply the YAML file 01_knative_serving_eventing_kafka_setup.yaml to create knative instances
oc create -f 01_knative_serving_eventing_kafka_setup.yaml\n

Note

Those instances can also be deployed through the OpenShift Console if you prefer to use a UI. In this case, follow the Serverless deployment instructions (this section and the following ones).

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-2-verify-knative-instances","title":"Step 2: Verify Knative Instances","text":"
oc get po -n knative-serving\noc get po -n knative-eventing\n
  • Pod with prefix kafka-controller-manager represents Knative Kafka Event Source.
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#part-2-knative-serving","title":"Part 2: Knative Serving","text":"

Knative Serving is your serverless business logic that you would like to execute based on the event generated by Kafka.

For example purpose we are using a simple greeter service here. Depending on your use case you will replace that with your own business logic.

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-1-create-knative-serving","title":"Step 1: Create Knative Serving","text":"
  • From the deployment folder, in the YAML file 02_knative_service.yaml, replace the placeholder YOUR_NAMESPACE with your namespace, and apply the file to create knative serving.
oc create -f 02_knative_service.yaml\n
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-2-verify-knative-serving","title":"Step 2: Verify Knative Serving","text":"
oc get serving\n
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#part-3-knative-eventing","title":"Part 3: Knative Eventing","text":"

Knative Eventing enables developers to use an event-driven architecture with serverless applications. An event-driven architecture is based on the concept of decoupled relationships between event producers that create events, and event sinks, or consumers, that receive them.

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-1-kafka-topic","title":"Step 1: Kafka topic","text":"
  • Create a Kafka topic where the events will be sent. In this example, the topic will be example_topic.
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-2-create-knative-eventing","title":"Step 2: Create Knative Eventing","text":"
  • To create a Knative Eventing, we need to create a Kafka Event Source. Before you apply the following YAML file, 03_knative_kafka_source.yaml, please make sure to edit namespace and bootstrapServers to match your Kafka cluster. Also make sure to use the correct Knative Service (serving) that you have created in the previous step (greeter in this example).
oc create -f 03_knative_kafka_source.yaml\n
"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#step-3-verify-knative-eventing","title":"Step 3: Verify Knative Eventing","text":"
oc get kafkasource\n

At this point, as soon as new messages are received in Kafka topic example_topic, Knative Eventing will trigger the Knative Service greeter to execute the business logic, allowing you to have event-driven serverless application running on OpenShift Container Platform.

"},{"location":"patterns/kafka/kafka-to-serverless/kafka-to-serverless/#part-4-testing","title":"Part 4: Testing","text":"
  • Optional: to view the logs of Knative Serving you can install stern to them from the CLI, or use the OpenShift Web Console.
oc get ksvc\nstern --selector=serving.knative.dev/service=greeter -c user-container\n
  • Launch a temporary Kafka CLI (kafkacat) in a new terminal
oc run kafkacat -i -t --image debezium/tooling --restart=Never\n
  • From the kafkacat container shell, generate kafka messages in the topic example_topic of your Kafka cluster. Here we are generating Kafka messages with CloudEvents (CE) specification.
for i in {1..50} ; do sleep 10 ; \\\necho '{\"message\":\"Hello Red Hat\"}' | kafkacat -P -b core-kafka-kafka-bootstrap -t example_topic \\\n  -H \"content-type=application/json\" \\\n  -H \"ce-id=CE-001\" \\\n  -H \"ce-source=/kafkamessage\"\\\n  -H \"ce-type=dev.knative.kafka.event\" \\\n  -H \"ce-specversion=1.0\" \\\n  -H \"ce-time=2018-04-05T03:56:24Z\"\ndone ;\n

The above command will generate 50 Kafka messages every 10 seconds. Knative Eventing will pick up the messages and invoke the greeter Knative service, that you can verify from the logs of Knative Serving.

"},{"location":"patterns/starproxy/starproxy/","title":"Starburst/Trino Proxy","text":""},{"location":"patterns/starproxy/starproxy/#what-it-is","title":"What it is","text":"

Starproxy is a fully HTTP compliant proxy that is designed to sit between clients and a Trino/Starburst cluster. The motivation for developing a solution like this is laid out in some prior art below:

  • Facebook Engineering Blog - Static Analysis
  • Strata Conference Talk
  • Uber Case Study - Prism

The most attractive items to us are probably:

  • Enabling host based security
  • Detecting \"bad\" queries and blocking/deprioritizing them with custom rules
  • Load balancing across regions
"},{"location":"patterns/starproxy/starproxy/#how-it-works","title":"How it works","text":"

First and foremost, starproxy is an http proxy implemented in rust using a combination of axum/hyper.

  1. Parse the query AST, then check a variety of rules:

    • inbound CIDR rule checking
    • checking for predicates in queries
    • identifying select * queries with no limit, among other rules
  2. If rules are violated they can be associated with actions, like tagging the query as low priority. This is done by modifying the request headers and injecting special tags. Rules can also outright block requests by returning error status codes to the client directly.

"},{"location":"predictive-ai/what-is-predictive-ai/","title":"What is Predictive AI?","text":"

Predictive AI generally has the following characteristics:

  • A model generally created based on specific or narrow set of data.
  • Aims to make predictions on the likehood of an outcome, based on certain conditions and inputs.
  • e.g. \"how likely is this person to default on their loan in the next 5 years, given what we know about them?\"
"},{"location":"tools-and-applications/airflow/airflow/","title":"Apache Airflow","text":""},{"location":"tools-and-applications/airflow/airflow/#what-is-it","title":"What is it?","text":"

Apache Airflow is a platform created by the community to programmatically author, schedule and monitor workflows. It has become popular because of how easy it is to use and how extendable it is, covering a wide variety of tasks and allowing you to connect your workflows with virtually any technology. Since it's a Python framework it has also gathered a lot of interest from the Data Science field.

One important concept used in Airflow is DAGs (Directed Acyclical Graphs). A DAG is a graph without any cycles. In other words, a node in your graph may never point back to a node higher up in your workflow. DAGs are used to model your workflows/pipelines, which essentially means that you are building and executing graphs when working with Airflow. You can read more about DAGs here: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html

The key features of Airflow are:

  • Webserver: It's a user interface where you can see the status of your jobs, as well as inspect, trigger, and debug your DAGs and tasks. It also gives a database interface and lets you read logs from the remote file store.
  • Scheduler: The Scheduler is a component that monitors and manages all your tasks and DAGs, it checks their status and triggers them in the correct order once their dependencies are complete.
  • Executors: It handles running your task when they are assigned by the scheduler. It can either run the tasks inside the scheduler or push task execution out to workers. Airflow supports a variety of different executors which you can choose between.
  • Metadata database: The metadata database is used by the executor, webserver, and scheduler to store state.

"},{"location":"tools-and-applications/airflow/airflow/#installing-apache-airflow-on-openshift","title":"Installing Apache Airflow on OpenShift","text":"

Airflow can be run as a pip package, through docker, or a Helm chart. The official Helm chart can be found here: https://airflow.apache.org/docs/apache-airflow/stable/installation/index.html#using-official-airflow-helm-chart

See OpenDataHub Airflow - Example Helm Values

A modified version of the Helm chart which can be installed on OpenShift 4.12: https://github.com/eformat/openshift-airflow

"},{"location":"tools-and-applications/apache-nifi/apache-nifi/","title":"Apache NiFi","text":""},{"location":"tools-and-applications/apache-nifi/apache-nifi/#what-is-it","title":"What is it?","text":"

Apache NiFi is an open-source data integration tool that helps automate the flow of data between systems. It is designed to be easy to use and allows users to quickly and efficiently process, transmit, and securely distribute data. NiFi provides a web-based interface for monitoring and controlling data flows, as well as a library of processors for common data manipulation tasks such as filtering, routing, and transformation. It is highly configurable and can be used in a variety of scenarios including data ingestion, ETL, and dataflow management.

Nifi is a powerful tool to move data between systems and can handle real-time data with ease. It can be used in conjunction with other big data technologies such as Apache Kafka and Apache Spark to create a complete data pipeline. It supports a wide range of protocols and data formats, making it a versatile solution for any organization looking to manage and process large amounts of data.

"},{"location":"tools-and-applications/apache-nifi/apache-nifi/#installing-apache-nifi-on-openshift","title":"Installing Apache Nifi on OpenShift","text":"

The easiest way to install it is to follow the instructions available on the Nifi on OpenShift repo.

Contrary to other recipes or the images you can find on the Nifi project, the container images available on this repo are all based on UBI8 and follow OpenShift guidelines and constraints, like running with minimal privileges.

Several deployment options are available:

  • Choice on the number of nodes to deploy,
  • Basic, OIDC or LDAP authentication.
"},{"location":"tools-and-applications/apache-spark/apache-spark/","title":"Apache Spark","text":""},{"location":"tools-and-applications/apache-spark/apache-spark/#what-is-it","title":"What is it?","text":"

Apache Spark is an open-source, distributed computing system used for big data processing. It can process large amounts of data quickly and efficiently, and handle both batch and streaming data. Spark uses the in-memory computing concept, which allows it to process data much faster than traditional disk-based systems.

Spark supports a wide range of programming languages including Java, Python, and Scala. It provides a number of high-level libraries and APIs, such as Spark SQL, Spark Streaming, and MLlib, that make it easy for developers to perform complex data processing tasks. Spark SQL allows for querying structured data using SQL and the DataFrame API, Spark Streaming allows for processing real-time data streams, and MLlib is a machine learning library for building and deploying machine learning models. Spark also supports graph processing and graph computation through GraphX and GraphFrames.

"},{"location":"tools-and-applications/apache-spark/apache-spark/#working-with-spark-on-openshift","title":"Working with Spark on OpenShift","text":"

Spark can be fully containerized. Therefore a standalone Spark cluster can of course be installed on OpenShift. However, it sorts of breaks the cloud-native approach brought by Kubernetes of ephemeral workloads. There are in fact many ways to work with Spark on OpenShift, either with Spark-on-Kubernetes operator, or directly through PySpark or spark-submit commands.

In this Spark on OpenShift repository, you will find all the instructions to work with Spark on OpenShift.

It includes:

  • pre-built UBI-based Spark images including the drivers to work with S3 storage,
  • instructions and examples to build your own images (to include your own libraries for example),
  • instructions to deploy the Spark history server to gather your processing logs,
  • instructions to deploy the Spark on Kubernetes operator,
  • Prometheus and Grafana configuration to monitor your data processing and operator in real time,
  • instructions to work without the operator, from a Notebook or a Terminal, inside or outside the OpenShift Cluster,
  • various examples to test your installation and the different methods.
"},{"location":"tools-and-applications/ensemble-serving/ensemble-serving/","title":"Ensemble Models with Triton and KServe","text":"

Ensemble models are a feature of the Triton Model Server. They represent a pipeline of one or more models and the connection of input and output tensors between those models. Ensemble models are intended to be used to encapsulate a procedure that involves multiple models, such as \u201cdata preprocessing -> inference -> data postprocessing\u201d.

Using Ensemble models for this purpose can avoid the overhead of transferring intermediate tensors and minimize the number of requests that must be sent to Triton.

Full reference

Note

In this repo you will find the full recipe to deploy the Triton runtime with the Single Model Serving Stack, and an example of an Ensemble model to test it.

The following instructions are the walkthrough for this installation and test.

"},{"location":"tools-and-applications/ensemble-serving/ensemble-serving/#requirements","title":"Requirements","text":"

Deploy Triton as a custom Single Model Serving Runtime in OpenShift AI. You can import a REST Interface version, or a gRPC one.

  • On the OpenShift AI dashboard, go to Settings->Serving runtimes

  • Add a Servig runtime:

  • Select the type:

  • Paste the content of the runtime definition:

  • You can do the same with the gRPC version:

  • You now have the two runtimes available:

"},{"location":"tools-and-applications/ensemble-serving/ensemble-serving/#model-deployment","title":"Model Deployment","text":"

This deployment is based on the example model you can find in the model01 of the repo.

  • Copy the whole content of the model folder (so normally multiple models, plus the Ensemble definition) to an object store bucket.

  • In OpenShift AI, create a Data Connection pointing to the bucket.

  • Serve the model in OpenShift AI using the custom runtime you imported, pointing it to the data connection.

  • After a few seconds/minutes, the model is served and an inference endpoint is available.

  • You can also deploy the gRPC version in the same manner if wou want.

"},{"location":"tools-and-applications/ensemble-serving/ensemble-serving/#test","title":"Test","text":"
  • You can use the notebook test-ensemble-rest.ipynb to test the endpoint if you deployed the REST version of the runtime. Another notebook is available for gRPC.

"},{"location":"tools-and-applications/minio/minio/","title":"Minio","text":""},{"location":"tools-and-applications/minio/minio/#what-is-it","title":"What is it?","text":"

Minio is a high-performance, S3 compatible object store. It can be deployed on a wide variety of platforms, and it comes in multiple flavors.

"},{"location":"tools-and-applications/minio/minio/#why-this-guide","title":"Why this guide?","text":"

This guide is a very quick way of deploying the community version of Minio in order to quickly setup a fully standalone Object Store, in an OpenShift Cluster. This can then be used for various prototyping tasks that require Object Storage.

Note that nothing in this guide should be used in production-grade environments. Also, Minio is not included in RHOAI, and Red Hat does not provide support for Minio.

"},{"location":"tools-and-applications/minio/minio/#pre-requisites","title":"Pre-requisites","text":"
  • Access to an OpenShift cluster
  • Namespace-level admin permissions, or permission to create your own project
"},{"location":"tools-and-applications/minio/minio/#deploying-minio-on-openshift","title":"Deploying Minio on OpenShift","text":""},{"location":"tools-and-applications/minio/minio/#create-a-data-science-project-optional","title":"Create a Data Science Project (Optional)","text":"

If you already have your own Data Science Project, or OpenShift project, you can skip this step.

  1. If your cluster already has Red Hat OpenShift AI installed, you can use the Dashboard Web Interface to create a Data Science project.
  2. Simply navigate to Data Science Projects
  3. And click Create Project
  4. Choose a name for your project (here, Showcase) and click Create:

  5. Make sure to make a note of the Resource name, in case it's different from the name.

"},{"location":"tools-and-applications/minio/minio/#log-on-to-your-project-in-openshift-console","title":"Log on to your project in OpenShift Console","text":"
  1. Go to your cluster's OpenShift Console:

  2. Make sure you use the Administrator view, not the developer view.

  3. Go to Workloads then Pods, and confirm the selected project is the right one

  4. You now have a project in which to deploy Minio

"},{"location":"tools-and-applications/minio/minio/#deploy-minio-in-your-project","title":"Deploy Minio in your project","text":"
  1. Click on the + (\"Import YAML\") button:

  2. Paste the following YAML in the box, but don't press ok yet!:

    ---\nkind: PersistentVolumeClaim\napiVersion: v1\nmetadata:\n  name: minio-pvc\nspec:\n  accessModes:\n    - ReadWriteOnce\n  resources:\n    requests:\n      storage: 20Gi\n  volumeMode: Filesystem\n---\nkind: Secret\napiVersion: v1\nmetadata:\n  name: minio-secret\nstringData:\n  # change the username and password to your own values.\n  # ensure that the user is at least 3 characters long and the password at least 8\n  minio_root_user: minio\n  minio_root_password: minio123\n---\nkind: Deployment\napiVersion: apps/v1\nmetadata:\n  name: minio\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: minio\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: minio\n    spec:\n      volumes:\n        - name: data\n          persistentVolumeClaim:\n            claimName: minio-pvc\n      containers:\n        - resources:\n            limits:\n              cpu: 250m\n              memory: 1Gi\n            requests:\n              cpu: 20m\n              memory: 100Mi\n          readinessProbe:\n            tcpSocket:\n              port: 9000\n            initialDelaySeconds: 5\n            timeoutSeconds: 1\n            periodSeconds: 5\n            successThreshold: 1\n            failureThreshold: 3\n          terminationMessagePath: /dev/termination-log\n          name: minio\n          livenessProbe:\n            tcpSocket:\n              port: 9000\n            initialDelaySeconds: 30\n            timeoutSeconds: 1\n            periodSeconds: 5\n            successThreshold: 1\n            failureThreshold: 3\n          env:\n            - name: MINIO_ROOT_USER\n              valueFrom:\n                secretKeyRef:\n                  name: minio-secret\n                  key: minio_root_user\n            - name: MINIO_ROOT_PASSWORD\n              valueFrom:\n                secretKeyRef:\n                  name: minio-secret\n                  key: minio_root_password\n          ports:\n            - containerPort: 9000\n              protocol: TCP\n            - containerPort: 9090\n              protocol: TCP\n          imagePullPolicy: IfNotPresent\n          volumeMounts:\n            - name: data\n              mountPath: /data\n              subPath: minio\n          terminationMessagePolicy: File\n          image: >-\n            quay.io/minio/minio:RELEASE.2023-06-19T19-52-50Z\n          args:\n            - server\n            - /data\n            - --console-address\n            - :9090\n      restartPolicy: Always\n      terminationGracePeriodSeconds: 30\n      dnsPolicy: ClusterFirst\n      securityContext: {}\n      schedulerName: default-scheduler\n  strategy:\n    type: Recreate\n  revisionHistoryLimit: 10\n  progressDeadlineSeconds: 600\n---\nkind: Service\napiVersion: v1\nmetadata:\n  name: minio-service\nspec:\n  ipFamilies:\n    - IPv4\n  ports:\n    - name: api\n      protocol: TCP\n      port: 9000\n      targetPort: 9000\n    - name: ui\n      protocol: TCP\n      port: 9090\n      targetPort: 9090\n  internalTrafficPolicy: Cluster\n  type: ClusterIP\n  ipFamilyPolicy: SingleStack\n  sessionAffinity: None\n  selector:\n    app: minio\n---\nkind: Route\napiVersion: route.openshift.io/v1\nmetadata:\n  name: minio-api\nspec:\n  to:\n    kind: Service\n    name: minio-service\n    weight: 100\n  port:\n    targetPort: api\n  wildcardPolicy: None\n  tls:\n    termination: edge\n    insecureEdgeTerminationPolicy: Redirect\n---\nkind: Route\napiVersion: route.openshift.io/v1\nmetadata:\n  name: minio-ui\nspec:\n  to:\n    kind: Service\n    name: minio-service\n    weight: 100\n  port:\n    targetPort: ui\n  wildcardPolicy: None\n  tls:\n    termination: edge\n    insecureEdgeTerminationPolicy: Redirect\n

  3. By default, the size of the storage is 20 GB. (see line 11). Change it if you need to.

  4. If you want to, edit lines 21-22 to change the default user/password.
  5. Press Create.
  6. You should see:

  7. And there should now be a running minio pod:

  8. As well as two minio routes:

  9. The -api route is for programmatic access to Minio

  10. The -ui route is for browser-based access to Minio
  11. Your Minio Object Store is now deployed, but we still need to create at least one bucket in it, to make it useful.
"},{"location":"tools-and-applications/minio/minio/#creating-a-bucket-in-minio","title":"Creating a bucket in Minio","text":""},{"location":"tools-and-applications/minio/minio/#log-in-to-minio","title":"Log in to Minio","text":"
  1. Locate the minio-ui Route, and open its location URL in a web browser:
  2. When prompted, log in

    • if you kept the default values, then:
    • user: minio
    • pass: minio123

  3. You should now be logged into your Minio instance.

"},{"location":"tools-and-applications/minio/minio/#create-a-bucket","title":"Create a bucket","text":"
  1. Click on Create a Bucket

  2. Choose a name for your bucket (for example mybucket) and click Create Bucket:

  3. Repeat those steps to create as many buckets as you will need.

"},{"location":"tools-and-applications/minio/minio/#create-a-matching-data-connection-for-minio","title":"Create a matching Data Connection for Minio","text":"
  1. Back in RHOAI, inside of your Data Science Project, Click on Add data connection:

  2. Then, fill out the required field to match with your newly-deployed Minio Object Storage

  3. You now have a Data Connection that maps to your mybucket bucket in your Minio Instance.

  4. This data connection can be used, among other things
    • In your Workbenches
    • For your Model Serving
    • For your Pipeline Server Configuration
"},{"location":"tools-and-applications/minio/minio/#validate","title":"Validate","text":"

To test if everything is working correctly, you can access the workbench associated with your Data Connection and run the following commands (i.e., inside a Jupyter notebook):

  1. Install and import MinIO Python Client SDK

    !pip install minio\n
    from minio import Minio\nfrom minio.error import S3Error\nimport  os\nimport datetime\n

  2. Access Data Connection properties as environment variables:

    # MinIO client doesn't like URLs with procotol/schema, so use\n# yourendpoint.com instead of https://yourtendpoint.com\nAWS_S3_ENDPOINT = os.getenv(\"AWS_S3_ENDPOINT\")\nAWS_ACCESS_KEY_ID = os.getenv(\"AWS_ACCESS_KEY_ID\")\nAWS_SECRET_ACCESS_KEY = os.getenv(\"AWS_SECRET_ACCESS_KEY\")\nAWS_S3_BUCKET = os.getenv(\"AWS_S3_BUCKET\")\n
  3. Create the MinIO client

    # Create the MinIO client\nclient = Minio(\n    AWS_S3_ENDPOINT,\n    access_key=AWS_ACCESS_KEY_ID,\n    secret_key=AWS_SECRET_ACCESS_KEY,\n    secure=True  # Set to True if you are using HTTPS\n)\n
  4. Test the connection by listing all buckets

    #List all buckets\ntry:\n    buckets = client.list_buckets()\n    for bucket in buckets:\n        print(bucket.name, bucket.creation_date)\nexcept S3Error as e:\n    print(\"Error occurred: \", e)\n
  5. Create a sample local file

    # Create File\nFILE_ON_DISK = 'file.txt'\n\nfile = open(f\"{FILE_ON_DISK}\", \"w\")\nfile.write('Hello there %s recorded at %s.\\n' % (FILE_ON_DISK, datetime.datetime.now()))\nfile.close()\n
  6. Upload a file to MinIO

    # Upload a File \nfile_path = FILE_ON_DISK\nobject_name = 'target-file.txt'\n\ntry:\n    client.fput_object(AWS_S3_BUCKET, object_name, file_path)\n    print(f\"'{object_name}' is successfully uploaded as object to bucket '{bucket_name}'.\")\nexcept S3Error as e:\n    print(\"Error occurred: \", e)\n
  7. Download a file from MinIO

    # Download a file \nobject_name = 'target-file.txt'\nfile_path = 'file-froms3.txt'\n\ntry:\n client.fget_object(AWS_S3_BUCKET, object_name, file_path)\n print(f\"'{object_name}' is successfully downloaded to '{file_path}'.\")\nexcept S3Error as e:\n print(\"Error occurred: \", e)\n
  8. List objects in our bucket

    # Download a file \nobject_name = 'target-file.txt'\nfile_path = 'file-froms3.txt'\n\ntry:\n    client.fget_object(AWS_S3_BUCKET, object_name, file_path)\n    print(f\"'{object_name}' is successfully downloaded to '{file_path}'.\")\nexcept S3Error as e:\n    print(\"Error occurred: \", e)\n

For more complete and detailed information about MinIO Python Client SDK usage, please check the official documentation.

"},{"location":"tools-and-applications/minio/minio/#notes-and-faq","title":"Notes and FAQ","text":"
  • As long as you are using the Route URLs, a Minio running in one namespace can be used by any other application, even running in another namespace, or even in another cluster altogether.
"},{"location":"tools-and-applications/minio/minio/#uninstall-instructions","title":"Uninstall instructions:","text":"

This will completely remove Minio and all its content. Make sure you have a backup of the things your need before doing so!

  1. Track down those objects created earlier:

  2. Delete them all.

"},{"location":"tools-and-applications/mlflow/mlflow/","title":"MLFlow","text":""},{"location":"tools-and-applications/mlflow/mlflow/#what-is-it","title":"What is it?","text":"

MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. MLflow currently offers four components: Read more here: https://mlflow.org/

"},{"location":"tools-and-applications/mlflow/mlflow/#helm-installation-into-openshift-namespace","title":"Helm installation into OpenShift namespace","text":""},{"location":"tools-and-applications/mlflow/mlflow/#pre-requisites","title":"Pre-requisites","text":"
  • Install the \"Crunchy Postgres for Kubernetes\" operator (can be found in OperatorHub) - To store the MLFlow config
  • Install the \"OpenShift Data Foundation\" operator (can be found in OperatorHub) - To provide S3 storage for the experiments and models
"},{"location":"tools-and-applications/mlflow/mlflow/#install","title":"Install","text":"
<Create an OpenShift project, either through the OpenShift UI or 'oc new-project project-name'>\nhelm repo add strangiato https://strangiato.github.io/helm-charts/\nhelm repo update\n<Log in to the correct OpenShift project through 'oc project project-name'>\nhelm upgrade -i mlflow-server strangiato/mlflow-server\n
"},{"location":"tools-and-applications/mlflow/mlflow/#additional-options","title":"Additional Options","text":"

The MLFlow Server helm chart provides a number of customizable options when deploying MLFlow. These options can be configured using the --set flag with helm install or helm upgrade to set options directly on the command line or through a values.yaml file using the --values flag.

For a full list of configurable options, see the helm chart documentation:

https://github.com/strangiato/helm-charts/tree/main/charts/mlflow-server#values

"},{"location":"tools-and-applications/mlflow/mlflow/#opendatahub-dashboard-application-tile","title":"OpenDataHub Dashboard Application Tile","text":"

As discussed in the Dashboard Configuration, ODH/RHOAI allows administrators to add a custom application tile for additional components on the cluster.

The MLFlow Server helm chart supports creation of the Dashboard Application tile as a configurable value. If MLFlow Server is installed in the same namespace as ODH/RHOAI you can install the dashboard tile run the following command:

helm upgrade -i mlflow-server strangiato/mlflow-server \\\n    --set odhApplication.enabled=true\n

The MLFlow Server helm chart also supports installing the odhApplication object in a different namespace, if MLFlow Server is not installed in the same namespace as ODH/RHOAI:

helm upgrade -i mlflow-server strangiato/mlflow-server \\\n    --set odhApplication.enabled=true \\\n    --set odhApplication.namespaceOverride=redhat-ods-applications\n

After enabling the odhApplication component, wait 1-2 minutes and the tile should appear in the Explorer view of the dashboard.

Note

This feature requires ODH v1.4.1 or newer

"},{"location":"tools-and-applications/mlflow/mlflow/#test-mlflow","title":"Test MLFlow","text":"
  • Go to the OpenShift Console and switch to Developer view.
  • Go to the Topology view and make sure that you are on the MLFlow project.
  • Check that the MLFlow circle is dark blue (this means it has finished deploying).
  • Press the \"External URL\" link in the top right corner of the MLFlow circle to open up the MLFlow UI.
  • Run helm test mlflow-server in your command prompt to test MLFlow. If successful, you should see a new experiment called \"helm-test\" show up in the MLFlow UI with 3 experiments inside it.
"},{"location":"tools-and-applications/mlflow/mlflow/#adding-mlflow-to-training-code","title":"Adding MLFlow to Training Code","text":"
import mlflow\nfrom sklearn.linear_model import LogisticRegression\n\n# Set tracking URI\nmlflow.set_tracking_uri(\u201chttps://<route-to-mlflow>\u201d)\n\n# Setting the experiment\nmlflow.set_experiment(\"my-experiment\")\n\nif __name__ == \"__main__\":\n    # Enabling automatic logging for scikit-learn runs\n    mlflow.sklearn.autolog()\n\n    # Starting a logging run\n    with mlflow.start_run():\n        # train\n
"},{"location":"tools-and-applications/mlflow/mlflow/#source-code","title":"Source Code","text":"

MLFlow Server Source Code: https://github.com/strangiato/mlflow-server

MLFlow Server Helm Chart Source Code: https://github.com/strangiato/helm-charts/tree/main/charts/mlflow-server

"},{"location":"tools-and-applications/mlflow/mlflow/#demos","title":"Demos","text":"
  • Credit Card Fraud Detection pipeline using MLFlow together with RHOAI: Demo
"},{"location":"tools-and-applications/rclone/rclone/","title":"Rclone","text":""},{"location":"tools-and-applications/rclone/rclone/#what-is-it","title":"What is it?","text":"

Rclone is a program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

Users call rclone \"The Swiss army knife of cloud storage\", and \"Technology indistinguishable from magic\".

Rclone really looks after your data. It preserves timestamps and verifies checksums at all times. Transfers over limited bandwidth; intermittent connections, or subject to quota can be restarted, from the last good file transferred. You can check the integrity of your files. Where possible, rclone employs server-side transfers to minimize local bandwidth use and transfers from one provider to another without using local disk.

Rclone is mature, open-source software originally inspired by rsync and written in Go. The friendly support community is familiar with varied use cases.

The implementation described here is a containerized version of Rclone to run on OpenShift, alongside or integrated within ODH/RHOAI.

"},{"location":"tools-and-applications/rclone/rclone/#deployment","title":"Deployment","text":""},{"location":"tools-and-applications/rclone/rclone/#integrated-in-open-data-hub-or-openshift-ai","title":"Integrated in Open Data Hub or OpenShift AI","text":"

Use this method if you want to use Rclone from the ODH/RHOAI launcher or in a Data Science Project.

  • In the Cluster Settings menu, import the image quay.io/guimou/rclone-web-openshift:odh-rhoai_latest. You can name it Rclone.
  • In your DSP project, create a new workbench using the Rclone image. You can set the storage size as minimal as it's only there to store the configuration of the endpoints.

Tip

The minimal size allowed by the dashboard for a storage volume is currently 1GB, which is way more than what is required for the Rclone configuration. So you can also create a much smaller PVC manually in the namespace corresponding to your Data Science Project, for example 100MB or less, and select this volume when creating the workbench.

  • Launch Rclone from the link once it's deployed!
  • After the standard authentication, you end up on the Rclone Login page. There is nothing to enter, but I have not found yet how to bypass it. So simply click on \"Login\".
"},{"location":"tools-and-applications/rclone/rclone/#standalone-deployment","title":"Standalone deployment","text":"

Use this method if you want to use Rclone on its own in a namespace. You can still optionally make a shortcut appear in the ODH/RHOAI dashboard.

  • Create a project/namespace for your installation.
  • Clone or head to this repo.
  • From the deploy folder, apply the different YAML files:
    • 01-pvc.yaml: creates a persistent volume to hold the configuration
    • 02-deployment.yaml: creates the deployment. Modify admin account and password if you want to restrict access. You should!
    • 03-service.yaml, 04-route.yaml: create the external access so that you can connect to the Web UI.
    • Optionally, to create a tile on the ODH/RHOAI dashboard:
      • modify the 05-tile.yaml file with the address of the Route that was created previously (namespace and name of the Route object).
      • the will appear under the available applications in the dashboard. Select it and click on \"Enable\" to make it appear in the \"Enabled\" menu.
"},{"location":"tools-and-applications/rclone/rclone/#configuration","title":"Configuration","text":"

In this example, we will create an S3 configuration that connects to a bucket on the MCG from OpenShift Data Foundation. So you must have created this bucket in advance and have all the information about it: endpoint, access and secret keys, bucket name.

  • In Rclone, click on \"Configs\" to create the new Remote.
  • Create new configuration, give it a name, and select \"Amazon S3 Compliant Storage Providers...\", which includes Ceph and MCG (even if not listed).
  • Enter the connection info. You only have to enter the Access key and Secret, as well as the Endpoint in \"Endpoint for S3 API\". This last info is automatically copied in other fields, that's normal.
  • Finalize the config by clicking on \"Next\" at the bottom.

Now that you have the Remote set up, you can go on the Explorer, select the Remote, and browse it!

"},{"location":"tools-and-applications/rclone/rclone/#usage-example","title":"Usage Example","text":"

In this simple example, we will transfer a dump sample from Wikipedia. Wikimedia publishes those dumps daily, and they are mirrored by different organizations. In a \"standard\" setup, loading those information into your object store would not be really practical, sometimes involving downloading it first locally to then push it to your storage.

This is how we can do it with Rclone.

  • Create your Bucket Remote as described in Configuration.
  • Create another remote of type \"HTTP\", and enter the address of one of the mirrors. Here I used https://dumps.wikimedia.your.org/wikidatawiki/.
  • Open the Explorer view, set it in dual-pane layout. In the first pane open your Bucket Remote, and in the other one the HTTP. This is what it will look like:
  • Browse to the folder you want, select a file or a folder, and simply drag and drop it from the Wikidump to your bucket. You can select a big one to make things more interesting!
  • Head for the dashboard where you will see the file transfer happening in the background.

That's it! Nothing to install, high speed optimized transfer, and you could even do multiple transfers in the background,...

"},{"location":"tools-and-applications/riva/riva/","title":"NVIDIA RIVA","text":"

NVIDIA\u00ae Riva is a GPU-accelerated SDK for building Speech AI applications that are customized for your use case and deliver real-time performance.

Riva offers pretrained speech models in NVIDIA NGC\u2122 that can be fine-tuned with the NVIDIA NeMo on a custom data set, accelerating the development of domain-specific models by 10x.

Models can be easily exported, optimized, and deployed as a speech service on premises or in the cloud with a single command using Helm charts.

Riva\u2019s high-performance inference is powered by NVIDIA TensorRT\u2122 optimizations and served using the NVIDIA Triton\u2122 Inference Server, which are both part of the NVIDIA AI platform.

Riva services are available as gRPC-based microservices for low-latency streaming, as well as high-throughput offline use cases.

Riva is fully containerized and can easily scale to hundreds and thousands of parallel streams.

"},{"location":"tools-and-applications/riva/riva/#deployment","title":"Deployment","text":"

The guide to deploy Riva on Kubernetes has to be adapted for OpenShift. Here are the different steps.

"},{"location":"tools-and-applications/riva/riva/#prerequisites","title":"Prerequisites","text":"
  1. You have access and are logged into NVIDIA NGC. For step-by-step instructions, refer to the NGC Getting Started Guide. Specifically you will need your API Key from NVIDIA NGC.
  2. You have at least one worker node with an NVIDIA Volta\u2122, NVIDIA Turing\u2122, or an NVIDIA Ampere architecture-based GPU. For more information, refer to the Support Matrix.
  3. The Node Feature Discovery and the NVIDIA operators have been properly installed and configured on your OpenShift Cluster to enable your GPU(s). Full instructions here
  4. The Pod that will be deployed will consume about 10GB of RAM. Make sure you have enough resources on your node (on top of the GPU itself), and you don't have limits in place that would restrict this. GPU memory consumption will be about 12GB with all models loaded.
"},{"location":"tools-and-applications/riva/riva/#installation","title":"Installation","text":"

Included in the NGC Helm Repository is a chart designed to automate deployment to a Kubernetes cluster. This chart must be modified for OpenShift.

The Riva Speech AI Helm Chart deploys the ASR, NLP, and TTS services automatically. The Helm chart performs a number of functions:

  • Pulls Docker images from NGC for the Riva Speech AI server and utility containers for downloading and converting models.
  • Downloads the requested model artifacts from NGC as configured in the values.yaml file.
  • Generates the Triton Inference Server model repository.
  • Starts the Riva Speech AI server as configured in a Kubernetes pod.
  • Exposes the Riva Speech AI server as a configured service.

Examples of pretrained models are released with Riva for each of the services. The Helm chart comes preconfigured for downloading and deploying all of these models.

Installation Steps:

  1. Download the Helm chart

    export NGC_API_KEY=<your_api_key>\nhelm fetch https://helm.ngc.nvidia.com/nvidia/riva/charts/riva-api-2.11.0.tgz \\\n        --username=\\$oauthtoken --password=$NGC_API_KEY --untar\n
  2. Switch to the newly created folder, riva-api

  3. In the templates folder, modify the file deployment.yaml. For both the container riva-speech-api and the initContainer riva-model-init you must add the following security context information:

    securityContext:\n    allowPrivilegeEscalation: false\n    capabilities:\n      drop: [\"ALL\"]\n    seccompProfile:\n      type: \"RuntimeDefault\"\n    runAsNonRoot: true\n
  4. The file deployment.yaml should now look like this:

    ...\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: {{ template \"riva-server.fullname\" . }}\n  ...\nspec:\n  ...\n  template:\n    ...\n    spec:\n      containers:\n        - name: riva-speech-api\n          securityContext:\n            allowPrivilegeEscalation: false\n            capabilities:\n              drop: [\"ALL\"]\n            seccompProfile:\n              type: \"RuntimeDefault\"\n            runAsNonRoot: true\n          image: {{ $server_image }}\n          ...\n      initContainers:\n        - name: riva-model-init\n          securityContext:\n            allowPrivilegeEscalation: false\n            capabilities:\n              drop: [\"ALL\"]\n            seccompProfile:\n              type: \"RuntimeDefault\"\n            runAsNonRoot: true\n          image: {{ $servicemaker_image }}\n          ...\n
  5. At the root of riva-api, modify the file values.yaml:

    1. You will need to convert your API Key to a password value. In a Terminal run:

      echo -n $NGC_API_KEY | base64 -w0\n
    2. In the ngcCredentials section ov values.yaml, enter the password you obtained above and your email

    3. In the modelRepoGenerator section, for the modelDeployKey value, enter dGx0X2VuY29kZQ==. (This value is obtained from the command echo -n tlt_encode | base64 -w0.
    4. In the persistentVolumeClaim section, set usePVC to true. This is very important as it will disable the hostPath configuration for storage that is not permitted by default on OpenShift.
    5. If you don't have a storageClass set as default, or want to you another one, enter the name of the class you want to use in storageClassName. Otherwise leave this field empty and the default class will be used.
    6. Optionally, modify the storageSize.
    7. Leave the ingress section as is, we will create an OpenShift Route later.
    8. Optionally you can modify other values in the file to enable/disable certain models, or modify their configuration.
  6. Log into your OpenShift cluster from a Terminal, and create a project riva-api:

    oc new-project riva-api\n
  7. Move up one folder (so outside of the riva-api folder), and install NVIDIA Riva with the modified Helm chart:

    helm install riva-api riva-api\n

The deployment will now start.

Info

Beware that the deployment can be really long the first time, about 45mn if you have all the models and features selected. Containers and models have to be downloaded and configured. Please be patient...

"},{"location":"tools-and-applications/riva/riva/#usage","title":"Usage","text":"

The Helm chart had automatically created a Service, riva-api in the namespace where you have deployed it. If you followed this guide, this should also be riva-api. So within the OpenShift cluster, the API is accessible at riva-api.riva-api.svc.cluster.local.

Different ports are accessible:

  • http (8000): HTTP port of the Triton server.
  • grpc (8001): gRPC port of the Triton server.
  • metrics (8002): port for the metrics of the Triton server.
  • speech-grpc (50051): gRPC port of the Speech that exposes directly the different services you can use. This is normally the one that you will use.

If you want to use the API outside of the OpenShift cluster, you will have to create one or multiple Routes to those different endpoints.

"},{"location":"tools-and-applications/riva/riva/#example","title":"Example","text":"
  • On the same cluster where NVIDIA Riva is deployed, deploy RHOAI or ODH and launch a Notebook (Standard DataScience is enough).
  • Clone the NVIDIA Riva tutorials repository at https://github.com/nvidia-riva/tutorials
  • Open a Terminal and install the client with pip install nvidia-riva-client:

(depending on the base image you used this may yield errors that you can ignore most of times).

  • In the tutorials folder, open the notebook asr-basics.ipynb.
  • In the cell that defines the uri of the API server, modify the default (localhost) for the address of the API server: riva-api.riva-api.svc.cluster.local

  • Run the notebook!

Note

In this example, only the first part of the notebook will work as only the English models have been deployed. You would have to adapt the configuration for other languages.

"},{"location":"whats-new/whats-new/","title":"What's new?","text":"

2024-08-21: Add Model serving type modification instructions.

2024-07-29: Add Kueue preemption example.

2024-07-16: Add ODH Tools and Extensions Companion documentation.

2024-07-16: Add GPU pruner documentation.

2024-07-02: Add Llama2 fine-tuning with Ray and DeepSpeed demo.

2024-06-25: Add GPU partitioning and sharing.

2024-06-18: Add Accelerator profiles documentation.

2024-01-03: Add Ensemble models demo.

2024-03-28: Add Zero to Workbench documentation.

2024-03-04: Refactoring of the site. Update on the LLMs deployment and RAG demo. Single Stack Serving certificate how-to.

2023-12-21: Name change for Red Hat OpenShift Data Science, which becomes Red Hat OpenShift AI.

2023-10-16: Add documentation for LLMs deployment and RAG demo.

2023-08-01: Update to Spark documentation to include usage without the operator Tools and Applications->Apache Spark

2023-07-05: Add documentation on Time Slicing and Autoscaling for NVIDIA GPUs ODH/RHOAI How-Tos->NVIDIA GPUs

2023-07-05: New example of how to configure a Custom Serving Runtime with Triton.

2023-07-03: New Minio tutorial on how to quickly deploy a simple Object Storage inside your OpenShift Project, for quick prototyping.

2023-06-30: New NVIDIA GPU installation documentation with Node tainting in ODH/RHOAI How-Tos->NVIDIA GPUs

2023-06-02: NVIDIA Riva documentation in Tools and Applications->NVIDIA Riva

NVIDIA\u00ae Riva is a GPU-accelerated SDK for building Speech AI applications that are customized for your use case and deliver real-time performance.

2023-02-06: Rclone documentation in Tools and Applications->Rclone.

Rclone is a program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

2023-02-02: Addition of VSCode and RStudio images to custom workbenches.

2023-01-22: Addition of StarProxy to Patterns->Starburst/Trino Proxy.

Starproxy is a fully HTTP compliant proxy that is designed to sit between clients and a Trino/Starburst cluster.

"}]} \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml new file mode 100644 index 00000000..dc306522 --- /dev/null +++ b/sitemap.xml @@ -0,0 +1,211 @@ + + + + https://ai-on-openshift.io/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/codellama-continue/codellama-continue/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/credit-card-fraud-detection-mlflow/credit-card-fraud/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/financial-fraud-detection/financial-fraud-detection/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/llama2-finetune/llama2-finetune/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/llm-chat-doc/llm-chat-doc/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/podman-ai-lab-to-rhoai/podman-ai-lab-to-rhoai/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/retail-object-detection/retail-object-detection/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/smart-city/smart-city/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/stable-diffusion/stable-diffusion/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/telecom-customer-churn-airflow/telecom-customer-churn-airflow/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/water-pump-failure-prediction/water-pump-failure-prediction/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/xray-pipeline/xray-pipeline/ + 2024-10-16 + + + https://ai-on-openshift.io/demos/yolov5-training-serving/yolov5-training-serving/ + 2024-10-16 + + + https://ai-on-openshift.io/generative-ai/llm-serving/ + 2024-10-16 + + + https://ai-on-openshift.io/generative-ai/what-is-generative-ai/ + 2024-10-16 + + + https://ai-on-openshift.io/getting-started/opendatahub/ + 2024-10-16 + + + https://ai-on-openshift.io/getting-started/openshift-ai/ + 2024-10-16 + + + https://ai-on-openshift.io/getting-started/openshift/ + 2024-10-16 + + + https://ai-on-openshift.io/getting-started/why-this-site/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/accelerator-profiles/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/configuration/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/connect-vscode-to-rhoai-wb/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/custom-notebooks/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/custom-runtime-triton/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/gitops/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/gpu-pruner/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/kserve-timeout/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/kserve-uwm-dashboard-metrics/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/model-serving-type-modification/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/nvidia-gpus/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/odh-tools-and-extensions-companion/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/openshift-group-management/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/single-stack-serving-certificate/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/from-zero-to-workbench/using-cli/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/from-zero-to-workbench/using-developer-hub/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/from-zero-to-workbench/using-ui/ + 2024-10-16 + + + https://ai-on-openshift.io/odh-rhoai/kueue-preemption/readme/ + 2024-10-16 + + + https://ai-on-openshift.io/patterns/bucket-notifications/bucket-notifications/ + 2024-10-16 + + + https://ai-on-openshift.io/patterns/kafka/kafka-to-object-storage/kafka-to-object-storage/ + 2024-10-16 + + + https://ai-on-openshift.io/patterns/kafka/kafka-to-serverless/kafka-to-serverless/ + 2024-10-16 + + + https://ai-on-openshift.io/patterns/starproxy/starproxy/ + 2024-10-16 + + + https://ai-on-openshift.io/predictive-ai/what-is-predictive-ai/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/airflow/airflow/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/apache-nifi/apache-nifi/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/apache-spark/apache-spark/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/ensemble-serving/ensemble-serving/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/minio/minio/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/mlflow/mlflow/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/rclone/rclone/ + 2024-10-16 + + + https://ai-on-openshift.io/tools-and-applications/riva/riva/ + 2024-10-16 + + + https://ai-on-openshift.io/whats-new/whats-new/ + 2024-10-16 + + \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz new file mode 100644 index 00000000..392a2936 Binary files /dev/null and b/sitemap.xml.gz differ diff --git a/stylesheets/extra.css b/stylesheets/extra.css new file mode 100644 index 00000000..29bb1575 --- /dev/null +++ b/stylesheets/extra.css @@ -0,0 +1,24 @@ +img { + border: 1px solid #cccccc; + transition: transform ease-in-out 0.3s; +} + +div.tx-hero__content img { + border: none; +} + +div.tx-hero__image img { + border: none; +} + +.noborder { + border: none; +} + +a.md-logo img { + border: none; +} + +.md-nav__item--section>.md-nav__link[for] { + color:grey; +} \ No newline at end of file diff --git a/theme_override/home.html b/theme_override/home.html new file mode 100644 index 00000000..fb153ec1 --- /dev/null +++ b/theme_override/home.html @@ -0,0 +1,359 @@ + + +{% extends "main.html" %} +{% block tabs %} +{{ super() }} + + + +
+
+
+
+ +
+
+ +

The one-stop shop for AI/ML,
Data Science and Data Engineering
on OpenShift!

+ + Get started + +
+
+
+
+ + + + + + +{% endblock %} +{% block content %}{% endblock %} +{% block footer %}{% endblock %} diff --git a/theme_override/main.html b/theme_override/main.html new file mode 100644 index 00000000..b4120ffa --- /dev/null +++ b/theme_override/main.html @@ -0,0 +1,29 @@ + +{% extends "base.html" %} + + +{% block footer %} +
+ +
+{% endblock %} \ No newline at end of file diff --git a/tools-and-applications/airflow/airflow/index.html b/tools-and-applications/airflow/airflow/index.html new file mode 100644 index 00000000..a32b96ee --- /dev/null +++ b/tools-and-applications/airflow/airflow/index.html @@ -0,0 +1,2875 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Apache Airflow - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Apache Airflow

+

What is it?

+

logoApache Airflow is a platform created by the community to programmatically author, schedule and monitor workflows.
+It has become popular because of how easy it is to use and how extendable it is, covering a wide variety of tasks and allowing you to connect your workflows with virtually any technology.
+Since it's a Python framework it has also gathered a lot of interest from the Data Science field.

+

One important concept used in Airflow is DAGs (Directed Acyclical Graphs).
+A DAG is a graph without any cycles. In other words, a node in your graph may never point back to a node higher up in your workflow.
+DAGs are used to model your workflows/pipelines, which essentially means that you are building and executing graphs when working with Airflow.
+You can read more about DAGs here: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html

+

The key features of Airflow are:

+
    +
  • Webserver: It's a user interface where you can see the status of your jobs, as well as inspect, trigger, and debug your DAGs and tasks. It also gives a database interface and lets you read logs from the remote file store.
  • +
  • Scheduler: The Scheduler is a component that monitors and manages all your tasks and DAGs, it checks their status and triggers them in the correct order once their dependencies are complete.
  • +
  • Executors: It handles running your task when they are assigned by the scheduler. It can either run the tasks inside the scheduler or push task execution out to workers. Airflow supports a variety of different executors which you can choose between.
  • +
  • Metadata database: The metadata database is used by the executor, webserver, and scheduler to store state.
  • +
+

graph

+

Installing Apache Airflow on OpenShift

+

Airflow can be run as a pip package, through docker, or a Helm chart.
+The official Helm chart can be found here: https://airflow.apache.org/docs/apache-airflow/stable/installation/index.html#using-official-airflow-helm-chart

+

See OpenDataHub Airflow - Example Helm Values

+

A modified version of the Helm chart which can be installed on OpenShift 4.12: https://github.com/eformat/openshift-airflow

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/airflow/img/graph.png b/tools-and-applications/airflow/img/graph.png new file mode 100644 index 00000000..860fbff7 Binary files /dev/null and b/tools-and-applications/airflow/img/graph.png differ diff --git a/tools-and-applications/airflow/img/logo.png b/tools-and-applications/airflow/img/logo.png new file mode 100644 index 00000000..ff203ce8 Binary files /dev/null and b/tools-and-applications/airflow/img/logo.png differ diff --git a/tools-and-applications/apache-nifi/apache-nifi/index.html b/tools-and-applications/apache-nifi/apache-nifi/index.html new file mode 100644 index 00000000..896d8b75 --- /dev/null +++ b/tools-and-applications/apache-nifi/apache-nifi/index.html @@ -0,0 +1,2867 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Apache NiFi - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Apache NiFi

+

What is it?

+

Apache NiFi is an open-source data integration tool that helps automate the flow of data between systems. It is designed to be easy to use and allows users to quickly and efficiently process, transmit, and securely distribute data. NiFi provides a web-based interface for monitoring and controlling data flows, as well as a library of processors for common data manipulation tasks such as filtering, routing, and transformation. It is highly configurable and can be used in a variety of scenarios including data ingestion, ETL, and dataflow management.

+

NiFi preview

+

Nifi is a powerful tool to move data between systems and can handle real-time data with ease. It can be used in conjunction with other big data technologies such as Apache Kafka and Apache Spark to create a complete data pipeline. It supports a wide range of protocols and data formats, making it a versatile solution for any organization looking to manage and process large amounts of data.

+

Installing Apache Nifi on OpenShift

+

The easiest way to install it is to follow the instructions available on the Nifi on OpenShift repo.

+

Nifi on OpenShift

+

Contrary to other recipes or the images you can find on the Nifi project, the container images available on this repo are all based on UBI8 and follow OpenShift guidelines and constraints, like running with minimal privileges.

+

Several deployment options are available:

+
    +
  • Choice on the number of nodes to deploy,
  • +
  • Basic, OIDC or LDAP authentication.
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/apache-nifi/img/nifi-openshift.png b/tools-and-applications/apache-nifi/img/nifi-openshift.png new file mode 100644 index 00000000..3a988c06 Binary files /dev/null and b/tools-and-applications/apache-nifi/img/nifi-openshift.png differ diff --git a/tools-and-applications/apache-nifi/img/nifi-prview.png b/tools-and-applications/apache-nifi/img/nifi-prview.png new file mode 100644 index 00000000..440cb224 Binary files /dev/null and b/tools-and-applications/apache-nifi/img/nifi-prview.png differ diff --git a/tools-and-applications/apache-spark/apache-spark/index.html b/tools-and-applications/apache-spark/apache-spark/index.html new file mode 100644 index 00000000..47ca8a5c --- /dev/null +++ b/tools-and-applications/apache-spark/apache-spark/index.html @@ -0,0 +1,2870 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Apache Spark - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Apache Spark

+

What is it?

+

SparkApache Spark is an open-source, distributed computing system used for big data processing. It can process large amounts of data quickly and efficiently, and handle both batch and streaming data. Spark uses the in-memory computing concept, which allows it to process data much faster than traditional disk-based systems.

+

Spark supports a wide range of programming languages including Java, Python, and Scala. It provides a number of high-level libraries and APIs, such as Spark SQL, Spark Streaming, and MLlib, that make it easy for developers to perform complex data processing tasks. Spark SQL allows for querying structured data using SQL and the DataFrame API, Spark Streaming allows for processing real-time data streams, and MLlib is a machine learning library for building and deploying machine learning models. Spark also supports graph processing and graph computation through GraphX and GraphFrames.

+

Working with Spark on OpenShift

+

Spark can be fully containerized. Therefore a standalone Spark cluster can of course be installed on OpenShift. However, it sorts of breaks the cloud-native approach brought by Kubernetes of ephemeral workloads. There are in fact many ways to work with Spark on OpenShift, either with Spark-on-Kubernetes operator, or directly through PySpark or spark-submit commands.

+

In this Spark on OpenShift repository, you will find all the instructions to work with Spark on OpenShift.

+

It includes:

+
    +
  • pre-built UBI-based Spark images including the drivers to work with S3 storage,
  • +
  • instructions and examples to build your own images (to include your own libraries for example),
  • +
  • instructions to deploy the Spark history server to gather your processing logs,
  • +
  • instructions to deploy the Spark on Kubernetes operator,
  • +
  • Prometheus and Grafana configuration to monitor your data processing and operator in real time,
  • +
  • instructions to work without the operator, from a Notebook or a Terminal, inside or outside the OpenShift Cluster,
  • +
  • various examples to test your installation and the different methods.
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/apache-spark/img/spark-logo.png b/tools-and-applications/apache-spark/img/spark-logo.png new file mode 100644 index 00000000..fdfdde7f Binary files /dev/null and b/tools-and-applications/apache-spark/img/spark-logo.png differ diff --git a/tools-and-applications/ensemble-serving/ensemble-serving/index.html b/tools-and-applications/ensemble-serving/ensemble-serving/index.html new file mode 100644 index 00000000..214e2fde --- /dev/null +++ b/tools-and-applications/ensemble-serving/ensemble-serving/index.html @@ -0,0 +1,2941 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Ensemble Serving - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Ensemble Models with Triton and KServe

+

Ensemble models are a feature of the Triton Model Server. They represent a pipeline of one or more models and the connection of input and output tensors between those models. Ensemble models are intended to be used to encapsulate a procedure that involves multiple models, such as “data preprocessing -> inference -> data postprocessing”.

+

Using Ensemble models for this purpose can avoid the overhead of transferring intermediate tensors and minimize the number of requests that must be sent to Triton.

+

Full reference

+
+

Note

+

In this repo you will find the full recipe to deploy the Triton runtime with the Single Model Serving Stack, and an example of an Ensemble model to test it.

+
+

The following instructions are the walkthrough for this installation and test.

+

Requirements

+

Deploy Triton as a custom Single Model Serving Runtime in OpenShift AI. You can import a REST Interface version, or a gRPC one.

+
    +
  • +

    On the OpenShift AI dashboard, go to Settings->Serving runtimes

    +

    serving-runtimes.png

    +
  • +
  • +

    Add a Servig runtime:

    +

    dd-serving-runtime.png

    +
  • +
  • +

    Select the type:

    +

    runtime-confg-1.png

    +
  • +
  • +

    Paste the content of the runtime definition:

    +

    create-rest.png

    +
  • +
  • +

    You can do the same with the gRPC version:

    +

    create-grpc.png

    +
  • +
  • +

    You now have the two runtimes available:

    +

    two-runtimes.png

    +
  • +
+

Model Deployment

+

This deployment is based on the example model you can find in the model01 of the repo.

+
    +
  • +

    Copy the whole content of the model folder (so normally multiple models, plus the Ensemble definition) to an object store bucket.

    +

    model-upload.png

    +
  • +
  • +

    In OpenShift AI, create a Data Connection pointing to the bucket.

    +

    data-connection.png

    +
  • +
  • +

    Serve the model in OpenShift AI using the custom runtime you imported, pointing it to the data connection.

    +

    model-server-deploy.png +model-configuration-1.png +model-configuration-2.png

    +
  • +
  • +

    After a few seconds/minutes, the model is served and an inference endpoint is available.

    +

    model-served.png

    +
  • +
  • +

    You can also deploy the gRPC version in the same manner if wou want.

    +
  • +
+

Test

+
    +
  • +

    You can use the notebook test-ensemble-rest.ipynb to test the endpoint if you deployed the REST version of the runtime. Another notebook is available for gRPC.

    +

    notebook-1.png +notebook-2.png

    +
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/ensemble-serving/img/add-serving-runtime.png b/tools-and-applications/ensemble-serving/img/add-serving-runtime.png new file mode 100644 index 00000000..3459c8da Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/add-serving-runtime.png differ diff --git a/tools-and-applications/ensemble-serving/img/create-grpc.png b/tools-and-applications/ensemble-serving/img/create-grpc.png new file mode 100644 index 00000000..a2364248 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/create-grpc.png differ diff --git a/tools-and-applications/ensemble-serving/img/create-rest.png b/tools-and-applications/ensemble-serving/img/create-rest.png new file mode 100644 index 00000000..aa6b4cb8 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/create-rest.png differ diff --git a/tools-and-applications/ensemble-serving/img/data-connection.png b/tools-and-applications/ensemble-serving/img/data-connection.png new file mode 100644 index 00000000..21949f1d Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/data-connection.png differ diff --git a/tools-and-applications/ensemble-serving/img/model-configuration-1.png b/tools-and-applications/ensemble-serving/img/model-configuration-1.png new file mode 100644 index 00000000..0b53cfbe Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/model-configuration-1.png differ diff --git a/tools-and-applications/ensemble-serving/img/model-configuration-2.png b/tools-and-applications/ensemble-serving/img/model-configuration-2.png new file mode 100644 index 00000000..3ca5fa3d Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/model-configuration-2.png differ diff --git a/tools-and-applications/ensemble-serving/img/model-served.png b/tools-and-applications/ensemble-serving/img/model-served.png new file mode 100644 index 00000000..c601058b Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/model-served.png differ diff --git a/tools-and-applications/ensemble-serving/img/model-server-deploy.png b/tools-and-applications/ensemble-serving/img/model-server-deploy.png new file mode 100644 index 00000000..87758724 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/model-server-deploy.png differ diff --git a/tools-and-applications/ensemble-serving/img/model-upload.png b/tools-and-applications/ensemble-serving/img/model-upload.png new file mode 100644 index 00000000..ce172a57 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/model-upload.png differ diff --git a/tools-and-applications/ensemble-serving/img/notebook-1.png b/tools-and-applications/ensemble-serving/img/notebook-1.png new file mode 100644 index 00000000..25376407 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/notebook-1.png differ diff --git a/tools-and-applications/ensemble-serving/img/notebook-2.png b/tools-and-applications/ensemble-serving/img/notebook-2.png new file mode 100644 index 00000000..3fb641bd Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/notebook-2.png differ diff --git a/tools-and-applications/ensemble-serving/img/runtime-confg-1.png b/tools-and-applications/ensemble-serving/img/runtime-confg-1.png new file mode 100644 index 00000000..d40450d0 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/runtime-confg-1.png differ diff --git a/tools-and-applications/ensemble-serving/img/serving-runtimes.png b/tools-and-applications/ensemble-serving/img/serving-runtimes.png new file mode 100644 index 00000000..4338fef2 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/serving-runtimes.png differ diff --git a/tools-and-applications/ensemble-serving/img/two-runtimes.png b/tools-and-applications/ensemble-serving/img/two-runtimes.png new file mode 100644 index 00000000..9ca0e5c4 Binary files /dev/null and b/tools-and-applications/ensemble-serving/img/two-runtimes.png differ diff --git a/tools-and-applications/minio/img/add.connection.png b/tools-and-applications/minio/img/add.connection.png new file mode 100644 index 00000000..f1a9938c Binary files /dev/null and b/tools-and-applications/minio/img/add.connection.png differ diff --git a/tools-and-applications/minio/img/connection.details.png b/tools-and-applications/minio/img/connection.details.png new file mode 100644 index 00000000..90f2b416 Binary files /dev/null and b/tools-and-applications/minio/img/connection.details.png differ diff --git a/tools-and-applications/minio/img/create.bucket.01.png b/tools-and-applications/minio/img/create.bucket.01.png new file mode 100644 index 00000000..f2f01f18 Binary files /dev/null and b/tools-and-applications/minio/img/create.bucket.01.png differ diff --git a/tools-and-applications/minio/img/create.bucket.02.png b/tools-and-applications/minio/img/create.bucket.02.png new file mode 100644 index 00000000..2f214836 Binary files /dev/null and b/tools-and-applications/minio/img/create.bucket.02.png differ diff --git a/tools-and-applications/minio/img/create.project.png b/tools-and-applications/minio/img/create.project.png new file mode 100644 index 00000000..a06d9429 Binary files /dev/null and b/tools-and-applications/minio/img/create.project.png differ diff --git a/tools-and-applications/minio/img/import.yaml.png b/tools-and-applications/minio/img/import.yaml.png new file mode 100644 index 00000000..3f4f5a30 Binary files /dev/null and b/tools-and-applications/minio/img/import.yaml.png differ diff --git a/tools-and-applications/minio/img/minio.login.png b/tools-and-applications/minio/img/minio.login.png new file mode 100644 index 00000000..d48f0768 Binary files /dev/null and b/tools-and-applications/minio/img/minio.login.png differ diff --git a/tools-and-applications/minio/img/openshift.console.png b/tools-and-applications/minio/img/openshift.console.png new file mode 100644 index 00000000..0165635a Binary files /dev/null and b/tools-and-applications/minio/img/openshift.console.png differ diff --git a/tools-and-applications/minio/img/resources.created.png b/tools-and-applications/minio/img/resources.created.png new file mode 100644 index 00000000..5e972680 Binary files /dev/null and b/tools-and-applications/minio/img/resources.created.png differ diff --git a/tools-and-applications/minio/img/routes.png b/tools-and-applications/minio/img/routes.png new file mode 100644 index 00000000..d7e4b654 Binary files /dev/null and b/tools-and-applications/minio/img/routes.png differ diff --git a/tools-and-applications/minio/img/running.pod.png b/tools-and-applications/minio/img/running.pod.png new file mode 100644 index 00000000..f7cdd1d4 Binary files /dev/null and b/tools-and-applications/minio/img/running.pod.png differ diff --git a/tools-and-applications/minio/img/workloads.pods.png b/tools-and-applications/minio/img/workloads.pods.png new file mode 100644 index 00000000..c32bf3c1 Binary files /dev/null and b/tools-and-applications/minio/img/workloads.pods.png differ diff --git a/tools-and-applications/minio/minio/index.html b/tools-and-applications/minio/minio/index.html new file mode 100644 index 00000000..f8d7c192 --- /dev/null +++ b/tools-and-applications/minio/minio/index.html @@ -0,0 +1,3492 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Minio - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

Minio

+

What is it?

+

Minio is a high-performance, S3 compatible object store. It can be deployed on a wide variety of platforms, and it comes in multiple flavors.

+

Why this guide?

+

This guide is a very quick way of deploying the community version of Minio in order to quickly setup a fully standalone Object Store, in an OpenShift Cluster. This can then be used for various prototyping tasks that require Object Storage.

+

Note that nothing in this guide should be used in production-grade environments. Also, Minio is not included in RHOAI, and Red Hat does not provide support for Minio.

+

Pre-requisites

+
    +
  • Access to an OpenShift cluster
  • +
  • Namespace-level admin permissions, or permission to create your own project
  • +
+

Deploying Minio on OpenShift

+

Create a Data Science Project (Optional)

+

If you already have your own Data Science Project, or OpenShift project, you can skip this step.

+
    +
  1. If your cluster already has Red Hat OpenShift AI installed, you can use the Dashboard Web Interface to create a Data Science project.
  2. +
  3. Simply navigate to Data Science Projects
  4. +
  5. And click Create Project
  6. +
  7. +

    Choose a name for your project (here, Showcase) and click Create:

    +

    alt_text

    +
  8. +
  9. +

    Make sure to make a note of the Resource name, in case it's different from the name.

    +
  10. +
+

Log on to your project in OpenShift Console

+
    +
  1. +

    Go to your cluster's OpenShift Console:

    +

    alt_text

    +
  2. +
  3. +

    Make sure you use the Administrator view, not the developer view.

    +
  4. +
  5. +

    Go to Workloads then Pods, and confirm the selected project is the right one

    +

    alt_text

    +
  6. +
  7. +

    You now have a project in which to deploy Minio

    +
  8. +
+

Deploy Minio in your project

+
    +
  1. +

    Click on the + ("Import YAML") button:

    +

    alt_text

    +
  2. +
  3. +

    Paste the following YAML in the box, but don't press ok yet!: +

    ---
    +kind: PersistentVolumeClaim
    +apiVersion: v1
    +metadata:
    +  name: minio-pvc
    +spec:
    +  accessModes:
    +    - ReadWriteOnce
    +  resources:
    +    requests:
    +      storage: 20Gi
    +  volumeMode: Filesystem
    +---
    +kind: Secret
    +apiVersion: v1
    +metadata:
    +  name: minio-secret
    +stringData:
    +  # change the username and password to your own values.
    +  # ensure that the user is at least 3 characters long and the password at least 8
    +  minio_root_user: minio
    +  minio_root_password: minio123
    +---
    +kind: Deployment
    +apiVersion: apps/v1
    +metadata:
    +  name: minio
    +spec:
    +  replicas: 1
    +  selector:
    +    matchLabels:
    +      app: minio
    +  template:
    +    metadata:
    +      creationTimestamp: null
    +      labels:
    +        app: minio
    +    spec:
    +      volumes:
    +        - name: data
    +          persistentVolumeClaim:
    +            claimName: minio-pvc
    +      containers:
    +        - resources:
    +            limits:
    +              cpu: 250m
    +              memory: 1Gi
    +            requests:
    +              cpu: 20m
    +              memory: 100Mi
    +          readinessProbe:
    +            tcpSocket:
    +              port: 9000
    +            initialDelaySeconds: 5
    +            timeoutSeconds: 1
    +            periodSeconds: 5
    +            successThreshold: 1
    +            failureThreshold: 3
    +          terminationMessagePath: /dev/termination-log
    +          name: minio
    +          livenessProbe:
    +            tcpSocket:
    +              port: 9000
    +            initialDelaySeconds: 30
    +            timeoutSeconds: 1
    +            periodSeconds: 5
    +            successThreshold: 1
    +            failureThreshold: 3
    +          env:
    +            - name: MINIO_ROOT_USER
    +              valueFrom:
    +                secretKeyRef:
    +                  name: minio-secret
    +                  key: minio_root_user
    +            - name: MINIO_ROOT_PASSWORD
    +              valueFrom:
    +                secretKeyRef:
    +                  name: minio-secret
    +                  key: minio_root_password
    +          ports:
    +            - containerPort: 9000
    +              protocol: TCP
    +            - containerPort: 9090
    +              protocol: TCP
    +          imagePullPolicy: IfNotPresent
    +          volumeMounts:
    +            - name: data
    +              mountPath: /data
    +              subPath: minio
    +          terminationMessagePolicy: File
    +          image: >-
    +            quay.io/minio/minio:RELEASE.2023-06-19T19-52-50Z
    +          args:
    +            - server
    +            - /data
    +            - --console-address
    +            - :9090
    +      restartPolicy: Always
    +      terminationGracePeriodSeconds: 30
    +      dnsPolicy: ClusterFirst
    +      securityContext: {}
    +      schedulerName: default-scheduler
    +  strategy:
    +    type: Recreate
    +  revisionHistoryLimit: 10
    +  progressDeadlineSeconds: 600
    +---
    +kind: Service
    +apiVersion: v1
    +metadata:
    +  name: minio-service
    +spec:
    +  ipFamilies:
    +    - IPv4
    +  ports:
    +    - name: api
    +      protocol: TCP
    +      port: 9000
    +      targetPort: 9000
    +    - name: ui
    +      protocol: TCP
    +      port: 9090
    +      targetPort: 9090
    +  internalTrafficPolicy: Cluster
    +  type: ClusterIP
    +  ipFamilyPolicy: SingleStack
    +  sessionAffinity: None
    +  selector:
    +    app: minio
    +---
    +kind: Route
    +apiVersion: route.openshift.io/v1
    +metadata:
    +  name: minio-api
    +spec:
    +  to:
    +    kind: Service
    +    name: minio-service
    +    weight: 100
    +  port:
    +    targetPort: api
    +  wildcardPolicy: None
    +  tls:
    +    termination: edge
    +    insecureEdgeTerminationPolicy: Redirect
    +---
    +kind: Route
    +apiVersion: route.openshift.io/v1
    +metadata:
    +  name: minio-ui
    +spec:
    +  to:
    +    kind: Service
    +    name: minio-service
    +    weight: 100
    +  port:
    +    targetPort: ui
    +  wildcardPolicy: None
    +  tls:
    +    termination: edge
    +    insecureEdgeTerminationPolicy: Redirect
    +

    +
  4. +
  5. +

    By default, the size of the storage is 20 GB. (see line 11). Change it if you need to.

    +
  6. +
  7. If you want to, edit lines 21-22 to change the default user/password.
  8. +
  9. Press Create.
  10. +
  11. +

    You should see:

    +

    alt_text

    +
  12. +
  13. +

    And there should now be a running minio pod:

    +

    alt_text

    +
  14. +
  15. +

    As well as two minio routes:

    +

    alt_text

    +
  16. +
  17. +

    The -api route is for programmatic access to Minio

    +
  18. +
  19. The -ui route is for browser-based access to Minio
  20. +
  21. Your Minio Object Store is now deployed, but we still need to create at least one bucket in it, to make it useful.
  22. +
+

Creating a bucket in Minio

+

Log in to Minio

+
    +
  1. Locate the minio-ui Route, and open its location URL in a web browser:
  2. +
  3. +

    When prompted, log in

    +
      +
    • if you kept the default values, then:
    • +
    • user: minio
    • +
    • pass: minio123
    • +
    +

    alt_text

    +
  4. +
  5. +

    You should now be logged into your Minio instance.

    +
  6. +
+

Create a bucket

+
    +
  1. +

    Click on Create a Bucket

    +

    alt_text

    +
  2. +
  3. +

    Choose a name for your bucket (for example mybucket) and click Create Bucket:

    +

    alt_text

    +
  4. +
  5. +

    Repeat those steps to create as many buckets as you will need.

    +
  6. +
+

Create a matching Data Connection for Minio

+
    +
  1. +

    Back in RHOAI, inside of your Data Science Project, Click on Add data connection:

    +

    alt_text

    +
  2. +
  3. +

    Then, fill out the required field to match with your newly-deployed Minio Object Storage

    +

    alt_text

    +
  4. +
  5. +

    You now have a Data Connection that maps to your mybucket bucket in your Minio Instance.

    +
  6. +
  7. This data connection can be used, among other things
      +
    • In your Workbenches
    • +
    • For your Model Serving
    • +
    • For your Pipeline Server Configuration
    • +
    +
  8. +
+

Validate

+

To test if everything is working correctly, you can access the workbench associated with your Data Connection and run the following commands (i.e., inside a Jupyter notebook):

+
    +
  1. +

    Install and import MinIO Python Client SDK

    +

    !pip install minio
    +
    +
    from minio import Minio
    +from minio.error import S3Error
    +import  os
    +import datetime
    +

    +
  2. +
  3. +

    Access Data Connection properties as environment variables:

    +
    # MinIO client doesn't like URLs with procotol/schema, so use
    +# yourendpoint.com instead of https://yourtendpoint.com
    +AWS_S3_ENDPOINT = os.getenv("AWS_S3_ENDPOINT")
    +AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
    +AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")
    +AWS_S3_BUCKET = os.getenv("AWS_S3_BUCKET")
    +
    +
  4. +
  5. +

    Create the MinIO client

    +
    # Create the MinIO client
    +client = Minio(
    +    AWS_S3_ENDPOINT,
    +    access_key=AWS_ACCESS_KEY_ID,
    +    secret_key=AWS_SECRET_ACCESS_KEY,
    +    secure=True  # Set to True if you are using HTTPS
    +)
    +
    +
  6. +
  7. +

    Test the connection by listing all buckets

    +
    #List all buckets
    +try:
    +    buckets = client.list_buckets()
    +    for bucket in buckets:
    +        print(bucket.name, bucket.creation_date)
    +except S3Error as e:
    +    print("Error occurred: ", e)
    +
    +
  8. +
  9. +

    Create a sample local file

    +
    # Create File
    +FILE_ON_DISK = 'file.txt'
    +
    +file = open(f"{FILE_ON_DISK}", "w")
    +file.write('Hello there %s recorded at %s.\n' % (FILE_ON_DISK, datetime.datetime.now()))
    +file.close()
    +
    +
  10. +
  11. +

    Upload a file to MinIO

    +
    # Upload a File 
    +file_path = FILE_ON_DISK
    +object_name = 'target-file.txt'
    +
    +try:
    +    client.fput_object(AWS_S3_BUCKET, object_name, file_path)
    +    print(f"'{object_name}' is successfully uploaded as object to bucket '{bucket_name}'.")
    +except S3Error as e:
    +    print("Error occurred: ", e)
    +
    +
  12. +
  13. +

    Download a file from MinIO

    +
    # Download a file 
    +object_name = 'target-file.txt'
    +file_path = 'file-froms3.txt'
    +
    +try:
    + client.fget_object(AWS_S3_BUCKET, object_name, file_path)
    + print(f"'{object_name}' is successfully downloaded to '{file_path}'.")
    +except S3Error as e:
    + print("Error occurred: ", e)
    +
    +
  14. +
  15. +

    List objects in our bucket

    +
    # Download a file 
    +object_name = 'target-file.txt'
    +file_path = 'file-froms3.txt'
    +
    +try:
    +    client.fget_object(AWS_S3_BUCKET, object_name, file_path)
    +    print(f"'{object_name}' is successfully downloaded to '{file_path}'.")
    +except S3Error as e:
    +    print("Error occurred: ", e)
    +
    +
  16. +
+

For more complete and detailed information about MinIO Python Client SDK usage, please check the official documentation.

+

Notes and FAQ

+
    +
  • As long as you are using the Route URLs, a Minio running in one namespace can be used by any other application, even running in another namespace, or even in another cluster altogether.
  • +
+

Uninstall instructions:

+

This will completely remove Minio and all its content. Make sure you have a backup of the things your need before doing so!

+
    +
  1. +

    Track down those objects created earlier:

    +

    alt_text

    +
  2. +
  3. +

    Delete them all.

    +
  4. +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/mlflow/img/MLFlow_capabilities.png b/tools-and-applications/mlflow/img/MLFlow_capabilities.png new file mode 100644 index 00000000..241649e8 Binary files /dev/null and b/tools-and-applications/mlflow/img/MLFlow_capabilities.png differ diff --git a/tools-and-applications/mlflow/img/enabled-tile.png b/tools-and-applications/mlflow/img/enabled-tile.png new file mode 100644 index 00000000..c554bc70 Binary files /dev/null and b/tools-and-applications/mlflow/img/enabled-tile.png differ diff --git a/tools-and-applications/mlflow/mlflow/index.html b/tools-and-applications/mlflow/mlflow/index.html new file mode 100644 index 00000000..cea372b1 --- /dev/null +++ b/tools-and-applications/mlflow/mlflow/index.html @@ -0,0 +1,3107 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + MLflow - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + + + + + +
+
+ + + + +

MLFlow

+

What is it?

+

MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. MLflow currently offers four components: +The 4 capabilities of MLFlow. Source: https://mlflow.org/ +Read more here: https://mlflow.org/

+

Helm installation into OpenShift namespace

+

Pre-requisites

+
    +
  • Install the "Crunchy Postgres for Kubernetes" operator (can be found in OperatorHub) - To store the MLFlow config
  • +
  • Install the "OpenShift Data Foundation" operator (can be found in OperatorHub) - To provide S3 storage for the experiments and models
  • +
+

Install

+
<Create an OpenShift project, either through the OpenShift UI or 'oc new-project project-name'>
+helm repo add strangiato https://strangiato.github.io/helm-charts/
+helm repo update
+<Log in to the correct OpenShift project through 'oc project project-name'>
+helm upgrade -i mlflow-server strangiato/mlflow-server
+
+

Additional Options

+

The MLFlow Server helm chart provides a number of customizable options when deploying MLFlow. These options can be configured using the --set flag with helm install or helm upgrade to set options directly on the command line or through a values.yaml file using the --values flag.

+

For a full list of configurable options, see the helm chart documentation:

+

https://github.com/strangiato/helm-charts/tree/main/charts/mlflow-server#values

+
OpenDataHub Dashboard Application Tile
+

As discussed in the Dashboard Configuration, ODH/RHOAI allows administrators to add a custom application tile for additional components on the cluster.

+

Enabled tile

+

The MLFlow Server helm chart supports creation of the Dashboard Application tile as a configurable value. If MLFlow Server is installed in the same namespace as ODH/RHOAI you can install the dashboard tile run the following command:

+
helm upgrade -i mlflow-server strangiato/mlflow-server \
+    --set odhApplication.enabled=true
+
+

The MLFlow Server helm chart also supports installing the odhApplication object in a different namespace, if MLFlow Server is not installed in the same namespace as ODH/RHOAI:

+
helm upgrade -i mlflow-server strangiato/mlflow-server \
+    --set odhApplication.enabled=true \
+    --set odhApplication.namespaceOverride=redhat-ods-applications
+
+

After enabling the odhApplication component, wait 1-2 minutes and the tile should appear in the Explorer view of the dashboard.

+
+

Note

+

This feature requires ODH v1.4.1 or newer

+
+

Test MLFlow

+
    +
  • Go to the OpenShift Console and switch to Developer view.
  • +
  • Go to the Topology view and make sure that you are on the MLFlow project.
  • +
  • Check that the MLFlow circle is dark blue (this means it has finished deploying).
  • +
  • Press the "External URL" link in the top right corner of the MLFlow circle to open up the MLFlow UI.
  • +
  • Run helm test mlflow-server in your command prompt to test MLFlow. If successful, you should see a new experiment called "helm-test" show up in the MLFlow UI with 3 experiments inside it.
  • +
+

Adding MLFlow to Training Code

+
import mlflow
+from sklearn.linear_model import LogisticRegression
+
+# Set tracking URI
+mlflow.set_tracking_uri(https://<route-to-mlflow>)
+
+# Setting the experiment
+mlflow.set_experiment("my-experiment")
+
+if __name__ == "__main__":
+    # Enabling automatic logging for scikit-learn runs
+    mlflow.sklearn.autolog()
+
+    # Starting a logging run
+    with mlflow.start_run():
+        # train
+
+

Source Code

+

MLFlow Server Source Code: +https://github.com/strangiato/mlflow-server

+

MLFlow Server Helm Chart Source Code: +https://github.com/strangiato/helm-charts/tree/main/charts/mlflow-server

+

Demos

+
    +
  • Credit Card Fraud Detection pipeline using MLFlow together with RHOAI: Demo
  • +
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/rclone/img/config-step-1.png b/tools-and-applications/rclone/img/config-step-1.png new file mode 100644 index 00000000..5f084bda Binary files /dev/null and b/tools-and-applications/rclone/img/config-step-1.png differ diff --git a/tools-and-applications/rclone/img/config-step2.png b/tools-and-applications/rclone/img/config-step2.png new file mode 100644 index 00000000..83baed2b Binary files /dev/null and b/tools-and-applications/rclone/img/config-step2.png differ diff --git a/tools-and-applications/rclone/img/configs.png b/tools-and-applications/rclone/img/configs.png new file mode 100644 index 00000000..62075a72 Binary files /dev/null and b/tools-and-applications/rclone/img/configs.png differ diff --git a/tools-and-applications/rclone/img/example-2-panes.png b/tools-and-applications/rclone/img/example-2-panes.png new file mode 100644 index 00000000..71abe505 Binary files /dev/null and b/tools-and-applications/rclone/img/example-2-panes.png differ diff --git a/tools-and-applications/rclone/img/explorer-2.png b/tools-and-applications/rclone/img/explorer-2.png new file mode 100644 index 00000000..dc2b3921 Binary files /dev/null and b/tools-and-applications/rclone/img/explorer-2.png differ diff --git a/tools-and-applications/rclone/img/explorer.png b/tools-and-applications/rclone/img/explorer.png new file mode 100644 index 00000000..d33bde09 Binary files /dev/null and b/tools-and-applications/rclone/img/explorer.png differ diff --git a/tools-and-applications/rclone/img/import-rclone-image.png b/tools-and-applications/rclone/img/import-rclone-image.png new file mode 100644 index 00000000..3c4839d3 Binary files /dev/null and b/tools-and-applications/rclone/img/import-rclone-image.png differ diff --git a/tools-and-applications/rclone/img/launch.png b/tools-and-applications/rclone/img/launch.png new file mode 100644 index 00000000..ec1fc1c8 Binary files /dev/null and b/tools-and-applications/rclone/img/launch.png differ diff --git a/tools-and-applications/rclone/img/login.png b/tools-and-applications/rclone/img/login.png new file mode 100644 index 00000000..4b33b8b7 Binary files /dev/null and b/tools-and-applications/rclone/img/login.png differ diff --git a/tools-and-applications/rclone/img/rclone-logo.svg b/tools-and-applications/rclone/img/rclone-logo.svg new file mode 100644 index 00000000..35360ef4 --- /dev/null +++ b/tools-and-applications/rclone/img/rclone-logo.svg @@ -0,0 +1,45 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tools-and-applications/rclone/img/transfer.png b/tools-and-applications/rclone/img/transfer.png new file mode 100644 index 00000000..aea3a047 Binary files /dev/null and b/tools-and-applications/rclone/img/transfer.png differ diff --git a/tools-and-applications/rclone/img/workbench-1.png b/tools-and-applications/rclone/img/workbench-1.png new file mode 100644 index 00000000..7e7dae47 Binary files /dev/null and b/tools-and-applications/rclone/img/workbench-1.png differ diff --git a/tools-and-applications/rclone/img/workbench-2.png b/tools-and-applications/rclone/img/workbench-2.png new file mode 100644 index 00000000..5590b082 Binary files /dev/null and b/tools-and-applications/rclone/img/workbench-2.png differ diff --git a/tools-and-applications/rclone/rclone/index.html b/tools-and-applications/rclone/rclone/index.html new file mode 100644 index 00000000..41b2d780 --- /dev/null +++ b/tools-and-applications/rclone/rclone/index.html @@ -0,0 +1,3009 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Rclone - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

Rclone

+

What is it?

+

Spark

+

Rclone is a program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

+

Users call rclone "The Swiss army knife of cloud storage", and "Technology indistinguishable from magic".

+

Rclone really looks after your data. It preserves timestamps and verifies checksums at all times. Transfers over limited bandwidth; intermittent connections, or subject to quota can be restarted, from the last good file transferred. You can check the integrity of your files. Where possible, rclone employs server-side transfers to minimize local bandwidth use and transfers from one provider to another without using local disk.

+

Rclone is mature, open-source software originally inspired by rsync and written in Go. The friendly support community is familiar with varied use cases.

+

The implementation described here is a containerized version of Rclone to run on OpenShift, alongside or integrated within ODH/RHOAI.

+

Deployment

+

Integrated in Open Data Hub or OpenShift AI

+

Use this method if you want to use Rclone from the ODH/RHOAI launcher or in a Data Science Project.

+
    +
  • In the Cluster Settings menu, import the image quay.io/guimou/rclone-web-openshift:odh-rhoai_latest. You can name it Rclone. +Import image
  • +
  • In your DSP project, create a new workbench using the Rclone image. You can set the storage size as minimal as it's only there to store the configuration of the endpoints. +Workbench +Workbench
  • +
+
+

Tip

+

The minimal size allowed by the dashboard for a storage volume is currently 1GB, which is way more than what is required for the Rclone configuration. So you can also create a much smaller PVC manually in the namespace corresponding to your Data Science Project, for example 100MB or less, and select this volume when creating the workbench.

+
+
    +
  • Launch Rclone from the link once it's deployed! +Launch
  • +
  • After the standard authentication, you end up on the Rclone Login page. There is nothing to enter, but I have not found yet how to bypass it. So simply click on "Login". +Login
  • +
+

Standalone deployment

+

Use this method if you want to use Rclone on its own in a namespace. You can still optionally make a shortcut appear in the ODH/RHOAI dashboard.

+
    +
  • Create a project/namespace for your installation.
  • +
  • Clone or head to this repo.
  • +
  • From the deploy folder, apply the different YAML files:
      +
    • 01-pvc.yaml: creates a persistent volume to hold the configuration
    • +
    • 02-deployment.yaml: creates the deployment. Modify admin account and password if you want to restrict access. You should!
    • +
    • 03-service.yaml, 04-route.yaml: create the external access so that you can connect to the Web UI.
    • +
    • Optionally, to create a tile on the ODH/RHOAI dashboard:
        +
      • modify the 05-tile.yaml file with the address of the Route that was created previously (namespace and name of the Route object).
      • +
      • the will appear under the available applications in the dashboard. Select it and click on "Enable" to make it appear in the "Enabled" menu.
      • +
      +
    • +
    +
  • +
+

Configuration

+

In this example, we will create an S3 configuration that connects to a bucket on the MCG from OpenShift Data Foundation. So you must have created this bucket in advance and have all the information about it: endpoint, access and secret keys, bucket name.

+
    +
  • In Rclone, click on "Configs" to create the new Remote. +Configs
  • +
  • Create new configuration, give it a name, and select "Amazon S3 Compliant Storage Providers...", which includes Ceph and MCG (even if not listed). +Step 1
  • +
  • Enter the connection info. You only have to enter the Access key and Secret, as well as the Endpoint in "Endpoint for S3 API". This last info is automatically copied in other fields, that's normal. +Step 2
  • +
  • Finalize the config by clicking on "Next" at the bottom.
  • +
+

Now that you have the Remote set up, you can go on the Explorer, select the Remote, and browse it! +Explorer +Explorer

+

Usage Example

+

In this simple example, we will transfer a dump sample from Wikipedia. Wikimedia publishes those dumps daily, and they are mirrored by different organizations. In a "standard" setup, loading those information into your object store would not be really practical, sometimes involving downloading it first locally to then push it to your storage.

+

This is how we can do it with Rclone.

+
    +
  • Create your Bucket Remote as described in Configuration.
  • +
  • Create another remote of type "HTTP", and enter the address of one of the mirrors. Here I used https://dumps.wikimedia.your.org/wikidatawiki/.
  • +
  • Open the Explorer view, set it in dual-pane layout. In the first pane open your Bucket Remote, and in the other one the HTTP. This is what it will look like: +Explorer
  • +
  • Browse to the folder you want, select a file or a folder, and simply drag and drop it from the Wikidump to your bucket. You can select a big one to make things more interesting!
  • +
  • Head for the dashboard where you will see the file transfer happening in the background. +Transfer
  • +
+

That's it! Nothing to install, high speed optimized transfer, and you could even do multiple transfers in the background,...

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/tools-and-applications/riva/img/api.png b/tools-and-applications/riva/img/api.png new file mode 100644 index 00000000..a1c3725a Binary files /dev/null and b/tools-and-applications/riva/img/api.png differ diff --git a/tools-and-applications/riva/img/asr.png b/tools-and-applications/riva/img/asr.png new file mode 100644 index 00000000..96f94463 Binary files /dev/null and b/tools-and-applications/riva/img/asr.png differ diff --git a/tools-and-applications/riva/img/nvidia-riva-client.png b/tools-and-applications/riva/img/nvidia-riva-client.png new file mode 100644 index 00000000..9fcfecc5 Binary files /dev/null and b/tools-and-applications/riva/img/nvidia-riva-client.png differ diff --git a/tools-and-applications/riva/riva/index.html b/tools-and-applications/riva/riva/index.html new file mode 100644 index 00000000..f58bfcc7 --- /dev/null +++ b/tools-and-applications/riva/riva/index.html @@ -0,0 +1,3078 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + NVIDIA Riva - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

NVIDIA RIVA

+

NVIDIA® Riva is a GPU-accelerated SDK for building Speech AI applications that are customized for your use case and deliver real-time performance.

+

Riva offers pretrained speech models in NVIDIA NGC™ that can be fine-tuned with the NVIDIA NeMo on a custom data set, accelerating the development of domain-specific models by 10x.

+

Models can be easily exported, optimized, and deployed as a speech service on premises or in the cloud with a single command using Helm charts.

+

Riva’s high-performance inference is powered by NVIDIA TensorRT™ optimizations and served using the NVIDIA Triton™ Inference Server, which are both part of the NVIDIA AI platform.

+

Riva services are available as gRPC-based microservices for low-latency streaming, as well as high-throughput offline use cases.

+

Riva is fully containerized and can easily scale to hundreds and thousands of parallel streams.

+

Deployment

+

The guide to deploy Riva on Kubernetes has to be adapted for OpenShift. Here are the different steps.

+

Prerequisites

+
    +
  1. You have access and are logged into NVIDIA NGC. For step-by-step instructions, refer to the NGC Getting Started Guide. Specifically you will need your API Key from NVIDIA NGC.
  2. +
  3. You have at least one worker node with an NVIDIA Volta™, NVIDIA Turing™, or an NVIDIA Ampere architecture-based GPU. For more information, refer to the Support Matrix.
  4. +
  5. The Node Feature Discovery and the NVIDIA operators have been properly installed and configured on your OpenShift Cluster to enable your GPU(s). Full instructions here
  6. +
  7. The Pod that will be deployed will consume about 10GB of RAM. Make sure you have enough resources on your node (on top of the GPU itself), and you don't have limits in place that would restrict this. GPU memory consumption will be about 12GB with all models loaded.
  8. +
+

Installation

+

Included in the NGC Helm Repository is a chart designed to automate deployment to a Kubernetes cluster. This chart must be modified for OpenShift.

+

The Riva Speech AI Helm Chart deploys the ASR, NLP, and TTS services automatically. The Helm chart performs a number of functions:

+
    +
  • Pulls Docker images from NGC for the Riva Speech AI server and utility containers for downloading and converting models.
  • +
  • Downloads the requested model artifacts from NGC as configured in the values.yaml file.
  • +
  • Generates the Triton Inference Server model repository.
  • +
  • Starts the Riva Speech AI server as configured in a Kubernetes pod.
  • +
  • Exposes the Riva Speech AI server as a configured service.
  • +
+

Examples of pretrained models are released with Riva for each of the services. The Helm chart comes preconfigured for downloading and deploying all of these models.

+

Installation Steps:

+
    +
  1. +

    Download the Helm chart

    +
    export NGC_API_KEY=<your_api_key>
    +helm fetch https://helm.ngc.nvidia.com/nvidia/riva/charts/riva-api-2.11.0.tgz \
    +        --username=\$oauthtoken --password=$NGC_API_KEY --untar
    +
    + +
  2. +
  3. +

    Switch to the newly created folder, riva-api

    +
  4. +
  5. +

    In the templates folder, modify the file deployment.yaml. For both the container riva-speech-api and the initContainer riva-model-init you must add the following security context information:

    +
    securityContext:
    +    allowPrivilegeEscalation: false
    +    capabilities:
    +      drop: ["ALL"]
    +    seccompProfile:
    +      type: "RuntimeDefault"
    +    runAsNonRoot: true
    +
    + +
  6. +
  7. +

    The file deployment.yaml should now look like this:

    +
    ...
    +apiVersion: apps/v1
    +kind: Deployment
    +metadata:
    +  name: {{ template "riva-server.fullname" . }}
    +  ...
    +spec:
    +  ...
    +  template:
    +    ...
    +    spec:
    +      containers:
    +        - name: riva-speech-api
    +          securityContext:
    +            allowPrivilegeEscalation: false
    +            capabilities:
    +              drop: ["ALL"]
    +            seccompProfile:
    +              type: "RuntimeDefault"
    +            runAsNonRoot: true
    +          image: {{ $server_image }}
    +          ...
    +      initContainers:
    +        - name: riva-model-init
    +          securityContext:
    +            allowPrivilegeEscalation: false
    +            capabilities:
    +              drop: ["ALL"]
    +            seccompProfile:
    +              type: "RuntimeDefault"
    +            runAsNonRoot: true
    +          image: {{ $servicemaker_image }}
    +          ...
    +
    + +
  8. +
  9. +

    At the root of riva-api, modify the file values.yaml:

    +
      +
    1. +

      You will need to convert your API Key to a password value. In a Terminal run:

      +
      echo -n $NGC_API_KEY | base64 -w0
      +
      + +
    2. +
    3. +

      In the ngcCredentials section ov values.yaml, enter the password you obtained above and your email

      +
    4. +
    5. In the modelRepoGenerator section, for the modelDeployKey value, enter dGx0X2VuY29kZQ==. (This value is obtained from the command echo -n tlt_encode | base64 -w0.
    6. +
    7. In the persistentVolumeClaim section, set usePVC to true. This is very important as it will disable the hostPath configuration for storage that is not permitted by default on OpenShift.
    8. +
    9. If you don't have a storageClass set as default, or want to you another one, enter the name of the class you want to use in storageClassName. Otherwise leave this field empty and the default class will be used.
    10. +
    11. Optionally, modify the storageSize.
    12. +
    13. Leave the ingress section as is, we will create an OpenShift Route later.
    14. +
    15. Optionally you can modify other values in the file to enable/disable certain models, or modify their configuration.
    16. +
    +
  10. +
  11. +

    Log into your OpenShift cluster from a Terminal, and create a project riva-api:

    +
    oc new-project riva-api
    +
    + +
  12. +
  13. +

    Move up one folder (so outside of the riva-api folder), and install NVIDIA Riva with the modified Helm chart:

    +
    helm install riva-api riva-api
    +
    + +
  14. +
+

The deployment will now start.

+
+

Info

+

Beware that the deployment can be really long the first time, about 45mn if you have all the models and features selected. Containers and models have to be downloaded and configured. Please be patient...

+
+

Usage

+

The Helm chart had automatically created a Service, riva-api in the namespace where you have deployed it. If you followed this guide, this should also be riva-api. So within the OpenShift cluster, the API is accessible at riva-api.riva-api.svc.cluster.local.

+

Different ports are accessible:

+
    +
  • http (8000): HTTP port of the Triton server.
  • +
  • grpc (8001): gRPC port of the Triton server.
  • +
  • metrics (8002): port for the metrics of the Triton server.
  • +
  • speech-grpc (50051): gRPC port of the Speech that exposes directly the different services you can use. This is normally the one that you will use.
  • +
+

If you want to use the API outside of the OpenShift cluster, you will have to create one or multiple Routes to those different endpoints.

+

Example

+
    +
  • On the same cluster where NVIDIA Riva is deployed, deploy RHOAI or ODH and launch a Notebook (Standard DataScience is enough).
  • +
  • Clone the NVIDIA Riva tutorials repository at https://github.com/nvidia-riva/tutorials
  • +
  • Open a Terminal and install the client with pip install nvidia-riva-client:
  • +
+

client

+

(depending on the base image you used this may yield errors that you can ignore most of times).

+
    +
  • In the tutorials folder, open the notebook asr-basics.ipynb.
  • +
  • In the cell that defines the uri of the API server, modify the default (localhost) for the address of the API server: riva-api.riva-api.svc.cluster.local
  • +
+

api

+
    +
  • Run the notebook!
  • +
+

asr

+
+

Note

+

In this example, only the first part of the notebook will work as only the English models have been deployed. You would have to adapt the configuration for other languages.

+
+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file diff --git a/whats-new/whats-new/index.html b/whats-new/whats-new/index.html new file mode 100644 index 00000000..c69a68a2 --- /dev/null +++ b/whats-new/whats-new/index.html @@ -0,0 +1,2797 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + Latest updates - AI on OpenShift + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + + + +
+ + + + + + + +
+ +
+ + + + +
+
+ + + +
+
+
+ + + + + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + +

What's new?

+

2024-08-21: Add Model serving type modification instructions.

+

2024-07-29: Add Kueue preemption example.

+

2024-07-16: Add ODH Tools and Extensions Companion documentation.

+

2024-07-16: Add GPU pruner documentation.

+

2024-07-02: Add Llama2 fine-tuning with Ray and DeepSpeed demo.

+

2024-06-25: Add GPU partitioning and sharing.

+

2024-06-18: Add Accelerator profiles documentation.

+

2024-01-03: Add Ensemble models demo.

+

2024-03-28: Add Zero to Workbench documentation.

+

2024-03-04: Refactoring of the site. Update on the LLMs deployment and RAG demo. Single Stack Serving certificate how-to.

+

2023-12-21: Name change for Red Hat OpenShift Data Science, which becomes Red Hat OpenShift AI.

+

2023-10-16: Add documentation for LLMs deployment and RAG demo.

+

2023-08-01: Update to Spark documentation to include usage without the operator Tools and Applications->Apache Spark

+

2023-07-05: Add documentation on Time Slicing and Autoscaling for NVIDIA GPUs ODH/RHOAI How-Tos->NVIDIA GPUs

+

2023-07-05: New example of how to configure a Custom Serving Runtime with Triton.

+

2023-07-03: New Minio tutorial on how to quickly deploy a simple Object Storage inside your OpenShift Project, for quick prototyping.

+

2023-06-30: New NVIDIA GPU installation documentation with Node tainting in ODH/RHOAI How-Tos->NVIDIA GPUs

+

2023-06-02: NVIDIA Riva documentation in Tools and Applications->NVIDIA Riva

+

NVIDIA® Riva is a GPU-accelerated SDK for building Speech AI applications that are customized for your use case and deliver real-time performance.

+

2023-02-06: Rclone documentation in Tools and Applications->Rclone.

+

Rclone is a program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

+
+

2023-02-02: Addition of VSCode and RStudio images to custom workbenches.

+
+

2023-01-22: Addition of StarProxy to Patterns->Starburst/Trino Proxy.

+

Starproxy is a fully HTTP compliant proxy that is designed to sit between clients and a Trino/Starburst cluster.

+ + + + + + + + + + + + + + + + + + +
+
+ + + +
+ + + +
+ + + +
+
+
+
+ + + + + + + + + + + + + \ No newline at end of file

Learn more