From 5b5b21edbcc281c3261a0ef6fe9ebd0f132de5b1 Mon Sep 17 00:00:00 2001 From: drasko Date: Wed, 20 Mar 2024 14:56:40 +0000 Subject: [PATCH] Deployed de32611 with MkDocs version: 1.4.2 --- .nojekyll | 0 404.html | 493 ++ CNAME | 1 + agent/index.html | 606 ++ architecture/index.html | 613 ++ assets/images/favicon.png | Bin 0 -> 1870 bytes assets/javascripts/bundle.51d95adb.min.js | 29 + assets/javascripts/bundle.51d95adb.min.js.map | 8 + assets/javascripts/lunr/min/lunr.ar.min.js | 1 + assets/javascripts/lunr/min/lunr.da.min.js | 18 + assets/javascripts/lunr/min/lunr.de.min.js | 18 + assets/javascripts/lunr/min/lunr.du.min.js | 18 + assets/javascripts/lunr/min/lunr.es.min.js | 18 + assets/javascripts/lunr/min/lunr.fi.min.js | 18 + assets/javascripts/lunr/min/lunr.fr.min.js | 18 + assets/javascripts/lunr/min/lunr.hi.min.js | 1 + assets/javascripts/lunr/min/lunr.hu.min.js | 18 + assets/javascripts/lunr/min/lunr.it.min.js | 18 + assets/javascripts/lunr/min/lunr.ja.min.js | 1 + assets/javascripts/lunr/min/lunr.jp.min.js | 1 + assets/javascripts/lunr/min/lunr.ko.min.js | 1 + assets/javascripts/lunr/min/lunr.multi.min.js | 1 + assets/javascripts/lunr/min/lunr.nl.min.js | 18 + assets/javascripts/lunr/min/lunr.no.min.js | 18 + assets/javascripts/lunr/min/lunr.pt.min.js | 18 + assets/javascripts/lunr/min/lunr.ro.min.js | 18 + assets/javascripts/lunr/min/lunr.ru.min.js | 18 + .../lunr/min/lunr.stemmer.support.min.js | 1 + assets/javascripts/lunr/min/lunr.sv.min.js | 18 + assets/javascripts/lunr/min/lunr.ta.min.js | 1 + assets/javascripts/lunr/min/lunr.th.min.js | 1 + assets/javascripts/lunr/min/lunr.tr.min.js | 18 + assets/javascripts/lunr/min/lunr.vi.min.js | 1 + assets/javascripts/lunr/min/lunr.zh.min.js | 1 + assets/javascripts/lunr/tinyseg.js | 206 + assets/javascripts/lunr/wordcut.js | 6708 +++++++++++++++++ .../workers/search.e5c33ebb.min.js | 42 + .../workers/search.e5c33ebb.min.js.map | 8 + assets/stylesheets/main.558e4712.min.css | 1 + assets/stylesheets/main.558e4712.min.css.map | 1 + assets/stylesheets/palette.2505c338.min.css | 1 + .../stylesheets/palette.2505c338.min.css.map | 1 + attestation/index.html | 516 ++ cli/index.html | 712 ++ hal/index.html | 601 ++ img/hal.png | Bin 0 -> 36869 bytes index.html | 597 ++ install/index.html | 539 ++ manager/index.html | 791 ++ search/search_index.json | 1 + sitemap.xml | 48 + sitemap.xml.gz | Bin 0 -> 260 bytes tee/index.html | 576 ++ 53 files changed, 13380 insertions(+) create mode 100644 .nojekyll create mode 100644 404.html create mode 100644 CNAME create mode 100644 agent/index.html create mode 100644 architecture/index.html create mode 100644 assets/images/favicon.png create mode 100644 assets/javascripts/bundle.51d95adb.min.js create mode 100644 assets/javascripts/bundle.51d95adb.min.js.map create mode 100644 assets/javascripts/lunr/min/lunr.ar.min.js create mode 100644 assets/javascripts/lunr/min/lunr.da.min.js create mode 100644 assets/javascripts/lunr/min/lunr.de.min.js create mode 100644 assets/javascripts/lunr/min/lunr.du.min.js create mode 100644 assets/javascripts/lunr/min/lunr.es.min.js create mode 100644 assets/javascripts/lunr/min/lunr.fi.min.js create mode 100644 assets/javascripts/lunr/min/lunr.fr.min.js create mode 100644 assets/javascripts/lunr/min/lunr.hi.min.js create mode 100644 assets/javascripts/lunr/min/lunr.hu.min.js create mode 100644 assets/javascripts/lunr/min/lunr.it.min.js create mode 100644 assets/javascripts/lunr/min/lunr.ja.min.js create mode 100644 assets/javascripts/lunr/min/lunr.jp.min.js create mode 100644 assets/javascripts/lunr/min/lunr.ko.min.js create mode 100644 assets/javascripts/lunr/min/lunr.multi.min.js create mode 100644 assets/javascripts/lunr/min/lunr.nl.min.js create mode 100644 assets/javascripts/lunr/min/lunr.no.min.js create mode 100644 assets/javascripts/lunr/min/lunr.pt.min.js create mode 100644 assets/javascripts/lunr/min/lunr.ro.min.js create mode 100644 assets/javascripts/lunr/min/lunr.ru.min.js create mode 100644 assets/javascripts/lunr/min/lunr.stemmer.support.min.js create mode 100644 assets/javascripts/lunr/min/lunr.sv.min.js create mode 100644 assets/javascripts/lunr/min/lunr.ta.min.js create mode 100644 assets/javascripts/lunr/min/lunr.th.min.js create mode 100644 assets/javascripts/lunr/min/lunr.tr.min.js create mode 100644 assets/javascripts/lunr/min/lunr.vi.min.js create mode 100644 assets/javascripts/lunr/min/lunr.zh.min.js create mode 100644 assets/javascripts/lunr/tinyseg.js create mode 100644 assets/javascripts/lunr/wordcut.js create mode 100644 assets/javascripts/workers/search.e5c33ebb.min.js create mode 100644 assets/javascripts/workers/search.e5c33ebb.min.js.map create mode 100644 assets/stylesheets/main.558e4712.min.css create mode 100644 assets/stylesheets/main.558e4712.min.css.map create mode 100644 assets/stylesheets/palette.2505c338.min.css create mode 100644 assets/stylesheets/palette.2505c338.min.css.map create mode 100644 attestation/index.html create mode 100644 cli/index.html create mode 100644 hal/index.html create mode 100644 img/hal.png create mode 100644 index.html create mode 100644 install/index.html create mode 100644 manager/index.html create mode 100644 search/search_index.json create mode 100644 sitemap.xml create mode 100644 sitemap.xml.gz create mode 100644 tee/index.html diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 0000000..e69de29 diff --git a/404.html b/404.html new file mode 100644 index 0000000..cd7de52 --- /dev/null +++ b/404.html @@ -0,0 +1,493 @@ + + + + + + + + + + + + + + + + + + + + Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ +

404 - Not found

+ +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/CNAME b/CNAME new file mode 100644 index 0000000..06916d6 --- /dev/null +++ b/CNAME @@ -0,0 +1 @@ +docs.cocos.ultraviolet.rs diff --git a/agent/index.html b/agent/index.html new file mode 100644 index 0000000..a63e572 --- /dev/null +++ b/agent/index.html @@ -0,0 +1,606 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + Agent - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Agent#

+

The agent is responsible for the life cycle of the computation, i.e., running the computation and sending events about the status of the computation within the TEE. The agent is found inside the VM (TEE), and each computation within the TEE has its own agent. When a computation run request is sent from from the manager, manager creates a VM where the agent is found and sends the computation manifest to the agent.

+

Agent Events#

+

As the computation in the agent undergoes different operations, it sends events to the manager so that the user can monitor the computation from either the UI or other client. Events sent to the manager include computation running, computation finished, computation failed, and computation stopped.

+

Vsock Connection Between Agent & Manager#

+

Agent sends agent events to the manager via vsock. The manager listens to the vsock and forwards the events via gRPC. The agent events are used to show the status of the computation inside the TEE so that a user can be aware of what is happening inside the TEE.

+

Security#

+

To run a computation in the agent, a signed certificate is required. The certificate is used to verify the user who is running the computation. The certificate is sent to the agent by the manager, and the agent verifies the certificate before running the computation.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/architecture/index.html b/architecture/index.html new file mode 100644 index 0000000..e7f4afd --- /dev/null +++ b/architecture/index.html @@ -0,0 +1,613 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + Architecture - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Architecture#

+

Cocos AI system is a distributed platform for running secure multi-party computations.

+

It has 2 parts:

+
    +
  • Manager, that acts as a bridge between the web and the TEE and controls the creation and management of computations in the TEE.
    • +
  • In-TEE (Trusted Execution Environment) fimrware, otherwise called Agent
    • +
+

The system architecture is illustrated in the image below.

+

Agent#

+

Agent defines firmware which goes into the TEE and is used to control and monitor computation within TEE and enable secure and encrypted communication with outside world (in order to fetch the data and provide the result of the computation). The Agent contains a gRPC server that listens for requests from gRPC clients. Communication between the Manager and Agent is done via vsock. The Agent sends events to the Manager via vsock, which then forwards these via gRPC. Agent contains a gRPC server that exposes useful functions that can be accessed by other gRPC clients such as the CLI.

+

Manager#

+

Manager is a gRPC client that listens to requests sent through gRPC and sends them to Agent via vsock. Manager creates a secure enclave and loads the computation where the agent resides. The connection between Manager and Agent is through vsock, through which channel agent sends events periodically to manager, who forwards these via gRPC.

+

CLI#

+

CoCoS CLI is used to access the agent within the secure enclave. CLI communicates to agent using gRPC, with funcitons such as algo to provide the algorithm to be run, data to provide the data to be used in the computation, and run to start the computation. It also has functions to fetch and validate the attestation report of the enclave.

+

For more information on CLI, please refer to CLI docs.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/assets/images/favicon.png b/assets/images/favicon.png new file mode 100644 index 0000000000000000000000000000000000000000..1cf13b9f9d978896599290a74f77d5dbe7d1655c GIT binary patch literal 1870 zcmV-U2eJ5xP)Gc)JR9QMau)O=X#!i9;T z37kk-upj^(fsR36MHs_+1RCI)NNu9}lD0S{B^g8PN?Ww(5|~L#Ng*g{WsqleV}|#l zz8@ri&cTzw_h33bHI+12+kK6WN$h#n5cD8OQt`5kw6p~9H3()bUQ8OS4Q4HTQ=1Ol z_JAocz`fLbT2^{`8n~UAo=#AUOf=SOq4pYkt;XbC&f#7lb$*7=$na!mWCQ`dBQsO0 zLFBSPj*N?#u5&pf2t4XjEGH|=pPQ8xh7tpx;US5Cx_Ju;!O`ya-yF`)b%TEt5>eP1ZX~}sjjA%FJF?h7cX8=b!DZl<6%Cv z*G0uvvU+vmnpLZ2paivG-(cd*y3$hCIcsZcYOGh{$&)A6*XX&kXZd3G8m)G$Zz-LV z^GF3VAW^Mdv!)4OM8EgqRiz~*Cji;uzl2uC9^=8I84vNp;ltJ|q-*uQwGp2ma6cY7 z;`%`!9UXO@fr&Ebapfs34OmS9^u6$)bJxrucutf>`dKPKT%%*d3XlFVKunp9 zasduxjrjs>f8V=D|J=XNZp;_Zy^WgQ$9WDjgY=z@stwiEBm9u5*|34&1Na8BMjjgf3+SHcr`5~>oz1Y?SW^=K z^bTyO6>Gar#P_W2gEMwq)ot3; zREHn~U&Dp0l6YT0&k-wLwYjb?5zGK`W6S2v+K>AM(95m2C20L|3m~rN8dprPr@t)5lsk9Hu*W z?pS990s;Ez=+Rj{x7p``4>+c0G5^pYnB1^!TL=(?HLHZ+HicG{~4F1d^5Awl_2!1jICM-!9eoLhbbT^;yHcefyTAaqRcY zmuctDopPT!%k+}x%lZRKnzykr2}}XfG_ne?nRQO~?%hkzo;@RN{P6o`&mMUWBYMTe z6i8ChtjX&gXl`nvrU>jah)2iNM%JdjqoaeaU%yVn!^70x-flljp6Q5tK}5}&X8&&G zX3fpb3E(!rH=zVI_9Gjl45w@{(ITqngWFe7@9{mX;tO25Z_8 zQHEpI+FkTU#4xu>RkN>b3Tnc3UpWzPXWm#o55GKF09j^Mh~)K7{QqbO_~(@CVq! zS<8954|P8mXN2MRs86xZ&Q4EfM@JB94b=(YGuk)s&^jiSF=t3*oNK3`rD{H`yQ?d; ztE=laAUoZx5?RC8*WKOj`%LXEkgDd>&^Q4M^z`%u0rg-It=hLCVsq!Z%^6eB-OvOT zFZ28TN&cRmgU}Elrnk43)!>Z1FCPL2K$7}gwzIc48NX}#!A1BpJP?#v5wkNprhV** z?Cpalt1oH&{r!o3eSKc&ap)iz2BTn_VV`4>9M^b3;(YY}4>#ML6{~(4mH+?%07*qo IM6N<$f(jP3KmY&$ literal 0 HcmV?d00001 diff --git a/assets/javascripts/bundle.51d95adb.min.js b/assets/javascripts/bundle.51d95adb.min.js new file mode 100644 index 0000000..b20ec68 --- /dev/null +++ b/assets/javascripts/bundle.51d95adb.min.js @@ -0,0 +1,29 @@ +"use strict";(()=>{var Hi=Object.create;var xr=Object.defineProperty;var Pi=Object.getOwnPropertyDescriptor;var $i=Object.getOwnPropertyNames,kt=Object.getOwnPropertySymbols,Ii=Object.getPrototypeOf,Er=Object.prototype.hasOwnProperty,an=Object.prototype.propertyIsEnumerable;var on=(e,t,r)=>t in e?xr(e,t,{enumerable:!0,configurable:!0,writable:!0,value:r}):e[t]=r,P=(e,t)=>{for(var r in t||(t={}))Er.call(t,r)&&on(e,r,t[r]);if(kt)for(var r of kt(t))an.call(t,r)&&on(e,r,t[r]);return e};var sn=(e,t)=>{var r={};for(var n in e)Er.call(e,n)&&t.indexOf(n)<0&&(r[n]=e[n]);if(e!=null&&kt)for(var n of kt(e))t.indexOf(n)<0&&an.call(e,n)&&(r[n]=e[n]);return r};var Ht=(e,t)=>()=>(t||e((t={exports:{}}).exports,t),t.exports);var Fi=(e,t,r,n)=>{if(t&&typeof t=="object"||typeof t=="function")for(let o of $i(t))!Er.call(e,o)&&o!==r&&xr(e,o,{get:()=>t[o],enumerable:!(n=Pi(t,o))||n.enumerable});return e};var yt=(e,t,r)=>(r=e!=null?Hi(Ii(e)):{},Fi(t||!e||!e.__esModule?xr(r,"default",{value:e,enumerable:!0}):r,e));var fn=Ht((wr,cn)=>{(function(e,t){typeof wr=="object"&&typeof cn!="undefined"?t():typeof define=="function"&&define.amd?define(t):t()})(wr,function(){"use strict";function e(r){var n=!0,o=!1,i=null,a={text:!0,search:!0,url:!0,tel:!0,email:!0,password:!0,number:!0,date:!0,month:!0,week:!0,time:!0,datetime:!0,"datetime-local":!0};function s(T){return!!(T&&T!==document&&T.nodeName!=="HTML"&&T.nodeName!=="BODY"&&"classList"in T&&"contains"in T.classList)}function f(T){var Ke=T.type,We=T.tagName;return!!(We==="INPUT"&&a[Ke]&&!T.readOnly||We==="TEXTAREA"&&!T.readOnly||T.isContentEditable)}function c(T){T.classList.contains("focus-visible")||(T.classList.add("focus-visible"),T.setAttribute("data-focus-visible-added",""))}function u(T){T.hasAttribute("data-focus-visible-added")&&(T.classList.remove("focus-visible"),T.removeAttribute("data-focus-visible-added"))}function p(T){T.metaKey||T.altKey||T.ctrlKey||(s(r.activeElement)&&c(r.activeElement),n=!0)}function m(T){n=!1}function d(T){s(T.target)&&(n||f(T.target))&&c(T.target)}function h(T){s(T.target)&&(T.target.classList.contains("focus-visible")||T.target.hasAttribute("data-focus-visible-added"))&&(o=!0,window.clearTimeout(i),i=window.setTimeout(function(){o=!1},100),u(T.target))}function v(T){document.visibilityState==="hidden"&&(o&&(n=!0),B())}function B(){document.addEventListener("mousemove",z),document.addEventListener("mousedown",z),document.addEventListener("mouseup",z),document.addEventListener("pointermove",z),document.addEventListener("pointerdown",z),document.addEventListener("pointerup",z),document.addEventListener("touchmove",z),document.addEventListener("touchstart",z),document.addEventListener("touchend",z)}function re(){document.removeEventListener("mousemove",z),document.removeEventListener("mousedown",z),document.removeEventListener("mouseup",z),document.removeEventListener("pointermove",z),document.removeEventListener("pointerdown",z),document.removeEventListener("pointerup",z),document.removeEventListener("touchmove",z),document.removeEventListener("touchstart",z),document.removeEventListener("touchend",z)}function z(T){T.target.nodeName&&T.target.nodeName.toLowerCase()==="html"||(n=!1,re())}document.addEventListener("keydown",p,!0),document.addEventListener("mousedown",m,!0),document.addEventListener("pointerdown",m,!0),document.addEventListener("touchstart",m,!0),document.addEventListener("visibilitychange",v,!0),B(),r.addEventListener("focus",d,!0),r.addEventListener("blur",h,!0),r.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&r.host?r.host.setAttribute("data-js-focus-visible",""):r.nodeType===Node.DOCUMENT_NODE&&(document.documentElement.classList.add("js-focus-visible"),document.documentElement.setAttribute("data-js-focus-visible",""))}if(typeof window!="undefined"&&typeof document!="undefined"){window.applyFocusVisiblePolyfill=e;var t;try{t=new CustomEvent("focus-visible-polyfill-ready")}catch(r){t=document.createEvent("CustomEvent"),t.initCustomEvent("focus-visible-polyfill-ready",!1,!1,{})}window.dispatchEvent(t)}typeof document!="undefined"&&e(document)})});var un=Ht(Sr=>{(function(e){var t=function(){try{return!!Symbol.iterator}catch(c){return!1}},r=t(),n=function(c){var u={next:function(){var p=c.shift();return{done:p===void 0,value:p}}};return r&&(u[Symbol.iterator]=function(){return u}),u},o=function(c){return encodeURIComponent(c).replace(/%20/g,"+")},i=function(c){return decodeURIComponent(String(c).replace(/\+/g," "))},a=function(){var c=function(p){Object.defineProperty(this,"_entries",{writable:!0,value:{}});var m=typeof p;if(m!=="undefined")if(m==="string")p!==""&&this._fromString(p);else if(p instanceof c){var d=this;p.forEach(function(re,z){d.append(z,re)})}else if(p!==null&&m==="object")if(Object.prototype.toString.call(p)==="[object Array]")for(var h=0;hd[0]?1:0}),c._entries&&(c._entries={});for(var p=0;p1?i(d[1]):"")}})})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Sr);(function(e){var t=function(){try{var o=new e.URL("b","http://a");return o.pathname="c d",o.href==="http://a/c%20d"&&o.searchParams}catch(i){return!1}},r=function(){var o=e.URL,i=function(f,c){typeof f!="string"&&(f=String(f)),c&&typeof c!="string"&&(c=String(c));var u=document,p;if(c&&(e.location===void 0||c!==e.location.href)){c=c.toLowerCase(),u=document.implementation.createHTMLDocument(""),p=u.createElement("base"),p.href=c,u.head.appendChild(p);try{if(p.href.indexOf(c)!==0)throw new Error(p.href)}catch(T){throw new Error("URL unable to set base "+c+" due to "+T)}}var m=u.createElement("a");m.href=f,p&&(u.body.appendChild(m),m.href=m.href);var d=u.createElement("input");if(d.type="url",d.value=f,m.protocol===":"||!/:/.test(m.href)||!d.checkValidity()&&!c)throw new TypeError("Invalid URL");Object.defineProperty(this,"_anchorElement",{value:m});var h=new e.URLSearchParams(this.search),v=!0,B=!0,re=this;["append","delete","set"].forEach(function(T){var Ke=h[T];h[T]=function(){Ke.apply(h,arguments),v&&(B=!1,re.search=h.toString(),B=!0)}}),Object.defineProperty(this,"searchParams",{value:h,enumerable:!0});var z=void 0;Object.defineProperty(this,"_updateSearchParams",{enumerable:!1,configurable:!1,writable:!1,value:function(){this.search!==z&&(z=this.search,B&&(v=!1,this.searchParams._fromString(this.search),v=!0))}})},a=i.prototype,s=function(f){Object.defineProperty(a,f,{get:function(){return this._anchorElement[f]},set:function(c){this._anchorElement[f]=c},enumerable:!0})};["hash","host","hostname","port","protocol"].forEach(function(f){s(f)}),Object.defineProperty(a,"search",{get:function(){return this._anchorElement.search},set:function(f){this._anchorElement.search=f,this._updateSearchParams()},enumerable:!0}),Object.defineProperties(a,{toString:{get:function(){var f=this;return function(){return f.href}}},href:{get:function(){return this._anchorElement.href.replace(/\?$/,"")},set:function(f){this._anchorElement.href=f,this._updateSearchParams()},enumerable:!0},pathname:{get:function(){return this._anchorElement.pathname.replace(/(^\/?)/,"/")},set:function(f){this._anchorElement.pathname=f},enumerable:!0},origin:{get:function(){var f={"http:":80,"https:":443,"ftp:":21}[this._anchorElement.protocol],c=this._anchorElement.port!=f&&this._anchorElement.port!=="";return this._anchorElement.protocol+"//"+this._anchorElement.hostname+(c?":"+this._anchorElement.port:"")},enumerable:!0},password:{get:function(){return""},set:function(f){},enumerable:!0},username:{get:function(){return""},set:function(f){},enumerable:!0}}),i.createObjectURL=function(f){return o.createObjectURL.apply(o,arguments)},i.revokeObjectURL=function(f){return o.revokeObjectURL.apply(o,arguments)},e.URL=i};if(t()||r(),e.location!==void 0&&!("origin"in e.location)){var n=function(){return e.location.protocol+"//"+e.location.hostname+(e.location.port?":"+e.location.port:"")};try{Object.defineProperty(e.location,"origin",{get:n,enumerable:!0})}catch(o){setInterval(function(){e.location.origin=n()},100)}}})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Sr)});var Qr=Ht((Lt,Kr)=>{/*! + * clipboard.js v2.0.11 + * https://clipboardjs.com/ + * + * Licensed MIT © Zeno Rocha + */(function(t,r){typeof Lt=="object"&&typeof Kr=="object"?Kr.exports=r():typeof define=="function"&&define.amd?define([],r):typeof Lt=="object"?Lt.ClipboardJS=r():t.ClipboardJS=r()})(Lt,function(){return function(){var e={686:function(n,o,i){"use strict";i.d(o,{default:function(){return ki}});var a=i(279),s=i.n(a),f=i(370),c=i.n(f),u=i(817),p=i.n(u);function m(j){try{return document.execCommand(j)}catch(O){return!1}}var d=function(O){var w=p()(O);return m("cut"),w},h=d;function v(j){var O=document.documentElement.getAttribute("dir")==="rtl",w=document.createElement("textarea");w.style.fontSize="12pt",w.style.border="0",w.style.padding="0",w.style.margin="0",w.style.position="absolute",w.style[O?"right":"left"]="-9999px";var k=window.pageYOffset||document.documentElement.scrollTop;return w.style.top="".concat(k,"px"),w.setAttribute("readonly",""),w.value=j,w}var B=function(O,w){var k=v(O);w.container.appendChild(k);var F=p()(k);return m("copy"),k.remove(),F},re=function(O){var w=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body},k="";return typeof O=="string"?k=B(O,w):O instanceof HTMLInputElement&&!["text","search","url","tel","password"].includes(O==null?void 0:O.type)?k=B(O.value,w):(k=p()(O),m("copy")),k},z=re;function T(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?T=function(w){return typeof w}:T=function(w){return w&&typeof Symbol=="function"&&w.constructor===Symbol&&w!==Symbol.prototype?"symbol":typeof w},T(j)}var Ke=function(){var O=arguments.length>0&&arguments[0]!==void 0?arguments[0]:{},w=O.action,k=w===void 0?"copy":w,F=O.container,q=O.target,Le=O.text;if(k!=="copy"&&k!=="cut")throw new Error('Invalid "action" value, use either "copy" or "cut"');if(q!==void 0)if(q&&T(q)==="object"&&q.nodeType===1){if(k==="copy"&&q.hasAttribute("disabled"))throw new Error('Invalid "target" attribute. Please use "readonly" instead of "disabled" attribute');if(k==="cut"&&(q.hasAttribute("readonly")||q.hasAttribute("disabled")))throw new Error(`Invalid "target" attribute. You can't cut text from elements with "readonly" or "disabled" attributes`)}else throw new Error('Invalid "target" value, use a valid Element');if(Le)return z(Le,{container:F});if(q)return k==="cut"?h(q):z(q,{container:F})},We=Ke;function Ie(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?Ie=function(w){return typeof w}:Ie=function(w){return w&&typeof Symbol=="function"&&w.constructor===Symbol&&w!==Symbol.prototype?"symbol":typeof w},Ie(j)}function Ti(j,O){if(!(j instanceof O))throw new TypeError("Cannot call a class as a function")}function nn(j,O){for(var w=0;w0&&arguments[0]!==void 0?arguments[0]:{};this.action=typeof F.action=="function"?F.action:this.defaultAction,this.target=typeof F.target=="function"?F.target:this.defaultTarget,this.text=typeof F.text=="function"?F.text:this.defaultText,this.container=Ie(F.container)==="object"?F.container:document.body}},{key:"listenClick",value:function(F){var q=this;this.listener=c()(F,"click",function(Le){return q.onClick(Le)})}},{key:"onClick",value:function(F){var q=F.delegateTarget||F.currentTarget,Le=this.action(q)||"copy",Rt=We({action:Le,container:this.container,target:this.target(q),text:this.text(q)});this.emit(Rt?"success":"error",{action:Le,text:Rt,trigger:q,clearSelection:function(){q&&q.focus(),window.getSelection().removeAllRanges()}})}},{key:"defaultAction",value:function(F){return yr("action",F)}},{key:"defaultTarget",value:function(F){var q=yr("target",F);if(q)return document.querySelector(q)}},{key:"defaultText",value:function(F){return yr("text",F)}},{key:"destroy",value:function(){this.listener.destroy()}}],[{key:"copy",value:function(F){var q=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body};return z(F,q)}},{key:"cut",value:function(F){return h(F)}},{key:"isSupported",value:function(){var F=arguments.length>0&&arguments[0]!==void 0?arguments[0]:["copy","cut"],q=typeof F=="string"?[F]:F,Le=!!document.queryCommandSupported;return q.forEach(function(Rt){Le=Le&&!!document.queryCommandSupported(Rt)}),Le}}]),w}(s()),ki=Ri},828:function(n){var o=9;if(typeof Element!="undefined"&&!Element.prototype.matches){var i=Element.prototype;i.matches=i.matchesSelector||i.mozMatchesSelector||i.msMatchesSelector||i.oMatchesSelector||i.webkitMatchesSelector}function a(s,f){for(;s&&s.nodeType!==o;){if(typeof s.matches=="function"&&s.matches(f))return s;s=s.parentNode}}n.exports=a},438:function(n,o,i){var a=i(828);function s(u,p,m,d,h){var v=c.apply(this,arguments);return u.addEventListener(m,v,h),{destroy:function(){u.removeEventListener(m,v,h)}}}function f(u,p,m,d,h){return typeof u.addEventListener=="function"?s.apply(null,arguments):typeof m=="function"?s.bind(null,document).apply(null,arguments):(typeof u=="string"&&(u=document.querySelectorAll(u)),Array.prototype.map.call(u,function(v){return s(v,p,m,d,h)}))}function c(u,p,m,d){return function(h){h.delegateTarget=a(h.target,p),h.delegateTarget&&d.call(u,h)}}n.exports=f},879:function(n,o){o.node=function(i){return i!==void 0&&i instanceof HTMLElement&&i.nodeType===1},o.nodeList=function(i){var a=Object.prototype.toString.call(i);return i!==void 0&&(a==="[object NodeList]"||a==="[object HTMLCollection]")&&"length"in i&&(i.length===0||o.node(i[0]))},o.string=function(i){return typeof i=="string"||i instanceof String},o.fn=function(i){var a=Object.prototype.toString.call(i);return a==="[object Function]"}},370:function(n,o,i){var a=i(879),s=i(438);function f(m,d,h){if(!m&&!d&&!h)throw new Error("Missing required arguments");if(!a.string(d))throw new TypeError("Second argument must be a String");if(!a.fn(h))throw new TypeError("Third argument must be a Function");if(a.node(m))return c(m,d,h);if(a.nodeList(m))return u(m,d,h);if(a.string(m))return p(m,d,h);throw new TypeError("First argument must be a String, HTMLElement, HTMLCollection, or NodeList")}function c(m,d,h){return m.addEventListener(d,h),{destroy:function(){m.removeEventListener(d,h)}}}function u(m,d,h){return Array.prototype.forEach.call(m,function(v){v.addEventListener(d,h)}),{destroy:function(){Array.prototype.forEach.call(m,function(v){v.removeEventListener(d,h)})}}}function p(m,d,h){return s(document.body,m,d,h)}n.exports=f},817:function(n){function o(i){var a;if(i.nodeName==="SELECT")i.focus(),a=i.value;else if(i.nodeName==="INPUT"||i.nodeName==="TEXTAREA"){var s=i.hasAttribute("readonly");s||i.setAttribute("readonly",""),i.select(),i.setSelectionRange(0,i.value.length),s||i.removeAttribute("readonly"),a=i.value}else{i.hasAttribute("contenteditable")&&i.focus();var f=window.getSelection(),c=document.createRange();c.selectNodeContents(i),f.removeAllRanges(),f.addRange(c),a=f.toString()}return a}n.exports=o},279:function(n){function o(){}o.prototype={on:function(i,a,s){var f=this.e||(this.e={});return(f[i]||(f[i]=[])).push({fn:a,ctx:s}),this},once:function(i,a,s){var f=this;function c(){f.off(i,c),a.apply(s,arguments)}return c._=a,this.on(i,c,s)},emit:function(i){var a=[].slice.call(arguments,1),s=((this.e||(this.e={}))[i]||[]).slice(),f=0,c=s.length;for(f;f{"use strict";/*! + * escape-html + * Copyright(c) 2012-2013 TJ Holowaychuk + * Copyright(c) 2015 Andreas Lubbe + * Copyright(c) 2015 Tiancheng "Timothy" Gu + * MIT Licensed + */var is=/["'&<>]/;Jo.exports=as;function as(e){var t=""+e,r=is.exec(t);if(!r)return t;var n,o="",i=0,a=0;for(i=r.index;i0&&i[i.length-1])&&(c[0]===6||c[0]===2)){r=0;continue}if(c[0]===3&&(!i||c[1]>i[0]&&c[1]=e.length&&(e=void 0),{value:e&&e[n++],done:!e}}};throw new TypeError(t?"Object is not iterable.":"Symbol.iterator is not defined.")}function W(e,t){var r=typeof Symbol=="function"&&e[Symbol.iterator];if(!r)return e;var n=r.call(e),o,i=[],a;try{for(;(t===void 0||t-- >0)&&!(o=n.next()).done;)i.push(o.value)}catch(s){a={error:s}}finally{try{o&&!o.done&&(r=n.return)&&r.call(n)}finally{if(a)throw a.error}}return i}function D(e,t,r){if(r||arguments.length===2)for(var n=0,o=t.length,i;n1||s(m,d)})})}function s(m,d){try{f(n[m](d))}catch(h){p(i[0][3],h)}}function f(m){m.value instanceof Xe?Promise.resolve(m.value.v).then(c,u):p(i[0][2],m)}function c(m){s("next",m)}function u(m){s("throw",m)}function p(m,d){m(d),i.shift(),i.length&&s(i[0][0],i[0][1])}}function mn(e){if(!Symbol.asyncIterator)throw new TypeError("Symbol.asyncIterator is not defined.");var t=e[Symbol.asyncIterator],r;return t?t.call(e):(e=typeof xe=="function"?xe(e):e[Symbol.iterator](),r={},n("next"),n("throw"),n("return"),r[Symbol.asyncIterator]=function(){return this},r);function n(i){r[i]=e[i]&&function(a){return new Promise(function(s,f){a=e[i](a),o(s,f,a.done,a.value)})}}function o(i,a,s,f){Promise.resolve(f).then(function(c){i({value:c,done:s})},a)}}function A(e){return typeof e=="function"}function at(e){var t=function(n){Error.call(n),n.stack=new Error().stack},r=e(t);return r.prototype=Object.create(Error.prototype),r.prototype.constructor=r,r}var $t=at(function(e){return function(r){e(this),this.message=r?r.length+` errors occurred during unsubscription: +`+r.map(function(n,o){return o+1+") "+n.toString()}).join(` + `):"",this.name="UnsubscriptionError",this.errors=r}});function De(e,t){if(e){var r=e.indexOf(t);0<=r&&e.splice(r,1)}}var Fe=function(){function e(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}return e.prototype.unsubscribe=function(){var t,r,n,o,i;if(!this.closed){this.closed=!0;var a=this._parentage;if(a)if(this._parentage=null,Array.isArray(a))try{for(var s=xe(a),f=s.next();!f.done;f=s.next()){var c=f.value;c.remove(this)}}catch(v){t={error:v}}finally{try{f&&!f.done&&(r=s.return)&&r.call(s)}finally{if(t)throw t.error}}else a.remove(this);var u=this.initialTeardown;if(A(u))try{u()}catch(v){i=v instanceof $t?v.errors:[v]}var p=this._finalizers;if(p){this._finalizers=null;try{for(var m=xe(p),d=m.next();!d.done;d=m.next()){var h=d.value;try{dn(h)}catch(v){i=i!=null?i:[],v instanceof $t?i=D(D([],W(i)),W(v.errors)):i.push(v)}}}catch(v){n={error:v}}finally{try{d&&!d.done&&(o=m.return)&&o.call(m)}finally{if(n)throw n.error}}}if(i)throw new $t(i)}},e.prototype.add=function(t){var r;if(t&&t!==this)if(this.closed)dn(t);else{if(t instanceof e){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(r=this._finalizers)!==null&&r!==void 0?r:[]).push(t)}},e.prototype._hasParent=function(t){var r=this._parentage;return r===t||Array.isArray(r)&&r.includes(t)},e.prototype._addParent=function(t){var r=this._parentage;this._parentage=Array.isArray(r)?(r.push(t),r):r?[r,t]:t},e.prototype._removeParent=function(t){var r=this._parentage;r===t?this._parentage=null:Array.isArray(r)&&De(r,t)},e.prototype.remove=function(t){var r=this._finalizers;r&&De(r,t),t instanceof e&&t._removeParent(this)},e.EMPTY=function(){var t=new e;return t.closed=!0,t}(),e}();var Or=Fe.EMPTY;function It(e){return e instanceof Fe||e&&"closed"in e&&A(e.remove)&&A(e.add)&&A(e.unsubscribe)}function dn(e){A(e)?e():e.unsubscribe()}var Ae={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var st={setTimeout:function(e,t){for(var r=[],n=2;n0},enumerable:!1,configurable:!0}),t.prototype._trySubscribe=function(r){return this._throwIfClosed(),e.prototype._trySubscribe.call(this,r)},t.prototype._subscribe=function(r){return this._throwIfClosed(),this._checkFinalizedStatuses(r),this._innerSubscribe(r)},t.prototype._innerSubscribe=function(r){var n=this,o=this,i=o.hasError,a=o.isStopped,s=o.observers;return i||a?Or:(this.currentObservers=null,s.push(r),new Fe(function(){n.currentObservers=null,De(s,r)}))},t.prototype._checkFinalizedStatuses=function(r){var n=this,o=n.hasError,i=n.thrownError,a=n.isStopped;o?r.error(i):a&&r.complete()},t.prototype.asObservable=function(){var r=new U;return r.source=this,r},t.create=function(r,n){return new wn(r,n)},t}(U);var wn=function(e){ne(t,e);function t(r,n){var o=e.call(this)||this;return o.destination=r,o.source=n,o}return t.prototype.next=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.next)===null||o===void 0||o.call(n,r)},t.prototype.error=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.error)===null||o===void 0||o.call(n,r)},t.prototype.complete=function(){var r,n;(n=(r=this.destination)===null||r===void 0?void 0:r.complete)===null||n===void 0||n.call(r)},t.prototype._subscribe=function(r){var n,o;return(o=(n=this.source)===null||n===void 0?void 0:n.subscribe(r))!==null&&o!==void 0?o:Or},t}(E);var Et={now:function(){return(Et.delegate||Date).now()},delegate:void 0};var wt=function(e){ne(t,e);function t(r,n,o){r===void 0&&(r=1/0),n===void 0&&(n=1/0),o===void 0&&(o=Et);var i=e.call(this)||this;return i._bufferSize=r,i._windowTime=n,i._timestampProvider=o,i._buffer=[],i._infiniteTimeWindow=!0,i._infiniteTimeWindow=n===1/0,i._bufferSize=Math.max(1,r),i._windowTime=Math.max(1,n),i}return t.prototype.next=function(r){var n=this,o=n.isStopped,i=n._buffer,a=n._infiniteTimeWindow,s=n._timestampProvider,f=n._windowTime;o||(i.push(r),!a&&i.push(s.now()+f)),this._trimBuffer(),e.prototype.next.call(this,r)},t.prototype._subscribe=function(r){this._throwIfClosed(),this._trimBuffer();for(var n=this._innerSubscribe(r),o=this,i=o._infiniteTimeWindow,a=o._buffer,s=a.slice(),f=0;f0?e.prototype.requestAsyncId.call(this,r,n,o):(r.actions.push(this),r._scheduled||(r._scheduled=ut.requestAnimationFrame(function(){return r.flush(void 0)})))},t.prototype.recycleAsyncId=function(r,n,o){var i;if(o===void 0&&(o=0),o!=null?o>0:this.delay>0)return e.prototype.recycleAsyncId.call(this,r,n,o);var a=r.actions;n!=null&&((i=a[a.length-1])===null||i===void 0?void 0:i.id)!==n&&(ut.cancelAnimationFrame(n),r._scheduled=void 0)},t}(Ut);var On=function(e){ne(t,e);function t(){return e!==null&&e.apply(this,arguments)||this}return t.prototype.flush=function(r){this._active=!0;var n=this._scheduled;this._scheduled=void 0;var o=this.actions,i;r=r||o.shift();do if(i=r.execute(r.state,r.delay))break;while((r=o[0])&&r.id===n&&o.shift());if(this._active=!1,i){for(;(r=o[0])&&r.id===n&&o.shift();)r.unsubscribe();throw i}},t}(Wt);var we=new On(Tn);var R=new U(function(e){return e.complete()});function Dt(e){return e&&A(e.schedule)}function kr(e){return e[e.length-1]}function Qe(e){return A(kr(e))?e.pop():void 0}function Se(e){return Dt(kr(e))?e.pop():void 0}function Vt(e,t){return typeof kr(e)=="number"?e.pop():t}var pt=function(e){return e&&typeof e.length=="number"&&typeof e!="function"};function zt(e){return A(e==null?void 0:e.then)}function Nt(e){return A(e[ft])}function qt(e){return Symbol.asyncIterator&&A(e==null?void 0:e[Symbol.asyncIterator])}function Kt(e){return new TypeError("You provided "+(e!==null&&typeof e=="object"?"an invalid object":"'"+e+"'")+" where a stream was expected. You can provide an Observable, Promise, ReadableStream, Array, AsyncIterable, or Iterable.")}function Ki(){return typeof Symbol!="function"||!Symbol.iterator?"@@iterator":Symbol.iterator}var Qt=Ki();function Yt(e){return A(e==null?void 0:e[Qt])}function Gt(e){return ln(this,arguments,function(){var r,n,o,i;return Pt(this,function(a){switch(a.label){case 0:r=e.getReader(),a.label=1;case 1:a.trys.push([1,,9,10]),a.label=2;case 2:return[4,Xe(r.read())];case 3:return n=a.sent(),o=n.value,i=n.done,i?[4,Xe(void 0)]:[3,5];case 4:return[2,a.sent()];case 5:return[4,Xe(o)];case 6:return[4,a.sent()];case 7:return a.sent(),[3,2];case 8:return[3,10];case 9:return r.releaseLock(),[7];case 10:return[2]}})})}function Bt(e){return A(e==null?void 0:e.getReader)}function $(e){if(e instanceof U)return e;if(e!=null){if(Nt(e))return Qi(e);if(pt(e))return Yi(e);if(zt(e))return Gi(e);if(qt(e))return _n(e);if(Yt(e))return Bi(e);if(Bt(e))return Ji(e)}throw Kt(e)}function Qi(e){return new U(function(t){var r=e[ft]();if(A(r.subscribe))return r.subscribe(t);throw new TypeError("Provided object does not correctly implement Symbol.observable")})}function Yi(e){return new U(function(t){for(var r=0;r=2;return function(n){return n.pipe(e?_(function(o,i){return e(o,i,n)}):me,Oe(1),r?He(t):zn(function(){return new Xt}))}}function Nn(){for(var e=[],t=0;t=2,!0))}function fe(e){e===void 0&&(e={});var t=e.connector,r=t===void 0?function(){return new E}:t,n=e.resetOnError,o=n===void 0?!0:n,i=e.resetOnComplete,a=i===void 0?!0:i,s=e.resetOnRefCountZero,f=s===void 0?!0:s;return function(c){var u,p,m,d=0,h=!1,v=!1,B=function(){p==null||p.unsubscribe(),p=void 0},re=function(){B(),u=m=void 0,h=v=!1},z=function(){var T=u;re(),T==null||T.unsubscribe()};return g(function(T,Ke){d++,!v&&!h&&B();var We=m=m!=null?m:r();Ke.add(function(){d--,d===0&&!v&&!h&&(p=jr(z,f))}),We.subscribe(Ke),!u&&d>0&&(u=new et({next:function(Ie){return We.next(Ie)},error:function(Ie){v=!0,B(),p=jr(re,o,Ie),We.error(Ie)},complete:function(){h=!0,B(),p=jr(re,a),We.complete()}}),$(T).subscribe(u))})(c)}}function jr(e,t){for(var r=[],n=2;ne.next(document)),e}function K(e,t=document){return Array.from(t.querySelectorAll(e))}function V(e,t=document){let r=se(e,t);if(typeof r=="undefined")throw new ReferenceError(`Missing element: expected "${e}" to be present`);return r}function se(e,t=document){return t.querySelector(e)||void 0}function _e(){return document.activeElement instanceof HTMLElement&&document.activeElement||void 0}function tr(e){return L(b(document.body,"focusin"),b(document.body,"focusout")).pipe(ke(1),l(()=>{let t=_e();return typeof t!="undefined"?e.contains(t):!1}),N(e===_e()),Y())}function Be(e){return{x:e.offsetLeft,y:e.offsetTop}}function Yn(e){return L(b(window,"load"),b(window,"resize")).pipe(Ce(0,we),l(()=>Be(e)),N(Be(e)))}function rr(e){return{x:e.scrollLeft,y:e.scrollTop}}function dt(e){return L(b(e,"scroll"),b(window,"resize")).pipe(Ce(0,we),l(()=>rr(e)),N(rr(e)))}var Bn=function(){if(typeof Map!="undefined")return Map;function e(t,r){var n=-1;return t.some(function(o,i){return o[0]===r?(n=i,!0):!1}),n}return function(){function t(){this.__entries__=[]}return Object.defineProperty(t.prototype,"size",{get:function(){return this.__entries__.length},enumerable:!0,configurable:!0}),t.prototype.get=function(r){var n=e(this.__entries__,r),o=this.__entries__[n];return o&&o[1]},t.prototype.set=function(r,n){var o=e(this.__entries__,r);~o?this.__entries__[o][1]=n:this.__entries__.push([r,n])},t.prototype.delete=function(r){var n=this.__entries__,o=e(n,r);~o&&n.splice(o,1)},t.prototype.has=function(r){return!!~e(this.__entries__,r)},t.prototype.clear=function(){this.__entries__.splice(0)},t.prototype.forEach=function(r,n){n===void 0&&(n=null);for(var o=0,i=this.__entries__;o0},e.prototype.connect_=function(){!zr||this.connected_||(document.addEventListener("transitionend",this.onTransitionEnd_),window.addEventListener("resize",this.refresh),xa?(this.mutationsObserver_=new MutationObserver(this.refresh),this.mutationsObserver_.observe(document,{attributes:!0,childList:!0,characterData:!0,subtree:!0})):(document.addEventListener("DOMSubtreeModified",this.refresh),this.mutationEventsAdded_=!0),this.connected_=!0)},e.prototype.disconnect_=function(){!zr||!this.connected_||(document.removeEventListener("transitionend",this.onTransitionEnd_),window.removeEventListener("resize",this.refresh),this.mutationsObserver_&&this.mutationsObserver_.disconnect(),this.mutationEventsAdded_&&document.removeEventListener("DOMSubtreeModified",this.refresh),this.mutationsObserver_=null,this.mutationEventsAdded_=!1,this.connected_=!1)},e.prototype.onTransitionEnd_=function(t){var r=t.propertyName,n=r===void 0?"":r,o=ya.some(function(i){return!!~n.indexOf(i)});o&&this.refresh()},e.getInstance=function(){return this.instance_||(this.instance_=new e),this.instance_},e.instance_=null,e}(),Jn=function(e,t){for(var r=0,n=Object.keys(t);r0},e}(),Zn=typeof WeakMap!="undefined"?new WeakMap:new Bn,eo=function(){function e(t){if(!(this instanceof e))throw new TypeError("Cannot call a class as a function.");if(!arguments.length)throw new TypeError("1 argument required, but only 0 present.");var r=Ea.getInstance(),n=new Ra(t,r,this);Zn.set(this,n)}return e}();["observe","unobserve","disconnect"].forEach(function(e){eo.prototype[e]=function(){var t;return(t=Zn.get(this))[e].apply(t,arguments)}});var ka=function(){return typeof nr.ResizeObserver!="undefined"?nr.ResizeObserver:eo}(),to=ka;var ro=new E,Ha=I(()=>H(new to(e=>{for(let t of e)ro.next(t)}))).pipe(x(e=>L(Te,H(e)).pipe(C(()=>e.disconnect()))),J(1));function de(e){return{width:e.offsetWidth,height:e.offsetHeight}}function ge(e){return Ha.pipe(S(t=>t.observe(e)),x(t=>ro.pipe(_(({target:r})=>r===e),C(()=>t.unobserve(e)),l(()=>de(e)))),N(de(e)))}function bt(e){return{width:e.scrollWidth,height:e.scrollHeight}}function ar(e){let t=e.parentElement;for(;t&&(e.scrollWidth<=t.scrollWidth&&e.scrollHeight<=t.scrollHeight);)t=(e=t).parentElement;return t?e:void 0}var no=new E,Pa=I(()=>H(new IntersectionObserver(e=>{for(let t of e)no.next(t)},{threshold:0}))).pipe(x(e=>L(Te,H(e)).pipe(C(()=>e.disconnect()))),J(1));function sr(e){return Pa.pipe(S(t=>t.observe(e)),x(t=>no.pipe(_(({target:r})=>r===e),C(()=>t.unobserve(e)),l(({isIntersecting:r})=>r))))}function oo(e,t=16){return dt(e).pipe(l(({y:r})=>{let n=de(e),o=bt(e);return r>=o.height-n.height-t}),Y())}var cr={drawer:V("[data-md-toggle=drawer]"),search:V("[data-md-toggle=search]")};function io(e){return cr[e].checked}function qe(e,t){cr[e].checked!==t&&cr[e].click()}function je(e){let t=cr[e];return b(t,"change").pipe(l(()=>t.checked),N(t.checked))}function $a(e,t){switch(e.constructor){case HTMLInputElement:return e.type==="radio"?/^Arrow/.test(t):!0;case HTMLSelectElement:case HTMLTextAreaElement:return!0;default:return e.isContentEditable}}function Ia(){return L(b(window,"compositionstart").pipe(l(()=>!0)),b(window,"compositionend").pipe(l(()=>!1))).pipe(N(!1))}function ao(){let e=b(window,"keydown").pipe(_(t=>!(t.metaKey||t.ctrlKey)),l(t=>({mode:io("search")?"search":"global",type:t.key,claim(){t.preventDefault(),t.stopPropagation()}})),_(({mode:t,type:r})=>{if(t==="global"){let n=_e();if(typeof n!="undefined")return!$a(n,r)}return!0}),fe());return Ia().pipe(x(t=>t?R:e))}function Me(){return new URL(location.href)}function ot(e){location.href=e.href}function so(){return new E}function co(e,t){if(typeof t=="string"||typeof t=="number")e.innerHTML+=t.toString();else if(t instanceof Node)e.appendChild(t);else if(Array.isArray(t))for(let r of t)co(e,r)}function M(e,t,...r){let n=document.createElement(e);if(t)for(let o of Object.keys(t))typeof t[o]!="undefined"&&(typeof t[o]!="boolean"?n.setAttribute(o,t[o]):n.setAttribute(o,""));for(let o of r)co(n,o);return n}function fr(e){if(e>999){let t=+((e-950)%1e3>99);return`${((e+1e-6)/1e3).toFixed(t)}k`}else return e.toString()}function fo(){return location.hash.substring(1)}function uo(e){let t=M("a",{href:e});t.addEventListener("click",r=>r.stopPropagation()),t.click()}function Fa(){return b(window,"hashchange").pipe(l(fo),N(fo()),_(e=>e.length>0),J(1))}function po(){return Fa().pipe(l(e=>se(`[id="${e}"]`)),_(e=>typeof e!="undefined"))}function Nr(e){let t=matchMedia(e);return Zt(r=>t.addListener(()=>r(t.matches))).pipe(N(t.matches))}function lo(){let e=matchMedia("print");return L(b(window,"beforeprint").pipe(l(()=>!0)),b(window,"afterprint").pipe(l(()=>!1))).pipe(N(e.matches))}function qr(e,t){return e.pipe(x(r=>r?t():R))}function ur(e,t={credentials:"same-origin"}){return ve(fetch(`${e}`,t)).pipe(ce(()=>R),x(r=>r.status!==200?Tt(()=>new Error(r.statusText)):H(r)))}function Ue(e,t){return ur(e,t).pipe(x(r=>r.json()),J(1))}function mo(e,t){let r=new DOMParser;return ur(e,t).pipe(x(n=>n.text()),l(n=>r.parseFromString(n,"text/xml")),J(1))}function pr(e){let t=M("script",{src:e});return I(()=>(document.head.appendChild(t),L(b(t,"load"),b(t,"error").pipe(x(()=>Tt(()=>new ReferenceError(`Invalid script: ${e}`))))).pipe(l(()=>{}),C(()=>document.head.removeChild(t)),Oe(1))))}function ho(){return{x:Math.max(0,scrollX),y:Math.max(0,scrollY)}}function bo(){return L(b(window,"scroll",{passive:!0}),b(window,"resize",{passive:!0})).pipe(l(ho),N(ho()))}function vo(){return{width:innerWidth,height:innerHeight}}function go(){return b(window,"resize",{passive:!0}).pipe(l(vo),N(vo()))}function yo(){return Q([bo(),go()]).pipe(l(([e,t])=>({offset:e,size:t})),J(1))}function lr(e,{viewport$:t,header$:r}){let n=t.pipe(X("size")),o=Q([n,r]).pipe(l(()=>Be(e)));return Q([r,t,o]).pipe(l(([{height:i},{offset:a,size:s},{x:f,y:c}])=>({offset:{x:a.x-f,y:a.y-c+i},size:s})))}(()=>{function e(n,o){parent.postMessage(n,o||"*")}function t(...n){return n.reduce((o,i)=>o.then(()=>new Promise(a=>{let s=document.createElement("script");s.src=i,s.onload=a,document.body.appendChild(s)})),Promise.resolve())}var r=class{constructor(n){this.url=n,this.onerror=null,this.onmessage=null,this.onmessageerror=null,this.m=a=>{a.source===this.w&&(a.stopImmediatePropagation(),this.dispatchEvent(new MessageEvent("message",{data:a.data})),this.onmessage&&this.onmessage(a))},this.e=(a,s,f,c,u)=>{if(s===this.url.toString()){let p=new ErrorEvent("error",{message:a,filename:s,lineno:f,colno:c,error:u});this.dispatchEvent(p),this.onerror&&this.onerror(p)}};let o=new EventTarget;this.addEventListener=o.addEventListener.bind(o),this.removeEventListener=o.removeEventListener.bind(o),this.dispatchEvent=o.dispatchEvent.bind(o);let i=document.createElement("iframe");i.width=i.height=i.frameBorder="0",document.body.appendChild(this.iframe=i),this.w.document.open(),this.w.document.write(` + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + + + + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/cli/index.html b/cli/index.html new file mode 100644 index 0000000..3bf3a9f --- /dev/null +++ b/cli/index.html @@ -0,0 +1,712 @@ + + + + + + + + + + + + + + + + + + + + + + + + CLI - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Agent CLI#

+

The CLI allows you to perform various tasks related to the computation and management of algorithms and datasets. The CLI is a gRPC client for the agent service.

+

Build#

+

To build the CLI, follow these steps:

+
    +
  1. Clone the repository: go get github.com/ultravioletrs/cocos.
    2. +
  2. Navigate to the project root: cd cocos.
    3. +
  3. Build the CLI binary: make cli.
    4. +
  4. Install the CLI binary to bin: make install-cli.
    5. +
+

Usage#

+

Run Computation#

+

To run a computation, use the following command:

+
./build/cocos-cli run --computation '{"name": "my-computation"}'
+
+

Upload Algorithm#

+

To upload an algorithm, use the following command:

+
./build/cocos-cli algo /path/to/algorithm
+
+

Upload Dataset#

+

To upload a dataset, use the following command:

+
./build/cocos-cli data /path/to/dataset.csv
+
+

Retrieve Result#

+

To retrieve the computation result, use the following command:

+
./build/cocos-cli result
+
+

Installation#

+

To install the CLI locally, i.e. for the current user:

+

Run make install-cli.

+

Notes#

+

The CLI supports various configuration flags and options.

+

Use the --help flag with any command to see additionalinformation.

+

The CLI uses gRPC for communication with the Agent service.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/hal/index.html b/hal/index.html new file mode 100644 index 0000000..88c1be5 --- /dev/null +++ b/hal/index.html @@ -0,0 +1,601 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + HAL - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Hardware Abstraction Layer (HAL)#

+

HAL is a layer of programming that allows the software to interact with the hardware device at a general level rather than at the detailed hardware level. Cocos uses HAL and AMD SEV-SNP as an abstraction layer for confidential computing.

+

AMD SEV-SNP creates secure virtual machines (SVMs). VMs are usually used to run an operating system (e.g., Ubuntu and its applications). To avoid using a whole OS, HAL uses:

+
    +
  • Linux kernel v6.6 - vmlinuz archive with the standard Linux kernel v6.6 with support for AMD SEV.
    • +
  • File system - the initial RAM file system (initramfs) that is used as the root file system of the VM.
    • +
+

This way, applications can be executed in the SVM, and the whole HAL SVM is entirely in RAM, protected by SEV-SNP. Being a RAM-only SVM means that secrets that are kept in the SVM will be destroyed when the SVM stops working.

+

How is HAL constructed?#

+

HAL is made using the tool Buildroot. Buildroot is used to create efficient, embedded Linux systems, and we use it to create the compressed image of the kernel (vmlinuz) and the initial file system (initramfs).

+

HAL configuration for Buildroot also includes Python runtime and agent software support. You can read more about the Agent software here.

+

How does it work?#

+

HAL is combined with AMD SEV-SNP to provide a fully encrypted VM that can be verified using remote attestation. You can read more about the attestation process here.

+

Cocos uses QEMU and Open Virtual Machine Firmware (OVMF) to boot the confidential VM. During boot with SEV-SNP, the AMD Secure Processor (AMD SP) measures (calculates the hash) of the contents of the VM to insert that hash into the attestation report. This measurement is proof of what is currently running inside the VM. The problem with SEV is that it only measures the Open Virtual Machine Firmware (OVMF). To solve this, we have built OVMF so that OVMF contains hashes of the vmlinuz and initrams. Once the OVMF is loaded, it will load the vmlinuz and initramfs into memory, but it will continue the boot process only if the hashes of the vmlinuz and initramfs match the hashes stored in OVMF. This way, the attestation report will contain the measurement of OVMF, with the hashes, and OVMF will guarantee that the correct kernel and file system are booted. The whole process can be seen in the following diagram. The green color represents the trusted part of the system, while the red is untrusted:

+

hal

+

This process guarantees that the whole VM is secure and can be verified.

+

After the kernel boots, the agent is started and ready for work.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/img/hal.png b/img/hal.png new file mode 100644 index 0000000000000000000000000000000000000000..e7d70b70b81c7cf27afa7b03690d0e2fdd09bbd3 GIT binary patch literal 36869 zcmeFY2|ShEyEl%=l$pqs%|>LNr^F^1GG!hz&$f9UGGv=FZF2}AQ;}I^sEClvGNdSE zmdsAE|3TiL=b(U|#Vj(;)p@z^-Ixij<2GV}47 zIy-Y&*qU0qnmV{~Ia<1dCh*?D$->sk*3#m*4<8So04EP0Cm)|CFAp=nv>+e&kC%sw zUqD3vxWB2jrQ^wj%1*wvaJVTmpPV2U518tbE-x=LKMefRGV^eB_c;E2!`9Km8?2 z9JdIIo@~b2^_0#DSyY3Dt?%hpP8eqGVQXROb~4-XRPIhraCckhKR22?IXYUJpGfs& zMpIW;C$B$uvvPu;?CzwGGuZf_W=Az>{Dq~cRvlXlcbntBB7!H&L$%9W+FIM3QU>Bb zUZsQSY3Gv$Hygl2ujBDQoBqnjQ*OFCIf40p6WZ~M-~UIMhm0j0#q3i)p3ngb_wOJ3 z2RB^*FpdArbZR$^EzP8rJcaFa>|FG9%)LB}RZi8*(-iJ;!g6LlAvoA5%)-_a?8D97 z=R}c(Ts%+=fLS>?x^uey)CrXO@(MWvy#mI0`X2nUM)i|d)zAd*y=(!?`FNz2fve!r zxTejip`r$UnVUMBnt_@5cx2SHPDuhT{iJ$4!Li7GEr8F}$-~j&r$<4#11}p}cS{Xt zQ($uZvR**ZL7R=c0~|b|UZa+IqPe_6sAu3(WSrnm$0o>cC1P%AZVuYr++CgQEq{As zCL$;RGzqmOl-)X;{^I_LK!7iBw>&i_RIh(sD^3*%n4_hWgQdHx5196JIO-*e%!wrm z0Ppzo+3S~H^9!A{+5E24rvpt-Y{~kMxlz;oGx`SloT%%+zW%?_>c3SbKo>>&H&xPB zK5qIilA(p5rHF;V-^oyf&x~IPC3TdjY=PsDhTB@BoVU9ZN==Se_1`O5-cy17Tax8F zbz}c?BztmP{3cmjM_YGr-dnl-D*{H@(0|TIg?LOwd4<6=C}aN3M6G`Q1MPn?UOri2 zA>i!)n?x+|XXEAL{~O~y75neTduq`Cb0mCn{{JT7Q=j~wJqGM8T^&*Iedr#<}t zbIQYesuaKbhyQWPqp7H%EugNZ@1bF(;O6G$;^PS8{TmFSs;w;dKif0>y!d0x@lq`< z{t{071=G1XdAR;Wl3>W+qMH-!=Wgn14MNGbg=&hH?K^&%Z=_f2$FHMNfaRAaf5_&p!g2f2TQ} zm<1~4u{<@3W6MHW!XLKuyIK5Uf<@Kedg1(I(YwsgA+VOTPO4X0p0(7^Z#@f$Bg)AHvNCv z6a5wCYg@Xavb28<<3NG~mBjh&6gNEyq0NE!KqWH&5?ediT3DcRMSmOg{TT!NB{_pi z)|_OT{-q?#|5kt|c$#N9kw34}z>NO$T* zo{!@Z;;!N2_tle{321G6i;#)pJiWsCHaCr`1amT`xLc?^2IREGyeKqiOd!G!;Y0JxZxV65! zjL5?lN5unz2FbzK?#ztc+6}-|zjIO1vMg;UFG-YyvXdnDEKX2R6YfVJU(xCIO^3q+ zY-h8Vdjn@_4%}%A_Me%wx)g<%0!fTAf2lK^zsqttDgVxOS}*Lb5Hy@1srzIi9`@N2 z$>D`rXE9$mOn=&WdNkP}GbdGz$pD>n4vPNRn{UjhvrV78E~NC@o^i6A)rK0i+@+|X zrM;2creDdhKdP<_}(>tGcG7UWYwv7YI zXzN$r8DTR-Xhhp%v^`n*-nioDaHQ%H@Z{p~ZR!1ZvLIwcNIdU{!o?qzqKgGe*%CB6 zZRmm5?bbFQ4{JVGqp<7G#U-G<@;OA~sBTnvqko0@N6lB&HNP*E?9dlS#rPfzqw^tC z`(78N>{H5ukOk9K2_uZM*uRt%ipUia*!bBAD{8@9tmpbat&nTA3i1GOHj zWymU!K}bwxHV8jymJv42^&Y;iUYu3|W`bwAdNA+y$nY7S$Kr6ziytp;6(QRVc(}2! zu?7}y1%^+Zr=pTi#?vZ@PUGu;y-yS$?>wh&xt|#Lnvxi@X%1b_HEO z`Ysv*5%VT2BEn+qpiGPVyPmerEJDbRqALikue(cpe!r})zd>ug=*Dh;j{Kp~mN3DA zDY1iObtIp47cD!qZfjmqqfI5>!*BRz?8l?*@g3{ofQ2m&VY_t)8vW<)%m~c0sa?1Y z`2I5DJwJT%u9v!G2|3Xc$SI(~n&m>wTk2xVXi|2%=@^^kUiIC$6ddG*uK8S^;?cxI zdIDAzG;86ke*H^Zu`*o(v`ry=ru|% zsi+x_JIGg5QDhj5EJFA1?Yo-1PSeeyXC>JwSI5Dn!898T<9hmt5+2{gq^>Ao^7d>) zUrRV<*amHLTI8+j^=U#^Pfr#-arFul=7kUYetUK6!xwQP4Thci3IsE3-p6XaUAYqy zQfwD*H)!pBvF`Si5ZIsIebUdiu|b)$DHF@afnt1QF4U3RF*5RVw1^7HPeaLX)$4u3 z$g?6n9Pz$MGbmzUABoCNd~bd`q4O@klA^PJrY58K`hHgR=J)e^mpb=1?M?Yfck#a# zo~5Af4+|Vhv{q3SduPw_;pvi;pXy2uvs)@m-lU{eG}y1FtWnJO_WqWn2di-Gdbfx- zHUre17R}thZmZ`$yI0R-)_aZlk#mM2Qqh>uJhePo;xBt?*%LjW_Ib?MT_kn@dFoJDt;Fg6K~fOh|$&WZv}EQ7k5L$iB~g zU*uNSy`dG(okvdk@=zkGwqyc!`Dy8cbY}Z_j%N6pk2wx*RMC`4BQAW;=`czMotvG| zk!L??+@UFO8u$x$>I(_gZZC8XWQWDt}PkSJ%BU7VDUQ z?xJPHO?ksc_v8k4!CZ9>xxUX?A+|lmpOPf5xdIX$d3Ulfr*ZF%^&GgA8UHBueuKz* zkhe+z$vLwVFPRzsRnefD^`-ckMMn3}_^Cr=MaD1CJ78~f^Y8Xg)T4-`72%g7fI zp-arknsim9l|>j?uNVi8Gi9a+ja~XVrNA4lnJH8I;tU|-6U?~5#Lpve7 zQG3s;(YUN_(wR1w1RV!AM(Qq2!1uwmT)oEvt2Yc@Da$-{#zGBXAjZegxtJB;wXpSq z;^4#TP=yF>xQ#3t490{UfBZr|mCtAB3=q`8_fMVFHR!^}WctWPJbmYDtb>UZeTFml z$xGles1e;vLhef${0@E6*DAG}RS074H51?jq2cH%er}2RR91FHjNiiK^#wAH(x|CY zCU9!fJteY>S=&9dsyWEEQkcooXNq$^^K)4yZ0HliakksL$-}$PNk=(x;)$D@7M5 zl`=T0&W?Y^_$d-8g5Yp&$G)7)8L^wi+e!+80cygYK&j2z*xu&BV3(S{Z!U+VCx1Je z!mgJqhT6W3Gy~#@r%a4b(4u@8&dPbvbbSNUc~wIc)-ZfqgIDR1e_&Fl-o<&GWDa#Y z6pd=e?<-$7y}n7*Z_NYDf?d6adlhwjsfqH1mF_<=udT*nO<606bjbdL#I?`g-*W~C zXWw6G(ua@=bW5z_rmm{>HZLy;x4o>q+HJ7dAzZCbAJwhG-U=PBoX{@uOtb z9cejODld=YdK~NJue%TB`L;FdcEUAKl0ZZ6#BC2L-ZC>ibgP3 zGm3gnhjGQ(QP^d&C^TEJU&;B&qM#scOryrld%yTO zbWVA!E%^oe0`9FhugX1^Wp+%#%EL}pzKB~QArb1m&+2%u^g7k!bTrz_H`Fg;LUKqM zezGD+1?9jnXqq??2*qxF{K)f(c*RR3$M0NlITmw1;fq6{aDGAtsHW?XjC>MDV)u-k3Xc2gzgf8N7XS4hCbE(!@qFj0_m5IUyDfZn8@VQVo~*c-gS#UUOhr zmOG#~hO7sQw?Rmr4Jx1t7bHPnv}1i?fS$W@7IXaZX=e2&DV?C805?=h>-lXj~-k%d6W z$w{d`uohx=lhcw~XkCntFUQM}(E<8lwF3eALGV+YU`eu&cqT2}z2XJq>czJ({Q%dZ z$)|T_tu^_*#P=@gIr?rausDbvU0$w#!BO4S5p$Tw+=hK~OS84ebxT{oLP^<9{~329 z{zl=e&rdZuXLZP7P3}MD1FOH=T&(-*Csh4paJRZ$YVC#|;^54|+(gD>oA>iBAf$Rg zdj?|aVMABTh(Pel5dFS|X1vI_gs=IB?{xvWuAA5uMy^d8VM!6xSy>o-0gNiM&vxT7-uNzp+`^`^xXnq#vuqK;jU^ z6fUbQ->UtRTGG-{S-x?CIBSO!IVooXFzGflSZbzsKTBm%~YEJy2JbC^Keb zV244`qnTP)#zxK#HEOthBSct-=*moexD=G{9t^ z8D=rOzD+QqT`8zTviZ$b^_AzH(q|p2c83Rvr4ktjtle*g!j}zdTj9npeSq-6z-2$CYXCM@xE8+u0@H; za;HjR)}mss#!lFjD z=~ey8egvjND;{IV*gHqe&>k3dIGWhKUg#Brw|cWcp|SYNl*^n4daO50$kv8cM@g${ zObnhd&c)+KN-jB_8?zT8;m8KKhM~ku3kSCm_jRBqrmTGXM(7P|sHbv);laK`i9EWr zQ;~a$+~DrisCqL^^>g-5TUW|(aI@D3lHP?rk3h;#rrc*)`EjJIM3S|@gOBUwXsR5GacRysZ;ILDqhcn;Ar(55;+nrOYTwQRty_E4(Cf2?V%NA6>vAlzy6W*XO0 z|8_duVDwSNppJc3Ev$PERz-g2=q+pe_8>a8W{+teo*o5ghrb){_b0Z!SnqhUCa z1hWPGLrMvo{>HNM5j?w=6R9~@@h?MRusgP%>okgIr@t?IsPDy7*&>{JP zmCy5sW)5BInus&I+_V+zz&|EcJ60ejzFYNZ-a2k~omGg0y=J6Y#ew$oO6N5yS=Z7y zxE)hSF?aHgMu7=oWryPO57ULzFJ<8s$re+kx7>r33v5%;EbnRRHQ+*Wv>wv6#aTe{uG(G$A##omw{fWW>NHz|#=1~f zaoekl<^(N-2*idK*E7yht8x!s52KsDSflR-s3~^rN$^~AUbLiSabC!Xz|q@gXl~+w zPFLY&SWsD1Bw7dqhvRwBSQ1Li3mGf1C`CksM8IH(kb^@eArf&uf`l05O4gdut$Q!7 zwq`RTzLj^RaHtBg<<3|%h9=BNsY`=n6iY%=>l#BYO6Tsp8JtkHzPR)ylZ?*Ceg^GP zR}_?4$z^Jk?QM_V-MU?|f&Qrw?}abI`40A>vhmS^bJ#}w6>-Dz?cFia^v{c=51+WN zu0qncdW7#f;oxe0@o0ZLQydd;Yrk)%#vsY|9Gk+Ggnnz|0J$sJo8=KvnSgit$ak+j zYM0Yq7sx%#!U>Ov;_T9(xxC{w zL(xYP+JbZV39UA8l)b9sj75O8eW&L=Wj>NhtVNyL40sltk8{#tZ0q-RLn%x>B+zQnyvJv$&k?iet`y3kwF&`L@do{tkluTl6IElTmngaN!WqWT?_T0 zN(ZyV`NUq;qMe`RUA>@%bb$rbqDzLb)?PgbuYP3hXul@1go1Qto;)oodcMmy zfs^D_W*OL*3!1ME3z{*O4Fn*S8sqXEgUc zm-1TOQhQ?_3m0`gEiD#|!gH%&fVS$b~|0Y@(YM!03sBV%%!IA2}b*B#BV; zG-?+JwqTakd}z9ELADA}Eo^vKP=l23l}nHUyH`=cpx7*R-Q(w)$xR2 z^a|sm1T+u)XW@J10&dMwJOnyODtjA+j=fc-VvsnU&P`?bHj)N=67=qNJ;^sGKM+zq zos{G-jaqr7es!&2+Ia!tM^iVE%705+mV^+E4?=`xYN$Kk!kGra{KE zEg@Dng+^T%{oY=$YK#v-#^cCzrgKA}B{^YYsEvMypqC;Lqw7fAPd2RxI#-t@*6VLK zf+QYMUo!{Dk3IPKFh_MYVSG@7#Ho^lecSq)hO>so`aJRQNaKT)u^3$;LpOM~3Hj<|eny8>g{zX;qesRM?JZ1?L2beH^t$H6Q{Yd+_ z*bev_j=fJvVZ%+DR#Zyjs~QM=PGX~1hoJ~jxtg6Q1^D8~wAT3ACHru0Kqee)Pcf*7 z5M`8aNJwXNJCsVN-;c?Rz7h3B16+K83y8Jc)C-mK_eY^2H%fvQ^&LQ=1Hn0L2kbgP}7Nrh|NoPc2 zr@1Za@(9Mzuj6nb5N|I5%mfe~N-2n6A`thL32SZPIwN&&w$Z?wXr5o>S5*N&_pSu; zwj||@&>Not6Z2L6Vr7dl1Y(^M=5S$5uhwyucLco2!~2c)dD5AkfHvcHH&*H@x>!|d z6i`#=5Fy2tXYjUuKNyfKS6Y4<>nB`;M?+LGyCot^EN8;C2E9h zbVTmepXz%Y=w{-KJV)|m*GTC5ac7toih;0|(B#*m+!;-X!LBAclG2KD$nb7`0v6PQ z|Hr04^nDHrMM!znzK1i9a4jNaDW8IESNyWG2W03WxVSl=n`)5`k{L_icw~dYwpso} zK?-8Z0!fPu9GNc7^)yBW&N#o6FB6AKR~{$c1>Oz6;?bO!W&nm>`prBegPKASYPMZN zcT0m+A_QDei(~pFGcl3~Y#6M4(3<9*whHTwN2m?R7S$kac#Dy5>5sD=RH;EB5~Kb7 zXvtsk*3PSwPTt!jP!ZvgY&LmBu2<4xpa2F9BobM8fOH=_!xg_ zz&(&WY!M-^TqD^ZnI3S&10qxdDtw|{AO`KAYVX1(GTZM@Q1qn-gA;%Po>~VnX-X`* z9zSnJX;D7KrU*S?0l@LUatb9E&a4^52EYYNi%Tr&z?&1ie4K?S_dF-?^)AK8 zJg1dPq+4ZL{exGHvsEM!4z7~THCK+`*RCIp?ql-avzAt0q^boQ0a8nVh5g5zi%3VK zm(TZgZ&!N3Ug>djxdQ{6nDqRsyGv>of8+wB>_7e42I80%yK!{KzW&& zQ`Cq>y3FvMXoOsjG;m4G9z57(HSnpu4qx*f6opSza$4=fuKCsJgd%(Cc&Wje;Y^It zE^k9W$F;;uGA7$~CESHdh^eC2y>GlZ+oPE_5&1IUZOXk;tc>*J2S39%w?D$Shr|3c z-w9CO(i3&cq?Wf&Rqv{?=zCr0}t253~o7|PHd=qLX#_uO+#EkI{Oi$NSd$ zc{7JC&NS>a{qH90W{FEdNx1MetBUdg>ik_lA9ZKGzW$;2aU|k1HJbfRc!~m zWEnUVdJ5E@c2j79#$3%EkwFoE3Uml8$1WBGTsZWlGEflGsl|u$DOU-=Xwq*3jlZBM zf&m!h436VE0yG8EcEVU}-)dr)ihM(7gUT+G#OD-jg`;?tX@!E*$@z@r)3Do-%4^>{ zMr38%>##`(tsszWtr6&JwFu=(8q-i&@k65_Sj0V%P8NG+Sz1|gQbyY8?TZzWBk{IK zI6UD>aaV2I#t*hba)^7l@>Y*;r9!;N`O0zFItC<1^ri?VSc@8JEm$wG{kmHUIv3wC z;hn?=LOzaOR<#Cj(?Y?m)on%=mNc=i-Pvs;YZWopi(OXdNnLZYyIC}tZmJ?=T3S;; zyg_jwKDy1$oQtGtx=QnEnfT0y(^`NnZUf6&soIIr50!LQi8o|{c8xk#`c!sQM?9UM zt>4kWOqjuf!Q37?w_{f3bHK)HYSmTAv5Yp-sBMqJHR3@%)>z z@4NGq`BHm|gC%NL6{pja&cYLpYA-Pm1E_#|>v{wSnX}L>8bOCkxyRUlO}69!zV^=X zc~M58AQT@mibh#G3ZHqY=Aqn{&b`sPxH5W?LWAxA9k+s zIh2`K&mY|Q;5+|gyOt}&Zm;*TSI>M;;_JEE6(0lPxUF;YK-GsIY+A4InKKMKDl1v^ z#3SdiI?ldD|@gx%lM6D`0=$JK`|8P zm~`(oa+_Yx^|%b|vgaAeKSl%xU*iRd)*kiStGi!XGV6>tjjhm@zgMX7!qj-(EQn#0 zR1e7(C4LR9gHd_DkgH4^xCCGM=4|5jga$N%$5l-3c6m3qv^BYnS@i=w@oKnu?|i%^ zG1t^-U%jxQE9S41;M4%EZS+qpYK_f6U@{U}guYG!=^l{H+@b!6FXysbE#o%!$$ zmS0~wKiORNVj#N)gEHO^*|fyjJS5VjZfgU|c~vy23TKx541NfqAyjM5A9|_MY3fg=l#kI8GDH7xM*eYZDFg#(mY1?X|3WlN{vdEN~vn<3=S$f z$Awf2ZRqQ@yUOR`=}+&c2b3P3fx)iGc<&&kEMU1>t6wble|C+6)Y zsP(q*?ptenl_^ibLN}sqw&m@zidd{R-w*8rm$@gdb})9?hJzkTy(LMLC8kl z*i7PRTNhsO3Z~dE64L5e24J&t!_vrP%PGV|jZZ&YZA;4VdN0%Ns+ePM1nfI;% z*Cb~P`}mLi2#K}so83TN)iFNY+fGPJtwD%t`Qc45utWOp#>GERDx!xm8;D)&Yi~6y z5c4eesfvt1=GGvIvnxMo(66(ZU8`hH$c?upv+XprunvP8B!}Kp7`15YxCqZ19Y=_& zRB>~XklG*VD~0%-%VDo!*9cLiR;#IdFG+l!m^3}~O>ZnC8L4q-*zTSx)^6O4}xNAK7>Ne9!v=tG|34>}|~ixuT$j#Y>IXJwN!HrX+Mco3%c4 zGo4e*S*J$b9TQs(KAGmlbuCR&#!V-0s8(W@Q$ipFzG_vF+-mesmPqjFeVkfWWYdv5 z{k{RBRc4UcU!bZ45X>1;)koJ^p!$>-rRj8j1Qy2$!kW97D-z!GP*SoA8$IPguzLr* z(V*vgTZeCagspi=rUuBGkMQNvKHq)hv)v^DPEoGW<-*h zgPnf0C*((Y2@8Pa_gcaiYDr1?K7B`C^tKF$NG?x|s^?fbCti~y!~_{&X?*Hb(+$t+$rIz_xR5yQf?6z5wrgC&Rcf z75|qGgc>wIoPpQ3&H`3-fFu08YfS|wWn8FUClok>gK2YnpeQKNaS&)9WS ze!*{+{9lqW9cOl5aENNBbtA3S?$A3%ud773`LnDA zmGa4m!SQiPZLpeT;?#F1hd`Vp$P$Mi95u6W@A^K$0@*IkOU>A*+5>PY6=u%B{FIsd zRT8D?Bd%3F`Sw_RpFHF{td2TtpEGl&?^_r}fNM*?bP$Y&Bf<_&Klu6GmXG~hwS4;3 zR2M^K>CX;iE1y@3wa$2eRaE3JB|oP?`2ti1&_<$olN(bUb4jgmHoH?sD50v<)W9SP z3Douu-v?A|g^A)uRHekY3BLgsr+zf|g-FwLc~=aKYj^d7B1?%t8Z4pN`S2t!m}x@3 zv{-W)23x!xP zMwQh(7mx|}1}U;;tEPis13c5jHb@bd92x1_9TF1aHAc?*OW+=wNM!oBpkLL-!;g|QAs88M#WA#isM$IL$q`iFIRiMZiQ;(Vftn+8 zgmTu)hYA(e5D5O|iAv>x=^GxdAdn){eTD`4iJR0JN)bFc0a~a^4 zs%Ub90vO-;P-75+UI(Q{Iesmf%EEJ8`n48iu;y^pHG|rE{j3uD7Yxvi2YA`2{i0%m z?V`>+vc2;6{gupq^}r}s9M;q3WG<)3=NbY!I2s~tWSw{yNKEX?D>V5)Vf zu=}$-s+=UolX*)rp|oNS7U7OEVeChr?s5R+xszyge@LTdGdT|5k9`;|qunCOV*EZgo6%fCQPq(|i$~HQ# ztIPf7^)ha`X{~=){-YXX1h1LMPji=I&<2Zu@pkwLwpi>2&9r7AWW^=x<3=sU3Qa*y z$rKhE3(Q=E3S@|r|u|cw!k0DoZarJCQ zgOtxBs0yUysaQ1UE`h9K2q?#TutvMKT2O*;?0ZTHf%N=1kN)M+SlYrg6%58&?u?Hj z$_})!_dHk<=+$b3kQI}BtpT6+5ZA~AdCob^^NzsUrKFhAu_;OOsj?efCux$Y%u5`( zVT|0{?gR!esQlp8*g%o>qJ9hy+rR198*wG z4$kL}lt-_%g08eLvP^S+5MtY6gjTMPI$C#{CRPoVlUzZ_b|i!uu;)V5pP|wgni$^0 zVJz`iyNzC0iRVN<)fK6rgv3Yh`$lp4aO}Z#!oiBQM0ziPrJO+(h|x*t8KxyU+_zIb zF9OyKxkYH=W{qF{;K(K55<>QVI2s*Ta|A08wJSvJ`daxR#oK4=#e~OfYBg8M$W z_7WimPB-e#ZA9E*utWB>8h4SAafGar^ZL#~0xws%u_a?kTK%Ed+IOC9Xs%>=H zh{Y|_`;@>YF{aTCOl!kkR9Mf`fwDg=C@ls`n<{|CkuX7pCCz9ISZn!t=^gUUEUC-; z_snvYvy4UrKe7TG{32tCQUM%PE&H+wlF}L6QeHPPt|&A$kAuN>BKr4|mP@WQ-7Nw4 z5?l0;4Iz9XBv__3v1SQEy708{sJ^vR*X?-P1Vvg zzX3AF71fA`8~i?7nLgvnI9ErS7#75|r%!qUIMrz2@)-@QEvAB(-XiTE+^`*SK>5nv zlnRj6{Ah#=lOM6S=;a2h%mY@W#>m25aldD#c__WBXMD5gL#m_KU*V10|X!nizx}&_}NC0<6Zm zSOzYHvCEN=Vt9~@-LB1EiUfBUSFWA84K8ST_|UP*Nd-lWWJd0c1Xzfhw(ML`kjI%Y zf@ha&l~aN{i!D;zjWd8VbC|U9@`E$F1Xg~pg_4xV(@aOq1R#(HomDX=2F*gkPda4e zTRbp(B>~>>hkQnZO}fRt7cTJb(2oe5lv#?t1bz20aEkl>==sX!=*K1?F;3xA4O|29 z1Wtc(EXV9;>5wzR zs!Ozfo0}w5%zEW}Jq#p3ayPA62d0LfCdqVKIw;b;?RwBBp3&QSeeKSt@A+2WJO}G` zU&>Z=@W`^lR8w4a|56CQ zqmG-{Hcz+l&4?!RSo8v>y%t{AX=)4L10yOG9p-XQb{F&j(6nvVf#bS!T@7C=_;GI!e(AashH-$J zGG_C$vq$}_;u%RC!9HSUOWg(Thkb{=QYMXL1RH*p6QbC1R;sPsP$Q}?P`qUGdD|fj9b&^ z;+grMCpGPPE*Z$gR#?X(y^6eQ+3ZO)c66lp#;z#*Dh{j}&!#9h{S|kNN*^SvR@c>E zD(S@wwkwgXaz2*JAytW&6*55q+d!`QZJEV(eeATe7ivqh*7(;4-@H3RhbmT)u&fL& z6rL6pUFp(UbRXqiLw1|<>SYzj&Ugbe3eb#*Pl3FIbrh*gyg-MQksxn;5W)P0;+C_Ae&LVwQP2?RV>r1eSV zC-Zs-KFgaRo(n#FI?_@6wz{>NWj5(uGgxlkl2)b!UD36QoIwu{bl=@?Y+sdhg7-9U zDa_E3=kxo;--zGPH@=(IkYY+F$>b8^6&PLoGJ9_?sSumHj9r7R@ezl4AeMrsuk1T{ z*_x5HCo7@O15BQGNIsRzV2I)33QNA{Fy*jElzm3rk1hpk~b#r6V(a~}BFz3H5CMp_l*1dAoK$#21tW@wcydqd~gg1CwgT~(5&fFxa zG3lJFUiT$;{bVIkIX=9^hS-6~dIOuLzVQZT_ksjHs)|PGqPTJmn#n-jOgi_u8EWRd z-ntze#n){|l`Bei_g;z@2rYU0Q`OirLeG0#Q3*y5CMPC-;D7dP*zSjob+p5@m4Vrf z$nkaF zT_UyGy@&R~Zx&%W0`KRsh^!`NUq}>95KYQ{i<|>xRU3Qn?biaAdzza<=_S1%9?ar+ zx4p^6!kaCe3RhKgcILstu2{8Bj|(Xd0QI1YuP3PlUvghfbuDDeD+%-_+gS)nR9Q+l z)Pwsh9IB*mmccE2jiEB?k_K+f_`In6+1@+v53VkrNwDU}uZW;&QB4MRy|UmgVPTMJ zZ+0)6m^7C69XudKQiHyCK!$62VeK%iv$ZwUaL1|eqjyK4Kljz11Wx?j{e6u}3;!vr zIv=GJM!jb%8U>2d;or*gSB<&ec&NP{Q-#Q@cE!DH<)K|7sy*@Q^uYG<5sS&z7nWLSxQZ!pf`gTjrE%IJ``&@h!xK+6B zM(D|`*D&g!4DqG=j1TbDpc)$shM2O$XE91L@8p-*Cdp{W_t%eclX|w;Zfn_bIE^*5 z5@qSPEmA#MJyYzep2iWA#1>_{KgKpdMG{rwC!gAq$aPf|mgm{NwSANdO6>yp8@jm^ z=d-e&T)84mwq31B_+~Kdy{SyA4Y-4*a`Ohax6xzYH|a3aljfEtr=j?P?^8i(ylD)= zH=1w?a=U-2LAi>?n)m*8;R7&IRG<>Qs+9K|=GjpsP&k^%}ey zl|-m1Uvd@$Q~(`^6(F!M<_Gym@6Dmq+DyH@^1vVSx{XiPBT%&`-Ul-mRW;wOEA&}> zE7tS{AQY3~J_u!~Qi4)o&D)#iAG};PE57P8>*01Ks!yjZ-QIJj2&FajDBS{;8Er&0ATWzolDZ68 zeYMR(S64T0GvTYDcp){+#TS$)NtXa#2neW#P5!{Q;#$ca0{K7vJ4GPQ1PD|gH*(pjMjMMBT|?(@r8nm!DWk+&f)?US4^E=yP$fOn7;YJVkx!6+m(&jJj~gU*R)Ss#wG zSqaq|-W$qDfBovxx%VF6v#Mr6vs-QBqXU!|ZgSXvZLu>1)gC$Fb$}UiEDu7FA};2s zM>TEMX9?jvZVB;vRK+XAq}12!2eXX}l@r9Kg43}KzG_a9!e9hf20?NGpk+X5Xh7+S zGa=n8O^zi+oI^+9-Z| zHv8)S`;4v7K0pdYsnl|vem{go!IfXE5XaUnJbO@$a*~6KR*@=2>iYR_MLi96!t?X% zdSUAhxOWwSc+*GSSh3ijFltx*E{Q!=D+|=SHHRsJJ*@@>QA1o7JFbZZn77>t*etAx z!M=3P0}l`i@APwPb9a>P)}cwAOKPDW1~nmSG9Z!;-jM<&dAheP!UBUj$KH*5%u2=( zaYVZlf=>-)qlOjFb;Z}p_cZIV>vFJwOb8B+AxJe)-3h9JO}@sh;9k@<4 zDLbXd3Sv3Dxw)86--a&rGMXnKIymgmVa<$&B2=iv7D>6Be1re`eb4*10v?0!0rjRx z@MJPU21Dh`n$fxg3DGvW-{N^&`}hQqn_*)>B@RrsF-u5DX;iz{{Vr@&-uL>UXT5xg zIc19tD4kof$$oO%yU%hZAxrRPYwd+UGJUHtLHV%B_V*4w-(Ec9t2F*pOg2S>&y-1o zKy>#}S5=HH^pD!V??y7#`@z#el_&u-?HLe>T*TF7gnpB(UkKcIOBWy!A~R+3i1&VBOZ>e z@3U&!AY`W3`$CZAE?hgyQ2QF|h9&Uhd z%fhaSf1ypzkSIZR2crAWo)6D#bX28qf782CT$}ZU z0JDUNXFqjRz`Xg!!eAAX$4*X&hl(0LrT~lc#-8Uy8@J=~in<@{cFe8UnR4K~5WujF zq4DWPJdQG$cTz4jDeUgcv4TPx87@|{e4%m%0~%8T?XPp%<0HvzwR>M z^TZpTm@D1)$O|*2r~1HUTErAOA6#a?nQOXE(ytCyeAl8Fu_Q~|_F6q772fhL@+CXC zBI}9~+nNh1LF5sh6&G1a6y9oRIGld>+KYTHlvVqKw3OK#1_F`UVzUwEI1V8nthdPHG<;_8AZGWt(oVDRs&u6N&p~5V*jzzd+|pMuBcv zzGMTBUfUHbjdHs?4{2)hK)K7TD-P0>;tTY3q^h`xF1dTEBzlqF)Nt`DrS@?u*jX>8}nTD^fjgR#4L!QUFdMWF(#9S|M7wtd4CZ8n|L$^hK z>n;)IT}w8VZm#Ym5d|~fy&blE#l*WtFMXARgM;1Ih9Y;I?}}iHL%$zvGSzK;PnuwQ zSx+>M*&wzfH0KcLR5mKST>f~Dv_L3Y{A`18jnY^0&x1p)qNcMCaTA$uq;l9V0NFk- z+-$#rODc7STDWp+VEv3AVQ<{T)&umsy+j>1ZuWaQ#F@_n(}zlRuQTn(8Ia&w-<+s0>x(#?4KlbA$Z*k)bNrrB!gb!mRz&JbcU zy%!sL=hCzI#~RHn4R@apUT1R~cXRPn&XQ11haS|qDlVnmH@ttVy|p98F!NJ;N5?3G zb(rG^GW05Bd7)$u_ZtS(XNBm}4HA6+tG%y&i}LIG1whWo!{P(fy+I5ypYeyNc z$}I0|r;X64s8MqOki_lFupuGbmShNdVk_vk5#sc9{RrOu%PD2xKM;czU%`_a9&!q@wwWH{rcfz{iA91APcs$bg6y z8pQh*D$O+(%@4Ed9`J^c00x1z`}>Me%>fQW#2KUu+sCU0__Jyi+nVYFD&G1goazz} zgA0YAGh8sk#oSH?s7b*B&E7+ILt%!k2}Iz+R8{~D9nA15T#PTMKkJYg-~X|!4k2Km z1J)cK8vM5?XY}9Q6$?0yEqaM`Gx!Q~05iVi2aY)ZP3u1vPz5;jGME*e@1cJMhV{15>K7AX^+#&!m7U5-)Alu5FfkONxy zyniv(^t~Ed3!2Hf#J7F+z4(BigR)#ef4YAxZ6|Rv-tT%cu9#B`03uoDxH_G)b50wM zmX8GB{uP=64wLLd|SPmA+Q&(Y8W02wWoWpoGn3TI(vg2A@mc8nY1*H(X& zP6FVVZaN&;py-m~?fcz{RqA&SwQbHvLME6IpKoTulN)?{Bf`O6b}a=4&D zV|T2#JjlY1iQDWkeu*hGHY- zb0`KxMR!M5tpp0#(%_KEM+p_rzQ@MPS3JBcSLIecTu~$dM`p+Os0oI!Z_e}ASXtP9 zu7%P-67Fs0*7r;yCXQ7vnasJNH2N$RUF}KoW76qfUSfbn*lg@CuZS$%!x+X66mvhO zsFBXWsh=4f*N@uv)s&L@qnE>IM34is2q{`wpade!4i736^l6idf?w`EI9Mr_kctmB zVr!Z((l=oq#WPO%V#ELtO5`JBA|3<^t+9w;uCaf-tWKoZ^=rKBCT<{5uCwX`lhLI_ zN6Qxh3O!#hF(80xgLt_N6`>l#%p?S}PXxnw+Y!exy(t(iBdCD5-RJlAHMRKFp@;bwMxN9Drb!eAx46d%Ht?40HT}i z2L@s654<-(;xC{p?aZ2iBXn3N=gMO^eow&_`?m3$nJii~;T65Ocmxz!0FEm#9m$+Z z&1H1s3u}tTsc~nOs35H1Q5N?FP#yV%lMreg)>WPaHTrKRy5ah|T9BQDTt0wja-yPB zLHVU@9yIGz?Q8dwjFgPLvk^3#QO9DJ0T>v#z$>|3Uo4yu%UcS&0}7?HE8exRO@=u} zn@R)Os!Mh3(@qvH6T6knrq2gWbkpgBxs03r5%(sGA(Nm<>BS+g82K^bp6D*Lng$DM zs;m-=&SS3M5sp9xW=&3CrW^sZa?*&|7ls)cio~aUb)rh-*GC$tWI_Gb=pb8eOUkjc z$wv7ar76IEV$)N$oFxKONpb#2`~ZmM|6lU{2g*nB3vW<>CY}QkQ65Cknn`GXgR8T9 zER~3l%r_D5kG}UXlpAPIod4Wy=QmM-?qe>6v@KPx=gXtJC=xvcb&#Ws0`i`dx$%?e%J1#*{r&?;sK*~d;^H_4f;X8I%|ULA=IP3erT6LnXcz;QJtj0 zt-WkwbO)I3r0ks{>YR=Snh^4?Wxd&$?T-a2mxBdfa$R)6{8T`%x zFs$0P%n5YVK8C5Uwe{l#Cf0q2UaeA*w|!)jQFXM^pM2GZF;Sy+uwqcdQgxaS|1^b% z7gi=svvLyO4fA=XWW4(EKvcB5j(lhK>rCN#OaI7ISp0CMk})@Vl8-qm7-mC{F@Cn~ zd9^Sz=>`z&u4XbIF6Yvre+=BGd|Hc7Of8cM8-CD_*9S{tym*oF-dpQ$FKvwE&1_%x zoQwPSTg>(Bh7XOyWT~xw0L~Ds+W@!YJj!S)=a1!MmXTWv6fD3s9@x6u@p;P4zS+qv zOf}48NeiMGx&eDWDf|4`+R+%rCYv17)LQTp(#9`*s0q-#=|RaFO+9C;U$4j`63TAy zy@4t!7h{Toy?c?=o44VOjsb|DNG5Gmm{!;Vw4KH_$^CQBWpj#YTrX(wfalF~VDu>) zm8R{>(?UCAKF8hwP6ART!oU-<-}JgPFc?G2|r-B#JnYO7iEr&ez{V}T980GqMlyQAmiC-<<#YS(O56^(n= zHh=R0sM5c*-6-D;#}sRIjcGK=rP76DvIFvejAs4uBA-==m6{|~BOhJvU!_oDFy}6t z)5R)H1WT)rOQRWeGF)LOdi)m#ZU0%OWdD5We#6FcW$tA=ttzZElr$D!_)RMD2>>g znAEp_>!t_`Ghuk1?T*0vV|7c)%}90+t9(fPDFK4KTryUVDpFp*P>la2mi^*iP$c}H zL2X(>Lb^7*_i$LMF-SvEEs;*}vq4oN@T6*k$X^%^8hR);Su*d9YYgwv{+Xs>3Bjxg zaHwv|cZot4++Yg>GCh#-J=-3k(@wWFL0FxAe@j%i73AW`ophrL99H8*1)d7hI5#nz|)gyr% zlqG*Vl=77&qC)f=umVK><6s*P=YQbGUZY0GhUIWo-3O2me^0bPk78hKGYPQr^M**j z{|cgedHn`7spyUAI|(^U4L=+pJ@E6tuu2oC3;@fwz(Q6+04U49k#+wX*#Z*?U%0a zyZ+gd^|xEkdDs~|tT^v3FFiB+o!sqc&V!WQiDgI7-EWuQU5Y_~KTl>;i6i~SDVk4< zi|q{Zx{8Hm==kubNX&6yqT^kJlFe0X__Lhz*Di|~_aEGU*NbsKdasp)CtmlmHAm=$ zLRmqvfMl4`&MA41z&p`~YD*?s`DC8tv@|-o%zW86rK728vxE5Ih?;}XTU_?nGcJiU zx!exd*X*u`*Sy9By6iLMORZ+9O}~(aW6S-lz?Gk2>bB+ZOnvIc{D`Q*@d}q#CLS5I z+Ak~QKK$iV94qB}r`eL(Qktm(BUl|~2 zSgA|}LlM_M-pDPtZ&!&Xv2Om#mbEvVRO*WpaI32su(*9D6f2nj&Ub}hSv~K`?z4|V z$_@4^G2Ju!B{~d-QyglYW3e%|zs=J-)~?i;zgpC&?g{qo#fCjEY%pIh(anyiT;IP7 z2)@3%K{xJ2dXuUa$_Pw*W?#dxp9g<6{qe<|6*%?xX^_sHT-n@(uhc~qs=5EHjlW&X z6C`Zsb@f_FarpcHW!%ifB5ChpTu$mC_zZa~U(f7u4r} z+Lt2Bdn5TH^<8~zaW9QD_)j1FNu|+%aP6K@`XrOu&ls)Rk3u-n$N9>mr@nW@*jRUb z5>dp=Ie|hanc*YZs@FZ8>Cz{E7&oJ+%_5<;xh2I~3@y|Tv8qKg`w{P|+(+qLFD^L; zDS3U~|0;Z5@&kYk88B4@?}s)zF<5_pBa?y<&D0sqQ8`+!=-W#bJj;eFrexA?ahKtr z$E(TT^2)p-AS06tZFGEb@qMmY-h4Iu)~+?K>eCk2`$lu(!RN2|s{jPvtCIEbzD}rt zbPE4{RTfnx0)L0s_9`p}^5$b8r}(=h?x6jP5*K#T>E&)IL!zY55NhQh5ca0qY9#H) z{$$O?YYAFti`k0A36rWy*kjP6tUG6S1-+V(hid~8ZbSpY@0{}~%oxchl%$HLYEoKA zIO2m=dIz})RT(@ARE{ZeYF*%^IoG_lWj4uT*t0XDhLAgAx0)NJ3qh#g##*x*hvg>a z%GDulipE78_0-B80T8tj!4EG4-n)-VhW$Gz1)pzjdHb4)mZnn9FzdA1N!EYpojttm z42~8WBP52U>$95<$Ak(-(-{NNH`@&6v2Eqmr4K{6@$ieY)16<3xq`=(^Mfk~$12kN zmw$jrk(Zb7VUcpc``^~WT6kpX+M1O=gVJ7jP(fFgZnphd2T9p9>Vt8$#D$r)8^c)U z>J$iFPanla#B-%ndQ2B-)-4Yf5rvRSr;QgAZuU|LdL%M>OsWw)KVb@J)rt!l1sqx% zhh2|lbSjZNr{lJaNOsfceevX;&fR4LiI2S#uNK4DuO=!#2*xmI(<@QNhTs4<)sJ;H zH?IdxOsK&Fw2Tmf$@An2BkYsZ^U6pacGL7@GHq|z2W@TbqEGE=qD%E@luxZjR0zJZ zQG7pD_wx#Qu0L7dxt<|Pt$b-uX|%17ETp*lzB*%A$Uc{nSL^{0HJu>O0RxPE`y9wF zrtH#o2PzpdoK%nIh|3s^XM9(wHGGk{EFga2%j&cNx0;h1ueG@R&}aB9k<;DVnebAw zPYD7cj(en9FdR<_PiJA)qW>*9coFK~U&`Th{`S*sy7yt_9N~*$gBH8iW)G(7^L%bv z2gM>p9HX9xJk~|YoHoVwoP!u6)i#2)UskJyyk36%vrr|g6(H3}~ zbnR{sVyetETD>0^cQ9DJ5szBr!g7KVUH`NZ!zW%*_}&y zXs0F&7+}I!-Pr~~@m!1V+?$yV&-Y<5LX@#HCd2;cU&-p7-$6q#o5#nI@`J-*Ry ziQqRR0gzD>`h-NBt?@5xkKJt6320*Sdm?5VC77TqaTLaV!`$*h8G4teKwQ!#1+n)V zh?>8d=n-9NuijFr(zIjyAt~uk4gcEJSJhqWi#6QNX`X-cQ@`&Lr(T&stL`N)AD^V` zaAb&hvGp_)94$`%S?;;Tu?P36DDx=VDAuXEr`kbqX%G9TWM3j?{XK3RY|ZIm|GT$A z@d@nqpzP&nt40yAXbT@Qp*llJ*R?aBFGJ@s;G@pP{X?P5omy@2uw;Sn3DGg`-gR=c zLgoaFc>_-eMGeGjOMW@F2}{iM_)GTb+%Bh8T6w%~+QwrXZ*VB`_d|pY+pUcp-MbTS zpb+VJ`)=rVrGHmi?6pH?;%c8nLy5@chd?BfPp!TOM_uy>BGF+o_TC}Sa=DA+Prw;* zT2Bz|ayty#Rq+9KD9!%qZFYt28LSg<(``B>#7+yqDfDJ=;6ICHxvk!6rhXxWocYHQ z2M1PixEvT5!+Lctrk(caTh$GXn*wcOHX9_i#)fO)ciEM&pvuFllo0P$CkyqQs|OAn z=`)Lp@vUCFLY{KD>ZR10ic|&?yK~QN9P^BCE{}cg2O4Gi@bugL*^7~B1qv_mJkcLs z997(8DP<%vRVm8Ao7OO;}BPWBWnlQ?hzAJKVF4wu@&uf7Whb9%AFv*5#( z%yIn*8&`+(57Bm86)eIm=>jd6k(USL1<{%ENy;_hf)C@GU*QD$!eJROPV-S~vF7~G zC!hKb*yC7D75Zav6h_$G&q#vvALrMa&AMJ%A-HvBn<5048aI0SWOVU{@8TGs<}YWo zb@IvPY1NsP6BCb8TqZWZ#uI7vNr1}K;3aq7vN#=+8omypj>u$RmBGNGuHjbUV zup9={BVz12tZW6ds9qop{LA)c}0M7V`hD4)J|_&E4WFHgb$VW0D*;qn`a zM_+VaMAv%9v2i_cG*jGi+RK@Q7aO5-1x3=z7ubJK=5(!rr%}Tsq`a&G>ix@C-iSp}7v3=&3d@`~vO@k!r0?Da_=LpVBtOgRG>$JF!%5sn^H zK)^mVwkFLh;{!0!tmnWkp7DU;CmJpe@-rZBF{Zp$9Woqs*Av62vhQ1&!~@Sb>=yO& zyTD*2Hq-Qbs`~L%46mVUi&oqwK6qxZm%2|U0559$oTnpS-)WAQ%I=E5DHsU$4#0)8 z6-R!IxT_zZu?jJnbFNtF?MDA`*p?&aE>hop_T2=UWetE?H?M|V?8L6DTIzz32*xC8 zoUs+WtNQ9f2Dh~V8Jz?J5^igX*@+5Xm6r+rep&0r4I)7=f#|Zp3Wm`36$=r7hOLR% z^xFUk3~$-?L^i5O7w;|IkFuU?$)0N(eKQpe9yRFaZ$hj1P;!RzF}p|SFv}lD%qh>F zl0BB|u2PS{pq!zcgjnC-uYI!-w~u;G3=YdCF4bkk;rlvW4RH`}Q!`xTNM(&r7|ji@ zGLMvw+I(3*krXUPEM-Df-+hJCQqS*sn#?laAZ_kUy14Ei>^vZ!@gv|_0feeKS8SE7G&FMrwBw+t)Ki@-85>&`M_- zvW>W{j?~ySh4giwvN!`M&Luq-sBIj~Zi9`LIe*JOrjXIey+S~>hCi`3qw|f_d9!D> zMF3|Uu)p#W1GNIwa_}}3cM>^OJ)f#c^+@EA8e_HFEFjuNx(glTUbf*zVRIKup%k@(t?{{>|N5@AtpEO+N%9Co)ZDCAL=&S}v+ z@9Z9WvE$UxO~R_!wKvj$hK9R>`bI#@QLF<9ZaE27keY!X#=DygfyR^N@)6F2-jF5f zF@4x%nMiN!N?)XEuV-)cdkqLg3+{Gs+FvG-g9jyK&p%UK^b@ZgHst9)SX7hEXBuU2 zaJT`-NFv-o}7n$?rKBtYcCvdkJ#&AYFSMnAPDBAr95TP4J!vq3Le-J#C2L? zUkuSR%n#lS{Ln)RMl)(4m4C0t%wnzal>o^Nh#wI^szMK0t(>`gZ?Np3`Tc?QDVKAp zRb1!Mq6EOP)BC`vqoX1{ zA%u6<7VVCpc@NcP&^V_CBeX-mUagpSIx51|^8~Q{E6$rn`_CK6O*8dECms}2R&*v1 zrC{lsqt&zh@c*jT+nTrs$i9 z?W{YzenfD;w?8KQ70qytwa{Gtkq_5EdY~&)C75 zF0bXrNCW+W(9i(Vs1wR=Djm^+K;!Kufojq`Dep=Ailvx}l=6}J(Amv263Lt)NT~Hoi0HX) zlw-(}iw$uC4XE7;dyd>-t(K@dJ`tD0>lX{XT@&)Ot$lK8HK_!7W!fo2+KD$R-Hrz* z?Z6o{dY&c6BvqhM^QiCIW;W<)v=vZPXh0dKBH$LjGwWs({hOrwfLKbDw9?N{$)>6} z@LrynznuSoaBg~sS-pk`ZPv3O`dGB(m&N{HY92biFH0Go>dXE_0@cPi=JI+|kBR^G z8_ur?nZhq)?=46`iI4sLgk4kPS`v)yCeLQ=hANhv?!J-C9c`~wvQEyY9wi$ddrOZz zy~C-d(>rQE+N~YxA;ESL$QFvBkrs^9Gta7+IDXD*Tz-7a^I zm|n_R4@~XG)wCX)LdB$7+TU7G+wi!s_n1lwx;9HCW%e3AyD;f8${!wTe}(9BkO>|=eyV;BgZ`HdV0UBcv^q=#+I8v zSG1BilBJOd&mOLJh*MmH@|tmHXR(jpb{+Q6{iZd(d>L(f-n1L-?6Eth(Rb}6-mOUx3ZupW5zug-kxd9_co*0jN$;Y z<0q3$)cwjHZ^9hoR80y^23**4y06v2D=lVBQayyultqX0bS6;flO8+iqe8z-cmbhi zf0}dqY&8U3F_FbMH`Gs+*GZ(tO0dBq!FBJqL%}z-%Mc^2LXNFIh&f)Iyq!`^rNzBi zBy(t#KxHNQ1y#%{*CZW7<4dCpVNy$4Iji@_S$4|bKIE1r(nczOSZ&P>rQA#9;??tC z7P+~q<=A^OBqi1AZ$Tb=CaRkTGcEGmb;oSWyCj=U6}UQfnzxMO$-Lr0I%w`bNE|O>{wa)wu-6@>sVV5j#1G zlFpa(_B)uwMpwP2lKzDC7|gatTFqSg&f}Tv_YD4~SyHj1Bi8tla9pr$R<(s<%O;hKn2(>_Y(GTBE=TS&+`IM!neEHuc4o`>=c_5ys5Ss*jRTx>OdibggJwQRwmZaem&DCYHe9Ai=lLON zll5Rksbp-Vk*2I8ld~zL<1%;Yti_B~O6o>>)oxb!6Sb6-z(>&SaH;1Gmz;@Hzpc!# z-YmP{+_r#p@XYJO)h#9#lGV-<{RDZx8*NUV{7H_KSIC~Kr>5+yFg{nS_BH&x`rhg6 zCzS*JxFKV_OQWID7g1MID5#g zevDab{1FqIQtANozeS8-1j~ zCzgGlsQm~vDws|fDfNg`M07L(AWHZPjTah5EG7sX`{8RJz@2|H4`>JjXor0RLkHS{ z$5rVuIwt;0T4Cf%+R4x4CjZ%xjYgj+mP&st>Jy4V3;=@|q)nhjGGJ)Yp%NL;8KLDx zMZm3-?hUiVT>9kk>VGm*BbMtwCh8seqAKmYx-BJwjJv!xsy4rF$yur|!n%=Q&( zRH=A05$adJP=Wx60%VJZ0nz|dNon|mGRh)<+SC1&D$uYDDvT_{ltsS80iG>sPE35^ zwrDhw_GqJShxmy9XxJa(r>WuNr>R%~0p`vD7?cA|h$v%}Mwa!-eg-;ISpj-6WAFW- zqKW`fc+v?_@K0uHPW{J3P}%2dwAf%Z7X}oO9n8W=L=wwS;aDz=&*fDay`46wQS||# zb-X16_?r;%Kjh6-07J=)Dg(O*)vzrRpdAboX@>z%(Tm^sfC4H53aA`OKCTaxWBWfF zvV%?ZofMkaZA2zR3J27(QA+3KRnj&`8;pnN=Bk-$Mc&0uqklp%p=~?fecyPrvccp_ zksw|tO0zMqaX9?G54Ft0ZqL_#Z&lueFZDTNYD^&{nGFVNMzHsE^wXGRm;CLGf1~`I-@fzc?gTv5VZfq+3`(_r&TwKE&z!0u^9h_B9I+i4l zf~*o|ub9T4=e@xO#DIoQ?j1gNS*cuUxz$ea)V$B(&&#niKs0H}0qV^fD)tFc_OOG!U8-^0+xxY9 zmq}aof-f4(1X$O?&1Eu{Bqt|V37N+0{6_!7XmLPy@DcbEtp+YS?_Es})QMT#(c?P+ zONJ%%4~WM_SD6BzSE?(Ll?lo=&42;q?@vBOrh-o3=lqlT^&byInc@TMG@LL*+B2Ju z<+9E;R8KCe~vUlT>Uv}pcN(OHP*40@Hh)|>7PRqy= zad2|D*Uodl;apu=HOcaCpk2^(c?d{uVFYZJhr`!xJ|}QurKqp~NJl5j0 z{n3k2U^h8#u_R-sqY*2Y1TsEdH(0 mU@?5++(fDQt!@a@zEExh!OMA9e=#)RCoQ2MULyLz=YIjr_Nr|F literal 0 HcmV?d00001 diff --git a/index.html b/index.html new file mode 100644 index 0000000..b567445 --- /dev/null +++ b/index.html @@ -0,0 +1,597 @@ + + + + + + + + + + + + + + + + + + + + + + + + Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

What is CocosAI#

+

CoCoS.ai is a distributed, microservice-based solution in the cloud that enables confidential and privacy-preserving AI/ML, i.e. execution of model training and algorithm inference on confidential data sets. Privacy-preservation is considered a “holy grail” of AI. It opens many possibilities, among which is a collaborative, trustworthy AI.

+

The final product enables data scientists to train AI and ML models on confidential data that is never revealed, and can be used for Secure Multi-Party Computation (SMPC). AI/ML on combined data sets that come from different sources will unlock huge value.

+

Collaborative AI drawio

+

Features#

+

CoCoS.ai is enabling the following features:

+
    +
  • Distributed computation orchestration over TEE-enabled machines
    • +
  • TEE enablement, deployment and monitoring
    • +
  • In-enclave agent, netowrking controller and other system software
    • +
  • Encrypted asynchronous data transfer and result delivery
    • +
  • API for programmable platform manipulation
    • +
+

License#

+

Apache-2.0

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/install/index.html b/install/index.html new file mode 100644 index 0000000..47c6f0d --- /dev/null +++ b/install/index.html @@ -0,0 +1,539 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + Install - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Install#

+

Before proceeding, install the following prerequisites:

+ +

Once everything is installed, execute the following command from project root:

+

To run CoCoS.ai, first download the cocos git repository:

+
git clone git@github.com:ultravioletrs/cocos.git
+cd cocos
+make
+
+

Finally - you can run the backend (within cocos directory):

+
make run
+
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/manager/index.html b/manager/index.html new file mode 100644 index 0000000..78d1bd3 --- /dev/null +++ b/manager/index.html @@ -0,0 +1,791 @@ + + + + + + + + + + + + + + + + + + + + + + Manager - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Manager#

+

Manager acts as the bridge between computation running in the VM and the user/organization. Once a computation is created by a user and the invited users have uploaded their public certificates and a run request is sent, the manager is responsible for creating the computation in the VM and managing the computation lifecycle. Communication to Manager is done via gRPC, while communication between Manager and Agent is done via vsock.

+

Vsock is used to send agent events from the computation in the agent to the manager. The manager then sends the events to via gRPC, and these are visible to the end user.

+

Manager <> Agent#

+

Agent runs a gRPC server, and CLI is a gRPC client of agent. The manager sends the computation to the agent via gRPC and the agent runs the computation while sending evnets back to manager on the status. The manager then sends the events it receives from agent via vsock through gRPC.

+

Setup and Test Manager <> Agent#

+
git clone https://github.com/ultravioletrs/cocos
+cd cocos
+
+

NB: all relative paths in this document are relative to cocos repository directory.

+

QEMU-KVM#

+

QEMU-KVM is a virtualization platform that allows you to run multiple operating systems on the same physical machine. It is a combination of two technologies: QEMU and KVM.

+
    +
  • QEMU is an emulator that can run a variety of operating systems, including Linux, Windows, and macOS.
    • +
  • KVM is a Linux kernel module that allows QEMU to run virtual machines.
    • +
+

To install QEMU-KVM on a Debian based machine, run

+
sudo apt update
+sudo apt install qemu-kvm
+
+

Create img directory in cmd/manager. Create tmp directory in cmd/manager.

+

Add V-sock#

+

The necessary kernel modules must be loaded on the hypervisor.

+
sudo modprobe vhost_vsock
+ls -l /dev/vhost-vsock
+# crw-rw-rw- 1 root kvm 10, 241 Jan 16 12:05 /dev/vhost-vsock
+ls -l /dev/vsock
+# crw-rw-rw- 1 root root 10, 121 Jan 16 12:05 /dev/vsock
+
+

Prepare Cocos HAL#

+

Cocos HAL for Linux is framework for building custom in-enclave Linux distribution. Use the instructions in Readme. +Once the image is built copy the kernel and rootfs image to cmd/manager/img from buildroot/output/images/bzImage and buildroot/output/images/rootfs.cpio.gz respectively.

+

Test VM Creation#

+
cd cmd/manager
+
+sudo find / -name OVMF_CODE.fd
+# => /usr/share/OVMF/OVMF_CODE.fd
+OVMF_CODE=/usr/share/OVMF/OVMF_CODE.fd
+
+sudo find / -name OVMF_VARS.fd
+# => /usr/share/OVMF/OVMF_VARS.fd
+OVMF_VARS=/usr/share/OVMF/OVMF_VARS.fd
+
+KERNEL="img/bzImage"
+INITRD="img/rootfs.cpio.gz"
+
+qemu-system-x86_64 \
+    -enable-kvm \
+    -cpu EPYC-v4 \
+    -machine q35 \
+    -smp 4 \
+    -m 2048M,slots=5,maxmem=10240M \
+    -no-reboot \
+    -drive if=pflash,format=raw,unit=0,file=$OVMF_CODE,readonly=on \
+    -netdev user,id=vmnic,hostfwd=tcp::7020-:7002 \
+    -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= \
+    -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3 -vnc :0 \
+    -kernel $KERNEL \
+    -append "earlyprintk=serial console=ttyS0" \
+    -initrd $INITRD \
+    -nographic \
+    -monitor pty \
+    -monitor unix:monitor,server,nowait
+
+

Once the VM is booted press enter and on the login use username root.

+

Build and Run Agent#

+

Agent is started automatically in the VM.

+
# List running processes and use 'grep' to filter for processes containing 'agent' in their names.
+ps aux | grep cocos-agent
+# This command helps verify that the 'agent' process is running.
+# The output shows the process ID (PID), resource usage, and other information about the 'cocos-agent' process.
+# For example: 118 root     cocos-agent
+
+

We can also check if Agent is reachable from the host machine:

+
# Use netcat (nc) to test the connection to localhost on port 7020.
+nc -zv localhost 7020
+# Output:
+# nc: connect to localhost (::1) port 7020 (tcp) failed: Connection refused
+# Connection to localhost (127.0.0.1) 7020 port [tcp/*] succeeded!
+
+

Conclusion#

+

Now you are able to use Manager with Agent. Namely, Manager will create a VM with a separate OVMF variables file on manager /run request.

+

OVMF#

+

We need Open Virtual Machine Firmware. OVMF is a port of Intel's tianocore firmware - an open source implementation of the Unified Extensible Firmware Interface (UEFI) - used by a qemu virtual machine. We need OVMF in order to run virtual machine with focal-server-cloudimg-amd64. When we install QEMU, we get two files that we need to start a VM: OVMF_VARS.fd and OVMF_CODE.fd. We will make a local copy of OVMF_VARS.fd since a VM will modify this file. On the other hand, OVMF_CODE.fd is only used as a reference, so we only record its path in an environment variable.

+
sudo find / -name OVMF_CODE.fd
+# => /usr/share/OVMF/OVMF_CODE.fd
+MANAGER_QEMU_OVMF_CODE_FILE=/usr/share/OVMF/OVMF_CODE.fd
+
+sudo find / -name OVMF_VARS.fd
+# => /usr/share/OVMF/OVMF_VARS.fd
+MANAGER_QEMU_OVMF_VARS_FILE=/usr/share/OVMF/OVMF_VARS.fd
+
+

NB: we set environment variables that we will use in the shell process where we run manager.

+

Deployment#

+

To start the service, execute the following shell script (note a server needs to be running see here):

+
# download the latest version of the service
+go get github.com/ultravioletrs/cocos
+
+cd $GOPATH/src/github.com/ultravioletrs/cocos
+
+# compile the manager
+make manager
+
+# copy binary to bin
+make install
+
+# set the environment variables and run the service
+MANAGER_GRPC_URL=localhost:7001
+MANAGER_LOG_LEVEL=debug \
+MANAGER_QEMU_USE_SUDO=false \
+MANAGER_QEMU_ENABLE_SEV=false \
+./build/cocos-manager
+
+

To enable AMD SEV support, start manager like this

+
MANAGER_GRPC_URL=localhost:7001
+MANAGER_LOG_LEVEL=debug \
+MANAGER_QEMU_USE_SUDO=true \
+MANAGER_QEMU_ENABLE_SEV=true \
+MANAGER_QEMU_SEV_CBITPOS=51 \
+./build/cocos-manager
+
+

Verifying VM Launch#

+

NB: To verify that the manager successfully launched the VM, you need to open two terminals on the same machine. In one terminal, you need to launch go run main.go (with the environment variables of choice) and in the other, you can run the verification commands.

+

To verify that the manager launched the VM successfully, run the following command:

+
ps aux | grep qemu-system-x86_64
+
+

You should get something similar to this

+
darko     324763 95.3  6.0 6398136 981044 ?      Sl   16:17   0:15 /usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty
+
+

If you run a command as sudo, you should get the output similar to this one

+
root       37982  0.0  0.0   9444  4572 pts/0    S+   16:18   0:00 sudo /usr/local/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 -machine memory-encryption=sev0 -nographic -monitor pty
+root       37989  122 13.1 5345816 4252312 pts/0 Sl+  16:19   0:04 /usr/local/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 -machine memory-encryption=sev0 -nographic -monitor pty
+
+

The two processes are due to the fact that we run the command /usr/bin/qemu-system-x86_64 as sudo, so there is one process for sudo command and the other for /usr/bin/qemu-system-x86_64.

+

Troubleshooting#

+

If the ps aux | grep qemu-system-x86_64 give you something like this

+
darko      13913  0.0  0.0      0     0 pts/2    Z+   20:17   0:00 [qemu-system-x86] <defunct>
+
+

means that the a QEMU virtual machine that is currently defunct, meaning that it is no longer running. More precisely, the defunct process in the output is also known as a "zombie" process.

+

You can troubleshoot the VM launch procedure by running directly qemu-system-x86_64 command. When you run manager with MANAGER_LOG_LEVEL=info env var set, it prints out the entire command used to launch a VM. The relevant part of the log might look like this

+
{"level":"info","message":"/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty","ts":"2023-08-14T18:29:19.2653908Z"}
+
+

You can run the command - the value of the "message" key - directly in the terminal:

+
/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty
+
+

and look for the possible problems. This problems can usually be solved by using the adequate env var assignments. Look in the manager/qemu/config.go file to see the recognized env vars. Don't forget to prepend MANAGER_QEMU_ to the name of the env vars.

+

Kill qemu-system-x86_64 Processes#

+

To kill any leftover qemu-system-x86_64 processes, use

+
pkill -f qemu-system-x86_64
+
+

The pkill command is used to kill processes by name or by pattern. The -f flag to specify that we want to kill processes that match the pattern qemu-system-x86_64. It sends the SIGKILL signal to all processes that are running qemu-system-x86_64.

+

If this does not work, i.e. if ps aux | grep qemu-system-x86_64 still outputs qemu-system-x86_64 related process(es), you can kill the unwanted process with kill -9 <PID>, which also sends a SIGKILL signal to the process.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file diff --git a/search/search_index.json b/search/search_index.json new file mode 100644 index 0000000..bda2006 --- /dev/null +++ b/search/search_index.json @@ -0,0 +1 @@ +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"What is CocosAI","text":"

CoCoS.ai is a distributed, microservice-based solution in the cloud that enables confidential and privacy-preserving AI/ML, i.e. execution of model training and algorithm inference on confidential data sets. Privacy-preservation is considered a \u201choly grail\u201d of AI. It opens many possibilities, among which is a collaborative, trustworthy AI.

The final product enables data scientists to train AI and ML models on confidential data that is never revealed, and can be used for Secure Multi-Party Computation (SMPC). AI/ML on combined data sets that come from different sources will unlock huge value.

"},{"location":"#features","title":"Features","text":"

CoCoS.ai is enabling the following features:

  • Distributed computation orchestration over TEE-enabled machines
  • TEE enablement, deployment and monitoring
  • In-enclave agent, netowrking controller and other system software
  • Encrypted asynchronous data transfer and result delivery
  • API for programmable platform manipulation
"},{"location":"#license","title":"License","text":"

Apache-2.0

"},{"location":"agent/","title":"Agent","text":"

The agent is responsible for the life cycle of the computation, i.e., running the computation and sending events about the status of the computation within the TEE. The agent is found inside the VM (TEE), and each computation within the TEE has its own agent. When a computation run request is sent from from the manager, manager creates a VM where the agent is found and sends the computation manifest to the agent.

"},{"location":"agent/#agent-events","title":"Agent Events","text":"

As the computation in the agent undergoes different operations, it sends events to the manager so that the user can monitor the computation from either the UI or other client. Events sent to the manager include computation running, computation finished, computation failed, and computation stopped.

"},{"location":"agent/#vsock-connection-between-agent-manager","title":"Vsock Connection Between Agent & Manager","text":"

Agent sends agent events to the manager via vsock. The manager listens to the vsock and forwards the events via gRPC. The agent events are used to show the status of the computation inside the TEE so that a user can be aware of what is happening inside the TEE.

"},{"location":"agent/#security","title":"Security","text":"

To run a computation in the agent, a signed certificate is required. The certificate is used to verify the user who is running the computation. The certificate is sent to the agent by the manager, and the agent verifies the certificate before running the computation.

"},{"location":"architecture/","title":"Architecture","text":"

Cocos AI system is a distributed platform for running secure multi-party computations.

It has 2 parts:

  • Manager, that acts as a bridge between the web and the TEE and controls the creation and management of computations in the TEE.
  • In-TEE (Trusted Execution Environment) fimrware, otherwise called Agent

The system architecture is illustrated in the image below.

"},{"location":"architecture/#agent","title":"Agent","text":"

Agent defines firmware which goes into the TEE and is used to control and monitor computation within TEE and enable secure and encrypted communication with outside world (in order to fetch the data and provide the result of the computation). The Agent contains a gRPC server that listens for requests from gRPC clients. Communication between the Manager and Agent is done via vsock. The Agent sends events to the Manager via vsock, which then forwards these via gRPC. Agent contains a gRPC server that exposes useful functions that can be accessed by other gRPC clients such as the CLI.

"},{"location":"architecture/#manager","title":"Manager","text":"

Manager is a gRPC client that listens to requests sent through gRPC and sends them to Agent via vsock. Manager creates a secure enclave and loads the computation where the agent resides. The connection between Manager and Agent is through vsock, through which channel agent sends events periodically to manager, who forwards these via gRPC.

"},{"location":"architecture/#cli","title":"CLI","text":"

CoCoS CLI is used to access the agent within the secure enclave. CLI communicates to agent using gRPC, with funcitons such as algo to provide the algorithm to be run, data to provide the data to be used in the computation, and run to start the computation. It also has functions to fetch and validate the attestation report of the enclave.

For more information on CLI, please refer to CLI docs.

"},{"location":"cli/","title":"Agent CLI","text":"

The CLI allows you to perform various tasks related to the computation and management of algorithms and datasets. The CLI is a gRPC client for the agent service.

"},{"location":"cli/#build","title":"Build","text":"

To build the CLI, follow these steps:

  1. Clone the repository: go get github.com/ultravioletrs/cocos.
  2. Navigate to the project root: cd cocos.
  3. Build the CLI binary: make cli.
  4. Install the CLI binary to bin: make install-cli.
"},{"location":"cli/#usage","title":"Usage","text":""},{"location":"cli/#run-computation","title":"Run Computation","text":"

To run a computation, use the following command:

./build/cocos-cli run --computation '{\"name\": \"my-computation\"}'\n
"},{"location":"cli/#upload-algorithm","title":"Upload Algorithm","text":"

To upload an algorithm, use the following command:

./build/cocos-cli algo /path/to/algorithm\n
"},{"location":"cli/#upload-dataset","title":"Upload Dataset","text":"

To upload a dataset, use the following command:

./build/cocos-cli data /path/to/dataset.csv\n
"},{"location":"cli/#retrieve-result","title":"Retrieve Result","text":"

To retrieve the computation result, use the following command:

./build/cocos-cli result\n
"},{"location":"cli/#installation","title":"Installation","text":"

To install the CLI locally, i.e. for the current user:

Run make install-cli.

"},{"location":"cli/#notes","title":"Notes","text":"

The CLI supports various configuration flags and options.

Use the --help flag with any command to see additionalinformation.

The CLI uses gRPC for communication with the Agent service.

"},{"location":"hal/","title":"Hardware Abstraction Layer (HAL)","text":"

HAL is a layer of programming that allows the software to interact with the hardware device at a general level rather than at the detailed hardware level. Cocos uses HAL and AMD SEV-SNP as an abstraction layer for confidential computing.

AMD SEV-SNP creates secure virtual machines (SVMs). VMs are usually used to run an operating system (e.g., Ubuntu and its applications). To avoid using a whole OS, HAL uses:

  • Linux kernel v6.6 - vmlinuz archive with the standard Linux kernel v6.6 with support for AMD SEV.
  • File system - the initial RAM file system (initramfs) that is used as the root file system of the VM.

This way, applications can be executed in the SVM, and the whole HAL SVM is entirely in RAM, protected by SEV-SNP. Being a RAM-only SVM means that secrets that are kept in the SVM will be destroyed when the SVM stops working.

"},{"location":"hal/#how-is-hal-constructed","title":"How is HAL constructed?","text":"

HAL is made using the tool Buildroot. Buildroot is used to create efficient, embedded Linux systems, and we use it to create the compressed image of the kernel (vmlinuz) and the initial file system (initramfs).

HAL configuration for Buildroot also includes Python runtime and agent software support. You can read more about the Agent software here.

"},{"location":"hal/#how-does-it-work","title":"How does it work?","text":"

HAL is combined with AMD SEV-SNP to provide a fully encrypted VM that can be verified using remote attestation. You can read more about the attestation process here.

Cocos uses QEMU and Open Virtual Machine Firmware (OVMF) to boot the confidential VM. During boot with SEV-SNP, the AMD Secure Processor (AMD SP) measures (calculates the hash) of the contents of the VM to insert that hash into the attestation report. This measurement is proof of what is currently running inside the VM. The problem with SEV is that it only measures the Open Virtual Machine Firmware (OVMF). To solve this, we have built OVMF so that OVMF contains hashes of the vmlinuz and initrams. Once the OVMF is loaded, it will load the vmlinuz and initramfs into memory, but it will continue the boot process only if the hashes of the vmlinuz and initramfs match the hashes stored in OVMF. This way, the attestation report will contain the measurement of OVMF, with the hashes, and OVMF will guarantee that the correct kernel and file system are booted. The whole process can be seen in the following diagram. The green color represents the trusted part of the system, while the red is untrusted:

This process guarantees that the whole VM is secure and can be verified.

After the kernel boots, the agent is started and ready for work.

"},{"location":"install/","title":"Install","text":"

Before proceeding, install the following prerequisites:

  • go (version 1.22.0)
  • Docker Compose (version 2.9.0)

Once everything is installed, execute the following command from project root:

To run CoCoS.ai, first download the cocos git repository:

git clone git@github.com:ultravioletrs/cocos.git\ncd cocos\nmake\n

Finally - you can run the backend (within cocos directory):

make run\n
"},{"location":"manager/","title":"Manager","text":"

Manager acts as the bridge between computation running in the VM and the user/organization. Once a computation is created by a user and the invited users have uploaded their public certificates and a run request is sent, the manager is responsible for creating the computation in the VM and managing the computation lifecycle. Communication to Manager is done via gRPC, while communication between Manager and Agent is done via vsock.

Vsock is used to send agent events from the computation in the agent to the manager. The manager then sends the events to via gRPC, and these are visible to the end user.

"},{"location":"manager/#manager-agent","title":"Manager <> Agent","text":"

Agent runs a gRPC server, and CLI is a gRPC client of agent. The manager sends the computation to the agent via gRPC and the agent runs the computation while sending evnets back to manager on the status. The manager then sends the events it receives from agent via vsock through gRPC.

"},{"location":"manager/#setup-and-test-manager-agent","title":"Setup and Test Manager <> Agent","text":"
git clone https://github.com/ultravioletrs/cocos\ncd cocos\n

NB: all relative paths in this document are relative to cocos repository directory.

"},{"location":"manager/#qemu-kvm","title":"QEMU-KVM","text":"

QEMU-KVM is a virtualization platform that allows you to run multiple operating systems on the same physical machine. It is a combination of two technologies: QEMU and KVM.

  • QEMU is an emulator that can run a variety of operating systems, including Linux, Windows, and macOS.
  • KVM is a Linux kernel module that allows QEMU to run virtual machines.

To install QEMU-KVM on a Debian based machine, run

sudo apt update\nsudo apt install qemu-kvm\n

Create img directory in cmd/manager. Create tmp directory in cmd/manager.

"},{"location":"manager/#add-v-sock","title":"Add V-sock","text":"

The necessary kernel modules must be loaded on the hypervisor.

sudo modprobe vhost_vsock\nls -l /dev/vhost-vsock\n# crw-rw-rw- 1 root kvm 10, 241 Jan 16 12:05 /dev/vhost-vsock\nls -l /dev/vsock\n# crw-rw-rw- 1 root root 10, 121 Jan 16 12:05 /dev/vsock\n
"},{"location":"manager/#prepare-cocos-hal","title":"Prepare Cocos HAL","text":"

Cocos HAL for Linux is framework for building custom in-enclave Linux distribution. Use the instructions in Readme. Once the image is built copy the kernel and rootfs image to cmd/manager/img from buildroot/output/images/bzImage and buildroot/output/images/rootfs.cpio.gz respectively.

"},{"location":"manager/#test-vm-creation","title":"Test VM Creation","text":"
cd cmd/manager\n\nsudo find / -name OVMF_CODE.fd\n# => /usr/share/OVMF/OVMF_CODE.fd\nOVMF_CODE=/usr/share/OVMF/OVMF_CODE.fd\n\nsudo find / -name OVMF_VARS.fd\n# => /usr/share/OVMF/OVMF_VARS.fd\nOVMF_VARS=/usr/share/OVMF/OVMF_VARS.fd\n\nKERNEL=\"img/bzImage\"\nINITRD=\"img/rootfs.cpio.gz\"\n\nqemu-system-x86_64 \\\n    -enable-kvm \\\n    -cpu EPYC-v4 \\\n    -machine q35 \\\n    -smp 4 \\\n    -m 2048M,slots=5,maxmem=10240M \\\n    -no-reboot \\\n    -drive if=pflash,format=raw,unit=0,file=$OVMF_CODE,readonly=on \\\n    -netdev user,id=vmnic,hostfwd=tcp::7020-:7002 \\\n    -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= \\\n    -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3 -vnc :0 \\\n    -kernel $KERNEL \\\n    -append \"earlyprintk=serial console=ttyS0\" \\\n    -initrd $INITRD \\\n    -nographic \\\n    -monitor pty \\\n    -monitor unix:monitor,server,nowait\n

Once the VM is booted press enter and on the login use username root.

"},{"location":"manager/#build-and-run-agent","title":"Build and Run Agent","text":"

Agent is started automatically in the VM.

# List running processes and use 'grep' to filter for processes containing 'agent' in their names.\nps aux | grep cocos-agent\n# This command helps verify that the 'agent' process is running.\n# The output shows the process ID (PID), resource usage, and other information about the 'cocos-agent' process.\n# For example: 118 root     cocos-agent\n

We can also check if Agent is reachable from the host machine:

# Use netcat (nc) to test the connection to localhost on port 7020.\nnc -zv localhost 7020\n# Output:\n# nc: connect to localhost (::1) port 7020 (tcp) failed: Connection refused\n# Connection to localhost (127.0.0.1) 7020 port [tcp/*] succeeded!\n
"},{"location":"manager/#conclusion","title":"Conclusion","text":"

Now you are able to use Manager with Agent. Namely, Manager will create a VM with a separate OVMF variables file on manager /run request.

"},{"location":"manager/#ovmf","title":"OVMF","text":"

We need Open Virtual Machine Firmware. OVMF is a port of Intel's tianocore firmware - an open source implementation of the Unified Extensible Firmware Interface (UEFI) - used by a qemu virtual machine. We need OVMF in order to run virtual machine with focal-server-cloudimg-amd64. When we install QEMU, we get two files that we need to start a VM: OVMF_VARS.fd and OVMF_CODE.fd. We will make a local copy of OVMF_VARS.fd since a VM will modify this file. On the other hand, OVMF_CODE.fd is only used as a reference, so we only record its path in an environment variable.

sudo find / -name OVMF_CODE.fd\n# => /usr/share/OVMF/OVMF_CODE.fd\nMANAGER_QEMU_OVMF_CODE_FILE=/usr/share/OVMF/OVMF_CODE.fd\n\nsudo find / -name OVMF_VARS.fd\n# => /usr/share/OVMF/OVMF_VARS.fd\nMANAGER_QEMU_OVMF_VARS_FILE=/usr/share/OVMF/OVMF_VARS.fd\n

NB: we set environment variables that we will use in the shell process where we run manager.

"},{"location":"manager/#deployment","title":"Deployment","text":"

To start the service, execute the following shell script (note a server needs to be running see here):

# download the latest version of the service\ngo get github.com/ultravioletrs/cocos\n\ncd $GOPATH/src/github.com/ultravioletrs/cocos\n\n# compile the manager\nmake manager\n\n# copy binary to bin\nmake install\n\n# set the environment variables and run the service\nMANAGER_GRPC_URL=localhost:7001\nMANAGER_LOG_LEVEL=debug \\\nMANAGER_QEMU_USE_SUDO=false \\\nMANAGER_QEMU_ENABLE_SEV=false \\\n./build/cocos-manager\n

To enable AMD SEV support, start manager like this 

MANAGER_GRPC_URL=localhost:7001\nMANAGER_LOG_LEVEL=debug \\\nMANAGER_QEMU_USE_SUDO=true \\\nMANAGER_QEMU_ENABLE_SEV=true \\\nMANAGER_QEMU_SEV_CBITPOS=51 \\\n./build/cocos-manager\n
"},{"location":"manager/#verifying-vm-launch","title":"Verifying VM Launch","text":"

NB: To verify that the manager successfully launched the VM, you need to open two terminals on the same machine. In one terminal, you need to launch go run main.go (with the environment variables of choice) and in the other, you can run the verification commands.

To verify that the manager launched the VM successfully, run the following command:

ps aux | grep qemu-system-x86_64\n

You should get something similar to this

darko     324763 95.3  6.0 6398136 981044 ?      Sl   16:17   0:15 /usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty\n

If you run a command as sudo, you should get the output similar to this one

root       37982  0.0  0.0   9444  4572 pts/0    S+   16:18   0:00 sudo /usr/local/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 -machine memory-encryption=sev0 -nographic -monitor pty\nroot       37989  122 13.1 5345816 4252312 pts/0 Sl+  16:19   0:04 /usr/local/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 -machine memory-encryption=sev0 -nographic -monitor pty\n

The two processes are due to the fact that we run the command /usr/bin/qemu-system-x86_64 as sudo, so there is one process for sudo command and the other for /usr/bin/qemu-system-x86_64.

"},{"location":"manager/#troubleshooting","title":"Troubleshooting","text":"

If the ps aux | grep qemu-system-x86_64 give you something like this

darko      13913  0.0  0.0      0     0 pts/2    Z+   20:17   0:00 [qemu-system-x86] <defunct>\n

means that the a QEMU virtual machine that is currently defunct, meaning that it is no longer running. More precisely, the defunct process in the output is also known as a \"zombie\" process.

You can troubleshoot the VM launch procedure by running directly qemu-system-x86_64 command. When you run manager with MANAGER_LOG_LEVEL=info env var set, it prints out the entire command used to launch a VM. The relevant part of the log might look like this

{\"level\":\"info\",\"message\":\"/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty\",\"ts\":\"2023-08-14T18:29:19.2653908Z\"}\n

You can run the command - the value of the \"message\" key - directly in the terminal:

/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty\n

and look for the possible problems. This problems can usually be solved by using the adequate env var assignments. Look in the manager/qemu/config.go file to see the recognized env vars. Don't forget to prepend MANAGER_QEMU_ to the name of the env vars.

"},{"location":"manager/#kill-qemu-system-x86_64-processes","title":"Kill qemu-system-x86_64 Processes","text":"

To kill any leftover qemu-system-x86_64 processes, use

pkill -f qemu-system-x86_64\n

The pkill command is used to kill processes by name or by pattern. The -f flag to specify that we want to kill processes that match the pattern qemu-system-x86_64. It sends the SIGKILL signal to all processes that are running qemu-system-x86_64.

If this does not work, i.e. if ps aux | grep qemu-system-x86_64 still outputs qemu-system-x86_64 related process(es), you can kill the unwanted process with kill -9 <PID>, which also sends a SIGKILL signal to the process.

"},{"location":"tee/","title":"TEE","text":"

A trusted execution environment (TEE) is a separate part of the main memory and the CPU that encrypts code/data and enables \"on the fly\" executions of the said encrypted code/data. One of the examples of TEEs is Intel Secure Guard Extensions (SGX) and AMD Secure Encrypted Virtualization (SEV).

"},{"location":"tee/#amd-sev","title":"AMD SEV","text":"

AMD SEV and its latest and most secure iteration, AMD Secure Encrypted Virtualization - Secure Nested Paging (SEV-SNP), is the AMD technology that isolates entire virtual machines (VMs). SEV-SNP encrypts the whole VM and provides confidentiality and integrity protection of the VM memory. This way, the hypervisor or any other application on the host machine cannot read the VM memory.

At Cocos, we use an in-memory VM called the Hardware Abstraction Layer (HAL). You can read more on HAL here.

One of the critical components of the SEV technology is the remote attestation. Remote attestation is a process in which one side (the attester) collects information about itself and sends that information to the client (or the relying party) for the relying party to assess the trustworthiness of the attester. If the attester is deemed trustworthy, the relying party will send confidential code/data or any secrets to the attester. You can read more on the attestation process here.

"}]} \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml new file mode 100644 index 0000000..caafc69 --- /dev/null +++ b/sitemap.xml @@ -0,0 +1,48 @@ + + + + https://docs.cocos.ai/ + 2024-03-20 + daily + + + https://docs.cocos.ai/agent/ + 2024-03-20 + daily + + + https://docs.cocos.ai/architecture/ + 2024-03-20 + daily + + + https://docs.cocos.ai/attestation/ + 2024-03-20 + daily + + + https://docs.cocos.ai/cli/ + 2024-03-20 + daily + + + https://docs.cocos.ai/hal/ + 2024-03-20 + daily + + + https://docs.cocos.ai/install/ + 2024-03-20 + daily + + + https://docs.cocos.ai/manager/ + 2024-03-20 + daily + + + https://docs.cocos.ai/tee/ + 2024-03-20 + daily + + \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz new file mode 100644 index 0000000000000000000000000000000000000000..d4b5f2641cf1eca8d91c85b8ae5f976fd1aba336 GIT binary patch literal 260 zcmV+f0sHf1|qH>O8VhPl8DfBt-d8O-r(iQOj_3OSA9PHNF%f;@#ZkK*n1rGFGh zdobq`3AwA{M5FM`r)Q8cFh|RT^cd<33km~cr=CBr*>MWMluo(Yg0xwJbf9($) K$2{?`1pok2v3*kj literal 0 HcmV?d00001 diff --git a/tee/index.html b/tee/index.html new file mode 100644 index 0000000..6d16c7c --- /dev/null +++ b/tee/index.html @@ -0,0 +1,576 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + TEE - Cocos + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

TEE#

+

A trusted execution environment (TEE) is a separate part of the main memory and the CPU that encrypts code/data and enables "on the fly" executions of the said encrypted code/data. One of the examples of TEEs is Intel Secure Guard Extensions (SGX) and AMD Secure Encrypted Virtualization (SEV).

+

AMD SEV#

+

AMD SEV and its latest and most secure iteration, AMD Secure Encrypted Virtualization - Secure Nested Paging (SEV-SNP), is the AMD technology that isolates entire virtual machines (VMs). SEV-SNP encrypts the whole VM and provides confidentiality and integrity protection of the VM memory. This way, the hypervisor or any other application on the host machine cannot read the VM memory.

+

At Cocos, we use an in-memory VM called the Hardware Abstraction Layer (HAL). You can read more on HAL here.

+

One of the critical components of the SEV technology is the remote attestation. Remote attestation is a process in which one side (the attester) collects information about itself and sends that information to the client (or the relying party) for the relying party to assess the trustworthiness of the attester. If the attester is deemed trustworthy, the relying party will send confidential code/data or any secrets to the attester. You can read more on the attestation process here.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + \ No newline at end of file