diff --git a/docs/README.md b/docs/README.md index e69de29b..adba95be 100644 --- a/docs/README.md +++ b/docs/README.md @@ -0,0 +1,12 @@ +# Atm0s Media Server + +A decentralized media server designed to handle media streaming on a global scale, making it suitable for large-scale applications but with minimal cost. + +It is developed by 8xFF, a group of independent developers who are passionate about building a new generation of media server and network infrastructure with decentralization in mind. While we have received support from various companies and individuals, we are not affiliated with any specific company. 8xFF is a community-driven project, and we welcome anyone interested in contributing to join us. + +For a deep dive into the technical aspects of network architecture, please refer to our [Smart-Routing](https://github.com/8xFF/atm0s-sdn/blob/master/docs/smart_routing.md) + +[](https://www.youtube.com/embed/QF8ZJq9xuSU) + +(Above is a demo video of the version used by Bluesea Network) diff --git a/docs/getting-started/installation/README.md b/docs/getting-started/installation/README.md index 508091f3..6d3d01e7 100644 --- a/docs/getting-started/installation/README.md +++ b/docs/getting-started/installation/README.md @@ -1,6 +1,6 @@ # Installation -To install, you can either: +Atm0s-media-server is built into a single executable file, it can be get by some ways: - Install from Docker diff --git a/docs/getting-started/installation/console_screen.png b/docs/getting-started/installation/console_screen.png new file mode 100644 index 00000000..61473bdf Binary files /dev/null and b/docs/getting-started/installation/console_screen.png differ diff --git a/docs/getting-started/installation/console_screen2.png b/docs/getting-started/installation/console_screen2.png new file mode 100644 index 00000000..bda2afb8 Binary files /dev/null and b/docs/getting-started/installation/console_screen2.png differ diff --git a/docs/getting-started/installation/docker-compose.md b/docs/getting-started/installation/docker-compose.md index 11195fd3..b075bbb7 100644 --- a/docs/getting-started/installation/docker-compose.md +++ b/docs/getting-started/installation/docker-compose.md @@ -1,4 +1,4 @@ -# Docker-compose +# Docker-compose (outdate, update needed) We can use `docker-compose` to deploy atm0s-media-server. diff --git a/docs/getting-started/installation/kubernetes.md b/docs/getting-started/installation/kubernetes.md index d08f7e55..37fe0259 100644 --- a/docs/getting-started/installation/kubernetes.md +++ b/docs/getting-started/installation/kubernetes.md @@ -1,4 +1,4 @@ -# Kubernetes +# Kubernetes (outdate, update needed) You can install into kubernetes cluster by Helm chart diff --git a/docs/getting-started/installation/multi-zones.md b/docs/getting-started/installation/multi-zones.md index cee122ad..a81fb4ef 100644 --- a/docs/getting-started/installation/multi-zones.md +++ b/docs/getting-started/installation/multi-zones.md @@ -12,9 +12,9 @@ Note that you can deploy multi connectors in some zones to handle room and peer ## Prerequisites -- Choose a different zone prefix for each zone. For example, if you have 3 zones, they can be named 0x000001, 0x000002, and 0x000003. +- Choose a different zone id for each zone, it is 32bit with format: 0x00_00_XX_00, example: 0, 256, 512 - Select a secret for all zones. ## Deploying each zone, same as a single-zone cluster -The deployment steps are the same as for a single-zone cluster with addition `--zone ZONE_ID` param. However, starting from second zone, you need to add the first zone gateway address as a seed node for each gateway node. +The deployment steps are the same as for a single-zone cluster with addition `--zone-id ZONE_ID` param. However, starting from second zone, you don't need to add console node, instead of that you can reuse single console node for all zones. diff --git a/docs/getting-started/installation/single-zone.md b/docs/getting-started/installation/single-zone.md index 36108bfe..407259b8 100644 --- a/docs/getting-started/installation/single-zone.md +++ b/docs/getting-started/installation/single-zone.md @@ -7,8 +7,10 @@ Limitations: maximum 256 nodes in the same zone. The cluster has the following nodes: -- Gateway nodes -- Media server nodes (WebRTC, SIP or RTMP depending on your needs) +- Console node: manage panel and act as a seed node for bootstrapping other nodes +- Gateway nodes: route signaling request into correct media node +- Media server nodes: handling media transport like WebRTC, RTP, RTMP ... +- Connector node: logging, hooks, record uri signer In this mode, the gateway will route requests to the best node based on the load, and some users in the same room may be routed to different media server nodes. @@ -27,145 +29,105 @@ Example we have a node index rules like: | Node type | Index range | | --------- | ------------ | -| Gateway | [0; 10) | -| SIP | [10; 60) | -| RTMP | [60; 90) | -| Connector | [90; 100) | -| WebRTC | [100 to 255] | +| Console | [0; 10) | +| Gateway | [10; 20) | +| Connector | [20; 30) | +| Media | [100 to 255] | -## Deploy some gateway nodes - -First pick index for your gateway nodes, for example 1 to 10 is for gateway nodes. +## Deploy a console nodes -Node1: +Console node will expose at least 1 tcp port and 1 udp port -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --http-port=8080 \ - --sdn-port=10010 \ - --zone-index=1 \ - --secret=insecure \ - gateway ``` - -After node1 started it will print out the node address like `10@/ip4/192.168.1.10/udp/10010/ip4/192.168.1.10/tcp/10010`, you can use it as a seed node for other nodes. - -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --http-port=8080 \ - --zone-index=2 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - gateway +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 0 \ + --sdn-port 10000 \ + --sdn-zone 0 \ + --http-port 8080 \ + console ``` -## Deploy some media webrtc nodes - -If you need WebRTC, Whip or Whep you need deploy some webrtc nodes, select index for your webrtc nodes, for example 100 to 255 is for webrtc nodes. +After node 0 started it will print out the node address like `0@/ip4/127.0.0.1/udp/10000`, you can use it as a seed node for other nodes. +The console node also have simple UI at `http://IP:8080`, you can login with above secret: `secr3t` -WebRTC 1: +![alt text](console_screen.png) +![alt text](console_screen2.png) -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=100 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - webrtc -``` +## Deploy some gateway nodes -WebRTC 2: +Gateway node will expose at least 1 tcp port and 1 udp port -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=101 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - webrtc ``` - -## Deploy some media sip nodes - -If you need SIP, you need deploy some sip nodes, select index for your sip nodes, for example 10 to 60 is for sip nodes. - -SIP 1: - -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=10 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - sip \ - --addr SERVER_IP:5060 +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 10 \ + --sdn-port 10010 \ + --sdn-zone 0 \ + --http-port 3000 \ + --seed 0@/ip4/192.168.1.10/udp/10010 \ + gateway ``` -SIP 1: - -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=11 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - sip \ - --addr SERVER_IP:5060 ``` - -## Deploy some media rtmp nodes - -If you need RTMP, you need deploy some rtmp nodes, select index for your rtmp nodes, for example 60 to 100 is for rtmp nodes. - -Rtmp 1: - -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=60 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - sip \ - --addr SERVER_IP:5060 +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 11 \ + --sdn-port 10011 \ + --sdn-zone 0 \ + --http-port 3001 \ + --seed 0@/ip4/192.168.1.10/udp/10011 \ + gateway ``` -Rtmp 2: +After gateway nodes started, it will print out the node address like: `10@/ip4/127.0.0.1/udp/10010` and `11@/ip4/127.0.0.1/udp/10011`, the address is used as a seed address for media and connector nodes -```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=60 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ - sip \ - --addr SERVER_IP:5060 -``` +After started gateway node also expose openapi ui for better integrating: ## Deploy some connector nodes -If you need RTMP, you need deploy some rtmp nodes, select index for your rtmp nodes, for example 60 to 100 is for rtmp nodes. - -Connector 1: +We need separated connector node which take care store logging data and response signed s3 uri for storing record file. +In default it will store data inside a sqlite file. For config other you can provide db-uri like `protocol://username:password@host/database`, with protocol is: `sqlite`, `mysql`, `postgres` ```bash -docker run -d --name atm0s-media-gateway-1 \ - --net=host ghcr.io/8xff/atm0s-media-gateway:master \ - --zone-index=60 \ - --secret=insecure \ - --seeds FIRST_GATEWAY_ADDR \ +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 30 \ + --sdn-port 10030 \ + --sdn-zone 0 \ + --seed 10@/ip4/127.0.0.1/udp/10010 \ + --seed 11@/ip4/127.0.0.1/udp/10011 \ connector \ - --mq-uri nats://NATS_IP:4222 + --s3-uri "http://minioadmin:minioadmin@127.0.0.1:9000/record" ``` -## Monitoring - -Each cluster nodes will expose a dashboard and prometheus metrics, you can use it to monitor your cluster. +## Deploy some media nodes -Example bellow is gateway node dashboard: `gateway_url/dashboard/` +```bash +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 11 \ + --sdn-port 10011 \ + --sdn-zone 0 \ + --http-port 3001 \ + --seed 10@/ip4/127.0.0.1/udp/10010 \ + --seed 11@/ip4/127.0.0.1/udp/10011 \ + media \ + --allow-private-ip +``` -![Monitoring](../../imgs/demo-monitor.png) +```bash +docker run -d --name main-console --net=host ghcr.io/8xff/atm0s-media-gateway:master \ + --secret secr3t \ + --node-id 12 \ + --sdn-port 10011 \ + --sdn-zone 0 \ + --http-port 3001 \ + --seed 10@/ip4/127.0.0.1/udp/10010 \ + --seed 11@/ip4/127.0.0.1/udp/10011 \ + media \ + --allow-private-ip +``` ## Testing your cluster diff --git a/docs/imgs/multi-zones-abstract.excalidraw.png b/docs/imgs/multi-zones-abstract.excalidraw.png index e0ab10fd..521a647c 100644 Binary files a/docs/imgs/multi-zones-abstract.excalidraw.png and b/docs/imgs/multi-zones-abstract.excalidraw.png differ diff --git a/docs/imgs/single-zone.excalidraw.png b/docs/imgs/single-zone.excalidraw.png index 602bab9f..d5388fe1 100644 Binary files a/docs/imgs/single-zone.excalidraw.png and b/docs/imgs/single-zone.excalidraw.png differ diff --git a/packages/media_connector/Cargo.toml b/packages/media_connector/Cargo.toml index ece4c56b..cefd1ed4 100644 --- a/packages/media_connector/Cargo.toml +++ b/packages/media_connector/Cargo.toml @@ -13,7 +13,12 @@ prost = { workspace = true } lru = "0.12" async-trait = "0.1" sea-orm-migration = "0.12" -sea-orm = { version = "0.12", features = ["sqlx-sqlite", "runtime-tokio-rustls"] } +sea-orm = { version = "0.12", features = [ + "sqlx-sqlite", + "sqlx-postgres", + "sqlx-mysql", + "runtime-tokio-rustls", +] } serde_json = "1.0" s3-presign = "0.0.2"