Skip to content

Commit

Permalink
update sriov get-started and coordinator docs (#2123)
Browse files Browse the repository at this point in the history
  • Loading branch information
@cyclinder
cyclinder authored Aug 2, 2023
1 parent f8098ec commit c38b9fa
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 28 deletions.
4 changes: 3 additions & 1 deletion docs/usage/coordinator-zh_CN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@ default via 10.6.0.1 dev eth0

- 因为 calico 会用到一些封装技术，性能会得到影响
- 大多都不支持 Pod 的 IP 地址固定
- 如果通过 Multus 为 Pod 附加多张网卡时，Pod 通信时常常遇到来回路径不一致的问题，导致 Pod 无法正常通信
- 如果通过 Multus 为 Pod 附加多张网卡时，Pod 通信时常常遇到数据包来回路径不一致的问题，导致 Pod 无法正常通信

> 当 Pod 附加了多张网卡时，极大可能出现 Pod 通信数据包来回路径不一致的问题。如果数据链路上存在一些安全设备，由于数据包的来回路径不一致，流量可能被安全设备认为是 "半连接"(没有 TCP SYN 报文的记录，但收到 TCP ACK 报文 )，在这种情况下，安全设备会阻断掉该连接，造成 Pod 通信异常。
上述问题我们可以通过设置 coordinator 运行在 overlay 模式解决。 在此模式下，`coordinator` 不会创建 veth 设备，而是设置一些策略路由，确保 Pod 访问 ClusterIP 时从 eth0(通常由 Calico、Cilium等CNI创建) 转发，Pod 访问集群外部目标时从 net1 (通常由 Macvlan、IPvlan 等CNI创建) 转发。

Expand Down
30 changes: 16 additions & 14 deletions docs/usage/install/underlay/get-started-sriov-zh_CN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ Sriov-network-operator 可以帮助我们自动安装、配置 sriov-cni 和 sri
```

> 如果组件未就绪, 可能需要给 sriov 工作节点打上 label: 'node-role.kubernetes.io/worker=""'
>
> sriov-network-operator 默认安装在 sriov-network-operator 命名空间下
>
> 安装 sriov-network-operator 后, 因为需要配置节点启用 SR-IOV 功能，可能会重启节点。如有需要，安装到工作节点。

2. 配置 sriov-network-operator
Expand Down Expand Up @@ -155,22 +157,24 @@ Sriov-network-operator 可以帮助我们自动安装、配置 sriov-cni 和 sri
> 如果您是国内用户，可以指定参数 `--set global.imageRegistryOverride=ghcr.m.daocloud.io` 避免 Spiderpool 的镜像拉取失败。
2. 创建 SpiderSubnet 实例。
2. 创建 SpiderIPPool 实例。
Pod 会从该子网中获取 IP，进行 Underlay 的网络通讯，所以该子网需要与接入的 Underlay 子网对应。
以下是创建相关的 SpiderSubnet 示例
以下是创建相关的 SpiderIPPool 示例
```shell
cat <<EOF | kubectl apply -f -
apiVersion: spiderpool.spidernet.io/v2beta1
kind: SpiderSubnet
kind: SpiderIPPool
metadata:
name: subnet-test
name: ippool-test
spec:
default: true
ips:
- "10.20.168.190-10.20.168.199"
subnet: 10.20.0.0/16
gateway: 10.20.0.1
multusName: kube-system/sriov-test
EOF
```
Expand All @@ -189,6 +193,8 @@ Sriov-network-operator 可以帮助我们自动安装、配置 sriov-cni 和 sri
resourceName: spidernet.io/sriov_netdevice
EOF
```
> 注意: SpiderIPPool.Spec.multusName: `kube-system/sriov-test` 要和创建的 SpiderMultusConfig 实例的 Name 和 Namespace 相匹配
## 创建应用
Expand All @@ -208,10 +214,6 @@ Sriov-network-operator 可以帮助我们自动安装、配置 sriov-cni 和 sri
template:
metadata:
annotations:
ipam.spidernet.io/subnet: |-
{
"ipv4": ["subnet-test"]
}
v1.multus-cni.io/default-network: kube-system/sriov-test
labels:
app: sriov-deploy
Expand Down Expand Up @@ -264,17 +266,17 @@ Sriov-network-operator 可以帮助我们自动安装、配置 sriov-cni 和 sri
sriov-deploy-9b4b9f6d9-xfsvj 1/1 Running 0 6m54s 10.20.168.190 master-11 <none> <none>
```
3. Spiderpool 自动为应用创建了 IP 固定池，应用的 IP 将会自动固定在该 IP 范围内
3. 应用的 IP 将会自动固定在该 IP 范围内:
```shell
~# kubectl get spiderippool
NAME VERSION SUBNET ALLOCATED-IP-COUNT TOTAL-IP-COUNT DEFAULT DISABLE
auto-sriov-deploy-v4-eth0-f5488b112fd9 4 10.20.0.0/16 2 2 false false
NAME VERSION SUBNET ALLOCATED-IP-COUNT TOTAL-IP-COUNT DEFAULT DISABLE
ippool-test 4 10.20.0.0/16 2 10 true false
~# kubectl get spiderendpoints
NAME INTERFACE IPV4POOL IPV4 IPV6POOL IPV6 NODE
sriov-deploy-9b4b9f6d9-mmpsm eth0 auto-sriov-deploy-v4-eth0-f5488b112fd9 10.20.168.191/16 worker-12
sriov-deploy-9b4b9f6d9-xfsvj eth0 auto-sriov-deploy-v4-eth0-f5488b112fd9 10.20.168.190/16 master-11
NAME INTERFACE IPV4POOL IPV4 IPV6POOL IPV6 NODE
sriov-deploy-9b4b9f6d9-mmpsm eth0 ippool-test 10.20.168.191/16 worker-12
sriov-deploy-9b4b9f6d9-xfsvj eth0 ippool-test 10.20.168.190/16 master-11
```
4. 测试 Pod 与 Pod 的通讯
Expand Down
28 changes: 15 additions & 13 deletions docs/usage/install/underlay/get-started-sriov.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ SriovNetwork helps us install sriov-cni and sriov-device-plugin components, maki
```

> You may need to label SR-IOV worker nodes using node-role.kubernetes.io/worker="" label, if not already.
>
> By default, SR-IOV Operator will be deployed in namespace 'openshift-sriov-network-operator'.
>
> After installation, the node may reboot automatically. If necessary, install sriov-network-operator to the designated worker nodes.

2. Configure sriov-network-operator
Expand Down Expand Up @@ -153,22 +155,24 @@ SriovNetwork helps us install sriov-cni and sriov-device-plugin components, maki
> If you are mainland user who is not available to access ghcr.io，You can specify the parameter `-set global.imageRegistryOverride=ghcr.m.daocloud.io` to avoid image pulling failures for Spiderpool.
2. Create a SpiderSubnet instance.
2. Create a SpiderIPPool instance.
The Pod will obtain an IP address from this subnet for underlying network communication, so the subnet needs to correspond to the underlying subnet that is being accessed.
Here is an example of creating a SpiderSubnet instance:：
```shell
cat <<EOF | kubectl apply -f -
apiVersion: spiderpool.spidernet.io/v2beta1
kind: SpiderSubnet
kind: SpiderIPPool
metadata:
name: subnet-test
name: ippool-test
spec:
default: true
ips:
- "10.20.168.190-10.20.168.199"
subnet: 10.20.0.0/16
gateway: 10.20.0.1
multusName: kube-system/sriov-test
EOF
```
Expand All @@ -187,6 +191,8 @@ SriovNetwork helps us install sriov-cni and sriov-device-plugin components, maki
resourceName: spidernet.io/sriov_netdevice
```
> Note: SpiderIPPool.Spec.multusName: 'kube-system/sriov-test' must be to match the Name and Namespace of the SpiderMultusConfig instance created.
## Create applications
1. Create test Pods and Services via the command below：
Expand All @@ -205,10 +211,6 @@ SriovNetwork helps us install sriov-cni and sriov-device-plugin components, maki
template:
metadata:
annotations:
ipam.spidernet.io/subnet: |-
{
"ipv4": ["subnet-test"]
}
v1.multus-cni.io/default-network: kube-system/sriov-test
labels:
app: sriov-deploy
Expand Down Expand Up @@ -261,17 +263,17 @@ SriovNetwork helps us install sriov-cni and sriov-device-plugin components, maki
sriov-deploy-9b4b9f6d9-xfsvj 1/1 Running 0 6m54s 10.20.168.190 master-11 <none> <none>
```
3. Spiderpool has created fixed IP pools for applications, ensuring that the applications' IPs are automatically fixed within the defined ranges.
3. Spiderpool ensuring that the applications' IPs are automatically fixed within the defined ranges.
```shell
~# kubectl get spiderippool
NAME VERSION SUBNET ALLOCATED-IP-COUNT TOTAL-IP-COUNT DEFAULT DISABLE
auto-sriov-deploy-v4-eth0-f5488b112fd9 4 10.20.0.0/16 2 2 false false
NAME VERSION SUBNET ALLOCATED-IP-COUNT TOTAL-IP-COUNT DEFAULT DISABLE
ippool-test 4 10.20.0.0/16 2 10 true false
~# kubectl get spiderendpoints
NAME INTERFACE IPV4POOL IPV4 IPV6POOL IPV6 NODE
sriov-deploy-9b4b9f6d9-mmpsm eth0 auto-sriov-deploy-v4-eth0-f5488b112fd9 10.20.168.191/16 worker-12
sriov-deploy-9b4b9f6d9-xfsvj eth0 auto-sriov-deploy-v4-eth0-f5488b112fd9 10.20.168.190/16 master-11
NAME INTERFACE IPV4POOL IPV4 IPV6POOL IPV6 NODE
sriov-deploy-9b4b9f6d9-mmpsm eth0 ippool-test 10.20.168.191/16 worker-12
sriov-deploy-9b4b9f6d9-xfsvj eth0 ippool-test 10.20.168.190/16 master-11
```
4. Test the communication between Pods:
Expand Down

0 comments on commit c38b9fa

Please sign in to comment.