From e9337a688276c30e50c82dc01f20163eff222df8 Mon Sep 17 00:00:00 2001
From: "tao.yang" <tao.yang@daocloud.io>
Date: Tue, 21 Nov 2023 18:03:04 +0800
Subject: [PATCH] docs: add egress docs for spiderpool

Signed-off-by: tao.yang <tao.yang@daocloud.io>
---
 docs/usage/egress-zh_CN.md                    | 261 ++++++++++++++++++
 docs/usage/egress.md                          | 261 ++++++++++++++++++
 .../install/underlay/get-started-macvlan.md   |   2 +-
 docs/usage/spider-ippool-zh_CN.md             |   1 -
 4 files changed, 523 insertions(+), 2 deletions(-)

diff --git a/docs/usage/egress-zh_CN.md b/docs/usage/egress-zh_CN.md
index b0760c0c60..c357437daa 100644
--- a/docs/usage/egress-zh_CN.md
+++ b/docs/usage/egress-zh_CN.md
@@ -1,3 +1,264 @@
 # Egress Policy
 
 [**English**](./egress.md) | **简体中文**
+
+## 介绍
+
+Spiderpool 是一个 Kubernetes 的 Underlay 网络解决方案，但在 Kubernetes 集群中，Pod 访问外部服务时，其出口 IP 地址不是固定的。在 Overlay 网络中，出口 IP 地址为 Pod 所在节点的地址，而在 Underlay 网络中，Pod 直接使用自身的 IP 地址与外部通信。因此，当 Pod 发生新的调度时，无论哪种网络模式，Pod 与外部通信时的 IP 地址都会发生变化。这种不稳定性给系统维护人员带来了 IP 地址管理的挑战。特别是在集群规模扩大以及需要进行网络故障诊断时，在集群外部，基于 Pod 原本的出口 IP 来管控出口流量很难实现。而 Spiderpool 可以搭配组件 [EgressGateway](https://github.com/spidernet-io/egressgateway) 完美解决 Underlay 网络下 Pod 出口流量管理的问题。
+
+## 项目功能
+
+EgressGateway 是一个开源的 Egress 网关，旨在解决在不同 CNI 网络模式下（Spiderpool、Calico、Flannel、Weave）出口 Egress IP 地址的问题。通过灵活配置和管理出口策略，为租户级或集群级工作负载设置 Egress IP，使得 Pod 访问外部网络时，系统会统一使用这个设置的 Egress IP 作为出口地址，从而提供了稳定的出口流量管理解决方案。但 EgressGateway 所有的规则都是生效在主机网络命名空间上的，要使 EgressGateway 策略生效，则 Pod 访问集群外部的流量，要经过主机的网络命名空间。因此可以通过 Spiderpool 的 `spidercoordinators` 的 `spec.hijackCIDR` 字段配置从主机转发的子网路由，再通过 [coordinator](../concepts/coordinator-zh_CN.md) 将匹配的流量从 veth pair 转发到主机上。使得所访问的外部流量从而被 EgressGateway 规则匹配，借此实现 underlay 网络下出口流量管理。
+
+Spiderpool 搭配 EgressGateway 具备如下的一些功能：
+
+- 解决 IPv4/IPv6 双栈连接问题，确保网络通信在不同协议栈下的无缝连接。
+- 解决 Egress 节点的高可用性问题，确保网络连通性不受单点故障的干扰。
+- 允许更精细的策略控制，可以通过 EgressGateway 灵活地过滤 Pods 的 Egress 策略，包括 Destination CIDR。
+- 允许过滤 Egress 应用（Pod），能够更精确地管理特定应用的出口流量。
+- 支持多个出口网关实例，能够处理多个网络分区或集群之间的通信。
+- 支持租户级别的 Egress IP。
+- 支持自动检测集群流量的 Egress 网关策略。
+- 支持命名空间默认 Egress 实例。
+- 可用于较低内核版本，适用于各种 Kubernetes 部署环境。
+
+## 实施要求
+
+1. 一套 Kubernetes 集群。
+
+2. 已安装 [Helm](https://helm.sh/docs/intro/install/)。
+
+## 步骤
+
+### 安装 Spiderpool
+
+可参考 [安装](./readme-zh_CN.md) 安装 Spiderpool，并创建 SpiderMultusConfig CR 与 IPPool CR.
+
+在安装完 Spiderpool 后，将集群外的服务地址添加到 `spiderpool.spidercoordinators` 的 'default' 对象的 'hijackCIDR' 中，使 Pod 访问这些外部服务时，流量先经过 Pod 所在的主机，从而被 EgressGateway 规则匹配。
+
+```shell
+# "10.6.168.63/32" 为外部服务地址。对于已经运行的 Pod，需要重启 Pod，这些路由规则才会在 Pod 中生效。
+~# kubectl patch spidercoordinators default  --type='merge' -p '{"spec": {"hijackCIDR": ["10.6.168.63/32"]}}'
+```
+
+### 安装 EgressGateway
+
+通过 helm 安装 EgressGateway
+
+```shell
+helm repo add egressgateway https://spidernet-io.github.io/egressgateway/
+helm repo update egressgateway
+helm install egressgateway egressgateway/egressgateway -n kube-system --set feature.tunnelIpv4Subnet="192.200.0.1/16" --set feature.enableGatewayReplyRoute=true  --wait --debug
+```
+
+> 如果需要使用 IPv6 ，可使用选项 `--set feature.enableIPv6=true` 开启，并设置 `feature.tunnelIpv6Subnet`, 值得注意的是在通过 `feature.tunnelIpv4Subnet` 与 `feature.tunnelIpv6Subnet` 配置 IPv4 或 IPv6 网段时，需要保证网段和集群内的其他地址不冲突。
+> `feature.enableGatewayReplyRoute` 为 true 时，将开启网关节点上的返回路由规则，在与 Spiderpool 搭配支持 underlay CNI 时，必须开启该选项。
+>
+> 如果您是国内用户，还可以指定参数 `--set global.imageRegistryOverride=ghcr.m.daocloud.io` 避免 EgressGateway 的镜像拉取失败。
+
+验证 EgressGateway 安装
+
+```bash
+~# kubectl get pod -n kube-system | grep egressgateway
+egressgateway-agent-4s8lt                   1/1     Running     0     29m
+egressgateway-agent-thzth                   1/1     Running     0     29m
+egressgateway-controller-77698899df-tln7j   1/1     Running     0     29m
+```
+
+更多安装细节，参考 [EgressGateway 安装](https://github.com/spidernet-io/egressgateway/blob/main/docs/usage/Install.zh.md)
+
+### 创建 EgressGateway 实例
+
+EgressGateway 定义了一组节点作为集群的出口网关，集群内的 egress 流量将会通过这组节点转发而出集群。因此，需要预先定义 EgressGateway 实例，以下的示例 Yaml 会创建一个 EgressGateway 实例，其中：
+
+- `spec.ippools.ipv4`：定义了一组 egress 的出口 IP 地址，需要根据具体环境的实际情况调整。并且 `spec.ippools.ipv4` 的 CIDR 应该与网关节点上的出口网卡（一般情况下是默认路由的网卡）的子网相同，否则，可能导致 egress 访问不通。
+
+- `spec.nodeSelector`：EgressGateway 提供的节点亲和性方式，当 `selector.matchLabels` 与节点匹配时，该节点将会被作为集群的出口网关，当 `selector.matchLabels` 与节点不匹配时，EgressGateway 会略过该节点，它将不会被作为集群的出口网关，它支持选择多个节点来实现高可用。
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressGateway
+metadata:
+  name: default
+spec:
+  ippools:
+    ipv4:
+    - "10.6.168.201-10.6.168.205"
+  nodeSelector:
+    selector:
+      matchLabels:
+        egressgateway: "true"
+EOF
+```
+
+给节点打上上述中 `nodeSelector.selector.matchLabels` 所指定的 Label，使节点能被 EgressGateway 选中，作为出口网关。
+
+```bash
+~# kubectl get node
+NAME                STATUS   ROLES           AGE     VERSION
+controller-node-1   Ready    control-plane   5d17h   v1.26.7
+worker-node-1       Ready    <none>          5d17h   v1.26.7
+
+~# kubectl label node worker-node-1 egressgateway="true"
+```
+
+创建完成后，查看 EgressGateway 状态。其中：
+
+- `spec.ippools.ipv4DefaultEIP` 代表该组 EgressGateway 的默认 VIP，它是会从 `spec.ippools.ipv4` 中随机选择的一个 IP 地址，它的作用是：当为应用创建 EgressPolicy 对象时，如果未指定 VIP 地址，则使用该默认 VIP
+
+- `status.nodeList` 代表识别到了符合 `spec.nodeSelector` 的节点及该节点对应的 EgressTunnel 对象的状态。
+
+```shell
+~# kubectl get EgressGateway default -o yaml
+...
+spec:
+  ippools:
+    ipv4DefaultEIP: 10.6.168.201
+...
+status:
+  nodeList:
+  - name: worker-node-1
+    status: Ready
+```
+
+### 创建应用和出口策略
+
+创建一个应用，它将用于测试 Pod 访问集群外部用途，并给它打上 label 以便与 EgressPolicy 关联，如下是示例 Yaml，其中：
+
+- `v1.multus-cni.io/default-network`：用于指定应用所使用的子网，该值对应的 Multus CR 需参考[安装](./readme-zh_CN.md)文档提前创建。
+
+- `ipam.spidernet.io/ippool`：指定 Pod 使用哪些的 SpiderIPPool 资源, 该值对应的 SpiderIPPool CR 需参考[安装](./readme-zh_CN.md)文档提前创建。
+
+```bash
+cat <<EOF | kubectl create -f -
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: test-app
+  name: test-app
+  namespace: default
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: test-app
+  template:
+    metadata:
+      labels:
+        app: test-app
+      annotations:
+        ipam.spidernet.io/ippool: |-
+            {      
+              "ipv4": ["v4-pool"]
+            }
+        v1.multus-cni.io/default-network: kube-system/macvlan-conf
+    spec:
+      containers:
+      - image: nginx
+        imagePullPolicy: IfNotPresent
+        name: test-app
+        ports:
+        - name: http
+          containerPort: 80
+          protocol: TCP
+EOF
+```
+
+EgressPolicy 实例用于定义哪些 Pod 的出口流量要经过 EgressGateway 节点转发，以及其它的配置细节。如下是为应用创建 EgressPolicy CR 对象的示例，其中。
+
+- `spec.egressGatewayName` 用于指定了使用哪一组 EgressGateway 。
+
+- `spec.appliedTo.podSelector` 用于指定本策略在集群内的哪些 Pod 上生效。
+
+- `namespace` 用于指定 EgressPolicy 对象所在租户，因为 EgressPolicy 是租户级别的，所以它务必创建在上述关联应用的相同 namespace 下，这样匹配的 Pod 访问任意集群外部的地址时，才能被 EgressGateway Node 转发。
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressPolicy
+metadata:
+ name: test
+ namespace: default
+spec:
+ egressGatewayName: default
+ appliedTo:
+  podSelector:
+   matchLabels:
+    app: "test-app"
+EOF
+```
+
+创建完成后，查看 EgressPolicy 的状态。其中：
+
+- `status.eip` 展示了该组应用出集群时使用的出口 IP 地址。
+
+- `status.node` 展示了哪一个 EgressGateway 的节点负责该 EgressPolicy 出口流量的转发。
+
+```bash
+~# kubectl get EgressPolicy -A
+NAMESPACE   NAME   GATEWAY   IPV4           IPV6   EGRESSNODE
+default     test   default   10.6.168.201          worker-node-1
+ 
+~# kubectl get EgressPolicy test -o yaml
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressPolicy
+metadata:
+  name: test
+  namespace: default
+spec:
+  appliedTo:
+    podSelector:
+      matchLabels:
+        app: test-app
+  egressIP:
+    allocatorPolicy: default
+    useNodeIP: false
+status:
+  eip:
+    ipv4: 10.6.168.201
+  node: worker-node-1
+```
+
+创建 EgressPolicy 对象后，会根据 EgressPolicy 选择的应用生成一个包含所有应用的 IP 地址集合的 EgressEndpointSlices 对象，当应用无法出口访问时，可以查看 EgressEndpointSlices 对象中的 IP 地址是否正常。
+
+```bash
+~# kubectl get egressendpointslices -A
+NAMESPACE   NAME         AGE
+default     test-4vbqf   41s
+
+~# kubectl get egressendpointslices test-kvlp6 -o yaml
+apiVersion: egressgateway.spidernet.io/v1beta1
+endpoints:
+- ipv4:
+  - 10.6.168.208
+  node: worker-node-1
+  ns: default
+  pod: test-app-f44846544-8dnzp
+kind: EgressEndpointSlice
+metadata:
+  name: test-4vbqf
+  namespace: default
+```
+
+### 测试
+
+在集群外部署应用 `nettools`，用于模拟一个集群外部的服务，而 `nettools` 会在 http 回复中返回请求者的源 IP 地址。
+
+```bash
+~# docker run -d --net=host ghcr.io/spidernet-io/egressgateway-nettools:latest /usr/bin/nettools-server -protocol web -webPort 8080
+```
+
+在集群内的测试应用：test-app 中，验证出口流量的效果，我们可以看到在该应用对应 Pod 中访问外部服务时，`nettools` 返回的源 IP 符合了 `EgressPolicy.status.eip` 的效果。
+
+```bash
+~# kubectl get pod -owide
+NAME                       READY   STATUS    RESTARTS      AGE     IP              NODE                NOMINATED NODE   READINESS GATES
+test-app-f44846544-8dnzp   1/1     Running   0             4m27s   10.6.168.208    worker-node-1       <none>           <none>
+
+~# kubectl exec -it test-app-f44846544-8dnzp bash
+
+~# curl 10.6.1.92:8080 # 集群外节点的 IP 地址 + webPort
+Remote IP: 10.6.168.201
+```
diff --git a/docs/usage/egress.md b/docs/usage/egress.md
index ca21a5c0be..87ca67d527 100644
--- a/docs/usage/egress.md
+++ b/docs/usage/egress.md
@@ -1,3 +1,264 @@
 # Egress Policy
 
 **English** | [**简体中文**](./egress-zh_CN.md)
+
+## Introduction
+
+Spiderpool is an Underlay networking solution for Kubernetes, but the egress IP address is not fixed in a Kubernetes cluster when a Pod accesses an external service. In an Overlay network, the egress IP address is the address of the node on which the Pod resides, whereas in an Underlay network, the Pod communicates directly with the outside world using its own IP address. Therefore, when a Pod undergoes new scheduling, the IP address of the Pod when communicating with the outside world will change regardless of the network mode. This instability creates IP address management challenges for system maintainers. Especially when the cluster scale increases and network troubleshooting is required, it is difficult to control the egress traffic based on the Pod's original egress IP outside the cluster. Spiderpool can be used with the component [EgressGateway](https://github.com/spidernet-io/egressgateway) to solve the problem of Pod egress traffic management in Underlay network.
+
+## Features of EgressGateway
+
+EgressGateway is an open source Egress gateway designed to solve the problem of exporting Egress IP addresses in different CNI network modes (Spiderpool, Calico, Flannel, Weave). By flexibly configuring and managing egress policies, Egress IP is set for tenant-level or cluster-level workloads, so that when a Pod accesses an external network, the system will uniformly use this set Egress IP as the egress address, thus providing a stable egress traffic management solution. However, all EgressGateway rules are effective on the host's network namespace. To make the EgressGateway policy effective, the traffic of Pods accessing the outside of the cluster has to go through the host's network namespace. Therefore, you can configure the subnet routes forwarded from the host via the `spec.hijackCIDR` field of `spidercoordinators` in Spiderpool, and then configure the subnet routes forwarded from the host via [coordinator](../concepts/coordinator.md) to forward matching traffic from the veth pair to the host. This enables egress traffic management on an underlay network by allowing access to external traffic to be matched by EgressGateway rules.
+
+Some of the features and benefits of Spiderpool with EgressGateway are as follows:
+
+* Solve IPv4 IPv6 dual-stack connectivity,ensuring seamless communication across different protocol stacks.
+* Solve the high availability of Egress Nodes, ensuring network connectivity remains unaffected by single-point failures.
+* Support finer-grained policy control, allowing flexible filtering of Pods' Egress policies, including Destination CIDR.
+* Support application-level control, allowing EgressGateway to filter Egress applications (Pods) for precise management of specific application outbound traffic.
+* Support multiple egress gateways instance,capable of handling communication between multiple network partitions or clusters.
+* Support namespaced egress IP.
+* Support automatic detection of cluster traffic for egress gateways policies.
+* Support namespace default egress instances.
+* Can be used in low kernel version, making EgressGateway suitable for various Kubernetes deployment environments.
+
+## Prerequisites
+
+1. A ready Kubernetes kubernetes.
+
+2. [Helm](https://helm.sh/docs/intro/install/) has been already installed.
+
+## Steps
+
+### Install Spiderpool
+
+Refer to [Installation](./readme.md) to install Spiderpool and create SpiderMultusConfig CR and IPPool CR.
+
+After installing Spiderpool, Add the service addresses outside the cluster to the 'hijackCIDR' field in the 'default' object of spiderpool.spidercoordinators. This ensures that when Pods access these external services, the traffic is routed through the host where the Pod is located, allowing the EgressGateway rules to match.
+
+```bash
+# For running Pods, you need to restart them for these routing rules to take effect within the Pods.
+~# kubectl patch spidercoordinators default  --type='merge' -p '{"spec": {"hijackCIDR": ["10.6.168.63/32"]}}'
+```
+
+### Install EgressGateway
+
+Installing EgressGateway via helm
+
+```shell
+helm repo add egressgateway https://spidernet-io.github.io/egressgateway/
+helm repo update egressgateway
+helm install egressgateway egressgateway/egressgateway -n kube-system --set feature.tunnelIpv4Subnet="192.200.0.1/16" --set feature.enableGatewayReplyRoute=true  --wait --debug
+```
+
+> If IPv6 is required, enable it with the option `-set feature.enableIPv6=true` and set `feature.tunnelIpv6Subnet`, it is worth noting that when configuring IPv4 or IPv6 segments via `feature.tunnelIpv4Subnet` and `feature. tunnelIpv6Subnet`, it is worth noting that when configuring IPv4 or IPv6 segments via `feature.tunnelIpv4Subnet` and `feature.tunnelIpv6Subnet`, you need to make sure that the segments don't conflict with any other addresses in the cluster.
+> `feature.enableGatewayReplyRoute` is true to enable return routing rules on gateway nodes, which must be enabled when pairing with Spiderpool to support underlay CNI.
+>
+> If you are mainland user who is not available to access ghcr.io，You can specify the parameter `-set global.imageRegistryOverride=ghcr.m.daocloud.io` to avoid image pulling failures for EgressGateway.
+
+Verifying EgressGateway Installation
+
+```bash
+~# kubectl get pod -n kube-system | grep egressgateway
+egressgateway-agent-4s8lt                   1/1     Running     0     29m
+egressgateway-agent-thzth                   1/1     Running     0     29m
+egressgateway-controller-77698899df-tln7j   1/1     Running     0     29m
+```
+
+For more installation details, refer to [EgressGateway Installation](https://github.com/spidernet-io/egressgateway/blob/main/docs/usage/Install.en.md)
+
+### Creating an instance of EgressGateway
+
+An EgressGateway defines a set of nodes that act as an egress gateway for the cluster, through which egress traffic within the cluster will be forwarded out of the cluster. Therefore, an EgressGateway instance needs to be pre-defined. The following example Yaml creates an EgressGateway instance.
+
+* `spec.ippools.ipv4`: defines a set of egress egress IP addresses, which need to be adjusted according to the actual situation of the specific environment. The CIDR of `spec.ippools.ipv4` should be the same as the subnet of the egress NIC on the gateway node (usually the NIC of the default route), or else the egress access may not work.
+
+* `spec.nodeSelector`: the node affinity method provided by EgressGateway, when `selector.matchLabels` matches with a node, the node will be used as the egress gateway for the cluster, when `selector.matchLabels` does not match with a node, the When `selector.matchLabels` does not match with a node, the EgressGateway skips that node and it will not be used as an egress gateway for the cluster, which supports selecting multiple nodes for high availability.
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressGateway
+metadata:
+  name: default
+spec:
+  ippools:
+    ipv4:
+    - "10.6.168.201-10.6.168.205"
+  nodeSelector:
+    selector:
+      matchLabels:
+        egressgateway: "true"
+EOF
+```
+
+Label the node with the Label specified in `nodeSelector.selector.matchLabels` above so that the node can be selected by the EgressGateway to act as an egress gateway.
+
+```bash
+~# kubectl get node
+NAME                STATUS   ROLES           AGE     VERSION
+controller-node-1   Ready    control-plane   5d17h   v1.26.7
+worker-node-1       Ready    <none>          5d17h   v1.26.7
+
+~# kubectl label node worker-node-1 egressgateway="true"
+```
+
+When creation is complete, check the EgressGateway status.
+
+* `spec.ippools.ipv4DefaultEIP` represents the default VIP of the EgressGateway for the group, which is an IP address that will be randomly selected from `spec.ippools.ipv4`, and its function is: when creating an EgressPolicy object for an application, if no VIP address is specified, the is used if no VIP address is specified when creating an EgressPolicy object for the application.
+
+* `status.nodeList` represents the status of the nodes identified as matching the `spec.nodeSelector` and the corresponding EgressTunnel object for that node.
+
+```shell
+~# kubectl get EgressGateway default -o yaml
+...
+spec:
+  ippools:
+    ipv4DefaultEIP: 10.6.168.201
+...
+status:
+  nodeList:
+  - name: worker-node-1
+    status: Ready
+```
+
+### Create Applications and Egress Policies
+
+Create an application that will be used to test Pod access for external cluster purposes and label it to be associated with the EgressPolicy, as shown in the following example Yaml.
+
+* `v1.multus-cni.io/default-network`: used to specify the subnet used by the application, the Multus CR corresponding to this value needs to be created in advance by referring to the [installation](./readme.md) document to create it in advance.
+
+* `ipam.spidernet.io/ippool`: Specify which SpiderIPPool resources are used by the Pod, the corresponding SpiderIPPool CR should be created in advance by referring to the [Installation](./readme.md).
+
+```bash
+cat <<EOF | kubectl create -f -
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: test-app
+  name: test-app
+  namespace: default
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: test-app
+  template:
+    metadata:
+      labels:
+        app: test-app
+      annotations:
+        ipam.spidernet.io/ippool: |-
+            {      
+              "ipv4": ["v4-pool"],
+            }
+        v1.multus-cni.io/default-network: kube-system/macvlan-conf
+    spec:
+      containers:
+      - image: nginx
+        imagePullPolicy: IfNotPresent
+        name: test-app
+        ports:
+        - name: http
+          containerPort: 80
+          protocol: TCP
+EOF
+```
+
+The EgressPolicy instance is used to define which Pods' egress traffic is to be forwarded through the EgressGateway node, as well as other configuration details. The following is an example of creating an EgressPolicy CR object for an application.
+
+* `spec.egressGatewayName` is used to specify which set of EgressGateways to use.
+
+* `spec.appliedTo.podSelector` is used to specify on which Pods within the cluster this policy takes effect.
+
+* `namespace` is used to specify the tenant where the EgressPolicy object resides. Because EgressPolicy is tenant-level, it must be created under the same namespace as the associated application, so that when the matching Pod accesses any address outside the cluster, it can be forwarded by the EgressGateway Node.
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressPolicy
+metadata:
+ name: test
+ namespace: default
+spec:
+ egressGatewayName: default
+ appliedTo:
+  podSelector:
+   matchLabels:
+    app: "test-app"
+EOF
+```
+
+When creation is complete, check the status of the EgressPolicy.
+
+* `status.eip` shows the egress IP address used by the group when applying out of the cluster.
+
+* `status.node` shows which EgressGateway node is responsible for forwarding traffic out of the EgressPolicy.
+
+```bash
+~# kubectl get EgressPolicy -A
+NAMESPACE   NAME   GATEWAY   IPV4           IPV6   EGRESSNODE
+default     test   default   10.6.168.201          worker-node-1
+ 
+~# kubectl get EgressPolicy test -o yaml
+apiVersion: egressgateway.spidernet.io/v1beta1
+kind: EgressPolicy
+metadata:
+  name: test
+  namespace: default
+spec:
+  appliedTo:
+    podSelector:
+      matchLabels:
+        app: test-app
+  egressIP:
+    allocatorPolicy: default
+    useNodeIP: false
+status:
+  eip:
+    ipv4: 10.6.168.201
+  node: worker-node-1
+```
+
+After creating the EgressPolicy object, an EgressEndpointSlices object containing a collection of IP addresses of all the applications will be generated according to the application selected by the EgressPolicy, so that you can check whether the IP addresses in the EgressEndpointSlices object are normal or not when the application cannot be accessed by export.
+
+```bash
+~# kubectl get egressendpointslices -A
+NAMESPACE   NAME         AGE
+default     test-4vbqf   41s
+
+~# kubectl get egressendpointslices test-kvlp6 -o yaml
+apiVersion: egressgateway.spidernet.io/v1beta1
+endpoints:
+- ipv4:
+  - 10.6.168.208
+  node: worker-node-1
+  ns: default
+  pod: test-app-f44846544-8dnzp
+kind: EgressEndpointSlice
+metadata:
+  name: test-4vbqf
+  namespace: default
+```
+
+### Test Results
+
+Deploy the application `nettools` outside the cluster to emulate a service outside the cluster, and `nettools` will return the source IP address of the requester in the http reply.
+
+```bash
+~# docker run -d --net=host ghcr.io/spidernet-io/egressgateway-nettools:latest /usr/bin/nettools-server -protocol web -webPort 8080
+```
+
+Verifying the effect of egress traffic in a test app within the cluster: test-app, we can see that the source IP returned by `nettools` complies with `EgressPolicy.status.eip` when accessing an external service in the Pod corresponding to this app.
+
+```bash
+~# kubectl get pod -owide
+NAME                       READY   STATUS    RESTARTS      AGE     IP              NODE                NOMINATED NODE   READINESS GATES
+test-app-f44846544-8dnzp   1/1     Running   0             4m27s   10.6.168.208    worker-node-1       <none>           <none>
+
+~# kubectl exec -it test-app-f44846544-8dnzp bash
+
+~# curl 10.6.168.63:8080 # IP address of the node outside the cluster + webPort
+Remote IP: 10.6.168.201
+```
diff --git a/docs/usage/install/underlay/get-started-macvlan.md b/docs/usage/install/underlay/get-started-macvlan.md
index 3a92f53e15..ace1384f5e 100644
--- a/docs/usage/install/underlay/get-started-macvlan.md
+++ b/docs/usage/install/underlay/get-started-macvlan.md
@@ -25,7 +25,7 @@ Spiderpool provides a solution for assigning static IP addresses in underlay net
     helm repo update spiderpool
     helm install spiderpool spiderpool/spiderpool --namespace kube-system --set multus.multusCNI.defaultCniCRName="macvlan-conf" 
     ```
-    
+
     > If Macvlan is not installed in your cluster, you can specify the Helm parameter `--set plugins.installCNI=true` to install Macvlan in your cluster.
     >
     > If you are mainland user who is not available to access ghcr.io，You can specify the parameter `-set global.imageRegistryOverride=ghcr.m.daocloud.io` to avoid image pulling failures for Spiderpool.
diff --git a/docs/usage/spider-ippool-zh_CN.md b/docs/usage/spider-ippool-zh_CN.md
index 87adc87fdb..06887f00ef 100644
--- a/docs/usage/spider-ippool-zh_CN.md
+++ b/docs/usage/spider-ippool-zh_CN.md
@@ -53,7 +53,6 @@ spec:
           command: ["/bin/sh", "-c", "trap : TERM INT; sleep infinity & wait"]
 ```
 
-
 ### 指定 IPPool 为应用分配 IP 地址
 
 > 对于以下指定使用 SpiderIPPool 规则的优先级，请参考 [IP 候选池规则](./../concepts/ipam-zh_CN.md#获取候选池)