forked from pivotal-cf/docs-pks
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathnsxt-deploy.html.md.erb
490 lines (331 loc) · 26.4 KB
/
nsxt-deploy.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
---
title: Deploying NSX-T for PKS
owner: PKS-NSXT
---
<strong><%= modified_date %></strong>
To deploy NSX-T for PKS, complete the following set of procedures, in the order presented.
Before you begin this procedure, ensure that you have successfully completed all preceding steps for installing PKS on vSphere with NSX-T, including:
<ul>
<li>
<a href="./vsphere-nsxt-requirements.html">vSphere with NSX-T Version Requirements</a>
</li>
<li>
<a href="./vsphere-nsxt-rpd-mpd.html">Hardware Requirements for PKS on vSphere with NSX-T</a>
</li>
<li>
<a href="./nsxt-topologies.html">NSX-T Deployment Topologies for PKS</a>
</li>
<li>
<a href="./nsxt-prepare-env.html">Preparing to Deploy PKS with NSX-T on vSphere</a>
</li>
</ul>
##<a id='deploy-nsx-manager'></a>Step 1: Deploy NSX Manager
The NSX Manager is provided as an OVA file named **NSX Unified Appliance** that you import into your vSphere environment and configure.
Complete either of the following procedures to deploy the NSX Manager appliance:
* [Deploy NSX Manager using the vSphere client](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-FA0ABBBD-34D8-4DA9-882D-085E7E0D269E.html)
* [Deploy NSX Manager using the ovftool CLI](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-5843917A-A1D1-4D19-B9B0-0FD4C900C31C.html)
To verify deployment of the NSX Manager:
1. Power on the NSX Manager VM.
1. Ping the NSX Manager VM. Get the IP address for the NSX Manager from the **Summary** tab in vCenter. Verify that you can ping the host. For example, run `ping 10.196.188.21`.
1. SSH to the VM. Use the IP address for the NSX Manager to remotely connect using SSH. From Unix hosts use the command `ssh admin@IP_ADDRESS_OF_NSX_MANAGER`. For example, run `ssh [email protected]`. On Windows use Putty and provide the IP address. Enter the CLI user name and password that you defined during OVA import.
1. Review NSX CLI usage. Once you are logged into the NSX Manager VM, enter `?` to view the command usage and options for the NSX CLI.
1. Connect to the NSX Manager web interface using a supported browser at the URL `https://IP_ADDRESS_OF_NSX_MANAGER`. For example, `https://10.16.176.10`.
##<a id='deploy-nsx-controllers'></a>Step 2: Deploy NSX Controllers
The NSX Controller provides communications for NSX-T components.
You must deploy at least one NSX Controller for PKS. Three NSX Controllers are recommended.
Complete either of the following procedures to deploy an NSX Controller:
* [Deploy NSX Controllers using the vSphere client](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-24428FD4-EC8F-4063-9CF9-D8136740963A.html)
* [Deploy NSX Controllers using the ovftool CLI](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-B7C068CD-F5C4-4C18-B2E3-539FCA5FACAA.html)
To verify deployment of the NSX Controller:
1. Power on the NSX Controller VM.
1. Ping the NSX Controller VM. Get the IP address for the NSX Controller from the **Summary** tab in vCenter. Make sure you use a routable IP. If necessary click **View all X IP addresses** to reveal the proper IP address. Verify that you can ping the Controller host. For example, run `ping 10.196.188.22`.
1. SSH to the VM. Use the IP address for the NSX Controller to remotely connect using SSH. From Unix hosts use the command `ssh admin@IP_ADDRESS_OF_NSX_CONTROLLER`. For example, run `ssh [email protected]`. On Windows use Putty and provide the IP address. Enter the CLI admin user name and password that you defined during installation.
1. Review NSX CLI usage. After you are logged into the NSX Controller VM, enter `?` to view the command usage and options for the NSX CLI.
<p class="note"><strong>Note</strong>: Repeat the deployment and verification procedure for each NSX Controller you intend to use for PKS.</p>
##<a id='create-nsx-cluster'></a>Step 3: Create NSX Clusters (Management and Control)
In this section you create NSX Clusters for the PKS Management Plane and Control Plane.
1. Complete this procedure to create the NSX Management Cluster: [Join NSX Controllers with the NSX Manager](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-05434745-7D74-4DA8-A68E-9FE17093DA7B.html#GUID-05434745-7D74-4DA8-A68E-9FE17093DA7B).
1. Complete this procedure to create the NSX Control Cluster: [Initialize Control Cluster](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-273F6344-7212-4105-9FBA-A872CD75803F.html).
1. If you are deploying more than one NSX Controller, complete this procedure: [Join Additional NSX Controllers with the Cluster Master](
https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-8A8394EB-9D4A-4F13-AE91-8CFDD10334D4.html).
To verify the creation of NSX Clusters:
1. Verify that the NSX Controller is `Connected` to the NSX Manager:
NSX-CONTROLLER-1> get managers
1. Verify that the status of the Control Cluster is `active`:
NSX-CONTROLLER-1> get control-cluster status
1. Verify that the Management Cluster is `STABLE`:
NSX-MGR-1-1-0> get management-cluster status
1. Verify the configuration of the NSX Clusters.
* Connect to the NSX Manager web interface using a supported browser at the URL `https://IP_ADDRESS_OF_NSX_MANAGER`. For example, `https://10.16.176.10`.
* Log in using your admin credentials.
* Select **Dashboard > System > Overview**.
* Confirm that the status of the NSX Manager and each NSX Controller is green.
<img src="images/nsxt/nsx-manager/install-nsx-manager-18.png">
##<a id='deploy-nsx-edges'></a>Step 4: Deploy NSX Edge Nodes
Edge Nodes provide the bridge between the virtual network environment implemented using NSX-T and the physical network. Edge Nodes for PKS run load balancers for PKS API traffic, Kubenetes pod LB services, and pod ingress controllers.
PKS supports active/standby Edge Node failover and requires at least two Edge Nodes. In addition, PKS requires the Edge Node Large VM (8 vCPU, 16 GB of RAM, and 120 GB of storage) or the bare metal Edge Node. See [Edge Node Requirements](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-11417AA2-5EBC-49C7-8A86-EB94604261A6.html) for details.
Complete either of the following procedures to deploy an NSX Edge Node:
- [Edge Node Installation using vSphere Client](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-AECC66D0-C968-4EF2-9CAD-7772B0245BF6.html)
- [Edge Node Installation using ofvtool CLI](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-BF5EC490-CB5D-4FA9-AEC8-BB1A85E55B01.html)
When deploying the Edge Node, be sure to connect the vNICs of the NSX Edge VMs to an appropriate PortGroup for your environment:
* **Network 0**: For management purposes. Connect the first Edge interface to your environment's PortGroup/VLAN where your Edge Management IP can route and communicate with the NSX Manager.
* **Network 1**: For TEP (Tunnel End Point). Connect the second Edge interface to your environment's PortGroup/VLAN where your GENEVE VTEPs can route and communicate with each other. Your **VTEP CIDR** should be routable to this PortGroup.
* **Network 2**: For uplink connectivity to external physical router. Connect the third Edge interface to your environment's PortGroup/VLAN where your T0 uplink interface is located.
* **Network 3**: Unused (select any port group)
For example:
<img src="images/nsxt/nsx-edge/install-nsx-edge-08.png">
To verify Edge Node deployment:
1. Power on the Edge Node VM.
1. Ping the Edge VM. Get the IP address for the NSX Manager from the **Summary** tab in vCenter. Verify that you can ping the host by running `ping IP_ADDRESS_OF_NSX_EDGE_NODE`. For example, run `ping 10.196.188.21`.
1. SSH to the Edge VM. Use the IP address for the NSX Manager to remotely connect using SSH. From Unix hosts use the command `ssh admin@IP_ADDRESS_OF_NSX_EDGE_NODE`. For example, run `ssh [email protected]`. On Windows use Putty and provide the IP address. Enter the CLI admin user name and password that you defined in the **Customize template > Application** section.
1. Review NSX CLI usage. After you are logged into the NSX Manager VM, enter `?` to view the command usage and options for the NSX CLI.
<p class="note"><strong>Note</strong>: Repeat the deployment and verification process for each NSX Edge Node you intend to use for PKS.</p>
##<a id='register-nsx-edges'></a>Step 5: Register NSX Edge Nodes with NSX Manager
To register an Edge Node with NSX Manager, complete this procedure: [Join NSX Edge with the Management Plane](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-11BB4CF9-BC1D-4A76-A32A-AD4C98CBF25B.html).
To verify Edge Node registration with NSX Manager:
1. SSH to the Edge Node and run the following command. Verify that the Status is `Connected`:
nsx-edge-1> get managers
1. In the NSX Manager Web UI, go to **Fabric > Nodes > Edges**. You should see each registered Edge Node.
<img src="images/nsxt/nsx-edge/register-nsx-edge-01.png">
<p class="note"><strong>Note</strong>: Repeat this procedure for each NSX Edge Node you are deploying for PKS.</p>
##<a id='enable-repo'></a>Step 6: Enable Repository Service on NSX Manager
To enable VIB installation from the NSX Manager repository, the repository service needs to be be enabled in NSX Manager.
1. SSH into NSX Manager by using the command `ssh admin@IP_ADDRESS_OF_NSX_MANAGER` (Unix) or Putty (Windows).
1. Run the following command:
nsx-manager> set service install-upgrade enable
##<a id='create-tep'></a>Step 7: Create TEP IP Pool
To create the TEP IP Pool, complete this procedure: [Create an IP Pool for Tunnel Endpoint IP Addresses](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-E7F7322D-D09B-481A-BD56-F1270D7C9692.html).
When creating the TEP IP Pool, refer to the following example:
<img src="images/nsxt/nsx-misc/tep-01.png">
To verify TEP IP Pool configuration:
1. In NSX Manager, select **Inventory > Groups > IP Pools**.
1. Verify that the TEP IP Pool you created is present.
<img src="images/nsxt/nsx-misc/tep-02.png">
##<a id='create-tz-overlay'></a>Step 8: Create Overlay Transport Zone
Create an Overlay Transport Zone (`TZ-Overlay`) for PKS control plane services and Kubernetes clusters associated with associated with VDS `hostswitch1`.
To create TZ-Overlay, complete this procedure: [Create Transport Zones](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-F739DC79-4358-49F4-9C58-812475F33A66.html).
When creating the TZ-Overlay for PKS, refer to the following example:
<img src="images/nsxt/nsx-misc/tz-overlay-01.png">
To verify TZ-Overlay creation:
1. In NSX Manager select **Fabric > Transport Zones**.
1. Verify that you see the TZ-Overlay transport zone you created:
<img src="images/nsxt/nsx-misc/tz-overlay-02.png">
##<a id='create-tz-vlan'></a>Step 9: Create VLAN Transport Zone
Create the VLAN Transport Zone (`TZ-VLAN`) for NSX Edge Node uplinks (ingress/egress) for PKS Kubernetes clusters associated with VDS `hostswitch2`.
To create TZ-VLAN, complete this procedure: [Create Transport Zones](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-F739DC79-4358-49F4-9C58-812475F33A66.html).
When creating the TZ-VLAN for PKS, refer to the following example:
<img src="images/nsxt/nsx-misc/tz-vlan-01.png">
To verify TZ-VLAN creation:
1. In NSX Manager select **Fabric > Transport Zones**.
1. Verify that you see the TZ-VLAN transport zone:
<img src="images/nsxt/nsx-misc/tz-vlan-02.png">
##<a id='create-uplink'></a>Step 10: Create Uplink Profile for Edge Nodes
To create an Uplink Profile, complete this procedure: [Create an Uplink Profile](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.install.doc/GUID-50FDFDFB-F660-4269-9503-39AE2BBA95B4.html).
When creating the Uplink Profile for PKS, refer to the following example:
<img src="images/nsxt/nsx-misc/edge-uplink-01.png">
To verify Uplink Profile creation:
1. In NSX Manager select **Fabric > Profiles > Uplink Profiles**.
1. Verify that you see the Edge Node uplink profile you created:
<img src="images/nsxt/nsx-misc/edge-uplink-02.png">
##<a id='create-edge-tns'></a>Step 11: Create Edge Transport Nodes
Create NSX Edge Transport Nodes which allow Edge Nodes to exchange virtual network traffic with other NSX nodes.
Be sure to add both the VLAN and OVERLAY NSX Transport Zones to the NSX Edge Transport Nodes and confirm NSX Controller and Manager connectivity. Use the MAC addresses of the Edge VM interfaces to deploy the virtual NSX Edges:
* Connect the OVERLAY N-VDS to the vNIC (`fp-eth#`) that matches the MAC address of the second NIC from your deployed Edge VM.
* Connect the VLAN N-VDS to the vNIC (`fp-eth#`) that matches the MAC address of the third NIC from your deployed Edge VM.
To create an Edge Transport Node for PKS:
1. Log in to NSX Manager (`https://IP_ADDRESS_OF_NSX_MANAGERs`).
1. Go to **Fabric > Nodes > Edges**.
1. Select an Edge Node.
1. Click Actions > **Configure as Transport Node**.
<img src="images/nsxt/nsx-misc/edge-transport-node-01.png">
1. In the General tab, enter a name and select both Transport Zones: TZ-Overlay (Overlay) and TZ-VLAN (VLAN).
<img src="images/nsxt/nsx-misc/edge-transport-node-02a.png">
1. Select the **Host Switches** tab.
1. Configure the first transport node switch. For example:
* **Edge Switch Name**: `hostswitch1`
* **Uplink Profile**: `edge-uplink-profile`
* **IP Assignment**: `Use IP Pool`
* **IP Pool**: `TEP-ESXi-POOL`
* **Virtual NICs**: `fp-eth0` (corresponds to Edge VM vnic1 (second vnic))
<img src="images/nsxt/nsx-misc/edge-transport-node-02.png">
1. Click **Add Host Switch**.
1. Configure the second transport node switch. For example:
* **Edge Switch Name**: `hostswitch2`
* **Uplink Profile**: `edge-uplink-profile`
* **Virtual NICs**: `fp-eth1` (corresponds to Edge VM vnic2 (third vnic))
<img src="images/nsxt/nsx-misc/edge-transport-node-03.png">
<p class="note"><strong>Note</strong>: Repeat this procedure for the second Edge Transport Node (Edge-TN2), as well as additional Edge Node pairs you deploy for PKS.</p>
To verify the creation of Edge Transport Nodes:
1. In NSX Manager, select **Fabric > Nodes > Edges**.
1. Verify that Controller Connectivity and Manager Connectivity are `UP` for both Edge Nodes.
<img src="images/nsxt/nsx-misc/edge-transport-node-08.png">
1. In NSX Manager, select **Fabric > Nodes > Transport Node**.
1. Verify that the configuration state is `Success`.
<img src="images/nsxt/nsx-misc/edge-transport-node-07.png">
1. SSH to each NSX Edge VM and verify that the Edge Transport Node is "connected" to the Controller.
nsx-edge-1> get controllers
##<a id='create-edge-cluster'></a>Step 12: Create Edge Cluster
Create an NSX Edge Cluster and add each Edge Transport Node to the Edge Cluster by completing this procedure: [Create an NSX Edge Cluster](https://docs.vmware.com/en/VMware-NSX-T/2.2/com.vmware.nsxt.install.doc/GUID-898099FC-4ED2-4553-809D-B81B494B67E7.html).
When creating the Edge Cluster for PKS, refer to the following example:
<img src="images/nsxt/nsx-misc/edge-cluster-01.png">
To verify Edge Cluster creation:
1. In NSX Manager, select **Fabric > Nodes > Edge Clusters**.
1. Verify that you see the new Edge Cluster.
<img src="images/nsxt/nsx-misc/edge-cluster-02.png">
1. Select **Edge Cluster > Related > Transport Nodes**.
1. Verify that all Edge Transport Nodes are members of the Edge Cluster.
<img src="images/nsxt/nsx-misc/edge-cluster-03.png">
1. SSH to NSX Edge Node 1 and run the following commands to verify proper connectivity.
nsx-edge-1> get vteps
nsx-edge-1> get host-switches
nsx-edge-1> get edge-cluster status
nsx-edge-1> get controller sessions
1. SSSH to NSX Edge Node 2 and repeat the above commands to verify proper connectivity.
1. Verify Edge-TN1 to Edge-TN2 connectivity (TEP to TEP).
nsx-edge-1> get logical-router
nsx-edge-1> vrf 0
nsx-edge-1(vrf)> ping IP-ADDRESS-EDGE-2
##<a id='create-t0-router'></a>Step 13: Create T0 Logical Router
Create a Tier-0 Logical Router for PKS. The [Tier-0 Logical Router](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.admin.doc/GUID-DAEF8457-8363-4F33-84DA-68AA36A2DE3C.html) is used to route data between the physical network and the NSX-T-defined virtual network.
To create a Tier-0 (T0) logical router:
1. Define a T0 logical switch with an ingress/egress uplink port. Attach the T0 LS to the VLAN Transport Zone.
1. Create a logical router port and assign to it a routable CIDR block, for example `10.172.1.0/28`, that your environment uses to route to all PKS assigned IP pools and IP blocks.
1. Connect the T0 router to the uplink VLAN logical switch.
1. Attach the T0 router to the Edge Cluster and set HA mode to **Active-Standby**. NAT rules are applied on the T0 by NCP. If the T0 router is not set in **Active-Standby** mode, the router does not support NAT rule configuration.
1. Lastly, configure T0 routing to the rest of your environment using the appropriate routing protocol for your environment or by using static routes.
### Create VLAN Logical Switch (LS)
1. In NSX Manager, go to **Switching > Switches**.
1. Click **Add** and create a VLAN logical switch (LS). For example:
<img src="images/nsxt/nsx-misc/t0-router-01.png">
1. Click **Save** and verify that you see the new LS:
<img src="images/nsxt/nsx-misc/t0-router-02.png">
### Create T0 Router Instance
1. In NSX Manager, go to **Routing > Routers**.
1. Click **Add** and select the **Tier-0 Router** option.
<img src="images/nsxt/nsx-misc/t0-router-03.png">
1. Create new T0 router as follows:
* **Name**: Enter a name for the T0 router, such as `T0-LR` or `t0-pks`, for example.
* **Edge Cluster**: Select the Edge Cluster, `edgecluster1` or `edge-cluster-pks`, for example.
* **High Availability Mode**: Select `Active-Standby` (required).
<img src="images/nsxt/nsx-misc/t0-router-04.png">
1. Click **Save** and verify you see the new T0 Router instance:
<img src="images/nsxt/nsx-misc/t0-router-05.png">
<p class="note"><strong>Note</strong>: Be sure to select Active/Standby. NAT rules are be applied on T0 by NCP. If not set Active-Standby, NCP will not be able to create NAT rules on the T0 Router.</p>
### Create T0 Router Port
1. In NSX Manager, go to **Routing > Routers**.
1. Select the T0 Router you just created.
1. Select **Configuration > Router Ports**.
1. Select the T0 Router and click **Add**.
<img src="images/nsxt/nsx-misc/t0-router-06.png">
1. Create new T0 router port. Attach the T0 router port to the uplink logical switch you created (`uplink-LS1`, for example). Assign an IP address and CIDR that your environment uses to route to all PKS assigned IP pools and IP blocks. For example:
* **Name**: `Uplink1`
* **Type**: Uplink
* **Transport Node**: `edge-TN1`
* **Logical Switch**: `uplink-LS1`
* **Logical Switch Port**: `uplink1-port`
* **IP Address/mask**: `10.40.206.24/25` (for example)
<img src="images/nsxt/nsx-misc/t0-router-07.png">
1. Click **Save** and verify that you see the new port interface:
<img src="images/nsxt/nsx-misc/t0-router-08.png">
### Define Default Static Route
Configure T0 routing to the rest of your environment using the appropriate routing protocol (if you are using no-NAT-mode), or using static routes (if you are using NAT-mode). The following example uses static routes for the T0 router. The CIDR used must route to the IP you just assigned to your T0 uplink interface.
1. Go to **Routing > Routers** and select the T0 Router.
1. Select **Routing > Static Routes** and click **Add**.
1. Create a new static route for the T0 router. For example:
* **Network**: `0.0.0.0/0`
* **Next Hop**: `10.40.206.125` (for example)
* **Admin Distance**: `1`
* **Logical Router Port**: `Uplink1`
<img src="images/nsxt/nsx-misc/t0-router-10.png">
1. Click **Save** and verify that see the newly created static route:
<img src="images/nsxt/nsx-misc/t0-router-11.png">
### Verify T0 Router Creation
The T0 router uplink IP should be reachable from the corporate network. From your local laptop or workstation, ping the uplink IP address. For example:
PING 10.40.206.24 (10.40.206.24): 56 data bytes
64 bytes from 10.40.206.24: icmp_seq=0 ttl=53 time=33.738 ms
64 bytes from 10.40.206.24: icmp_seq=1 ttl=53 time=36.965 ms
##<a id='create-edge-ha'></a> Step 14: Configure Edge Nodes for HA
Configure <a href="nsxt-prepare-env.html#nsx-edge-ha">high-availability (HA) for NSX Edge Nodes</a>. If the T0 Router is not correctly configured for HA, failover to the standby Edge Node will not occur.
Proper configuration requires two new uplinks on the T0 router: one attached to Edge TN1, and the other attached to Edge TN2. In addition, you need to create a VIP that is the IP address used for the T0 uplink defined when the T0 Router was created.
<img src="images/nsxt/nsx-misc/edge-node-ha-01.png">
### Create Uplink1 for Edge-TN1
On the T0 router, create the Uplink1 router port and attach it to Egde TN1. For example:
* **IP Address/Mask**: `10.40.206.10/25
* **URPF Mode**: None (optional)
* **Transport Node**: `edge-TN1`
* **Logical Switch**: `uplink-LS1`
<img src="images/nsxt/nsx-misc/edge-node-ha-02.png">
### Create Uplink2 for Edge-TN2
On the T0 router, create the Uplink2 router port and attach it to Egde TN2. For example:
* **IP Address/Mask**: `10.40.206.9/25`
* **URPF Mode**: None (optional)
* **Transport Node**: `edge-TN2`
* **Logical Switch**: `uplink-LS1`
<img src="images/nsxt/nsx-misc/edge-node-ha-03.png">
### Create HA VIP
Create an HA virtual IP (VIP) address. Once created the HA VIP becomes the official IP for the T0 router uplink. External router devices peering with the T0 router **must** use this IP address.
<p class="note"><strong>Note</strong>: The IP addresses for uplink-1, uplink-2 and HA VIP must belong to same subnet.</p>
1. On the T0 router, create the HA VIP. For example:
* **VIP Address**: `10.40.206.24/25`
* **Uplinks Ports**: `Uplink-1` and `Uplink-2`
<img src="images/nsxt/nsx-misc/edge-node-ha-05.png">
1. Verify creation of the HA VIP.
<img src="images/nsxt/nsx-misc/edge-node-ha-04.png">
### Create Static Route for HA
1. On the T0 router, create a static default route so that the next hop points to the HA VIP address. For example:
* **Network**: `0.0.0.0/0`
* **Next Hop**: `10.40.206.125`
* **Logical Router Port**: empty
<img src="images/nsxt/nsx-misc/edge-node-ha-05b.png">
1. Using vCenter, disconnect the last interface of each Edge Node VM (this interface can cause duplicate packets.)
<img src="images/nsxt/nsx-misc/edge-node-ha-06.png">
### Verify Edge Node HA
1. The T0 router should display both Edge TNs in active/standby pairing.
<img src="images/nsxt/nsx-misc/edge-node-ha-07.png">
1. Run the following commands to verify HA channels:
nsx-edge-n-1> get high-availability channels
nsx-edge-n-1> get high-availability channels stats
nsx-edge-n-1> get logical-router
nsx-edge-n-1> get logical-router ROUTER-UUID high-availability status
##<a id='prepare-esxi'></a> Step 15: Prepare ESXi Servers for the PKS Compute Cluster
For each ESXi host in the NSX-T Fabric to be used for PKS Compute purposes, create an associated transport node. For example, if you have three ESXi hosts in the NSX-T Fabric, create three nodes named `tnode-host-1`, `tnode-host-2`, and `tnode-host-3`. Add the Overlay Transport Zone to each ESXi Host Transport Node.
Prepare each ESXi server dedicated for the PKS Compute Cluster as a Transport Node. These instructions assume that for each participating ESXi host the ESXi hypervisor is installed and the `vmk0` is configured. In addition, each ESXi host must have at least one **free nic/vmnic** for use with NSX Host Transport Nodes that is not already in use by other vSwitches on the ESXi host. Make sure the `vmnic1` (second physical interface) of the ESXi host is not used. NSX will take ownership of it (opaque NSX vswitch will use it as uplink). For more information, see [Add a Hypervisor Host to the NSX-T Fabric](https://docs.vmware.com/en/VMware-NSX-T/2.2/com.vmware.nsxt.install.doc/GUID-8C0EEC08-3A63-4918-A5E2-7A94AD50B0E6.html) in the VMware NSX-T documentation.
### Add ESXi Host to NSX-T Fabric
Complete the following operation for each ESXi host to be used by the PKS Compute Cluster.
1. Go to **Fabric > Nodes > Hosts**.
1. Click **Add** and create a new host. For example:
* **IP Address**: 10.115.40.72
* **OS**: ESXi
* **Username**: root
* **Password**: PASSWORD
<img src="images/nsxt/nsx-misc/esxi-prep-01.png">
1. After clicking **Save**, click **Yes** if the following invalid thumbprint message appears.
<img src="images/nsxt/nsx-misc/esxi-prep-02.png">
1. NSX installs VIBs on the ESXi host. In a few moments, you should see the new defined host. Deployment status should show `NSX Installed` and Manager Connectivity should show `Up`.
<img src="images/nsxt/nsx-misc/esxi-prep-03.png">
### Create Transport Node
1. In NSX Manager, go to **Fabric > Nodes > Transport Nodes**.
1. Click **Add** and create a new Transport Node. For example:
* **Name**: ESXi-COMP-1-TN
* **Node**: ESXi-COMP-1
* **TZ**: TZ-Overlay
<img src="images/nsxt/nsx-misc/esxi-prep-04.png">
1. Select the **Host Switches** tab.
1. Configure a Host Switch. For example:
* **Host Switch Name**: `hostswitch1`
* **Uplink Profile**: `nsx-default-uplink-hostswitch-profile`
* **IP Assignment**: `Use IP Pool`
* **IP POOL**: `TEP-ESXi-POOL`
* **Physical NICs**: `vmnic1`
<img src="images/nsxt/nsx-misc/esxi-prep-05.png">
### Verify ESXi Host Preparation for PKS Compute Cluster
1. Verify that you see the ESXi Compute Transport Node:
<img src="images/nsxt/nsx-misc/esxi-prep-06.png">
1. Verify the status is `Up`.
<img src="images/nsxt/nsx-misc/esxi-prep-07.png">
<p class="note"><strong>Note</strong>: If you are using NSX-T 2.3, the status should be up. If you are using NSX-T 2.2, the status may incorrectly show as down (because the Tunnel Status is Down.) Either way, verify TEP communications as described in the next step.</p>
1. Make sure the NSX TEP vmk is created on ESXi host and TEP to TEP communication (with Edge TN for instance) works.
[root@ESXi-1:~] esxcfg-vmknic -l
[root@ESXi-1:~] vmkping ++netstack=vxlan <IP of the vmk10 interface> -d -s 1500
##<a id='next'></a> Next Step
After you complete this procedure, follow the instructions in <a href="./nsxt-prepare-mgmt-plane.html">Creating the PKS Management Plane</a>.