Skip to content

Commit 3ccc7a6

Browse files
carloryjdef
carlory
and
jdef
committed
update spec doc for groupcontroller service
Co-authored-by: jdef <2348332+jdef@users.noreply.github.com>
1 parent b01039c commit 3ccc7a6

File tree

1 file changed

+94
-65
lines changed

1 file changed

+94
-65
lines changed

‎spec.md

+94-65
Original file line numberDiff line numberDiff line change
@@ -80,85 +80,97 @@ A CO SHOULD be equipped to handle both centralized and headless plugins, as well
8080
Several of these possibilities are illustrated in the following figures.
8181

8282
```
83-
CO "Master" Host
84-
+-------------------------------------------+
85-
| |
86-
| +------------+ +------------+ |
87-
| | CO | gRPC | Controller | |
88-
| | +-----------> Plugin | |
89-
| +------------+ +------------+ |
90-
| |
91-
+-------------------------------------------+
92-
93-
CO "Node" Host(s)
94-
+-------------------------------------------+
95-
| |
96-
| +------------+ +------------+ |
97-
| | CO | gRPC | Node | |
98-
| | +-----------> Plugin | |
99-
| +------------+ +------------+ |
100-
| |
101-
+-------------------------------------------+
83+
CO "Master" Host
84+
+-------------------------------------------------+
85+
| |
86+
| +------------+ +------------------+ |
87+
| | CO | gRPC | Controller | |
88+
| | +--+--------> Plugin | |
89+
| +------------+ | +------------------+ |
90+
| | |
91+
| | +------------------+ |
92+
| | | Group Controller | |
93+
| +--------> Plugin | |
94+
| +------------------+ |
95+
| |
96+
+-------------------------------------------------+
97+
98+
CO "Node" Host(s)
99+
+-------------------------------------------------+
100+
| |
101+
| +------------+ +------------------+ |
102+
| | CO | gRPC | Node | |
103+
| | +-----------> Plugin | |
104+
| +------------+ +------------------+ |
105+
| |
106+
+-------------------------------------------------+
102107
103108
Figure 1: The Plugin runs on all nodes in the cluster: a centralized
104-
Controller Plugin is available on the CO master host and the Node
105-
Plugin is available on all of the CO Nodes.
109+
Controller Plugin (alpha) and Group Controller Plugin (alpha) are available
110+
on the CO master host and the Node Plugin is available on all of the
111+
CO Nodes.
106112
```
107113

108114
```
109-
CO "Node" Host(s)
110-
+-------------------------------------------+
111-
| |
112-
| +------------+ +------------+ |
113-
| | CO | gRPC | Controller | |
114-
| | +--+--------> Plugin | |
115-
| +------------+ | +------------+ |
116-
| | |
117-
| | |
118-
| | +------------+ |
119-
| | | Node | |
120-
| +--------> Plugin | |
121-
| +------------+ |
122-
| |
123-
+-------------------------------------------+
115+
CO "Node" Host(s)
116+
+-------------------------------------------------+
117+
| |
118+
| +------------------+ |
119+
| | Node | |
120+
| +--------> Plugin | |
121+
| | +------------------+ |
122+
| | |
123+
| +------------+ | +------------------+ |
124+
| | CO | |gRPC | Controller | |
125+
| | +-----------> Plugin | |
126+
| +------------+ | +------------------+ |
127+
| | |
128+
| | +------------------+ |
129+
| | | Group Controller | |
130+
| +--------> Plugin | |
131+
| +------------------+ |
132+
| |
133+
+-------------------------------------------------+
124134
125135
Figure 2: Headless Plugin deployment, only the CO Node hosts run
126-
Plugins. Separate, split-component Plugins supply the Controller
127-
Service and the Node Service respectively.
136+
Plugins. Separate, split-component Plugins supply the Node Service,
137+
the Controller Service (alpha) and the Group Controller Service (alpha)
138+
respectively.
128139
```
129140

130141
```
131-
CO "Node" Host(s)
132-
+-------------------------------------------+
133-
| |
134-
| +------------+ +------------+ |
135-
| | CO | gRPC | Controller | |
136-
| | +-----------> Node | |
137-
| +------------+ | Plugin | |
138-
| +------------+ |
139-
| |
140-
+-------------------------------------------+
142+
CO "Node" Host(s)
143+
+-------------------------------------------------+
144+
| |
145+
| +------------------+ |
146+
| +------------+ | Node | |
147+
| | CO | gRPC | Controller | |
148+
| | +-----------> Group Controller | |
149+
| +------------+ | Plugin | |
150+
| +------------------+ |
151+
| |
152+
+-------------------------------------------------+
141153
142154
Figure 3: Headless Plugin deployment, only the CO Node hosts run
143-
Plugins. A unified Plugin component supplies both the Controller
144-
Service and Node Service.
155+
Plugins. A unified Plugin component supplies both the Node Service,
156+
the Controller Service (alpha) and the Group Controller Service (alpha).
145157
```
146158

147159
```
148-
CO "Node" Host(s)
149-
+-------------------------------------------+
150-
| |
151-
| +------------+ +------------+ |
152-
| | CO | gRPC | Node | |
153-
| | +-----------> Plugin | |
154-
| +------------+ +------------+ |
155-
| |
156-
+-------------------------------------------+
160+
CO "Node" Host(s)
161+
+-------------------------------------------------+
162+
| |
163+
| +------------+ +------------------+ |
164+
| | CO | gRPC | Node | |
165+
| | +-----------> Plugin | |
166+
| +------------+ +------------------+ |
167+
| |
168+
+-------------------------------------------------+
157169
158170
Figure 4: Headless Plugin deployment, only the CO Node hosts run
159171
Plugins. A Node-only Plugin component supplies only the Node Service.
160-
Its GetPluginCapabilities RPC does not report the CONTROLLER_SERVICE
161-
capability.
172+
Its GetPluginCapabilities RPC does not report either the CONTROLLER_SERVICE
173+
capability or the GROUP_CONTROLLER_SERVICE capability.
162174
```
163175

164176
### Volume Lifecycle
@@ -264,11 +276,13 @@ This section describes the interface between COs and Plugins.
264276
### RPC Interface
265277

266278
A CO interacts with an Plugin through RPCs.
267-
Each SP MUST provide:
279+
Each SP MAY provide:
268280

269281
* **Node Plugin**: A gRPC endpoint serving CSI RPCs that MUST be run on the Node whereupon an SP-provisioned volume will be published.
270282
* **Controller Plugin**: A gRPC endpoint serving CSI RPCs that MAY be run anywhere.
271-
* In some circumstances a single gRPC endpoint MAY serve all CSI RPCs (see Figure 3 in [Architecture](#architecture)).
283+
* **Group Controller Plugin**: A gRPC endpoint serving CSI RPCs that MAY be run anywhere.
284+
285+
In some circumstances a single gRPC endpoint MAY serve all CSI RPCs (see Figure 3 in [Architecture](#architecture)).
272286

273287
```protobuf
274288
syntax = "proto3";
@@ -322,10 +336,11 @@ extend google.protobuf.ServiceOptions {
322336
}
323337
```
324338

325-
There are three sets of RPCs:
339+
There are four sets of RPCs:
326340

327-
* **Identity Service**: Both the Node Plugin and the Controller Plugin MUST implement this sets of RPCs.
341+
* **Identity Service**: Every Controller Plugin, Group Controller Plugin and Node Plugin MUST implement this sets of RPCs.
328342
* **Controller Service**: The Controller Plugin MUST implement this sets of RPCs.
343+
* **Group Controller Service**: The GroupController Plugin MUST implement this sets of RPCs.
329344
* **Node Service**: The Node Plugin MUST implement this sets of RPCs.
330345

331346
```protobuf
@@ -3066,6 +3081,20 @@ The CO MUST implement the specified error recovery behavior when it encounters t
30663081
| Snapshot list mismatch | 3 INVALID_ARGUMENT | Besides the general cases, this code SHOULD also be used to indicate when plugin supporting CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT detects a mismatch in the `snapshot_ids`. | If a mismatch is detected in the `snapshot_ids`, caller SHOULD use different `snapshot_ids`. |
30673082
| Volume group snapshot does not exist | 5 NOT_FOUND | Indicates that a volume group snapshot corresponding to the specified `group_snapshot_id` does not exist. | Caller MUST verify that the `group_snapshot_id` is correct and that the volume group snapshot is accessible and has not been deleted before retrying with exponential back off. |
30683083

3084+
#### RPC Interactions
3085+
3086+
##### `CreateVolumeGroupSnapshot`, `DeleteVolumeGroupSnapshot`, `GetVolumeGroupSnapshot`
3087+
3088+
The plugin-generated `group_snapshot_id` and `source_volume_ids` are a REQUIRED field for both the `DeleteVolumeGroupSnapshot` RPC and the `GetVolumeGroupSnapshot` PRC, as opposed to the CO-generated group snapshot `name` that is REQUIRED for the `CreateVolumeGroupSnapshot` RPC and the CO-generated volume `name` that is REQUIRED for the `CreateVolume` RPC.
3089+
3090+
A `CreateVolumeGroupSnapshot` operation SHOULD return with a `group_snapshot_id` when the group snapshot is cut successfully. If a `CreateVolumeGroupSnapshot` operation times out before the group snapshot is cut, leaving the CO without an ID with which to reference a group snapshot, and the CO also decides that it no longer needs/wants the group snapshot in question then the CO MAY choose one of the following paths:
3091+
3092+
1. Retry the `CreateVolumeGroupSnapshot` RPC to possibly obtain a group snapshot ID that may be used to execute a `DeleteVolumeGroupSnapshot` RPC; upon success execute `DeleteVolumeGroupSnapshot`. If the `CreateVolumeGroupSnapshot` RPC returns a server-side gRPC error, it means that SP do clean up and make sure no snapshots are leaked.
3093+
3094+
2. The CO takes no further action regarding the timed out RPC, a group snapshot is possibly leaked and the operator/user is expected to clean up. But this way isn't considered as a good practice.
3095+
3096+
For plugins that support snapshot post processing such as uploading, a `GetVolumeGroupSnapshot` operation SHALL return current information of the group snapshot with the given `group_snapshot_id`. When processing is complete, the `ready_to_use` parameter of the group snapshot from `GetVolumeGroupSnapshot` SHALL become `true`.
3097+
30693098
## Protocol
30703099

30713100
### Connectivity

0 commit comments

Comments
 (0)