forked from cloudfoundry/docs-running-cf
-
Notifications
You must be signed in to change notification settings - Fork 0
/
uaa-performance.html.md.erb
358 lines (309 loc) · 18.3 KB
/
uaa-performance.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
---
title: UAA Performance
owner: UAA
---
<!---
fancyBox is licensed under the GPLv3 license for all open source applications.
Do not use this code in commercial documentation
--->
<% if vars.platform_code == 'CF' %>
<script src="//code.jquery.com/jquery-3.2.1.min.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/fancybox/3.2.5/jquery.fancybox.min.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/fancybox/3.2.5/jquery.fancybox.min.js"></script>
This topic provides an overview of the configuration and results of performance tests conducted on the UAA API. It does not discuss log outputs or metrics.
For information about monitoring and diagnosing UAA performance over time, see <a href="../uaa/uaa-metrics.html">UAA Performance Metrics</a>.
The perfomance test results illustrate the baseline performance of a single instance and expected performance when horizontally scaling the number of UAA virtual machine (VM) instances.
The graphs in this topic are presented as time series and describe the effect of increasing concurrency on throughput and latency. Operators can use this information to determine the number of instances required to achieve a target throughput under reasonable latency.
##<a id='benchmark-setup'></a> Benchmarking
UAA testing for this topic was performed in the context of a Cloud Foundry environment instead of in full isolation tests.
The benchmarking test setup places the database, client, router, and UAA servers within the same network to minimize external network factors on performance. The test setup relies on Gorouter to load balance requests across multiple UAA instances.
The [Apache JMeter](http://jmeter.apache.org/) client increases step concurrency to reach the target currency for a given test run. The testing tools are packaged as a BOSH release and deployed to a dedicated VM. The performance tests exercise the following flows:
* **Authorization code** /oauth/authorize and /oauth/login flow
* **Client credentials** /oauth/token flow
* **Implicit** /oauth/authorize flow
* **Password** /oauth/token flow
Each request generated by JMeter is:
- Sent to the router HTTPS protocol
- Includes UAA URL as Host Header (used by Gorouter to decide the host)
- Includes client credentials as a URL parameter
- For authorization code and implicit grant types, an initial login is performed and JSESSIONID is passed for subsequent authorize and token requests
- For password grant type, user credentials are included as URL parameters
When JMeter receives a response from UAA, it:
- Validates response code to ensure request is successful.
JMeter stores each request entry to in an intermediate JTL(CSV) format file and this is used as an input to generate graphs.
We simulate increasing load by ramping up the number of concurrent threads in increments of 20 threads until we reach the equivalent of 200 threads per VM. This allows us to observe the behavior of UAA as the load increases to give us a better understanding of how UAA behavior changes as load increases.
##<a id='tools'></a> Tools
[JMeter test plan](https://github.com/cf-identity/jmeter): A collection of JMeter scripts. UAA testing for this topic uses these scripts with JMeter to load test, measure, and analyze performance. JMeter supports variable parameterization, response assertions and validation, and graph generation.
### Benchmarking Setup ###
- VM Instance Properties
- GCP Cloud Config Name: n1-standard-2
- GCP Machine Type: n1-standard-2
- CPU Information: 2 x Intel(R) Xeon(R) CPU @ 2.60GHz
- RAM Information: 7479 MB
- UAA Max Heap Size: 768 MB
- UAA Version: `v53.1` branch of UAA
- Database Instance Properties
- GCP Cloud Config Name: n1-standard-2
- CPU: 2 x Intel(R) Xeon(R) CPU @ 2.60GHz
- RAM Information: 7479 MB
- Allocated Storage: 10Gb of Persistent Disk
- Engine: MySQL
- Network Setup:
- MySQL deployed with CF deployment so the connection from UAA to database within the same network
- GCP does not document any network bandwidth on any vm types
- Gorouter is scaled to 8 GB memory to prevent router bottlenecks
- Data Configuration: Test dataset includes the following:
- 1 client
- 1 User
- 41 Groups
##<a id='results'></a> Performance Results
###<a id='client-credentials'></a> Client Credentials Grant Type
**Endpoint**: /oauth/token?grant_type=client_credentials
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Throughput</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-1.png"><img src="images/client-creds-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-throughput-1.png"><img src="images/client-creds-throughput-1.png" alt="Throughput Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-2.png"><img src="images/client-creds-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-throughput-2.png"><img src="images/client-creds-throughput-2.png" alt="Throughput Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-4.png"><img src="images/client-creds-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-throughput-4.png"><img src="images/client-creds-throughput-4.png" alt="Throughput Level 4"></a></td>
</tr>
</table>
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Latency</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-1.png"><img src="images/client-creds-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-latency-1.png"><img src="images/client-creds-latency-1.png" alt="Latency Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-2.png"><img src="images/client-creds-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-latency-2.png"><img src="images/client-creds-latency-2.png" alt="Latency Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-4.png"><img src="images/client-creds-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-latency-4.png"><img src="images/client-creds-latency-4.png" alt="Latency Level 4"></a></td>
</tr>
</table>
Server Performance metric like CPU utilization, heap memory usage, and time
spent in garbage collection. These metrics are collected by an agent colocated
with the UAA VM. Graphs are plotted as time series.
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>CPU</strong></th>
<th><strong>Heap Memory</strong></th>
<th><strong>Garbage Collection</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-1.png"><img src="images/client-creds-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-cpu-1.png"><img src="images/client-creds-cpu-1.png" alt="CPU Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-heap-1.png"><img src="images/client-creds-heap-1.png" alt="Heap Memory Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-garbage-1.png"><img src="images/client-creds-garbage-1.png" alt="Garbage Collection Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-2.png"><img src="images/client-creds-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-cpu-2.png"><img src="images/client-creds-cpu-2.png" alt="CPU Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-heap-2.png"><img src="images/client-creds-heap-2.png" alt="Heap Memory Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-garbage-2.png"><img src="images/client-creds-garbage-2.png" alt="Garbage Collection Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/client-creds-threads-4.png"><img src="images/client-creds-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-cpu-4.png"><img src="images/client-creds-cpu-4.png" alt="CPU Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-heap-4.png"><img src="images/client-creds-heap-4.png" alt="Heap Memory Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/client-creds-garbage-4.png"><img src="images/client-creds-garbage-4.png" alt="Garbage Collection Level 4"></a></td>
</tr>
</table>
###<a id='client-credentials'></a> Password Grant Type
**Endpoint**: /oauth/token?grant_type=client_credentials
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Throughput</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-1.png"><img src="images/password-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-throughput-1.png"><img src="images/password-grant-throughput-1.png" alt="Throughput Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-2.png"><img src="images/password-grant-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-throughput-2.png"><img src="images/password-grant-throughput-2.png" alt="Throughput Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-4.png"><img src="images/password-grant-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-throughput-4.png"><img src="images/password-grant-throughput-4.png" alt="Throughput Level 4"></a></td>
</tr>
</table>
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Latency</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-1.png"><img src="images/password-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-latency-1.png"><img src="images/password-grant-latency-1.png" alt="Latency Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-2.png"><img src="images/password-grant-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-latency-2.png"><img src="images/password-grant-latency-2.png" alt="Latency Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-4.png"><img src="images/password-grant-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-latency-4.png"><img src="images/password-grant-latency-4.png" alt="Latency Level 4"></a></td>
</tr>
</table>
Server Performance metric like CPU utilization, heap memory usage, and time
spent in garbage collection. These metrics are collected by an agent colocated
with the UAA VM. Graphs are plotted as time series.
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>CPU</strong></th>
<th><strong>Heap Memory</strong></th>
<th><strong>Garbage Collection</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-1.png"><img src="images/password-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-cpu-1.png"><img src="images/password-grant-cpu-1.png" alt="CPU Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-heap-1.png"><img src="images/password-grant-heap-1.png" alt="Heap Memory Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-garbage-1.png"><img src="images/password-grant-garbage-1.png" alt="Garbage Collection Level 1"></a></td>
</tr>
<tr>
<td>2</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-2.png"><img src="images/password-grant-threads-2.png" alt="Threads Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-cpu-2.png"><img src="images/password-grant-cpu-2.png" alt="CPU Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-heap-2.png"><img src="images/password-grant-heap-2.png" alt="Heap Memory Level 2"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-garbage-2.png"><img src="images/password-grant-garbage-2.png" alt="Garbage Collection Level 2"></a></td>
</tr>
<tr>
<td>4</td>
<td><a data-fancybox="gallery" href="images/password-grant-threads-4.png"><img src="images/password-grant-threads-4.png" alt="Threads Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-cpu-4.png"><img src="images/password-grant-cpu-4.png" alt="CPU Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-heap-4.png"><img src="images/password-grant-heap-4.png" alt="Heap Memory Level 4"></a></td>
<td><a data-fancybox="gallery" href="images/password-grant-garbage-4.png"><img src="images/password-grant-garbage-4.png" alt="Garbage Collection Level 4"></a></td>
</tr>
</table>
###<a id='client-credentials'></a> Authorization Code Grant Type
**Endpoint**: /oauth/authorize?grant_type=authorization_code and /oauth/token?grant_type=authorization_code
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Throughput</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-threads-1.png"><img src="images/auth-code-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-throughput-1.png"><img src="images/auth-code-grant-throughput-1.png" alt="Throughput Level 1"></a></td>
</tr>
</table>
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Latency</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-threads-1.png"><img src="images/auth-code-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-latency-1.png"><img src="images/auth-code-grant-latency-1.png" alt="Latency Level 1"></a></td>
</tr>
</table>
Server Performance metric like CPU utilization, heap memory usage, and time
spent in garbage collection. These metrics are collected by an agent colocated
with the UAA VM. Graphs are plotted as time series.
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>CPU</strong></th>
<th><strong>Heap Memory</strong></th>
<th><strong>Garbage Collection</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-threads-1.png"><img src="images/auth-code-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-cpu-1.png"><img src="images/auth-code-grant-cpu-1.png" alt="CPU Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-heap-1.png"><img src="images/auth-code-grant-heap-1.png" alt="Heap Memory Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/auth-code-grant-garbage-1.png"><img src="images/auth-code-grant-garbage-1.png" alt="Garbage Collection Level 1"></a></td>
</tr>
</table>
###<a id='client-credentials'></a> Implicit Grant Type
**Endpoint**: /oauth/token?grant_type=client_credentials
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Throughput</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/implicit-grant-threads-1.png"><img src="images/implicit-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/implicit-grant-throughput-1.png"><img src="images/implicit-grant-throughput-1.png" alt="Throughput Level 1"></a></td>
</tr>
</table>
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>Latency</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/implicit-grant-threads-1.png"><img src="images/implicit-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/implicit-grant-latency-1.png"><img src="images/implicit-grant-latency-1.png" alt="Latency Level 1"></a></td>
</tr>
</table>
Server Performance metric like CPU utilization, heap memory usage, and time
spent in garbage collection. These metrics are collected by an agent colocated
with the UAA VM. Graphs are plotted as time series.
<table class="nice">
<tr>
<th><strong>Instances</strong></th>
<th><strong>Threads</strong></th>
<th><strong>CPU</strong></th>
<th><strong>Heap Memory</strong></th>
<th><strong>Garbage Collection</strong></th>
</tr>
<tr>
<td>1</td>
<td><a data-fancybox="gallery" href="images/implicit-grant-threads-1.png"><img src="images/implicit-grant-threads-1.png" alt="Threads Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/implicit-grant-cpu-1.png"><img src="images/implicit-grant-cpu-1.png" alt="CPU Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/implicit-grant-heap-1.png"><img src="images/implicit-grant-heap-1.png" alt="Heap Memory Level 1"></a></td>
<td><a data-fancybox="gallery" href="images/implicit-grant-garbage-1.png"><img src="images/implicit-grant-garbage-1.png" alt="Garbage Collection Level 1"></a></td>
</tr>
</table>
<br>
<% end %>