Add fixes for release process #6704

andrewlock · 2025-02-24T13:29:23Z

Summary of changes

Increase HttpClient download timeout for Azure Devops release artifacts
Make sure destination temp directory has been created
Fix issue in GitLab release pipeline for Windows

Reason for change

Saw failures due to all of these issues in latest release.

Implementation details

Increase the timeout to download the release artifacts from "ridiculous" (100s) to "ludicrous" (6000s), in the hope that we don't hit it again. It's safe to just retry in the UI here anyway, so this will hopefully be sufficient.

Additionally, a recent additional check (the fleet installer) was failing because the target directory for the download hadn't been created, so fixed that.

Finally, the 3.11.0 release is broken because of a bug in the script, which only manifests when running for a tag

Test coverage

Ran the Build stage locally - they both failed previously and now both pass.

We can't easily test the gitlab change unfortunately

andrewlock · 2025-02-24T14:39:58Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (69ms)  : 65, 74
     .   : milestone, 69,
    master - mean (69ms)  : 66, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (997ms)  : 975, 1018
     .   : milestone, 997,
    master - mean (998ms)  : 977, 1018
     .   : milestone, 998,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (102ms)  : 100, 104
     .   : milestone, 102,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (671ms)  : 654, 688
     .   : milestone, 671,
    master - mean (673ms)  : 656, 690
     .   : milestone, 673,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (89ms)  : 87, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (628ms)  : 612, 644
     .   : milestone, 628,
    master - mean (634ms)  : 618, 651
     .   : milestone, 634,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (191ms)  : 186, 196
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (1,108ms)  : 1077, 1138
     .   : milestone, 1108,
    master - mean (1,110ms)  : 1078, 1142
     .   : milestone, 1110,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (271ms)  : 265, 276
     .   : milestone, 271,
    master - mean (274ms)  : 267, 282
     .   : milestone, 274,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (866ms)  : 825, 907
     .   : milestone, 866,
    master - mean (869ms)  : 829, 908
     .   : milestone, 869,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6704) - mean (263ms)  : 259, 268
     .   : milestone, 263,
    master - mean (262ms)  : 258, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (6704) - mean (846ms)  : 809, 882
     .   : milestone, 846,
    master - mean (848ms)  : 817, 880
     .   : milestone, 848,

andrewlock · 2025-02-24T15:06:55Z

Benchmarks Report for tracer 🐌

Benchmarks for #6704 compared to master:

1 benchmarks are slower, with geometric mean 1.130
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.19μs	46.7ns	353ns	0.0155	0.00387	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.4μs	56ns	317ns	0.0201	0.00503	0	5.81 KB
master	`StartStopWithChild`	net472	16.3μs	58.4ns	211ns	1.05	0.33	0.0966	6.21 KB
#6704	`StartStopWithChild`	net6.0	8.35μs	48ns	399ns	0.024	0.012	0	5.61 KB
#6704	`StartStopWithChild`	netcoreapp3.1	10.7μs	61.2ns	509ns	0.0204	0.0102	0	5.8 KB
#6704	`StartStopWithChild`	net472	16.1μs	57.4ns	222ns	1.05	0.307	0.102	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	488μs	459ns	1.78μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	647μs	611ns	2.37μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	859μs	807ns	3.12μs	0.422	3.3 KB
#6704	`WriteAndFlushEnrichedTraces`	net6.0	488μs	198ns	765ns	0	2.7 KB
#6704	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	666μs	303ns	1.13μs	0	2.7 KB
#6704	`WriteAndFlushEnrichedTraces`	net472	850μs	667ns	2.58μs	0.425	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	130μs	424ns	1.64μs	0.198	14.47 KB
master	`SendRequest`	netcoreapp3.1	143μs	485ns	1.88μs	0.208	17.27 KB
master	`SendRequest`	net472	0.000683ns	0.00027ns	0.00104ns	0	0 b
#6704	`SendRequest`	net6.0	130μs	472ns	1.77μs	0.132	14.47 KB
#6704	`SendRequest`	netcoreapp3.1	149μs	133ns	517ns	0.223	17.27 KB
#6704	`SendRequest`	net472	0.000196ns	0.000115ns	0.000429ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	582μs	2.8μs	10.9μs	0.553	0	0	41.61 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	699μs	3.98μs	32.3μs	0.338	0	0	41.72 KB
master	`WriteAndFlushEnrichedTraces`	net472	864μs	4.06μs	17.2μs	8.45	2.53	0.422	53.33 KB
#6704	`WriteAndFlushEnrichedTraces`	net6.0	558μs	2.85μs	12.7μs	0.568	0	0	41.59 KB
#6704	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	684μs	3.78μs	22.4μs	0.34	0	0	41.72 KB
#6704	`WriteAndFlushEnrichedTraces`	net472	850μs	3.1μs	11.6μs	8.08	2.55	0.425	53.32 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.27μs	1.45ns	5.61ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.76μs	1.51ns	5.86ns	0.0134	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.08μs	1.41ns	5.27ns	0.156	0.00105	987 B
#6704	`ExecuteNonQuery`	net6.0	1.27μs	1.32ns	5.11ns	0.0147	0	1.02 KB
#6704	`ExecuteNonQuery`	netcoreapp3.1	1.74μs	1.18ns	4.42ns	0.0132	0	1.02 KB
#6704	`ExecuteNonQuery`	net472	2.1μs	2.74ns	10.6ns	0.156	0.00105	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6704

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0	1.130	1,285.91	1,452.75

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.3μs	1.39ns	5.21ns	0.0137	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.53μs	1.66ns	6.19ns	0.0129	976 B
master	`CallElasticsearch`	net472	2.56μs	1.67ns	6.46ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.29μs	0.411ns	1.59ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.74μs	1.73ns	6.48ns	0.0138	1.02 KB
master	`CallElasticsearchAsync`	net472	2.53μs	1.64ns	6.36ns	0.167	1.05 KB
#6704	`CallElasticsearch`	net6.0	1.36μs	0.728ns	2.72ns	0.0135	976 B
#6704	`CallElasticsearch`	netcoreapp3.1	1.63μs	1.18ns	4.58ns	0.0131	976 B
#6704	`CallElasticsearch`	net472	2.55μs	1.51ns	5.86ns	0.157	995 B
#6704	`CallElasticsearchAsync`	net6.0	1.45μs	0.67ns	2.6ns	0.0131	952 B
#6704	`CallElasticsearchAsync`	netcoreapp3.1	1.75μs	1.16ns	4.33ns	0.014	1.02 KB
#6704	`CallElasticsearchAsync`	net472	2.59μs	0.6ns	2.25ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.43μs	0.743ns	2.88ns	0.0136	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.68μs	0.575ns	2.15ns	0.0123	952 B
master	`ExecuteAsync`	net472	1.92μs	0.512ns	1.98ns	0.145	915 B
#6704	`ExecuteAsync`	net6.0	1.31μs	0.297ns	1.07ns	0.0132	952 B
#6704	`ExecuteAsync`	netcoreapp3.1	1.71μs	1.55ns	6ns	0.0129	952 B
#6704	`ExecuteAsync`	net472	1.84μs	0.907ns	3.51ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.46μs	23.2ns	109ns	0.0334	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.38μs	1.54ns	5.57ns	0.0373	2.85 KB
master	`SendAsync`	net472	7.52μs	1.45ns	5.43ns	0.492	3.12 KB
#6704	`SendAsync`	net6.0	4.53μs	2.44ns	9.44ns	0.0317	2.31 KB
#6704	`SendAsync`	netcoreapp3.1	5.38μs	3.15ns	12.2ns	0.0379	2.85 KB
#6704	`SendAsync`	net472	7.43μs	2.43ns	9.08ns	0.492	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.62μs	0.698ns	2.61ns	0.0227	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.3μs	1.48ns	5.55ns	0.0218	1.64 KB
master	`EnrichedLog`	net472	2.51μs	2.1ns	8.15ns	0.249	1.57 KB
#6704	`EnrichedLog`	net6.0	1.55μs	0.769ns	2.88ns	0.0231	1.64 KB
#6704	`EnrichedLog`	netcoreapp3.1	2.19μs	1.99ns	7.7ns	0.022	1.64 KB
#6704	`EnrichedLog`	net472	2.59μs	1.98ns	7.39ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	114μs	208ns	807ns	0	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	178ns	688ns	0	0	4.28 KB
master	`EnrichedLog`	net472	150μs	125ns	483ns	0.673	0.224	4.46 KB
#6704	`EnrichedLog`	net6.0	115μs	162ns	607ns	0.0573	0	4.28 KB
#6704	`EnrichedLog`	netcoreapp3.1	117μs	248ns	961ns	0.0582	0	4.28 KB
#6704	`EnrichedLog`	net472	150μs	155ns	601ns	0.674	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.1μs	0.573ns	2.07ns	0.0311	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.21μs	1.92ns	6.92ns	0.0295	2.2 KB
master	`EnrichedLog`	net472	4.86μs	0.927ns	3.59ns	0.32	2.02 KB
#6704	`EnrichedLog`	net6.0	2.93μs	3.6ns	13.9ns	0.0307	2.2 KB
#6704	`EnrichedLog`	netcoreapp3.1	4.18μs	0.936ns	3.37ns	0.0291	2.2 KB
#6704	`EnrichedLog`	net472	4.87μs	2.1ns	8.13ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.39μs	0.78ns	3.02ns	0.016	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.72μs	2.24ns	8.68ns	0.0154	1.14 KB
master	`SendReceive`	net472	2.14μs	0.547ns	2.05ns	0.183	1.16 KB
#6704	`SendReceive`	net6.0	1.43μs	0.664ns	2.57ns	0.0158	1.14 KB
#6704	`SendReceive`	netcoreapp3.1	1.85μs	1.38ns	5.33ns	0.0153	1.14 KB
#6704	`SendReceive`	net472	2.15μs	0.461ns	1.73ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.7μs	0.975ns	3.52ns	0.0216	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.88μs	1.72ns	6.65ns	0.0213	1.65 KB
master	`EnrichedLog`	net472	4.25μs	4.09ns	15.8ns	0.323	2.04 KB
#6704	`EnrichedLog`	net6.0	2.76μs	1.56ns	5.82ns	0.0221	1.6 KB
#6704	`EnrichedLog`	netcoreapp3.1	4.03μs	1.72ns	6.67ns	0.0222	1.65 KB
#6704	`EnrichedLog`	net472	4.26μs	1.79ns	6.68ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	417ns	0.557ns	2.08ns	0.00809	576 B
master	`StartFinishSpan`	netcoreapp3.1	618ns	1.02ns	3.94ns	0.00772	576 B
master	`StartFinishSpan`	net472	671ns	1.18ns	4.58ns	0.0915	578 B
master	`StartFinishScope`	net6.0	472ns	0.831ns	3.22ns	0.00987	696 B
master	`StartFinishScope`	netcoreapp3.1	718ns	0.697ns	2.61ns	0.00924	696 B
master	`StartFinishScope`	net472	824ns	1.61ns	6.02ns	0.104	658 B
#6704	`StartFinishSpan`	net6.0	452ns	0.641ns	2.48ns	0.00815	576 B
#6704	`StartFinishSpan`	netcoreapp3.1	602ns	0.901ns	3.25ns	0.00764	576 B
#6704	`StartFinishSpan`	net472	661ns	1.6ns	6.19ns	0.0917	578 B
#6704	`StartFinishScope`	net6.0	479ns	0.364ns	1.41ns	0.00979	696 B
#6704	`StartFinishScope`	netcoreapp3.1	733ns	1.03ns	4ns	0.00925	696 B
#6704	`StartFinishScope`	net472	867ns	1.36ns	5.25ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	650ns	0.834ns	3.23ns	0.00961	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	922ns	2.04ns	7.91ns	0.00939	696 B
master	`RunOnMethodBegin`	net472	1.05μs	2.66ns	10.3ns	0.104	658 B
#6704	`RunOnMethodBegin`	net6.0	662ns	0.514ns	1.99ns	0.0097	696 B
#6704	`RunOnMethodBegin`	netcoreapp3.1	942ns	2.84ns	11ns	0.00939	696 B
#6704	`RunOnMethodBegin`	net472	1.09μs	0.744ns	2.88ns	0.104	658 B

Add fixes for release process

1195ca9

andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Feb 24, 2025

andrewlock requested a review from a team as a code owner February 24, 2025 13:29

bouwkast approved these changes Feb 24, 2025

View reviewed changes

Fix release

51013be

brettlangdon approved these changes Feb 24, 2025

View reviewed changes

lucaspimentel approved these changes Feb 24, 2025

View reviewed changes

andrewlock merged commit d2e3f0b into master Feb 24, 2025
143 of 151 checks passed

andrewlock deleted the andrew/ci/fix-release branch February 24, 2025 17:47

github-actions bot added this to the vNext-v3 milestone Feb 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fixes for release process #6704

Add fixes for release process #6704

andrewlock commented Feb 24, 2025 •

edited

Loading

andrewlock commented Feb 24, 2025

andrewlock commented Feb 24, 2025

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6704

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Add fixes for release process #6704

Add fixes for release process #6704

Conversation

andrewlock commented Feb 24, 2025 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

andrewlock commented Feb 24, 2025

Execution-Time Benchmarks Report ⏱️

andrewlock commented Feb 24, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6704

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

andrewlock commented Feb 24, 2025 •

edited

Loading