Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fleet installer] Allow pushing Windows SSI OCI images to prod #6716

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

andrewlock
Copy link
Member

Summary of changes

Removes the "don't publish Windows OCI images to prod" guardrails

Reason for change

We want to publish prod OCI images

Implementation details

Delete the guardrails

Warning

This line must also be removed from the one-pipeline so that it publishes Windows OCI images. I have created a PR to do that here. If that is merged then we should revert the change in this PR which explicitly uses that branch (as the branch will become stale)

Test coverage

There's not a good way to test this without doing a release...

Other details

We don't want to release this until we're happy with the interface we have defined between the fleet installer and the .net installer

@andrewlock andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages status:do-not-merge Work is done. Can review, but do not merge yet. area:ssi labels Feb 25, 2025
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Feb 25, 2025

Datadog Report

Branch report: andrew/win-ssi/release-prod-oci-images
Commit report: a827bbc
Test service: dd-trace-dotnet

✅ 0 Failed, 249337 Passed, 2408 Skipped, 19h 0m 0.99s Total Time
⌛ 2 Performance Regressions

⌛ Performance Regressions vs Default Branch (2)

  • Baseline - Samples.HttpMessageHandler.windows.netcoreapp31.json.scenarios 275.52ms (+4.27ms, +2%) - Details
  • Baseline - Samples.HttpMessageHandler.windows.net60.json.scenarios 266.95ms (+4.29ms, +2%) - Details

@andrewlock
Copy link
Member Author

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (999ms)  : 983, 1016
     .   : milestone, 999,
    master - mean (997ms)  : 976, 1019
     .   : milestone, 997,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (103ms)  : 100, 106
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (674ms)  : 657, 691
     .   : milestone, 674,
    master - mean (674ms)  : 657, 691
     .   : milestone, 674,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (89ms)  : 87, 90
     .   : milestone, 89,
    master - mean (89ms)  : 88, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (633ms)  : 612, 654
     .   : milestone, 633,
    master - mean (634ms)  : 614, 653
     .   : milestone, 634,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (191ms)  : 187, 196
     .   : milestone, 191,
    master - mean (192ms)  : 187, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (1,121ms)  : 1083, 1159
     .   : milestone, 1121,
    master - mean (1,110ms)  : 1084, 1135
     .   : milestone, 1110,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (276ms)  : 267, 284
     .   : milestone, 276,
    master - mean (272ms)  : 265, 278
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (877ms)  : 844, 910
     .   : milestone, 877,
    master - mean (869ms)  : 836, 902
     .   : milestone, 869,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6716) - mean (267ms)  : 260, 273
     .   : milestone, 267,
    master - mean (264ms)  : 260, 267
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6716) - mean (860ms)  : 827, 892
     .   : milestone, 860,
    master - mean (849ms)  : 819, 880
     .   : milestone, 849,

Loading

@andrewlock
Copy link
Member Author

Benchmarks Report for tracer 🐌

Benchmarks for #6716 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.29μs 47ns 358ns 0.0163 0.00814 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.5μs 57.4ns 349ns 0.0205 0.0103 0 5.8 KB
master StartStopWithChild net472 16.3μs 70.3ns 272ns 1.06 0.328 0.096 6.21 KB
#6716 StartStopWithChild net6.0 8.36μs 48.7ns 449ns 0.0134 0.00668 0 5.61 KB
#6716 StartStopWithChild netcoreapp3.1 10.3μs 59.8ns 507ns 0.0207 0 0 5.8 KB
#6716 StartStopWithChild net472 16μs 52.8ns 205ns 1.06 0.326 0.103 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 474μs 353ns 1.37μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 641μs 343ns 1.29μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 839μs 692ns 2.68μs 0.419 0 0 3.3 KB
#6716 WriteAndFlushEnrichedTraces net6.0 482μs 256ns 921ns 0 0 0 2.7 KB
#6716 WriteAndFlushEnrichedTraces netcoreapp3.1 664μs 291ns 1.09μs 0 0 0 2.7 KB
#6716 WriteAndFlushEnrichedTraces net472 844μs 667ns 2.5μs 0.428 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 131μs 365ns 1.41μs 0.196 0 0 14.47 KB
master SendRequest netcoreapp3.1 150μs 125ns 486ns 0.223 0 0 17.27 KB
master SendRequest net472 0.00044ns 0.000216ns 0.000808ns 0 0 0 0 b
#6716 SendRequest net6.0 132μs 524ns 2.03μs 0.199 0 0 14.47 KB
#6716 SendRequest netcoreapp3.1 153μs 133ns 514ns 0.153 0 0 17.27 KB
#6716 SendRequest net472 0.000738ns 0.000351ns 0.00131ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 564μs 2.95μs 14.2μs 0.551 0 0 41.61 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 3.67μs 22.3μs 0.326 0 0 41.62 KB
master WriteAndFlushEnrichedTraces net472 863μs 4μs 16μs 8.33 2.5 0.417 53.29 KB
#6716 WriteAndFlushEnrichedTraces net6.0 573μs 3.3μs 26.8μs 0.287 0 0 41.62 KB
#6716 WriteAndFlushEnrichedTraces netcoreapp3.1 703μs 4.05μs 30.9μs 0.338 0 0 41.79 KB
#6716 WriteAndFlushEnrichedTraces net472 841μs 2.21μs 8.27μs 8.45 2.53 0.422 53.3 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.26μs 0.747ns 2.8ns 0.0144 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.77μs 1.4ns 5.44ns 0.0138 0 0 1.02 KB
master ExecuteNonQuery net472 2.08μs 1.18ns 4.43ns 0.157 0.00104 0 987 B
#6716 ExecuteNonQuery net6.0 1.32μs 1.79ns 6.95ns 0.0145 0 0 1.02 KB
#6716 ExecuteNonQuery netcoreapp3.1 1.78μs 2.29ns 8.57ns 0.0135 0 0 1.02 KB
#6716 ExecuteNonQuery net472 2.02μs 1.25ns 4.68ns 0.157 0.00101 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.25μs 0.831ns 3.11ns 0.0139 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.51μs 0.595ns 2.22ns 0.0128 0 0 976 B
master CallElasticsearch net472 2.59μs 1.54ns 5.95ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.39μs 0.728ns 2.72ns 0.0133 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.488ns 1.83ns 0.0139 0 0 1.02 KB
master CallElasticsearchAsync net472 2.6μs 1.67ns 6.45ns 0.166 0 0 1.05 KB
#6716 CallElasticsearch net6.0 1.32μs 0.683ns 2.56ns 0.0133 0 0 976 B
#6716 CallElasticsearch netcoreapp3.1 1.57μs 1.37ns 5.14ns 0.0131 0 0 976 B
#6716 CallElasticsearch net472 2.6μs 2.16ns 8.37ns 0.157 0 0 995 B
#6716 CallElasticsearchAsync net6.0 1.4μs 0.487ns 1.82ns 0.0133 0 0 952 B
#6716 CallElasticsearchAsync netcoreapp3.1 1.6μs 0.592ns 2.22ns 0.0139 0 0 1.02 KB
#6716 CallElasticsearchAsync net472 2.64μs 1.52ns 5.87ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.29μs 0.616ns 2.22ns 0.0137 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.67μs 2.19ns 8.18ns 0.0124 0 0 952 B
master ExecuteAsync net472 1.87μs 0.624ns 2.16ns 0.145 0 0 915 B
#6716 ExecuteAsync net6.0 1.35μs 0.916ns 3.55ns 0.0129 0 0 952 B
#6716 ExecuteAsync netcoreapp3.1 1.71μs 1.08ns 3.9ns 0.0128 0 0 952 B
#6716 ExecuteAsync net472 1.95μs 1.23ns 4.74ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.4μs 2.02ns 7.81ns 0.033 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.29μs 1.8ns 6.73ns 0.0369 0 0 2.85 KB
master SendAsync net472 7.47μs 1.42ns 5.32ns 0.493 0 0 3.12 KB
#6716 SendAsync net6.0 4.35μs 5.23ns 19.6ns 0.0322 0 0 2.31 KB
#6716 SendAsync netcoreapp3.1 5.33μs 4.01ns 15.5ns 0.0377 0 0 2.85 KB
#6716 SendAsync net472 7.55μs 1.68ns 6.06ns 0.494 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.52μs 0.868ns 3.25ns 0.0229 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.2μs 1.3ns 4.86ns 0.022 0 0 1.64 KB
master EnrichedLog net472 2.56μs 1.82ns 6.82ns 0.249 0 0 1.57 KB
#6716 EnrichedLog net6.0 1.47μs 1.04ns 3.9ns 0.0228 0 0 1.64 KB
#6716 EnrichedLog netcoreapp3.1 2.1μs 0.835ns 3.01ns 0.0222 0 0 1.64 KB
#6716 EnrichedLog net472 2.65μs 1.45ns 5.62ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 114μs 171ns 664ns 0.057 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 117μs 143ns 535ns 0.0579 0 0 4.28 KB
master EnrichedLog net472 152μs 187ns 725ns 0.68 0.227 0 4.46 KB
#6716 EnrichedLog net6.0 114μs 164ns 636ns 0.0571 0 0 4.28 KB
#6716 EnrichedLog netcoreapp3.1 118μs 227ns 879ns 0.0593 0 0 4.28 KB
#6716 EnrichedLog net472 150μs 143ns 533ns 0.674 0.225 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.12μs 1.18ns 4.56ns 0.0312 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.04μs 7.58ns 29.4ns 0.028 0 0 2.2 KB
master EnrichedLog net472 4.9μs 9.7ns 36.3ns 0.319 0 0 2.02 KB
#6716 EnrichedLog net6.0 3.06μs 0.562ns 2.1ns 0.0306 0 0 2.2 KB
#6716 EnrichedLog netcoreapp3.1 4.24μs 1.16ns 4.33ns 0.0299 0 0 2.2 KB
#6716 EnrichedLog net472 4.82μs 1.3ns 5.05ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.35μs 3.96ns 14.8ns 0.0161 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.77μs 0.821ns 3.07ns 0.0151 0 0 1.14 KB
master SendReceive net472 2.09μs 1.42ns 5.49ns 0.183 0 0 1.16 KB
#6716 SendReceive net6.0 1.29μs 0.354ns 1.28ns 0.016 0 0 1.14 KB
#6716 SendReceive netcoreapp3.1 1.77μs 0.672ns 2.52ns 0.0151 0 0 1.14 KB
#6716 SendReceive net472 2.12μs 1.5ns 5.4ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.78μs 0.725ns 2.81ns 0.0223 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.99μs 2.57ns 9.96ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.33μs 5.71ns 22.1ns 0.322 0 0 2.04 KB
#6716 EnrichedLog net6.0 2.58μs 1.01ns 3.79ns 0.0232 0 0 1.6 KB
#6716 EnrichedLog netcoreapp3.1 3.75μs 1.31ns 5.06ns 0.0225 0 0 1.65 KB
#6716 EnrichedLog net472 4.39μs 10.3ns 39.8ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 387ns 0.622ns 2.41ns 0.00811 0 0 576 B
master StartFinishSpan netcoreapp3.1 601ns 1.19ns 4.61ns 0.00764 0 0 576 B
master StartFinishSpan net472 591ns 1.07ns 4.14ns 0.0916 0 0 578 B
master StartFinishScope net6.0 528ns 0.805ns 3.12ns 0.00978 0 0 696 B
master StartFinishScope netcoreapp3.1 682ns 1.44ns 5.59ns 0.00934 0 0 696 B
master StartFinishScope net472 851ns 1.53ns 5.92ns 0.104 0 0 658 B
#6716 StartFinishSpan net6.0 401ns 0.775ns 2.8ns 0.00811 0 0 576 B
#6716 StartFinishSpan netcoreapp3.1 604ns 0.71ns 2.75ns 0.00794 0 0 576 B
#6716 StartFinishSpan net472 573ns 1.12ns 4.34ns 0.0916 0 0 578 B
#6716 StartFinishScope net6.0 554ns 0.603ns 2.34ns 0.00965 0 0 696 B
#6716 StartFinishScope netcoreapp3.1 712ns 1.17ns 4.52ns 0.00926 0 0 696 B
#6716 StartFinishScope net472 840ns 1.22ns 4.71ns 0.105 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 685ns 1.51ns 5.84ns 0.00985 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 911ns 2.4ns 8.66ns 0.00917 0 0 696 B
master RunOnMethodBegin net472 1.11μs 3.1ns 12ns 0.104 0 0 658 B
#6716 RunOnMethodBegin net6.0 669ns 1.13ns 4.38ns 0.00961 0 0 696 B
#6716 RunOnMethodBegin netcoreapp3.1 893ns 1.12ns 4.33ns 0.00961 0 0 696 B
#6716 RunOnMethodBegin net472 1.18μs 3.35ns 13ns 0.104 0 0 658 B

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages area:ssi status:do-not-merge Work is done. Can review, but do not merge yet.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant