Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(ci): Fixes iOS binary size diff failure #4489

Merged
merged 2 commits into from
Jan 30, 2025
Merged

chore(ci): Fixes iOS binary size diff failure #4489

merged 2 commits into from
Jan 30, 2025

Conversation

antonis
Copy link
Collaborator

@antonis antonis commented Jan 28, 2025 •
edited
Loading

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

📜 Description

Increase iOS binary size diff by 100KB to avoid failing binary size checks.

💡 Motivation and Context

I've noticed that metrics (new, ios) check has started to fail consistently (though marginally) due to small changes in the size of the app.

For example it passes on the feedback widget feature branch

INFO: App io.sentry.rn.perf-test-app-sentry is 1.07 MiB larger than app io.sentry.rn.perf-test-app-plain
BinarySizeTest > app size() PASSED

but fails on the autoinject branch that does not include major code changes or large assets.

INFO: App io.sentry.rn.perf-test-app-sentry is 1.08 MiB larger than app io.sentry.rn.perf-test-app-plain
BinarySizeTest > app size() FAILED
    java.lang.AssertionError: 1127476 should be < 1126400

On main the test marginally passes:

INFO: App io.sentry.rn.perf-test-app-sentry is 1.07 MiB larger than app io.sentry.rn.perf-test-app-plain
BinarySizeTest > app size() PASSED

With this PR I suggest bumping the diff by 100KB which should give us some space for new SDK features without compromising the ability of the test to detect any unreasonable bump of the binary size.

Notes:

💚 How did you test it?

CI

📝 Checklist

  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 28, 2025 •
edited
Loading

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 342.82 ms 335.92 ms -6.90 ms
Size 17.75 MiB 20.11 MiB 2.37 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
cdf2f33 469.46 ms 462.17 ms -7.29 ms
7bc4d75 488.76 ms 473.28 ms -15.48 ms
e22745e 462.66 ms 458.10 ms -4.56 ms
f06c879 408.41 ms 424.54 ms 16.13 ms
e2b64fe 316.88 ms 330.23 ms 13.35 ms
c6f01ea 486.20 ms 486.98 ms 0.77 ms
34aba08 328.10 ms 342.84 ms 14.74 ms
b95b8af 454.05 ms 454.53 ms 0.48 ms
d8e8c67 448.79 ms 438.70 ms -10.09 ms
52c0562 453.04 ms 434.71 ms -18.33 ms

App size

Revision Plain With Sentry Diff
cdf2f33 17.74 MiB 20.08 MiB 2.34 MiB
7bc4d75 17.74 MiB 20.08 MiB 2.34 MiB
e22745e 17.74 MiB 20.08 MiB 2.34 MiB
f06c879 17.73 MiB 19.85 MiB 2.12 MiB
e2b64fe 17.73 MiB 19.80 MiB 2.07 MiB
c6f01ea 17.74 MiB 20.10 MiB 2.36 MiB
34aba08 17.73 MiB 19.80 MiB 2.07 MiB
b95b8af 17.73 MiB 20.11 MiB 2.37 MiB
d8e8c67 17.74 MiB 20.10 MiB 2.36 MiB
52c0562 17.73 MiB 20.11 MiB 2.38 MiB

Previous results on branch: antonis/ci-ios-binary-size

Startup times

Revision Plain With Sentry Diff
a202c7a 495.10 ms 494.98 ms -0.13 ms

App size

Revision Plain With Sentry Diff
a202c7a 17.75 MiB 20.11 MiB 2.37 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 28, 2025 •
edited
Loading

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 469.30 ms 614.86 ms 145.56 ms
Size 7.15 MiB 8.38 MiB 1.23 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
416f465+dirty 407.26 ms 451.31 ms 44.05 ms
9f0f6c8+dirty 365.53 ms 419.16 ms 53.63 ms
70e6261+dirty 395.08 ms 408.12 ms 13.04 ms
d43a46b+dirty 417.65 ms 472.98 ms 55.33 ms
2ec71da+dirty 375.64 ms 431.59 ms 55.95 ms
12427f4+dirty 379.48 ms 400.92 ms 21.44 ms
5446992+dirty 371.61 ms 390.00 ms 18.39 ms
baa882f+dirty 449.30 ms 540.40 ms 91.10 ms
d0bf494+dirty 253.73 ms 308.23 ms 54.49 ms
9a3ca65+dirty 344.96 ms 358.92 ms 13.96 ms

App size

Revision Plain With Sentry Diff
416f465+dirty 7.15 MiB 8.37 MiB 1.22 MiB
9f0f6c8+dirty 7.15 MiB 8.37 MiB 1.22 MiB
70e6261+dirty 7.15 MiB 8.21 MiB 1.07 MiB
d43a46b+dirty 7.15 MiB 8.34 MiB 1.19 MiB
2ec71da+dirty 7.15 MiB 8.38 MiB 1.23 MiB
12427f4+dirty 7.15 MiB 8.12 MiB 997.78 KiB
5446992+dirty 7.15 MiB 8.12 MiB 999.45 KiB
baa882f+dirty 7.15 MiB 8.34 MiB 1.19 MiB
d0bf494+dirty 7.15 MiB 8.04 MiB 910.85 KiB
9a3ca65+dirty 7.15 MiB 8.09 MiB 962.83 KiB

Previous results on branch: antonis/ci-ios-binary-size

Startup times

Revision Plain With Sentry Diff
a202c7a+dirty 398.12 ms 447.65 ms 49.53 ms

App size

Revision Plain With Sentry Diff
a202c7a+dirty 7.15 MiB 8.38 MiB 1.23 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 28, 2025 •
edited
Loading

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1222.51 ms 1230.16 ms 7.65 ms
Size 3.19 MiB 4.25 MiB 1.07 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
12427f4+dirty 1224.90 ms 1231.40 ms 6.50 ms
de59d3a+dirty 1241.17 ms 1249.16 ms 8.00 ms
63ed251+dirty 1223.27 ms 1222.94 ms -0.33 ms
5a22220+dirty 1246.18 ms 1249.61 ms 3.43 ms
416f465+dirty 1232.48 ms 1233.26 ms 0.78 ms
c398f67+dirty 1227.31 ms 1230.00 ms 2.69 ms
1332acb+dirty 1243.98 ms 1241.12 ms -2.86 ms
8fe7c9d+dirty 1227.63 ms 1245.28 ms 17.65 ms
5f03ae9+dirty 1237.79 ms 1241.02 ms 3.23 ms
9cab16b+dirty 1236.10 ms 1247.16 ms 11.06 ms

App size

Revision Plain With Sentry Diff
12427f4+dirty 2.92 MiB 3.44 MiB 533.29 KiB
de59d3a+dirty 2.92 MiB 3.67 MiB 772.59 KiB
63ed251+dirty 2.92 MiB 3.66 MiB 757.10 KiB
5a22220+dirty 2.92 MiB 3.48 MiB 575.81 KiB
416f465+dirty 2.92 MiB 3.67 MiB 772.44 KiB
c398f67+dirty 2.92 MiB 3.60 MiB 701.89 KiB
1332acb+dirty 2.92 MiB 3.67 MiB 772.45 KiB
8fe7c9d+dirty 3.19 MiB 4.24 MiB 1.06 MiB
5f03ae9+dirty 3.19 MiB 4.25 MiB 1.06 MiB
9cab16b+dirty 2.92 MiB 3.64 MiB 743.06 KiB

Previous results on branch: antonis/ci-ios-binary-size

Startup times

Revision Plain With Sentry Diff
a202c7a+dirty 1218.98 ms 1215.89 ms -3.09 ms

App size

Revision Plain With Sentry Diff
a202c7a+dirty 3.19 MiB 4.25 MiB 1.07 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 28, 2025 •
edited
Loading

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1219.10 ms 1217.38 ms -1.72 ms
Size 2.63 MiB 3.69 MiB 1.06 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
12427f4+dirty 1267.15 ms 1271.30 ms 4.15 ms
de59d3a+dirty 1223.73 ms 1236.28 ms 12.55 ms
63ed251+dirty 1232.55 ms 1238.77 ms 6.22 ms
5a22220+dirty 1209.49 ms 1220.94 ms 11.45 ms
416f465+dirty 1221.56 ms 1226.48 ms 4.92 ms
c398f67+dirty 1219.67 ms 1225.66 ms 5.99 ms
1332acb+dirty 1230.53 ms 1234.54 ms 4.01 ms
8fe7c9d+dirty 1241.83 ms 1244.35 ms 2.51 ms
5f03ae9+dirty 1232.29 ms 1230.92 ms -1.37 ms
9cab16b+dirty 1237.76 ms 1234.00 ms -3.76 ms

App size

Revision Plain With Sentry Diff
12427f4+dirty 2.36 MiB 2.88 MiB 530.38 KiB
de59d3a+dirty 2.36 MiB 3.11 MiB 760.16 KiB
63ed251+dirty 2.36 MiB 3.10 MiB 752.55 KiB
5a22220+dirty 2.36 MiB 2.92 MiB 570.21 KiB
416f465+dirty 2.36 MiB 3.11 MiB 759.80 KiB
c398f67+dirty 2.36 MiB 3.04 MiB 696.27 KiB
1332acb+dirty 2.36 MiB 3.11 MiB 759.86 KiB
8fe7c9d+dirty 2.63 MiB 3.68 MiB 1.04 MiB
5f03ae9+dirty 2.63 MiB 3.68 MiB 1.05 MiB
9cab16b+dirty 2.36 MiB 3.08 MiB 737.23 KiB

Previous results on branch: antonis/ci-ios-binary-size

Startup times

Revision Plain With Sentry Diff
a202c7a+dirty 1229.63 ms 1230.33 ms 0.71 ms

App size

Revision Plain With Sentry Diff
a202c7a+dirty 2.63 MiB 3.69 MiB 1.05 MiB

@antonis antonis marked this pull request as ready for review January 28, 2025 11:03
krystofwoldrich
krystofwoldrich approved these changes Jan 30, 2025
View reviewed changes
Copy link
Member

@krystofwoldrich krystofwoldrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@krystofwoldrich krystofwoldrich merged commit 1c9e040 into main Jan 30, 2025
70 checks passed
@krystofwoldrich krystofwoldrich deleted the antonis/ci-ios-binary-size branch January 30, 2025 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@krystofwoldrich krystofwoldrich krystofwoldrich approved these changes

@lucas-zimerman lucas-zimerman Awaiting requested review from lucas-zimerman lucas-zimerman is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@antonis @krystofwoldrich