Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: clear previously generated files #14524

Merged
merged 3 commits into from
Jul 23, 2024

Conversation

dbolduc
Copy link
Member

@dbolduc dbolduc commented Jul 20, 2024

Fixes #12587

Make the generate-libraries build a bit more hermetic by removing previously generated files. This is how I found #14519

@scotthart - my read is that the compute stuff is the old way of doing things and UPDATED_DISCOVERY_DOCUMENT=compute is the new way. You would know better.


This change is Reviewable

Copy link

codecov bot commented Jul 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.58%. Comparing base (b55de81) to head (06f65cf).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #14524      +/-   ##
==========================================
- Coverage   93.58%   93.58%   -0.01%     
==========================================
  Files        2316     2316              
  Lines      206957   206957              
==========================================
- Hits       193677   193671       -6     
- Misses      13280    13286       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dbolduc dbolduc marked this pull request as ready for review July 20, 2024 22:58
@dbolduc dbolduc requested a review from a team as a code owner July 20, 2024 22:58
@coryan
Copy link
Contributor

coryan commented Jul 21, 2024

Do you think this is a fix for #12587 ?

@dbolduc
Copy link
Member Author

dbolduc commented Jul 22, 2024

Do you think this is a fix for #12587 ?

Yes, it is. I remember us discussing it, but I couldn't find the issue when I searched for it. Thanks.

Copy link
Member

@scotthart scotthart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @dbolduc)


ci/cloudbuild/builds/generate-libraries.sh line 29 at r2 (raw file):

if [ -z "${GENERATE_GOLDEN_ONLY}" ]; then
  io::log_h2 "Removing all other previously generated files"
  git grep -l "Generated by the Codegen C++ plugin" google | xargs rm

This probably needs to also delete any generated files under the protos/ dir as well.

@dbolduc dbolduc force-pushed the hermetic-generate-libraries branch from f3f5b93 to 06f65cf Compare July 22, 2024 19:28
@dbolduc
Copy link
Member Author

dbolduc commented Jul 22, 2024

This probably needs to also delete any generated files under the protos/ dir as well.

Thanks, I did not think about this.

(You know more about this, but) I think the discovery to proto process depends on the existing protos to make sure we do not break backwards compatibility. If I remove the protos before regenerating them, some of the field indices change.

So I restored the code that removes obsolete compute protos.

Copy link
Member

@scotthart scotthart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. We do need the previous generation to compare message field id numbers.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @dbolduc)

Copy link
Member

@scotthart scotthart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 2 unresolved discussions (waiting on @dbolduc)


ci/cloudbuild/builds/generate-libraries.sh line 54 at r3 (raw file):

  rm "${newer_tmp_file}"

  io::log_h2 "Formatting and adding any new google/cloud/compute/v1/internal/common_*.proto"

IIRC, we need to format the new protos before we generate code from them because that could change line numbers in the proto files that we put in documentation links.

@dbolduc
Copy link
Member Author

dbolduc commented Jul 22, 2024

IIRC, we need to format the new protos before we generate code from them because that could change line numbers in the proto files that we put in documentation links.

We already format the protos a few lines down, before running the generator on generator_config.textproto, so it was redundant.

io::log_h2 "Formatting generated protos"
git ls-files -z -- '*.proto' |

Copy link
Member

@scotthart scotthart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 2 unresolved discussions (waiting on @dbolduc)

@dbolduc dbolduc merged commit 036bb44 into googleapis:main Jul 23, 2024
63 of 64 checks passed
@dbolduc dbolduc deleted the hermetic-generate-libraries branch July 23, 2024 00:40
cuiy0006 pushed a commit to cuiy0006/google-cloud-cpp that referenced this pull request Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

enhance generated code auditing
3 participants