Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: fix pipeline failure due to bigquery Provided Schema does not ma…
…tch Table. (#6344) fix kokoro job failing with error message: ``` + bq load --autodetect --project_id=cloud-java-metrics --source_format=CSV client_library_versions.cloud_java_client_library_release_dates cloud_java_client_library_release_dates.csv Upload complete.Waiting on bqjob_r28424c53fa9642fd_0000018b47445529_1 ... (0s) Current status: RUNNING Waiting on bqjob_r28424c53fa9642fd_0000018b47445529_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'cloud-java-metrics:bqjob_r28424c53fa9642fd_0000018b47445529_1': Provided Schema does not match Table cloud-java-metrics:client_library_versions.cloud_java_clien t_library_release_dates. Cannot add fields (field: string_field_0) Failure details: - It looks like you are appending to an existing table with autodetect enabled. Disabling autodetect may resolve this. ``` Relevant gcloud documentation: [bq load](https://cloud.google.com/bigquery/docs/reference/bq-cli-reference#bq_load) - remove `--autodetect` and use default false so destination table schema is used. - `--skip_leading_rows=1` since no need to infer schema as above. - add --null_marker="-" to deal with empty release dates from source, e.g. for `google-cloud-accessapproval:1.0.10` ![image](https://github.com/googleapis/java-cloud-bom/assets/3340152/7edc8ecf-7f0e-4b8a-969a-b7d5f475cfa0) Manual script run and successfully run script to upload data.
- Loading branch information