Test dynamic column names in resultsets

[jeeminso]

Summary of the changes / Why this is an improvement

Adding a test for crate/crate#17580.

Checklist

• Link to issue this PR refers to (if applicable): Fixes #???

[jeeminso]

Finally able to reproduce the bug scenario:

ERROR: test_upgrade_paths (test_upgrade.StorageCompatibilityTest.test_upgrade_paths)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 205, in _test_upgrade_path
    self._do_upgrade(cluster, nodes, paths, versions)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/src/crate/qa/tests.py", line 363, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 266, in _do_upgrade
    self.assert_data_persistence(version_def, nodes, digest, paths, accumulated_dynamic_column_names)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 328, in assert_data_persistence
    self.assertIn(name, accumulated_dynamic_column_names)
AssertionError: '16' not found in ['t_5_7_x', 't_5_9_11']  ** `16` must be the oid falsely returned instead of the column name

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 186, in test_upgrade_paths
    self._test_upgrade_path(path, nodes=1)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 219, in _test_upgrade_path
    raise Exception(msg).with_traceback(e.__traceback__)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 205, in _test_upgrade_path
    self._do_upgrade(cluster, nodes, paths, versions)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/src/crate/qa/tests.py", line 363, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 266, in _do_upgrade
    self.assert_data_persistence(version_def, nodes, digest, paths, accumulated_dynamic_column_names)
  File "/var/lib/jenkins/workspace/CrateDB/qa/crate_qa_on_pr/tests/bwc/test_upgrade.py", line 328, in assert_data_persistence
    self.assertIn(name, accumulated_dynamic_column_names)

2025-03-16T14:05:10 Start version: 5.4.x
2025-03-16T14:05:20 Upgrade to: 5.7.x
2025-03-16T14:05:27 Upgrade to: 5.9.11

[jeeminso]

0b8bd79 which picks up the fix passes https://jenkins.crate.io/job/CrateDB/job/qa/job/crate_qa_on_pr/563/execution/node/202/log/

102573c without the fix fails with AssertionError: '16' not found in ['t_5_7_x', 't_5_9_11'] https://jenkins.crate.io/job/CrateDB/job/qa/job/crate_qa_on_pr/559/execution/node/197/log/

Currently the version upgrades are limited to 5.4 > 5.7 > 5.9 to confirm the fix is valid.(Also there are many timeouts occurring...)

[jeeminso]

Hi @mfussenegger could you take a look at this test before I start resolving the timeout errors(which will come back when I revert the temporarily removed changes)

[jeeminso]

Hi @matriv could you review this?

[matriv]

Looks, good, thank you, left a comment.

[matriv]

This shouldn't happen, right? wouldn't be better to remove the check and have it throw an error if the result is null?

Also, please do a REFRESH after the INSERT, to make sure that we test all the values for cols.

[matriv]

Thank you!

[mfussenegger]

Isn't this going to stop producing the broken version because we had released a fix that caused it?

I'd think we'd need a dedicated version path that uses the broken version, and then moves to the newer ones which contain mitigations?

[jeeminso]

Isn't this going to stop producing the broken version because we had released a fix that caused it?

For all tables created in 5.4.x that are upgraded to 5.7.x and executed an alter followed by add column will have the unwanted oids. So I think the current test will created the broken metadata and continue to test the mitigation is in place for upcoming releases.

[mfussenegger]

Isn't this going to stop producing the broken version because we had released a fix that caused it?

For all tables created in 5.4.x that are upgraded to 5.7.x and executed an alter followed by add column will have the unwanted oids. So I think the current test will created the broken metadata and continue to test the mitigation is in place for upcoming releases.

Okay, I thought we had backported a fix to 5.7 that prevents that issue from happening?

[jeeminso]

Okay, I thought we had backported a fix to 5.7 that prevents that issue from happening?

The fix is crate/crate#17178 and is only merged to 5.9. The commit that caused the corrupted createdVersion is crate/crate@5a3fa0f which exists since 4.8.