Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support create table with primary key in mysql #24930

Merged

Conversation

hqbhoho
Copy link
Contributor

@hqbhoho hqbhoho commented Feb 6, 2025

Description

Add MysqlTableProperties and support create table with primary key in mysql.

Release notes

## MySQL
* Add support for creating tables with `primary_key` table property. 

@cla-bot cla-bot bot added the cla-signed label Feb 6, 2025
@hqbhoho hqbhoho marked this pull request as draft February 6, 2025 12:24
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch 2 times, most recently from bcf63d1 to 06363ce Compare February 7, 2025 09:22
@github-actions github-actions bot added the mysql MySQL connector label Feb 7, 2025
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from 06363ce to 943aa3e Compare February 7, 2025 10:44
@github-actions github-actions bot added the mariadb MariaDB connector label Feb 7, 2025
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from 7929945 to f5e0909 Compare February 7, 2025 11:51
@hqbhoho hqbhoho requested review from chenjian2664, ebyhr and wendigo and removed request for chenjian2664 and ebyhr February 7, 2025 12:23
@hqbhoho
Copy link
Contributor Author

hqbhoho commented Feb 7, 2025

I found In Ignite or phoenix will keep primary key information in table properties rather than column properties. So which way is better?

@hqbhoho hqbhoho requested a review from Praveen2112 February 7, 2025 12:25
@hqbhoho hqbhoho marked this pull request as ready for review February 7, 2025 15:42
Copy link
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix CI failures.

@github-actions github-actions bot added ignite Ignite connector sqlserver SQLServer connector phoenix5 labels Feb 8, 2025
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from d76f026 to ac4e8ef Compare February 8, 2025 02:04
@hqbhoho hqbhoho requested a review from ebyhr February 8, 2025 02:55
@hqbhoho
Copy link
Contributor Author

hqbhoho commented Feb 8, 2025

Please fix CI failures.

it's done!

Comment on lines 66 to 67
@JsonProperty("autoIncrement") boolean autoIncrement,
@JsonProperty("primaryKey") boolean primaryKey)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about for the datasources/tables which doesn't support primayKey/autoincrement ? Instead of implementing it directly for BaseJDBC module - Can we add them for each datasource and unify them once we are sure on how each datasource behave ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! In the java.sql package, DatabaseMetaData defines methods such as getPrimaryKeys and getColumns. The getColumns method returns information about whether a column is IS_AUTOINCREMENT. I believe this is a common JDBC specification. Therefore, I think it can be placed in the BaseJDBC module.Different data sources may have different implementations. If there is a problem, it is likely to be due to the implementation of the data source.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can add primary key first, for the autoIncrement it's seems specify to the connectors

Copy link
Contributor Author

@hqbhoho hqbhoho Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can add primary key first, for the autoIncrement it's seems specify to the connectors

Thanks for the feedback! As mentioned in the document, the attributes of a column should include data type, default value, isNullable, isAutoIncrement, comment and so on. In the future, I also want to add default value configurations.

In the current implementation, If user create table with primary key and autoincrement and the datasources/tables which doesn't support primayKey/autoincrement, these pieces of information will not be included in the generated table creation statement.Therefore, I believe this will not result in any erroneous impact, it's just that these configuration will not take effect if datasources/tables which doesn't support.

@Praveen2112 @chenjian2664 It's better to implement custom JdbcColumnHandle and ColumnPropertiesProvider for different JDBC connectors even if their contents are the same? If that's the case, I will open a new PR to implement only mysql first.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user create table with primary key and autoincrement and the datasources/tables which doesn't support primayKey/autoincrement, these pieces of information will not be included in the generated table creation statement

Would it be better for this behavior to throw exceptions in such cases?

Since you're modifying the base JDBC module, it's needs to consider carefully as different connectors may have varying driver behaviors. You might also need to update the base connector tests for all connectors, implementing this at the base level could ensure consistency(no matter the connector support or not support), while adding it to a specific connector might keep things cleaner.

I also suggest prioritizing primary key support first, as we've already involved this feature in some connectors, though not support create it directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better for this behavior to throw exceptions in such cases?

We should definitely throw an exception if the data source doesn’t support primary keys.

Copy link
Contributor Author

@hqbhoho hqbhoho Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better for this behavior to throw exceptions in such cases?

We should definitely throw an exception if the data source doesn’t support primary keys.

@chenjian2664 @ebyhr @Praveen2112 So It's better to add primary key in base JDBC module first and check all data source, throw an exception if the data source doesn’t support primary keys in this PR. And then open a new PR to support autoincrement, default value and other attribute in a specified data source such as mysql. Is this implementation feasible?
Or we can first implement these features (such as primary keys, auto-increment, default values, etc.) in MySQL, and then gradually extend them to other data sources.

Please give me suggestions,thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could add primary key in Mysql first

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could add primary key in Mysql first

thanks, I am re-implementing it and only support MySQL table creation with primary key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chenjian2664 @Praveen2112 @ebyhr could you help review when you have a time? thanks

@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch 3 times, most recently from 3d3d9f5 to 3c45452 Compare February 10, 2025 14:02
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch 3 times, most recently from bb2f28d to 5593109 Compare February 18, 2025 03:17
@hqbhoho hqbhoho requested a review from chenjian2664 February 18, 2025 03:54
Copy link
Contributor

@chenjian2664 chenjian2664 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good % comments, good job.

cc @ebyhr

@hqbhoho
Copy link
Contributor Author

hqbhoho commented Feb 21, 2025

@chenjian2664 Thank you for the assistance you've provided!
@ebyhr @Praveen2112 Could you confirm if this can be merged? Thanks!

@hqbhoho hqbhoho requested a review from hashhar February 27, 2025 12:43
Copy link

This pull request has gone a while without any activity. Ask for help on #core-dev on Trino slack.

@github-actions github-actions bot added the stale label Mar 20, 2025
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from 5593109 to aa530d1 Compare March 21, 2025 04:04
@ebyhr
Copy link
Member

ebyhr commented Mar 21, 2025

Please add a test that primary_key table property has an uppercase column name.

@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from aa530d1 to c3e1d3e Compare March 21, 2025 06:19
@hqbhoho
Copy link
Contributor Author

hqbhoho commented Mar 21, 2025

Please add a test that primary_key table property has an uppercase column name.

Done

@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from c3e1d3e to 0bbcd5e Compare March 21, 2025 07:05
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch 3 times, most recently from f197320 to 66a01ae Compare March 21, 2025 08:45
@hqbhoho hqbhoho force-pushed the feature/support_primary_key_with_jdbc_connector branch from 66a01ae to 1ef2ccf Compare March 21, 2025 09:13
@github-actions github-actions bot removed the stale label Mar 21, 2025
@ebyhr ebyhr force-pushed the feature/support_primary_key_with_jdbc_connector branch from 1ef2ccf to 87bdd5e Compare March 23, 2025 22:31
@ebyhr ebyhr merged commit 93fe6c4 into trinodb:master Mar 23, 2025
19 checks passed
@github-actions github-actions bot added this to the 475 milestone Mar 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed mysql MySQL connector
Development

Successfully merging this pull request may close these issues.

4 participants