Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TransactionError when manually commit() in use generator #730

Open
XuCcc opened this issue Oct 18, 2024 · 0 comments
Open

TransactionError when manually commit() in use generator #730

XuCcc opened this issue Oct 18, 2024 · 0 comments

Comments

@XuCcc
Copy link

XuCcc commented Oct 18, 2024

I have manually called commit() before yield, but I still get the TransactionError: You need to manually commit() changes before suspending the generator error.
From the DEBUG SQL logs, it seems like the transaction is being directly rolled back.

    @db_session
    def confirm(self, db_name: str = None, table_name: str = None, field_name: List[str] = None):
        engine_logger.info(f"search fields: "
                           f"{'database=' + db_name if db_name else ''} "
                           f"{'table=' + table_name if table_name else ''} "
                           f"{'field in (' + ','.join(field_name) + ')' if field_name else ''}")
        fields = select(f for f in RDSField if f.data_type in MYSQL_SAMPLE_DATA_TYPE)
        if db_name:
            fields = fields.filter(lambda f: f.db_name == db_name)
        if table_name:
            fields = fields.filter(lambda f: f.table_name == table_name)
        if field_name:
            fields = fields.filter(lambda f: f.name in field_name)
        engine_logger.info(f'find {fields.count()} fields to scan')
        commit()
        for field in fields:
            yield field.db_name, field.table_name, field.name, field.region
            # print(field.db_name, field.table_name, field.name, field.region)
2024-10-18 12:03:47.282 | INFO     | core.task:confirm:64 - search fields: database=caijing_bytepay_sign_db  
GET NEW CONNECTION
SELECT COUNT(*)
FROM `rdsfield` `f`
WHERE `f`.`data_type` IN (%s, %s, %s, %s, %s, %s)
  AND `f`.`db_name` = %s
['varchar', 'mediumtext', 'text', 'char', 'longtext', 'json', 'caijing_bytepay_sign_db']

COMMIT
SELECT `f`.`id`, `f`.`hash`, `f`.`name`, `f`.`region`, `f`.`db_name`, `f`.`table_name`, `f`.`data_type`, `f`.`comment`, `f`.`is_pk`, `f`.`is_sharding_key`, `f`.`create_time`, `f`.`latest_time`
FROM `rdsfield` `f`
WHERE `f`.`data_type` IN (%s, %s, %s, %s, %s, %s)
  AND `f`.`db_name` = %s
['varchar', 'mediumtext', 'text', 'char', 'longtext', 'json', 'caijing_bytepay_sign_db']

2024-10-18 12:03:47.317 | INFO     | core.task:confirm:75 - find 16 fields to scan
ROLLBACK
RELEASE CONNECTION
Traceback (most recent call last):
  File "/Users/bytedance/Projects/code/team/DBScan/demo.py", line 46, in <module>
    for i in task.confirm('caijing_bytepay_sign_db'):
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/orm/core.py", line 601, in new_gen_func
    output = wrapped_interact(iterator)
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/orm/core.py", line 587, in wrapped_interact
    rollback_and_reraise(sys.exc_info())
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/orm/core.py", line 361, in rollback_and_reraise
    reraise(*exc_info)
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/utils/utils.py", line 88, in reraise
    try: raise exc.with_traceback(tb)
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/orm/core.py", line 584, in wrapped_interact
    if cache.modified or cache.in_transaction: throw(TransactionError,
  File "/Users/bytedance/.local/share/virtualenvs/DBScan-X_8vXymQ/lib/python3.9/site-packages/pony/utils/utils.py", line 99, in throw
    raise exc
pony.orm.core.TransactionError: You need to manually commit() changes before suspending the generator

Process finished with exit code 1
  • pony ==0.7.19
  • python == 3.9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant