Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

accounts-db: throw error explicitly for next_account_offset calculation when overflows #2093

Merged
merged 2 commits into from
Jul 12, 2024

Conversation

HaoranYi
Copy link

@HaoranYi HaoranYi commented Jul 11, 2024

Problem

  1. When we scanning append_vec, next_account_offset calculation is not safe, and it assumes that calculation never overflow. If it overflows, the app will crash. Fortunately, this is only an issue for tests and debug tools. For production validator code, we always sanitize the storage when we load. Therefore, we should (hopefully)never hit overflow for new_account_offset calculation. However, it would still be nice to throw the error explicitly when we overflow.
  2. We don't have unit test to cover scan_index. And the current test coverage doesn't cover corner cases.
    image

Summary of Changes

  1. throw overflow error explicitly when calculating next_account_offset.
  2. add test to cover scan_index with the corner cases, such incomplete account's data and garbage data that cause overflow. (now split into Accounts-db: Add scan tests to cover bad accounts storage files #2123)
  3. improve test_scan_pubkey to cover mroe corner cases (now split into Accounts-db: Add scan tests to cover bad accounts storage files #2123).

Fixes #

@HaoranYi HaoranYi changed the title handle overflow for next_account_offset calculation accounts-db: handle overflow for next_account_offset calculation Jul 11, 2024
@HaoranYi HaoranYi force-pushed the accounts-db/next_account_offset branch 3 times, most recently from 1858a19 to c301752 Compare July 11, 2024 20:21
@HaoranYi HaoranYi force-pushed the accounts-db/next_account_offset branch from 6ffcc21 to c28a32a Compare July 12, 2024 18:52
@HaoranYi HaoranYi changed the title accounts-db: handle overflow for next_account_offset calculation accounts-db: throw error explicitly for next_account_offset calculation when overflows Jul 12, 2024
@HaoranYi HaoranYi force-pushed the accounts-db/next_account_offset branch from c28a32a to ef631e3 Compare July 12, 2024 19:03
@HaoranYi HaoranYi requested a review from brooksprumo July 12, 2024 19:28
@HaoranYi HaoranYi force-pushed the accounts-db/next_account_offset branch 2 times, most recently from 5097b80 to f389dd5 Compare July 12, 2024 19:46
@HaoranYi HaoranYi closed this Jul 12, 2024
@HaoranYi HaoranYi force-pushed the accounts-db/next_account_offset branch from f389dd5 to d7714fe Compare July 12, 2024 19:48
@HaoranYi HaoranYi requested a review from brooksprumo July 12, 2024 20:13
Copy link

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@HaoranYi HaoranYi added the automerge automerge Merge this Pull Request automatically once CI passes label Jul 12, 2024
@mergify mergify bot merged commit e6a1812 into anza-xyz:master Jul 12, 2024
41 checks passed
@HaoranYi HaoranYi deleted the accounts-db/next_account_offset branch July 12, 2024 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge automerge Merge this Pull Request automatically once CI passes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants