Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Write JSON state #997

Merged
merged 22 commits into from
Oct 29, 2024
Merged

Write JSON state #997

merged 22 commits into from
Oct 29, 2024

Commits on Oct 29, 2024

  1. Add types to DynamoDB restore flow. Add Persistable abstract class an… 

    …d to_json methods for classes in DynamoDB restore flow. Write an additional attribute to DynamoDB to capture non-pickled state_data.
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    b4a7e16 View commit details
    Browse the repository at this point in the history

  2. Remove old comments. Delete mcp state metadata.

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    c081772 View commit details
    Browse the repository at this point in the history

  3. Return these

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    88de418 View commit details
    Browse the repository at this point in the history

  4. Try itest validation without mcp_state

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    909e9b2 View commit details
    Browse the repository at this point in the history

  5. Fix isinstance check for Action to_json

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    1a5238b View commit details
    Browse the repository at this point in the history

  6. Comment cleanup. Add cleanup TODOs for TRON-2293. Rename val in dynam… 

    …odb_state_store to something a little more explanatory now that we have 2 versions
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    c18cce4 View commit details
    Browse the repository at this point in the history

  7. Remove shelve key test since we no longer save mcp_state or state_met… 

    …adata
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    7fb271d View commit details
    Browse the repository at this point in the history

  8. Remove deprecated itest. Rename some variables to make things a bit c… 

    …learer.
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    677d6e3 View commit details
    Browse the repository at this point in the history

  9. Remove unnecessary list call

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    e262cbc View commit details
    Browse the repository at this point in the history

  10. Replace ignore with proper types for sorted_groups

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    83d7c65 View commit details
    Browse the repository at this point in the history

  11. Remove skip_validation since removing MCP StateMetadata

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    002f935 View commit details
    Browse the repository at this point in the history

  12. Add more types and a few TODO tickets

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    f5ab20c View commit details
    Browse the repository at this point in the history

  13. Remove debug logs, add constant for transact_write_items, update tran… 

    …saction limit, and change setitem signature for take value tuple
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    00a50a9 View commit details
    Browse the repository at this point in the history

  14. Check that rendered_command and docker_image are not None before crea… 

    …ting task
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    0eca03f View commit details
    Browse the repository at this point in the history

  15. Remove comment from before my enlightenment

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    d727236 View commit details
    Browse the repository at this point in the history

  16. Add remaining types, add tickets to valid TODOs, and delete irrelevan… 

    …t TODOs
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    a02aedc View commit details
    Browse the repository at this point in the history

  17. Fix time_zone type. Remove unreachable code after typing trigger_down… 

    …streams
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    f26f855 View commit details
    Browse the repository at this point in the history

  18. Maintain most of the parsing logic as is. Add a few more tests. Add T… 

    …ODO and ticket for regex issues
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    c3c8c77 View commit details
    Browse the repository at this point in the history

  19. Add error handling in various to_json() funcs. Break JSON write out f… 

    …rom pickle write so that we maintain writing pickles if JSON fails
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    7a0cb98 View commit details
    Browse the repository at this point in the history

  20. Update to_json exception logging. On DynamoDB write failure only pass… 

    … the pickle back into the queue
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    e79ae25 View commit details
    Browse the repository at this point in the history

  21. Raise these so that we get the full picture on failure and can decide… 

    … what to do in dynamodb_state_store
    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    e8dcc8e View commit details
    Browse the repository at this point in the history

  22. Add serialization error counter

    @KaspariK
    KaspariK committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    b0186d1 View commit details
    Browse the repository at this point in the history