Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

operation_data usability #396

Open
DifferentialOrange opened this issue Nov 10, 2023 · 0 comments
Open

operation_data usability #396

DifferentialOrange opened this issue Nov 10, 2023 · 0 comments
Labels
question Further information is requested

Comments

@DifferentialOrange
Copy link
Member

DifferentialOrange commented Nov 10, 2023

*_many operation provides operation_data with each error so it would be possible to

  • track which exact record had been failed and
  • retry failed records afterward.

But operation_data that may be returned is inconsistent, may lose some info or contain additional values that may change the behavior (like bucket_id).

./doc/playground.lua
tarantool> crud.insert_many('customers', {{'asd'}})
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:179:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for ['asd'] is ['asd', 1550].

tarantool> crud.insert_object_many('customers', {{id = 'asd'}})
2023-11-10 17:17:50.891 [1571729] main/103/playground.lua/crud.common.schema schema.lua:108 W> Number of attempts to reload schema has been ended: 1
---
- null
- - line: 314
    class_name: InsertManyError
    err: 'Failed to flatten object: FlattenError: Object is specified in bad format:
      FlattenError: Field "name" isn''t nullable (set skip_nullability_check_on_flatten
      option to true to skip check)'
    file: '...y/Development/github/tarantool/crud/crud/insert_many.lua'
    operation_data:
      id: asd
    str: 'InsertManyError: Failed to flatten object: FlattenError: Object is specified
      in bad format: FlattenError: Field "name" isn''t nullable (set skip_nullability_check_on_flatten
      option to true to skip check)'
...

Operation data for {id = 'asd'} is {id = 'asd'}.

tarantool> crud.insert_object_many('customers', {{id = 'asd', name = 'asd', age = 2}})
2023-11-10 17:20:02.788 [1571729] main/103/playground.lua/crud.common.schema schema.lua:108 W> Number of attempts to reload schema has been ended: 1
---
- null
- - line: 58
    class_name: CallError
    space_schema_hash: 247485528
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:179:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function 'wrap_func_reload'\n\t...y/Development/github/tarantool/crud/crud/insert_many.lua:331:
      in function <...y/Development/github/tarantool/crud/crud/insert_many.lua:280>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550, 'asd', 2]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for {id = 'asd', name = 'asd', age = 2} is ['asd', 1550, 'asd', 2].

tarantool> crud.upsert_many('customers', { { {'asd'}, {{'=', 'age', 2}} } })
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Tuple field 1 (id) type does not match one required by operation: expected unsigned,
      got string'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/upsert_many.lua:193:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: ['asd', 1550]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Tuple field 1 (id) type does not match one required by operation:
      expected unsigned, got string'
...

Operation data for { {'asd'}, {{'=', 'age', 2}} } is ['asd', 1550].

tarantool> crud.upsert_many('customers', { { {1, box.NULL, 'name', 2}, {{'=!', 'age', 2}} } })
---
- null
- - line: 58
    class_name: CallError
    err: 'Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned an error:
      Unknown UPDATE operation #1: "=!"'
    file: '.../crud/crud/common/map_call_cases/batch_postprocessor.lua'
    stack: "stack traceback:\n\t.../crud/crud/common/map_call_cases/batch_postprocessor.lua:58:
      in function 'collect'\n\t...y/Development/github/tarantool/crud/crud/common/call.lua:132:
      in function 'map'\n\t...y/Development/github/tarantool/crud/crud/upsert_many.lua:193:
      in function 'method'\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:148:
      in function 'func'\n\t...Development/github/tarantool/crud/crud/common/schema.lua:93:
      in function <...Development/github/tarantool/crud/crud/common/schema.lua:88>\n\t[C]:
      in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:754:
      in function 'repl'\n\tbuiltin/box/console.lua:805: in function 'start'\n\t./doc/playground.lua:153:
      in main chunk"
    operation_data: [1, 477, 'name', 2]
    str: 'CallError: Failed for 00e44528-9a64-40e1-b180-fb2b14e369f0: Function returned
      an error: Unknown UPDATE operation #1: "=!"'
...

Operation data for { {1, box.NULL, 'name', 2}, {{'=!', 'age', 2}} } is [1, 477, 'name', 2], even though it's update operation that has been failed.

It seems that we need to properly consider the potential usability of operation_data and rework current implementation to satisfy these potentials.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant