Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Json Result length not requested length #298

Closed
1084540622 opened this issue Aug 21, 2019 · 1 comment
Closed

Json Result length not requested length #298

1084540622 opened this issue Aug 21, 2019 · 1 comment

Comments

@1084540622
Copy link

When parsing MySQL 8.0 binlog using python-mysql-replication. A Value of Json field is too long. Appearance of Result length not requested length phenomenon

Environment

  • Python Version: Python 3.5
  • MySQL Version: MySQL 8.0
  • requirements.txt: python-mysql-replication 0.19
Traceback (most recent call last):
  File "c:\Users\zhongbiao\.vscode\extensions\ms-python.python-2019.8.30787\pythonFiles\ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "c:\Users\zhongbiao\.vscode\extensions\ms-python.python-2019.8.30787\pythonFiles\lib\python\ptvsd\__main__.py", line 432, in main
    run()
  File "c:\Users\zhongbiao\.vscode\extensions\ms-python.python-2019.8.30787\pythonFiles\lib\python\ptvsd\__main__.py", line 316, in run_file
    runpy.run_path(target, run_name='__main__')
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\Users\zhongbiao\Desktop\mbinlog.py", line 45, in <module>
    main()
  File "c:\Users\zhongbiao\Desktop\mbinlog.py", line 22, in main
    for row in binlogevent.rows:
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\row_event.py", line 427, in rows
    self._fetch_rows()
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\row_event.py", line 422, in _fetch_rows
    self.__rows.append(self._fetch_one_row())
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\row_event.py", line 511, in _fetch_one_row
    row["before_values"] = self._read_column_data(self.columns_present_bitmap)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\row_event.py", line 181, in _read_column_data
    values[name] = self.packet.read_binary_json(column.length_size)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 352, in read_binary_json
    return self.read_binary_json_type(t, length)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 357, in read_binary_json_type
    return self.read_binary_json_object(length - 1, large)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 440, in read_binary_json_objectysqlreplication\packet.py", line 440, in read_binary_json_object
    data = self.read_binary_json_type(t, length)                                       ysqlreplication\packet.py", line 361, in read_binary_json_type
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 361, in read_binary_json_type                         ysqlreplication\packet.py", line 269, in read_variable_length_string
    return self.read_variable_length_string()
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 269, in read_variable_length_stringysqlreplication\packet.py", line 269, in read_variable_length_string
    return self.read(length)
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysqlreplication\packet.py", line 152, in read
    return data + self.packet.read(size - len(data))
  File "C:\Users\zhongbiao\AppData\Local\Programs\Python\Python35\lib\site-packages\pymysql\protocol.py", line 73, in read
    raise AssertionError(error)
AssertionError: Result length not requested length:
Expected=434314973550195661.  Actual=86307.  Position: 86335.  Data Length: 172642
juravlikk pushed a commit to juravlikk/python-mysql-replication that referenced this issue Dec 13, 2019
…onchelle#298

In case of large JSON documents leteral values should be retrieved with
`read_uint32` instead of `read_uint32` to avoid further AssertionError
@juravlikk
Copy link

Got the same and described into the #226 : #226 (comment)

baloo pushed a commit that referenced this issue Dec 14, 2019
In case of large JSON documents leteral values should be retrieved with
`read_uint32` instead of `read_uint32` to avoid further AssertionError
@baloo baloo closed this as completed Dec 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants