Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

ValueError: malformed node or string #368

Open
sobolevn opened this issue Jun 9, 2019 · 9 comments · May be fixed by #381
Open

ValueError: malformed node or string #368

sobolevn opened this issue Jun 9, 2019 · 9 comments · May be fixed by #381

Comments

@sobolevn
Copy link
Member

sobolevn commented Jun 9, 2019

Hi! I am running pydocstyle as a part of flake8-docstrings plugin which is a plugin for wemake-python-styleguide.

Original issue: wemake-services/wemake-python-styleguide#579

One of my users reported this bug:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/checker.py", line 669, in _run_checks
return checker.run_checks()
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/checker.py", line 608, in run_checks
self.run_ast_checks()
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/checker.py", line 504, in run_ast_checks
for (line_number, offset, text, check) in runner:
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8_docstrings.py", line 89, in run
for error in self._check_source():
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8_docstrings.py", line 78, in _check_source
ignore_decorators=None,
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/pydocstyle/checker.py", line 75, in check_source
definition.docstring)
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/pydocstyle/checker.py", line 114, in check_docstring_missing
docstring and is_blank(ast.literal_eval(docstring))):
File "/usr/lib/python3.7/ast.py", line 91, in literal_eval
return _convert(node_or_string)
File "/usr/lib/python3.7/ast.py", line 90, in _convert
return _convert_signed_num(node)
File "/usr/lib/python3.7/ast.py", line 63, in _convert_signed_num
return _convert_num(node)
File "/usr/lib/python3.7/ast.py", line 55, in _convert_num
raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.JoinedStr object at 0x7f1d8db27358>
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/ilya/.virtualenvs/ds-backend/bin/flake8", line 10, in 
sys.exit(main())
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/main/cli.py", line 18, in main
app.run(argv)
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/main/application.py", line 394, in run
self._run(argv)
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/main/application.py", line 382, in _run
self.run_checks()
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/main/application.py", line 301, in run_checks
self.file_checker_manager.run()
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/checker.py", line 328, in run
self.run_parallel()
File "/home/ilya/.virtualenvs/ds-backend/lib/python3.7/site-packages/flake8/checker.py", line 292, in run_parallel
for ret in pool_map:
File "/usr/lib/python3.7/multiprocessing/pool.py", line 354, in 
return (item for chunk in result for item in chunk)
File "/usr/lib/python3.7/multiprocessing/pool.py", line 748, in next
raise value
ValueError: malformed node or string: <_ast.JoinedStr object at 0x7f1d8db27358>

This happens here: https://github.com/PyCQA/pydocstyle/blob/master/src/pydocstyle/checker.py#L159

Reproduction code:

def function():
    f"""Template #{locals}"""
    ...
@sobolevn
Copy link
Member Author

I would say that something like this should be done:

def safe_literal_eval(node) -> Optional[str]:
     try:
         return ast.literal_eval(node)
     except ValueError:
         return None  # happens when literal eval cannot process node

And then check that instead of ast.literal_eval this function is used and None value is checked.

@lordmauve
Copy link
Contributor

Docstrings may not be f-strings. So actually the better fix is to specifically look for attempting to use a docstring as an f-string and issue a new error code.

@sobolevn
Copy link
Member Author

@lordmauve agreed, new rules are always a good idea 🙂

@lordmauve lordmauve linked a pull request Jul 17, 2019 that will close this issue
Open
@sobolevn sobolevn mentioned this issue Jul 28, 2019
Open
@bra-fsn
Copy link

bra-fsn commented Aug 13, 2020

Any progress on this? It's quite puzzling to run into this, because there's no indication on what/where the error is, just a big fat python backtrace.

@s-weigand
Copy link

s-weigand commented Sep 1, 2020 •
edited
Loading

Just came across this error, since we autogenerate some docstrings with f-strings using a decorator.
I could resolve the issue in our case by using --ignore-decorators.

pydocstyle <source_dir> --ignore-decorators=<name_of_decorator_gerating_docstring>

@s-weigand s-weigand mentioned this issue Sep 1, 2020
Merged
@sambhav
Copy link
Member

sambhav commented Sep 1, 2020

Any progress on this? It's quite puzzling to run into this, because there's no indication on what/where the error is, just a big fat python backtrace.

I plan to pick it up and release a fix this weekend :)

@sambhav
Copy link
Member

sambhav commented Sep 6, 2020

I have #381 updated. Waiting for a review so that we can merge it.

@leungyukshing
Copy link

Is this bug fixed?

@vlad0337187
Copy link

still present

@davemfish davemfish mentioned this issue Sep 19, 2023
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add error D303 when an f-string is used as a docstring lordmauve/pydocstyle
7 participants
@lordmauve @bra-fsn @sobolevn @s-weigand @vlad0337187 @sambhav @leungyukshing