You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello.
It would be helpful for one of my usecases of baron if it supported non ascii identifiers.
Currently (in 0.6.6 but probably also in the latest master) the following code: >>> baron.parse("fňuk=42")
Raises the following:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python3\lib\site-packages\baron\baron.py", line 49, in parse
tokens = tokenize(source_code, False)
File "C:\Python3\lib\site-packages\baron\baron.py", line 70, in tokenize
return mark_indentation(inner_group(space_group(_tokenize(group(split(pouet)), print_function))))
File "C:\Python3\lib\site-packages\baron\spliter.py", line 6, in split
return list(split_generator(sequence))
File "C:\Python3\lib\site-packages\baron\spliter.py", line 76, in split_generator
raise UntreatedError("Untreated elements: %s" % iterator.rest_of_the_sequence().__repr__()[:50])
baron.spliter.UntreatedError: Untreated elements: 'ňuk=42\n'
The reason looks quite clear, during the splitting the condition block with the identifier chars uses ascii only.
It would be probably easy to fix this error without taking into account any python 2.x compatibility, but if it is desired, then it is more complicated.
But anyway, could something be done with it?
The text was updated successfully, but these errors were encountered:
Hello.
It would be helpful for one of my usecases of baron if it supported non ascii identifiers.
Currently (in 0.6.6 but probably also in the latest master) the following code:
>>> baron.parse("fňuk=42")
Raises the following:
The reason looks quite clear, during the splitting the condition block with the identifier chars uses ascii only.
It would be probably easy to fix this error without taking into account any python 2.x compatibility, but if it is desired, then it is more complicated.
But anyway, could something be done with it?
The text was updated successfully, but these errors were encountered: