Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feature
backtrack n
is a parser that rewinds the input byn
bytes, or fails if not enough client-uncommitted bytes are available. It is sorted with the expert, undocumented parser since it requires a good understanding of the input to be used safely.I do understand that this feature may seem dangerous, yet I think it is legitimate and useful in some contexts (see rationale below). I would definitely understand if it's deemed too tricky to integrate upstream, I'm fine using our pinned version.
Rationale
We use
Angstrom.scan
to consume UTF-8 input. Thus, we must sometime read multiple bytes to get an actual unicode character. If we then try to reject that character by returningNone
, only the last byte is pushed back to the input, losing some information and probably making the input malformed UTF-8.backtrack
enables to roll back a few additional bytes to the start of the actual UTF-8 character. It is safe to use in such a case since we know there are at least that many uncommitted bytes to rollback.