How to parse a specific/min/max number of elements of a CharacterSet?

[Jeehut]

I found that CharacterSet conforms to Parser, but for some reason it seems to be built as a "any number of characters from the character set" parser, here's it's current implementation (source):

import Foundation

extension CharacterSet: Parser {
  @inlinable
  public func parse(_ input: inout Substring) -> Substring {
    let output = input.unicodeScalars.prefix(while: self.contains)
    input.unicodeScalars.removeFirst(output.count)
    return Substring(output)
  }
}

Wouldn't an extra struct that accepts arguments similar to Many (such as atLeast and atMost) make this more useful? Comparing this to regular expressions, where I can specify [abc]* but also [abc]{3,} it seems the library is less powerful. Or is there a way I can use Many and CharacterSet combined to build such a parser?

[Jeehut]

I worked around this limitation by implementing my own Characters parser like this:

public struct Characters: Parser {
    struct MinimumNotReachedError: Error {}

    let characterSet: CharacterSet
    let minimum: Int
    let maximum: Int

    public init(
      in characterSet: CharacterSet,
      atLeast minimum: Int = 0,
      atMost maximum: Int = .max
    ) {
      self.characterSet = characterSet
      self.minimum = minimum
      self.maximum = maximum
    }

    public func parse(_ input: inout Substring) throws -> Substring {
      let original = input
      var output: Substring.UnicodeScalarView = .init()

      while output.count < maximum, !input.unicodeScalars.isEmpty, self.characterSet.contains(input.unicodeScalars.first!) {
        output.append(input.unicodeScalars.removeFirst())
      }

      guard output.count >= minimum else {
        input = original
        throw MinimumNotReachedError()
      }

      return Substring(output)
    }
  }

Usage is something like this:

var input = "zh-Hans.lproj"
let locale = try Parse {
  Characters(in: CharacterSet(charactersIn: "AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz-"), atLeast: 2)
  ".lproj"
  End()
}.parse(&input)

If this is considered useful, I can post a PR to incude this into the library directly (or feel free to copy & paste).

[mbrandonw]

Having CharacterSet conform to Parser directly means it's very lightweight to use but also that it's not customizable. If you need deeper customization then I think you have to create a custom parser type like you have done.

You can also technically do this with Many and First, but it's a little messy:

try Many(atLeast: 2) {
  First<Substring.UnicodeScalarView>()
    .filter(CharacterSet.alphanumerics.contains)
}
.parse("ABC".unicodeScalars)

At one point we sketched out how to really deeply integrate the ideas of min/max into parsers so that you could chain on .mininum(x) and .maximum(y) anywhere in a parser chain and things like Many, Prefix, CharacterSet, etc. could pick up those arguments automatically. This was pretty powerful, but unfortunately hurt performance too much. It was designed using task local values in a similar style to SwiftUI's environment, and reading from task locals isn't too big of a hit, but writing to task locals is very slow (comparatively).

[Jeehut]

@mbrandonw Thank you for the quick & detailed answer. I get it now, I think I'll keep my custom parser then.

On the same note, it sounded like you are not interested in adding this custom parser into the library to keep it small. I am interested in exploring the area of regular expression to replace them by a more readable, declarative syntax based on swift-parsing. If I understand you right, you are (currently) not interested in such a feature added to the library directly, right? See also #153.

Because if you don't want this area explored within the library, I will probably just create a new library based on swift-parsing where I can then add the missing parts (like the above Characters parser and my matches extension from the other thread).

Also, I'm sure others will create other libs based on this (or already have?), so maybe you want to also consider showcasing libs based on this in the README (or somewhere else).

[mbrandonw]

Hi @Jeehut, I think at this time we are going to take it slow with adding new parsers, especially since we have a couple of big things on the horizon (printers, new result builder features, and some other stuff we are experimenting with), so don't want to accidentally back ourselves into a corner.

We think it would definitely be cool if you put some of your useful parsers in an external library, and we would happily create a section of the readme to link to other people's projects. 🙂

[Jeehut]

I can totally understand. Sounds good, I'll make sure to inform you when I had the time to tackle that project! 👍

[alexito4]

As a side note on this, I found the documentation on this parser a bit lacking. From the description I didn't get that it just tries to consume the characters, if no character match it just doesn't do anything, but it doesn't throw an error (which for some reason is what I was expecting)

Maybe is worth to improve the documentation overall indicating a bit more specifically what the behaviours in terms of erroring, doing a noop or backtracking.

[Jeehut]

@alexito4 You want to post a PR improving the docs on CharacterSet? 😁

I'm pretty sure both Brandon & Stephen are doing their best to keep up with things, but after all there's a reason it's open source, right? While I asked for if they want me to add new types, I'm pretty sure they won't have any objections against PRs that improve documentation. So just go for it! 👍

[alexito4]

Oh don't get me wrong, it was not a "i want you to work for me for free" type of request, apologies if my tone implied that.
I was just probing the waters to see if anyone else agreed, because it could totally be that I'm the only one being confused with this 😂

[Jeehut]

@alexito4 If something confuses you, you can expect others will be confused, too. Adding more perspectives and explicitness to documentation is nearly almost a good idea. I didn't see your comment as "trying to dump work on others", don't worry about that. I was just trying to nudge you to just do the documentation improvement instead of discussing it. 😄