How can I go around this Infinite loop ?

[mackoj]

Hi,

I'm trying to parse console output and extract the text attributes. For a simple line, it's correct but when I have multiple attributes it doesn't work properly in all cases.

The issue is with PrefixUpTo it does not seems to work with a parser junebash raise a similar issue recently. I do understand the performance implication but I didn't find how to do it differently. Maybe some of you have an idea?

BTW I did find another issue with the library, when I try to parse text that contains invisible characters like \u{1B} the error show ^ expected input but the issue is that it points to a bad character since \u{1B} is invisible maybe we should say in the expectation that the pointed character is invisible in this case using .unicodeScalars.contains(where: $0.generalCategory == .control ) for example.

Thanks,

A color is like this [32;1m is parsed with the color parser.

let terminalAttribute = Parse(TerminalColor.init) {
  "["
  Int.parser()
  Optionally {
    ";"
    Int.parser()
  }
  "m"
}

In one line of the log, there are multiple sections of text with optional text attributes and what I expect at the end is a [ParsedLog] for each line of the log.

public struct ParsedLog: CustomStringConvertible {
  var text: String
  var color: TerminalColor?
}

Console log example(without all the invisible characters)

Nodename:	LAS1-vFW01-gen2-mmg4-10-92-100-101
[32;1m  version: 1.49.0 [0m
[32;1m  version: 1.49.0 [34;1mfoo:[0m bar
[36mINFO[0m[08:45:33] [33;1mci runs in Secret Filtering mode[0m 
[33;1maaa[33;1mbbb[33;1mccc[33;1mddd
asdasdasd[33;1maaa[33;1mbbb[33;1mccc[33;1mddd
[08:45:33]asdasdasd[33;1maaa[0m bar

Output

input: Nodename:  LAS1-vFW01-gen2-mmg4-10-92-100-101
---
Output
text: Nodename:  LAS1-vFW01-gen2-mmg4-10-92-100-101
-------
input: [32;1m  version: 1.49.0 [0m
---
Output
text:   version: 1.49.0 , attributes: color: green, modifier: Optional(ConsoleParser.TerminalTextCode.bold)
-------
input: [32;1m  version: 1.49.0 [34;1mfoo:[0m bar
---
Output
text:   version: 1.49.0 , attributes: color: green, modifier: Optional(ConsoleParser.TerminalTextCode.bold)
text: foo:, attributes: color: blue, modifier: Optional(ConsoleParser.TerminalTextCode.bold)
text:  bar
-------
input: [36mINFO[0m[08:45:33] [33;1mci runs in Secret Filtering mode[0m
---
error: infinite loop
 --> input:1:12
1 | [36mINFO[0m[08:45:33] [33;1mci runs in Secret Filtering mode[0m
  |            ^ expected input to be consumed
Program ended with exit code: 0

The full exemple.

The issue is in textWithoutAttributes the PrefixUpTo("[") is making a infinite loop on the time [08:45:33]. Instead of PrefixUpTo("[") having PrefixUpTo(terminalAttribute) could fix it.

import Foundation
import Parsing

public struct ParsedLog: CustomStringConvertible {
  var text: String
  var color: TerminalColor?
  
  public var description: String {
    var out = "text: \(text)"
    if let color = color {
      out += ", attributes: \(color.description)"
    }
    return out
  }
  
  public init(string: String, color: TerminalColor? = nil) {
    self.text = string
    self.color = color
  }
  
  public init(substring: Substring, color: TerminalColor? = nil) {
    self.text = String(substring)
    self.color = color
  }
}

public let mainParser = Many {
  OneOf {
    textWithAttributes
    textWithoutAttributes
  }
}

// MARK: Parser Internal
let textWithAttributes = Parse {
  terminalAttribute
  PrefixUpTo("[").map(String.init)
  Optionally { endColor }
}.map { ParsedLog(string: $0.1, color: $0.0) }

let textWithoutAttributes = Parse {
  Not { terminalAttribute }
  OneOf {
    PrefixUpTo("[").map(String.init)
    Rest().map(String.init)
  }
}.map { ParsedLog(string: $0) }

let terminalAttribute = Parse(TerminalColor.init) {
  "["
  Int.parser()
  Optionally {
    ";"
    Int.parser()
  }
  "m"
}

let endColor = Skip {
  "[0m"
}

[mackoj]

After more investigation I did find a way around it. I did remplace PrefixUpTo("[").map(String.init) with PrefixUntil { startTerminalTextFormatting } .map(String.init).

I did this PrefixUntil to make things works. How do you think I can improve it ?

import Foundation
import Parsing

extension String: Error {}

public struct PrefixUntil<Input: Collection, Upstream: Parser>: Parser where Input.SubSequence == Input, Upstream.Input == Input {
  
  public let upstream: Upstream
  
  @inlinable
  public init(@ParserBuilder _ build: () -> Upstream) {
    self.upstream = build()
  }
  
  @inlinable
  public func parse(_ input: inout Input) throws -> Input {
    guard !input.isEmpty else { throw "a non-empty input(\(input))" }
    
    let originalInput = input
    var lastIndex = input.startIndex
    var out = try? self.upstream.parse(&input)
    while out == nil && lastIndex != input.endIndex {
      input.removeFirst()
      lastIndex = input.startIndex
      out = try? self.upstream.parse(&input)
    }
    return originalInput[..<lastIndex]
  }
}

extension PrefixUntil where Input == Substring {
  
  @_disfavoredOverload
  @inlinable
  public init(@ParserBuilder _ build: () -> Upstream) {
    self.upstream = build()
  }
}

extension PrefixUntil where Input == Substring.UTF8View {
  
  @_disfavoredOverload
  @inlinable
  public init(@ParserBuilder _ build: () -> Upstream) {
    self.upstream = build()
  }
}

[mackoj]

The implementation above is pretty buggy you have been warned.

[mackoj]

I'm still progressing on this...

I think I'm almost done but I still have issues 😭

[mackoj]

I'm having a infinite loop with this refined version. 🤔

I was hoping that the End stop the recursion without success so far...

error: infinite loop
 --> input:1:64
1 | …et Filtering mode[0m
  |                      ^ expected input to be consumed
Program ended with exit code: 0

import Foundation
import Parsing

// input: [36mINFO[0m[08:45:33] [33;1mci runs in Secret Filtering mode[0m
public let mainParser = Many(1...) {
  textWithAttributes
}

// MARK: Parser Internal
let textWithAttributes = Parse(ParsedLog.init(color:string:)) {
  Optionally { terminalFontModifier }
  OneOf {
    End().map { Optional<String>.none }
    Optionally { PrefixUntil { terminalFontModifier } .map(String.init) }
  }
}

let terminalFontModifier = Parse(TerminalAttribute.init) {
  "["
  Int.parser()
  Optionally {
    ";"
    Int.parser()
  }
  "m"
}

[mackoj]

I just have to add Peek { Prefix<Substring>(1) } before the other optional parser in order to be sure that the input wasn't empty.

[mackoj]

I have open a PR to upstream this parser fixed and faster #229