Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function for splitting conduit by sequence and not element #457

Open
Termina1 opened this issue Dec 4, 2020 · 1 comment
Open

Function for splitting conduit by sequence and not element #457

Termina1 opened this issue Dec 4, 2020 · 1 comment

Comments

@Termina1
Copy link

Termina1 commented Dec 4, 2020

Hi,
there is a function called splitOnUnboundedE, which splits conduit by some element of sequence, that this conduit contains.
Is there any way to split not by an element, but using a sequence? I stumbled upon it trying to do something similar to linesUnbounded, but I need so split file using sequence of double newline symbols.

Is it possible now with any existing combinator or if not, would such a function be useful for others to be in combinators library?

In case it is, I tried to modify splitOnUnboundedE, there are some inefficiencies, but is the idea may be useful?

splitOnSeq :: (Monad m, Seq.IsSequence seq, Eq (Element seq)) => seq -> ConduitT seq seq m ()
splitOnSeq f =
    start
  where
    start = await >>= maybe (return ()) (loop id)

    loop bldr t =
        if onull y
            then do
                mt <- await
                case mt of
                    Nothing -> let finalChunk = mconcat $ bldr [t]
                               in  unless (onull finalChunk) $ yield finalChunk
                    Just t' -> loop (bldr . (t:)) t'
            else yield (mconcat $ bldr [x]) >> loop id y
      where
        (x, y) = let (x : z) = Seq.splitSeq f t in
          (x, mconcat $ intersperse f z)
@snoyberg
Copy link
Owner

snoyberg commented Dec 5, 2020

I don't think it's possible. Generally this is a non-trivial problem to solve. Not impossible or crazy, just that doing this with good performance requires some thinking. I had to do something like that for multipart form data processing in wai-extra IIRC.

Overall, I'd be in favor of some kind of an addition that does this efficiently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants