What does HackerNews think of conduit?

A streaming data library

Language: Haskell

OP is somewhat conflating two different things: non-strict function evaluation and lazy IO. With lazy IO, you can get, for example, a String from a file. That string is actually a lazily-constructed chain of Cons cells, so if you're into following linked lists and processing files one char at a time, then it's fun to use. The dangerous bit comes in when you close the file after evaluating its contents as a string:

    fd <- open "/some/path"
    s <- readContentsLazy fd
    close fd
    pure $ processString s
Now, processString is getting a string with the file's contents, right? Nope, you have a cons cell that probably contains the first character of the file, and maybe even a few more up to the first page that got read from disk, but eventually as you're processing that string, you'll hit a point where your pure string processing actually tries to do IO on the file that isn't open anymore, and your perfect sane and pure string processing code will throw an exception. So, that's gross.

That's a real issue that will hit beginners. There's been a lot of work done to make ergonomic and performant libraries that handle this without issues; I think that right now pipes[0] and conduit[1] are the big ones, but it's a space that people like to play with.

[0] - https://hackage.haskell.org/package/pipes [1] - https://github.com/snoyberg/conduit

Reminds me of a stream-processing library that Michael Snoyman recently started working on:

https://github.com/snoyberg/zerem

For background, he authored Conduit, one of the most popular Haskell stream-processing libraries:

https://github.com/snoyberg/conduit

Working with streams of data often makes pointfree syntax beneficial, since you can start to see the omitted arguments "flow" through your chain of composed functions. If you write

   f # g = \x -> g (f x)
then you could implement functions like this:

   f :: Stream Int -> Stream Int
   f = drop 3       -- drop the first three elements
     # skipEvery 3  -- skip every third element
     # sumEvery 2   -- sum every consecutive pair of elements
     # map (* 2)    -- double everything
This would turn

   1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, ...
into

   2 * (4 + 5), 2 * (7 + 8), 2 * (10 + 11), ...
You can find many non-pointfree versions of similar streaming code here: https://github.com/snoyberg/conduit/