A new OTP behaviour named 'gen_stream' is proposed for R15.  It features a simple interface for consuming serial streams which reside on slow media, are generated computationally or may be infinite in length.  Internally, the stream may be prefetched and/or generated concurrently, allowing the application implementor the ability to declaratively define the concurrency characteristics, improving the performance characteristics of the data source.

The inspiration for gen_stream came from Tim Bray's Widefinder challenge and the ensuing discussions and submissions of the Erlang mailing list members in attempting to efficiently handle text file I/O and analysis.

Target audience: Anyone who has complained of slow serial I/O performance, was involved in or observed the WideFinder competition, or needs to efficiently generate or simulate infinite data streams or transform serial data on the fly.

Talk Objectives: Introduce the API and implementation of the gen_stream behaviour and its approach of prefetching data buffers by striping across processes to leverage overlapped I/O.