Subscribe to our Erlang Factory newsletter to receive the latest updates and news

Jay Nelson
Concurrently Serial Erlangist
DuoMark International, Inc

Jay Nelson has been working with Erlang under the guise of DuoMark International, Inc. since 2002 on various projects involving web-based casual gaming, backend social servers for iPhone apps and data aggregation using CouchDB in the cloud.

Jay's Website

Twitter: @duomark

Jay Nelson is Giving the Following Talks
Process-Striped Buffering with gen_stream

A new OTP behaviour named 'gen_stream' is proposed for R15.  It features a simple interface for consuming serial streams which reside on slow media, are generated computationally or may be infinite in length.  Internally, the stream may be prefetched and/or generated concurrently, allowing the application implementor the ability to declaratively define the concurrency characteristics, improving the performance characteristics of the data source.

The inspiration for gen_stream came from Tim Bray's Widefinder challenge and the ensuing discussions and submissions of the Erlang mailing list members in attempting to efficiently handle text file I/O and analysis.

Target audience: Anyone who has complained of slow serial I/O performance, was involved in or observed the WideFinder competition, or needs to efficiently generate or simulate infinite data streams or transform serial data on the fly.

Talk Objectives: Introduce the API and implementation of the gen_stream behaviour and its approach of prefetching data buffers by striping across processes to leverage overlapped I/O.