Subscribe to our Erlang Factory newsletter to receive the latest updates and news

Guanhua Ye
Developer, Architect @ TigerText

Developer, architect on real-time messaging system at TigerText Inc. Senior engineer with Yahoo for 5+ years, a veteran of Yahoo Messenger, interested in real-time communication, distributed system, and software design and architecture,
PhD in E.E, City University of New York.

Guanhua Ye is Giving the Following Talks
Distributed Producer and Consumer Framework with Guaranteed Message Delivery

Consumer error, crash or down is a classic tech problem for applications involves producer and consumer. When guaranteed message delivery is a mandatory at consumer side, it is essential to have a general architecture or framework in place thus free the consumer of complicated logic such as error recovery or message re-delivery. 

In this talk, we present a distributed framework which guarantees message delivery for producer/consumer applications. Producer and consumer can be distributed across network, no shared memory or state is needed, which fits Erlang process
model very well. This framework suites well for applications/tasks that can be abstracted using producer/consumer model or chained

A general guaranteed message delivery framework is built around open source no sql db redis. The framework has a redelivery mechanism, which redelivers message at configurable interval. Consumer feedback is essential in the control loop of the framework, which is used to remove message from retry or schedule retry at alternative pace. With the redeliver mechanism of the framework, consumer code does not need to worry about how to recover from a crash or downstream error while processing message/event. It can simply ignore or log an error and the system will put the same message/event in the consumer queue for redelivery at certain time later.

With building blocks, an Erlang lib for producer/consumer, we believe the system can benefit the community and developers for easily building system or applications that provides reliable message delivery.

Talk objectives: This distributed, reliable message delivery system has been working great for us. We'd like to share design ideas with the community and hopefully open source the whole system.

Target audience: Developers who build application/system that involves producer and consumer. And those who are interested in reliable message delivery.