Scalable Distributed Erlang

In this talk we present Scalable Distributed (SD) Erlang - an extension of distributed Erlang functional programming language for reliable scalability. The work is a part of the RELEASE project that aims to improve the scalability of Erlang programming language. We start by providing an overview of the RELEASE project and discussing distributed Erlang limitations. Then we introduce SD Erlang, its design, motivation, and the main two components, i.e. scalable groups and semi-explicit placement. The scalable groups (s_groups) enable scaling the network of Erlang nodes by eliminating transitive connections, i.e. a node may belong to multiple s_groups where each s_group node has transitive connections with the nodes from the same s_groups and non-transitive connections with other nodes. The semi-explicit placement enables to spawn processes on nodes either in a particular s_group, or with particular attributes (e.g. available hardware or software), or with certain parameters (e.g. least load). We’ll also cover the results of the preliminary validation, and SD Erlang operational semantics and its verification. We conclude the talk by providing a brief overview of the ongoing work and future plans.

Talk objectives:

- To present the RELEASE research project that aims to improve the scalability of Erlang, introduce s_groups to the Erlang community, and get feedback.

Target audience:

- To those who plan to scale Erlang applications beyond 60 Erlang nodes;

Slides
Video

Tutorial: Scalable Distributed Erlang

In this tutorial we present and demonstrate Scalable Distributed (SD) Erlang -- an extension of Distributed Erlang functional programming language for reliable scalability. We start by discussing distributed Erlang limitations and introducing scalable groups (s_groups). The s_groups enable scaling of Erlang nodes by eliminating transitive connections, i.e. a node may belong to multiple s_groups, and each s_group node has transitive connections with the nodes from the same s_groups and non-transitive connections with other nodes. Then we refactor a distributed Erlang program into an SD Erlang program using Orbit benchmark -- a generalization of a transitive closure computation. That is we group nodes in s_groups, add gateway processes and nodes that enable message passing between worker nodes from different s_groups without a direct connection between the nodes, and introduce an additional hashing to identify the s_group where a corresponding part of the distributed hash table is stored. Then we discuss semi-explicit placement that enables to spawn processes on nodes either in a particular s_group, or with particular attributes (e.g. available hardware or software), or with certain parameters (e.g. least load). We conclude the tutorial by providing a brief overview of our future plans.

Tutorial objectives:

- To introduce s_groups and semi-explicit placement, show how they can be used, and get feedback;

Target audience:

- To those who plan to scale Erlang applications beyond 60 Erlang nodes.

REGISTER HERE

About Natalia

Natalia Chechina received a PhD degree from Heriot-Watt University, UK in 2011. She is now a Research Associate at the University of Glasgow and leads Scalable Distributed Erlang work package (WP3) in the RELEASE project. The RELEASE project (A High-Level Paradigm for Reliable Large-Scale Server Software) sponsored by an EU FP7 STREP (287510) aims to improve the scalability of Erlang programming language - aiming to scale it out to systems with a million cores (maybe 10 000 Erlang nodes). The Glasgow team of the RELEASE project works at the language level, designing Scalable Distributed Erlang to improve the scalability of Distributed Erlang. Natalia’s main research interests are distributed and parallel computing, Erlang, mathematical and theoretical analysis, stochastic modelling, and overlay networks.

 

Back to conference page