Maestro - Orchestrating Large Scale Multiplayer Games

This talk is about our journey of going from monolithic game servers to component based game services. We will present the Maestro Platform that consists in a multi-component architecture, developed in Erlang, which aims to orchestrate an infrastructure of game servers. When faced with the requirement of building large scale real-time games, and considered our experiences with our past products, we came to the inevitable conclusion that we had to rethink the architecture of our game servers and the services to support them. That’s when the Maestro platform was born as a central piece development and deployment of new games (mobile and web).

The main goal of Maestro is to accommodate all our knowledge and technologies as pluggable components that can be combined based on the specific requirements of each game, as well as their acceptance in the market. We started our design by guaranteeing that the platform would help us automatically scale our game server infrastructure based on rules defined by the game itself (to achieve scalability and availability). The Maestro platform will be able to launch new instances of game logic arenas/rooms and monitor their health and status, perform scaling decisions, or even recover from unexpected crashes, by relaunching these managed arenas as a erlang supervisor would do with any gen_server. 

We separated the game play logic from the service logic in a way that our client development teams can program also the server side game play logic in their preferred language (mainly C++) almost as a black box. To accomplish this we use a defined high level protobuf protocol for client and for the game server logic and also a well defined API, both of them providing extensibility points for use on a game-by-game basis. 

The services layer provides components/services to manage the business logic (IAP’s, Analytics, Accounts) and also common used features support like Leaderboards, Matchmaking, Gifting, Social Networks and new ones that can be reused in other titles. 

This approach lead us to cutting down on re-implementing solutions to simply re-use and reapply. We are able to launch a global scale infrastructure for a new game with integration included within just one week of coordination and integration between game client and backend developers. This also give us room to improve a single product and concentrate our energy in improving only one “product” but affecting all the current and future products.

Talk objectives:

• Discuss the challenges, benefits of going from a monolithic server design to a component based service architecture
• Discuss how we rely on erlang to give us the scalability, resilience and availability we need for massive multiplayer online games
• Discuss some design patterns and architecture decision we have adopted 

Target audience:

• People interested in large scale deployment of erlang software servers 
• Erlang services architects and developers 
• Game industry backend developers

Slides

About Pedro

Pedro is a backend and gameplay developer at Miniclip, whose efforts for the past year have been focused on architecting ways of improving both our services ecosystem, as well as our game building tools. He always endeavours to use the best tools for the job, so he naturally gravitated to working with Erlang for infrastructure and services and C++ for gameplay performance. A game development enthusiast at heart, Pedro is always deeply interested in working out elegant solutions that are fast and easy to work with, while at the same time allowing for great technical quality in the end product.

GitHub: Enagan

Back to conference page