HORIZONTAL SCALABILITY

HORIZONTAL

SCALABILITY

»

INTRODUCTION

Architects for new MMO games worry about server performance. The architect has to strike a realistic balance between new cool gameplay with high player densi-tiy on one hand, and hard performance and cost limits on the other hand.

The architect must find a stable solution for the server side, while still fulfilling de-manding gameplay requirements from the game designers. This is a challenge.

By using the Pikko architecture, the si-tuation can be changed fundamentally. The server side will reach completely new levels of scalability and allow gameplay with very high player density.

Figure 1: Typical system layout when using a ver-tical scaling pattern, i. e. offloading specific core tasks to separate servers, like AI, and making the servers multithreaded.

»

THE PIKKO ARCHITECTURE

Pikko Server makes zone boundaries seam-less - thus making the need for heavy duty zones obsolete, as you can scale any zone. The game operator can add as many new game servers as needed, on the fly, without the player ever noticing that different game servers are involved during normal game-play. The load balancing between game servers is carried out dynamically so that even simple single threaded game servers suffice when Pikko makes them work to-gether. The result is horizontal scalability. Without Pikko, the architect is tempted to go with a multithreaded server architec-ture. This will scale to the capacity of your machine. That is, it scales vertically.

We deliver the Pikko architecture to make the server side horizontally scalable per zone. In addition, we have made it easier to program the server side of massive games with this approach. Distribution, load balan-cing and message passing is dynamically handled by Pikko while the actual gameplay can be designed in a single-thread environ-ment in the game engine of your choice.

»

VERTICAL ARCHITECTURE

PROBLEMS

Game servers get overloaded as the amount of traffic increases, especially in games with high player density or high object den-sity. The solution for scaling up this kind of architecture is traditionally handled in two ways, which are illustrated in figure 1:

1. Offloading the core game server with special dedicated servers for chat, login,

AI, handling client connections, persis-tence and so on.

2. Making the core game server explicitly multi-threaded, by using one main th-read for game logic, one separate for networking, serveral threads for physics, and so on.

All of these solutions work fine to a certain degree, but the offloading has a scalability limit that is reached rather quickly. For ex-ample, when offloading the game server by constructing a separate AI server, problems arise when the single threaded AI-server reaches 100 % CPU core load. Then the sca-lability problem is back. To remedy this one could offload once again, or create a multi-threaded AI-server. Then we would be back at solution number 2 and the explicit usage

of threads.

Solution number 2 is even more proble-matic. Making the core game server multi-threaded and also responsible for physics, networking, game logic scripts etc is hard, error-prone and risky. In the end this will lead to high costs for MMO producers and MMO operators. This is a subject that has made many server side game developers frustrated, stressed and stuck in time-con-suming bug tracking. See for example [1]_, [2]

and [3]_.

To summarize, a multi-threaded solution scales vertically. That is, it is limited to the capacity of the machine it runs on. In addi-tion, it is difficult and costly to develop and maintain this kind of server architecture.

Figure 2: The horizontally scaling layout of Pikko Server. This layout can have any number of connected cell servers. [1] http://gamedev.stackexchange.com/questions/7338/multi-threaded-games-best-practices-one-thread-for-logic-one-for-rendering-o [2] _{http://stackoverflow.com/questions/3834533/basic-design-of-a-multithreaded-game-server} [3] _{http://www.drdobbs.com/199200938}

3

»

THE HORIZONTAL PIKKO

ARCHITECTURE

By using the Pikko architecture on the server side, one can use a large number of identical single-threaded game servers. This is shown schematically in figure 2. The responsibility to distribute load over the many identical game servers is handled outside the game servers. The distribution logic is completely handled by the Pikko Server Application Level Gateway (ALG) and the algorithm for this distribution logic is based on the loca-tion of game objects.

The players of the game are not aware of the different servers. The player only sees a massive online game with lots and lots of game objects.

A Pikko approach makes the programming environment for game developers on the server side very appealing. Game server programmers can build the game server to run on a single thread and still get all the power of horizontal scalability. No tricky th-reads and locking in the game logic, physics engine or other subsystems. A distributed environment with many single threaded game servers has much better properties for scalability, redundancy and maintaina-bility than one using a few heavy-weight, multi-threaded servers. The server adminis-trator does not have to buy only heavy-duty machines to be able to host the game, but can find a cost-effective balance between a few strong machines and many light-weight ones. This is a freedom that few can enjoy today.

When the load on the server side is distri-buted horizontally, it is much easier to add

new game servers live, gradually remove and stop game servers live or update game servers without stopping the game. This makes the operations for the server side easier, and in the end, cheaper.

A Pikko architecture adapts dynamically to player activities. The result is a game with fewer limits for the players and at the same time, the design predictability in the game is greatly increased, which allows for games to have more freedom and even allows un-predictability.

»

CHAT AND AI

It is still possible, and sometimes very han-dy, to set up separate servers for chat, AI and other specific tasks when using Pikko Server, but this time without the pressure to do it just to offload the game servers. We suggest a different implementation for offloading AI. We suggest that AI is im-plemented and treated by Pikko Server as clients, and thus the AI also becomes hori-zontally scalable. The AI clients are hosted in the datacenter.

Text and voice chat solutions are normally trivial to offload and easily integrated into any game using Pikko on the server side. How networking scales is also an interes-ting topic within the Pikko architecture, as Pikko Server is the layer that handles the network communication with clients. Pikko Server is written in the Erlang language which enables the network layer to be put on as many cores and machines as desired. However, how Pikko scales depends in the end on how many points of synchronisation

the game is designed to have. Depending on this, the Pikko layer can be organized differently to adapt to the game characte-ristics.

»

DATABASES AND PIKKO

Any database fits the Pikko architecture. However, we believe that scalability is an important factor to consider when choo-sing a database. The database scales very well when using a modern distributed NoSQL database for high performance data like scores, inventory, persistent in-game items and destructable items. This kind of horizontally scalable database is a perfect match when using Pikko Server. We recom-mend the use of Riak, since it is based on the same technology as Pikko Server.

A horizontally scalable database can be combined with a traditional SQL data-base for important transactions with virtual goods, micro-transactions and other activi-ties with high demands for consistency and durability.

»

EASIER PROGRAMMING

MODEL

When using Pikko Server, the developer’s role changes slightly as they work with a simple API for handling game object han-dovers, boundary events and asynchronous messaging between distributed game ob-jects. Implementing this API is the small prize they pay for massive horizontal sca-lability.

The game developer needs to be aware of the situation that every game server will

only be aware of a subset of the game objects in the complete virtual world. For example, physics on the server side can only be done inside a boundary dictated by Pikko Server. When game events cross this boundary they will become function calls sent from one game server to another. The traffic between game servers is routed by Pikko Server.

This new programming model is appealing because the programmers can execute function calls locally, and Pikko Server will automatically send the call to other servers when needed. The game logic can be scrip-ted in a “fire and forget” paradigm, making it very easy to route between the game servers.

The exact API for this programming model depends on the game engine being used. An API for Unity is available, and an API for Unreal is work in progress (Feb 2011).

»

FURTHER INFORMATION

Please contact us for more details. We are happy to answer any and all questions you might have.

»

»

[email protected] +46 - 70 417 46 10