As always I want to paint a picture, if you look back at my previous article I am not a veteran of null neither do I have any desire to be. I have neither the skills nor the desire to own capitals beyond my beloved Dreadnaughts so this is almost an outsider’s commentary more than anything else. Now, in terms of technology, I recently graduated with a degree in computer networking and as a result, find myself being able to provide a slightly educated commentary on the outstanding issues that curtailed the highly anticipated titan battle.
Obviously, following the battle of M2, significant questions were asked of the server. I read over the dev blog by CCP themselves elaborating on the issues that occurred and one statistic that stood out for me was that 35% of the players present in the game were within 3 nodes of the server. A quick analysis of the existing infrastructure to carry the weight of all of New Eden’s pilots is what is referred to as a shard configuration.
What is a shard server I hear you ask? Put simply, a shard is a collection of databases linked together which virtually presents as one single source. This allows CCP to place all in data sets on various SQL databases whether it be market data or solar system information. You may have heard the word node being mentioned on numerous occasions, the nodes are the links between databases.
This brings us to the first issue and ties in with what happened with the server. Taking one chunk of the article “The second fight in M2-XFE was the next step in the evolution of these unprecedented fights. Three systems in Delve were the focus of the fight: 1DQ1-A (defender staging), T5ZI-S (attacker staging), and M2-XFE (battlefield). 1DQ1-A’s player count topped out at 4226, T5ZI-S hit 6723 and the player count in the target system of M2-XFE went the highest with 6739 pilots, breaking the Fury of FWST record of 6557. The three systems – which were each reinforced – saw a combined peak of 13,770 players around 23:23 EVE time. To put this in perspective, approximately 35% of all online pilots in the game were situated in just these 3 systems.” Cutting away the toxicity and general frustration, the main thread of discussion was what else exactly can we ask for?
Alongside this configuration, we have to bring into account the client-side of things. Combining the age of eve, alongside the fact that the previously mentioned shard setup means that the coding aspect has to be developed by hand, as a result, there are parts of solar systems that are now of the legal age to drink. The significant downside to that this code was not optimized to capitalize on the hardware at our disposal, multithreaded cores were futuristic at the time of development.
Strangely enough, I took a bit of time to listen to Trash Talk Tuesday, the well known Grath Telkin made an appearance expressing, in his typical brash and straight to the point manner, pointing out how following the meta of out-forming each other and trying to cram all these people in one place, what did we expect to happen? Just don’t ask him to work in customer service.
Another factor we have to bring into account when discussing this subject is the hardware performance, an aspect of the shard configuration means that when a call is performed, the server has to let all other ships in the system know about this. So the formula as mentioned on a stream by CCP recently is N squared. The N is the number of players within the solar system. So adding to dev-blog published we are asking the server to accommodate 189,612,900 calls at 23:23. This isn’t including various other parts of the “meat grinder” whether it be fighters, drones, and the likes.
I did note some commentary about the technology available to us, World of Warcraft being mentioned. Another quote I found (and this is significantly out of date): “Apparently, in order for you to be able to log in whenever you want, it takes roughly 20,000 computer systems, over a petabyte of storage, and over 4600 people. Using multiple data centers around the world, this works out to a total of 13,250 server blades, 75,000 CPU cores, and 112.5 terabytes of blade RAM.“ This is a quote from 2009, also bringing into account Activision/Blizzard is a multi-billion dollar corporation that enjoys the income of game franchises to fund said hardware. Being one who played World of Warcraft for a long time many moons I am fully aware that there are hundreds of servers to accommodate so many players.
I would encourage everyone to take a look at the server discussion that took place on the CCP Twitch channel, the wonderful CCP Aurora made a point of getting the link republished as a source of information to have a better understanding of what happened during that fight, and why it happened and what we have available.