Enabling Player-Created Online Worlds with Grid Computing and Streaming (Second Life)
This webpage documents my presentation of "Enabling Player-Created Online
Worlds with Grid Computing and Streaming" by Philip Rosedale and Cory
Ondrejka for CPSC 538A: Topics in Time-sensitive Distributed Systems,
taught by Charles 'Buck' Krasic.
Quick Links
Summary
"The hours and intensity with which players engage in games makes for very
fast consumption of content"
As players participate in a game they need to interact with new content in
order to stay interested and keep playing. Previously, game designers have
used the following ways of providing users with a reason to keep playing:
- Provide large amounts of content in the game itself (Final Fantasy)
- Make the content reusable, enhancing replay value (Gran Turismo)
- Make experience dependent on interaction with other human players (Quake)
These approaches do not work well for Massively Multiplayer Online Games (MMOG)
where players pay a monthly subscription fee to participate in the game.
Since it only takes a player around one month to consume 100 hours of content,
MMOG developers have tried to frequently update the interactive content of
their games to keep players subscribing. This approach is costly as new content must
be developed on short time scales.
This paper describes the challenges behind creating the Second Life MMOG.
Second Life makes the game players responsible for creating and modifying
in-game content. The problem that the paper addresses is: How can we scale
an online world to support tens of thousands of players and hundreds of
thousands of objects at one time such that all of the players have a consistent
view of the world around them, and can interact with each other by modifying the
world collaboratively in real-time?
The authors address the above problem by using two techniques. First, the
authors divide the world up into square tiles of 16 acres in area. Each tile
represents a simulation (or 'sim') running on a single server that simulates
the physics, manages all objects, behaviours, terrain and player actions in
the tile. Each sim is targeted to manage around
ten thousand objects. As players and objects move around the world their
in-game representations are transfered from server to server as they cross
tile boundaries. Servers only communicate with their four neighbouring servers.
This solves the scalability problem for Second Life; in order to grow the world
the developer only need to add more servers to the grid.
In order to solve the problem of distributing in-game content to the players
in real time, the authors decide to use streaming. Players play the game using
a small (~10MB) 'world viewer'
application. All information about the world (audio, video, geometry, textures) is
streamed to the viewer application; none of the content is actually stored on the
players machine. As players move around the grid they maintain streaming connections
only to the servers that they are near.
All data is sent over multiple UDP streams to
allow for a more controllable stream than TCP. Reliability of a stream
is built around loss detection and data correction instead of retransmission
to allow for timely delivery of content. The authors decided to cap streaming
bandwidth at 100kbps per client in order to avoid network congestion.
However, there is a lot of game state that needs to be transmitted to
clients at any one time. To fit all of the data into a 100kbps pipe,
Second Life uses wavelet compression on textures, ogg vorbis compression
on audio, and efficient compression of game geometry because of its simplicity
(avoiding generalized meshes). In addition, players only receive information that
is new to the clients (or has changed).
The main contributions of this paper are:
- The use of a grid of simulators to solve the problem of scale when users are
allowed to create and modify objects in a large online world.
- Streaming data to game players over the internet in real-time allows
users to modify the online world interactively and collaboratively with other players.
The Second Life world viewer can be downloaded from
http://secondlife.com/download/. It can
be played for free for the first 7 days.
Presentation Slides
The slides for my presentation (in ppt format) can be found
here.
Group Discussion
Questions about the paper:
- How does the game deal with diagonal movement? The game world is
separated into square tiles. Each tile represents 16 acres of landscape;
each tile is managed by a separate server. It is easy to see how player
information is handed off from one server to another when a player moves
horizontally or vertically across the world. The player maintains two
streaming connections (one to each server) as he/she crosses the tile
boundary. But suppose a player moves diagonally, perhaps walking across
a corner of a tile instead of an edge. A corner of one tile is close to four
different tiles. How does the game handle such a situation?
I believe that the player will maintain four streaming connections
(one to each server) as he/she walks across the corners of the four tiles.
This does not seem like a huge deal, but it did lead to an interesting
discussion about dividing up worlds into tiles. Perhaps square tiles are
not optimal for such a free roaming game. Maybe hexagonal tiles would be
a better fit.
- Compression is expensive. How did the authors of the game balance
time and space complexity when using compression? In other words, how did
they balance how effecient a compression algorithm is versus how long said
algorithm takes to run? Did they make specific choices of compression
algorithms to keep the real-time elements of the game?
For the most part the game designers used brute force. Essentially, they
throw computing power at the problem by requiring client computers
to have the following requirements:
- Graphics Cards: Nvidia Geforce 2 (32MB RAM) or higher, or
ATI Radeon 8500 (32MB RAM) or higher.
- Computer: 800MHZ or higher, 256MB RAM or more.
With so much computing power, the time it takes to efficiently compress
what they need is small enough not to matter.
- How far ahead can you see in the game? Can you see through
tile boundaries (ie. across server boundaries)?
Yes, you can. The
developers tried to make the world as seamless as possible; the
tile/server boundaries should go unnoticed unnoticed by a player as they
walk over or look through them. The reason a player can
be standing in one server while looking into another is because the player
maintains two streaming connections. One connection would
stream information from the server the player is located in while the
other connection would stream information from the server the player is
looking into.
- It was noted that a lot of the problems mentioned above go away if
you strategically put up walls across some of the tile boundaries. I
am not sure if this is done very often in Second Life, but I do know that one of
the goals cited by the developers was to make the world as free roaming
as possible. Erecting walls between areas would go against this goal.
- How does the grid deal with the failure of a machine? Does the
16 acres of land that was simulated by the failed server become unavailable?
Do neighbouring servers contain enough redundent information to fill in
for the failed server?
The paper does not discuss failure. It would seem that the inter-server
communication present in Second Life has one purpose: dealing with object and
player mobility. I do not believe that the servers exchange information
for the purpose of failure recovery and robustness. It is much more likely
that the area being simulated becomes unavailable while a replacement server
is added to the grid. It is possible that there are backup machines ready to
be swapped in quickly in the event of a failure.
- How do the landscape designers distribute the load across the world? In
other words, how can they be sure that one area is just as interesting as
another so that everyone won't crowd into a single area and overload the
server managing said area?
It is possible that there may be a hard limit on the number of objects
that can exist in one area at any one time. It is also possible that there
may be a hard limit on the number of players that can play in one area at any
one time.
- How realistic is it to push the task of generating content onto the users of the game?
First, the content generation tools provided with the game must be of high quality
and easy enough for any player to use. This is a difficult thing to do. More importantly, how
many users are going to be interested in authoring content? The content in
Second Life can be audio, geometry, textures, or behaviour (via scripts). Is the
average game player going to be interested in the time consuming task of creating
such things when they could be playing the game? This is an open question.
Discussion of future work:
In what ways could the developers improve the game?
- A TCP-friendly adaptive congestion control algorithm could be used instead of
simply capping client bandwidth at 100kbps. This could allow the game to make better
use of the clients available bandwidth, hopefully allowing for more detailed content
to be streamed such as more detailed graphics.
- Currently the game uses multiple UDP streams to send data from servers to clients.
Firewalls between the server and clients cause problems for game players because most
firewalls drop incoming UDP traffic by default. Perhaps the developers could work
towards supporting TCP so that clients will be able to play from behind firewalls.
Links
Daniel Ferstay
Last modified: Sun Mar 28 18:18:43 PST 2004