Three ways to gauge Second Life’s performance
By Eric Reuters
At the Second Life Community Convention in August, Linden Lab Chief Philip Rosedale kicked off the weekend apologizing for poor performance on the Second Life Grid, pledging the company would make improving the reliability of the virtual world its top priority. Rosedale said Linden’s plan was to publish performance metrics on the Web, and he hoped the new statistics would give the company feedback to gauge its performance.
But figuring out how to measure Second Life’s performance is a tricky question, and in the weeks since Rosedale’s announcement at least three different approaches have emerged.
Tateru Nino of the Second Life Insider debuted a new metric yesterday called the Grid Stability Index. Nino believes concurrency is the best indicator of reliability and does a statistical analysis on the concurrency numbers looking for unusual jumps or drops. “What we’re looking for is sudden anomalies,” Nino said. “Having done this by eye for more than a year, I can see which lumps represent difficulties — you see them, and within five or ten minutes are getting IMs and emails from users reporting certain difficulties.”
Nino’s Grid Stability Index, which she’ll start publishing daily as part of her Today in Second Life report, uses the evenness of the concurrency numbers to generate a numerical score for Second Life’s performance. On her scale, a score of zero represents a perfectly smooth day, while on a buggy day the score can shoot over a thousand. A good day in Second Life should earn below 1.0, she said. “Anything over ten would result in some hair-pulling on my part, certainly.”
Meta Linden, who compiles monthly statistics metrics for Linden Lab, added reliability statistics to July’s report. She measures three things: Client crashes, the frames per second Second Life’s servers can handle, and the frames per second experienced on the client side by Second Life’s users. When frames per second dip, residents experience a “jaggy” effect and even simple activities like walking can become difficult.
In July over 22% of Second Life sessions ended “abnormally,” with residents averaging 12.8 frames per second of animation.
Linden Lab announced another way to measure Second Life’s performance yesterday with a post by Ian Linden introducing new service quality metrics. A page devoted to Second Life’s availability will be maintained on Second Life Grid.Net. “The green bar is the amount of usage time we lost during planned outages, such as releases,” Ian Linden told Reuters. “The red bar is the amount of time we lost outside those windows.” Adding the two bars together represents the total time lost to system-wide problems.
The difference between Ian’s numbers and Meta’s numbers, Ian said, was that he tracks Second Life’s availability — whether users can log in — while Meta tracks performance, how smooth Second Life is for those able to log on. And the numbers don’t end there. “There are more metrics to come,” Ian said. “We’re trying to find the best way to measure inventory/content problems, but that’s still pretty tentative.”
With three different ways to measure Second Life’s performance and more in the works, watchers of the virtual world will have their hands full trying to measure the success or failure of Rosedale’s pledge to improve the quality of the Second Life experience. But Linden Lab is looking forwards to the discussion. “I’m glad to see people working on stuff like this, it keeps us honest,” Ian Linden said.
