Online Player Rankings Thread

[Xorlof]

Hello everyone! We're quite pleased to offer our first public release of the player rankings that we've been using internally for a while. We have rankings for all 177 players who have played in any of the 24 events (totaling more than 1600 games!) we have run, up to and including Season 19. Here, we are just posting the top 35 rankings.

For those who are curious how this works, we are using the [ame="http://en.wikipedia.org/wiki/TrueSkill"]Trueskill[/ame] algorithm. This algorithm is based on the idea that everyone has a true playing skill level, but it's difficult to figure out what it is from game results. So, the algorithm estimates a range of what your true playing skill could be. The more you play, the tighter the range of this estimate becomes. The ranking of a player represents a conservative estimate (i.e. towards the bottom of the estimate range) of what that player's true playing skill might be.

We also use the Trueskill Through Time extension of Trueskill (pdf) to calculate rankings more accurately. This means that if you play someone who's 0-0, and then they tear off a huge winning streak, the algorithm will give you credit, in retrospect, for having played a good player.

Anyway, without further delay, here are the top 35 rankings!


EDIT:

Additionally, we now have "recent rankings", comprised of the top 20 players over the last year*:



* Actually, going back to the beginning of the latest event that was at least a year ago, so generally a little more than a year.

 

[MegaSilver]

Yeah, I was at #25 but someone took it from me...

 

[bobinchese]

Congrats to the top 15, and thanks to the OnlineHS guys for getting this up and running for us!

Are the rest of the rankings going to be posted soon, or is it going to be more of a leaderboard type thing?

 

[dok]

Yeah, I was at #25 but someone took it from me...

It was bob, not Xorlof, who took it from you. Xorlof was merely the beneficiary.

Are the rest of the rankings going to be posted soon, or is it going to be more of a leaderboard type thing?

I doubt we will ever post the full rankings... posting the bottom third would amount to shaming those players (plus a bunch of those are 0-1 or 0-2 or something similarly insignificant). We might expand the published rankings a bit from top 25, and/or post the actual means and variances of each ranked player, at some point.

A personal note: of my 17 losses, 14 of them are to players in the top 25. 9 of them are to players in the top 10. To put that another way, I'm 7-9 against the rest of the top 10, 5-5 against 11-25... and 28-3 against everyone else.

 

[vegietarian18]

Heh, I get to trashtalk my dad now and say that advanced statistics prove that I'm better than him.

Does this include the C3G events and the team event?

 

[MegaSilver]

Does this include the C3G events?

No, SLH is all by it's lonesome in that regard.

We're the red-headed stepchild compared to this do-hickey and OHS, though with dok's help we're not in the Stone Age anymore.

 

[dok]

Does this include the C3G events

Nope.

and the team event?

Yep. Took some very sophisticated implementation work by Xorlof, but yes.

Thanks to Xorlof's hard work, we are really at the cutting edge of research on ranking algorithms. Xorlof got quite a bit of help from the researchers themselves in setting this up; we supposedly have the first ever implementation of the Trueskill Through Time algorithm outside of Microsoft's internals.

 

[MegaSilver]

Thanks to Xorlof's hard work, we are really at the cutting edge of research on ranking algorithms. Xorlof got quite a bit of help from the researchers themselves in setting this up; we supposedly have the first ever implementation of the Trueskill Through Time algorithm outside of Microsoft's internals.

O_O

Wow. That's awesome!

 

[sixthflagbearer]

That is really cool. Kinda gives more incentive to win, as if you lose your first 2 games, you still will want to try to win, versus just giving up. Glad I made the top 25, time to climb my way higher, and get ready to defend my title as Gencon Team Champion.

 

[bobinchese]

Yeah, I was at #25 but someone took it from me...

It was bob, not Xorlof, who took it from you. Xorlof was merely the beneficiary.

That's payback for keeping me out of playoffs last season

 

[MegaSilver]

Yeah, I was at #25 but someone took it from me...

It was bob, not Xorlof, who took it from you. Xorlof was merely the beneficiary.

That's payback for keeping me out of playoffs last season

Which I promptly lost that too in my first game.

 

[Serenity]

Looks awesome! Is there a Trueskill numerical value given by this system? It's always interesting to see in these systems where there are big gaps between the top 5-10 and the rest.

 

[dok]

Looks awesome! Is there a Trueskill numerical value given by this system? It's always interesting to see in these systems where there are big gaps between the top 5-10 and the rest.

Trueskill gives each player's skill estimate as a mean and a variance. The ratings are a 3-sigma lower bound on a player's skill estimate. Essentially it's the algorithm saying "I'm 99.7% sure that this player is at least this good".

(This is what Xorlof was saying in that paragraph about Trueskill, in a way that was a bit less mathy.)

There is one gap in the rankings, numerically, that currently dwarfs all the others. (And no, it's not between 1 and 2. TSIYW and I are ranked ridiculously close to one another, actually, and have traded the top spot back and forth a few times just in the last month or so.)

 

[greygnarl]

One thing that would be interesting is if we could add every known tournament into this too. It would take a lot of compiling and remembering who played who, but it would be an amazing piece of work.

EDIT: Just looked and I am 2-3 in games I actually played against the top 25.

 

[Dysole]

Sweetness!

I'm in the top 25. Did not expect that. (It's probably mostly due to beating Kinseth twice ~_^) More incentive to keep trying to win. Whee!

~Dysole, now wanting to try this algorithm out for CoN but knowing that he would probably take one look at it and go "And that is what the who now?"

 

[Kinseth]

Yeah, I was at #25 but someone took it from me...

It was bob, not Xorlof, who took it from you. Xorlof was merely the beneficiary.

Are the rest of the rankings going to be posted soon, or is it going to be more of a leaderboard type thing?

I doubt we will ever post the full rankings... posting the bottom third would amount to shaming those players (plus a bunch of those are 0-1 or 0-2 or something similarly insignificant). We might expand the published rankings a bit from top 25, and/or post the actual means and variances of each ranked player, at some point.

A personal note: of my 17 losses, 14 of them are to players in the top 25. 9 of them are to players in the top 10. To put that another way, I'm 7-9 against the rest of the top 10, 5-5 against 11-25... and 28-3 against everyone else.

I tallied mine up, vs the rest of the top 10 I am 4-5, vs 11-25 I am 5-8.

26-5 vs those outside of the top 25.

In the "Modern era"(Season 4 and beyond.) I have only lost to 2 players outside of the top 25. Rexmax2 and Hivelord.

 

[Dysole]

Rankings Against...

I decided to go check myself and see what my W-L was against the non top 25ers. If we count just the tournaments (and not any other fun little stuffs I participate in). 2-5 against the top 10. 2-3 against the top 25. (My W-L is off because ISB and ISB3 were on the same team against me) 5-0 against everyone else. I feel like I end up playing the top players a lot. ~_^

~Dysole, who if he reads the standing correctly for Season 8 means it will likely be happening again

 

[kevindola]

I have the least amount of games played in the top 25. Do I get the quality over quantity award?

I am 3-1 against the top 10.
I am 6-3 against the top 25.

9 of my 11 games have been against the top 25 (2-0 against the others)

 

[C$$]

Excellent work on implementing these rankings!

If I understand correctly, each person is represented by a 'skill number' and an approximation range, and players are currently ranked by their lower bound.

I'd be curious to see the skill number and approximation ranges then too. Or are they being intentionally withheld?

Cheers,
C$$

 

[awesomeunleashed]

I have the least amount of games played in the top 25. Do I get the quality over quantity award?

I am 3-1 against the top 10.
I am 6-3 against the top 25.

9 of my 11 games have been against the top 25 (2-0 against the others)

Yes, quality over quantity is exactly what got you there.

Excellent work on implementing these rankings!

If I understand correctly, each person is represented by a 'skill number' and an approximation range, and players are currently ranked by their lower bound.

I'd be curious to see the skill number and approximation ranges then too. Or are they being intentionally withheld?

Cheers,
C$$

I suspect there's a good chance we'll release the numbers at some point.

 

[dok]

"Intentionally withheld" is maybe a little strong. We are trying to keep things simple in terms of what we're releasing, at least initially.

You can pretty much figure out where people's means and variances are, at least relative to those near them in the rankings, just by looking at the rankings. The players with more games tend to have lower variances, although the more consistent your play is, the quicker your variance drops, as well. So, for example, TSIYW has a higher mean to his ranking than I do, but I have a lower variance, which is why I am (slightly) ahead of him.

There are some dynamics in the algorithm that also govern speed of convergence. There's a variable that basically talks about how likely it is that a higher-ranked player defeats a lower-ranked player. The higher that probability is, the faster a player's variance drops (because the algorithm is more confident that each game result reflects playing skill and not just luck or matchup or whatever). We have tuned this variable based on our results so far, but we still have not done a sophisticated optimization of that convergence dynamics. That is in the works.

 

[Foudzing]

So only games wins count, but not tournament wins or wins streak if I understand well?

 

[awesomeunleashed]

So only games wins count, but not tournament wins or wins streak if I understand well?

Yes, streaks and tournament wins do not factor extra into the rankings, as those things don't really have anything to do with skill beyond the individual games.

 

[dok]

So only games wins count, but not tournament wins or wins streak if I understand well?

Win streaks and tournament wins only count insofar as the wins count, correct. Of course, in the finals of an event you're probably going to be playing a good player, and beating good players is worth more.

It's not an accident that six of our tournament winners are in the top 8, and the other two are in the top 20.

 

[dok]

Congrats to Foudzing for gaining his second championship. His win over sparkle vaults him all the way to #3 in the rankings, making him the fifth person to hold the third spot in just the past few weeks (Filthy, Wanderer, Kinseth, and Infectedsloth have all taken turns as well). Sparkle still has a comfortable hold on the #2 spot. (The 2->3 gap was a very large one before; while it has shrunk it's still the largest gap in the rankings.)