Online Player Rankings Thread

[dok]

Why would that cause a substantial amount of mistakes?

I think that more often than not, players first seasons are worse than their later seasons, since they learn a lot from online play. How many people have gone 5-1 and then regressed to their mean and had multiple seasons of going 1-3 or 2-2? I'd bet that the number of people who started 5-1 and then continued at a similar winning rate is much greater.

This is a straw man. The question is not how many players started 5-1 and then had losing records going forward. The question is how many players started 3-1 or 4-1 or 5-1 and then regressed to winning 60% of their games. And the answer is, the majority.

It's true that weaker players tend to improve in later seasons, but players who start out with excellent records tend to regress towards the mean.

I am curious what happened with my dad's ranking then, if not the latter (the question was intended to be rhetorical).

Your dad's ranking didn't backslide (although we do gradually increase the variance of players who are inactive, he was not inactive for long enough for this to start to happen). I'd have to look at the individual players who passed him, but the short answer is that they had good results and the algorithm acquired more confidence in their rankings. So the conservative estimate of their ranking became higher. The people who are ahead of him are all people who have made playoff runs in recent seasons.

The algorithm's assessment of your dad is essentially "he seems like one of the very best players but we haven't seen enough results to be sure".

 

[vegietarian18]

You said 5-1, so I just went with the example in your post. I wasn't intending to turn your argument into a straw man.

"Regressing" to 60% wins isn't really the right term... 60% wins is what the #10 player, Filthy the Clown has. #6 and #7, Bengi and Kinseth are also pretty close to that. Below #11, I'm not going to calculate everyone's win percentage, but it looks like everyone with more than 20 games played has a win percentage of less than 60%.

So yeah, these players may regress towards the mean a bit, but that doesn't mean they aren't good players, if 60% is the mean they are regressing towards.

EDIT FOR YOUR EDIT: I guess I can understand that, although again, there is a disadvantage for not playing enough games. These are just rankings and just for fun, but they do advantage people who play more.

 

[dok]

Maybe some of the issue here is just the high rankings of Kinseth, Filthy, Bengi, and myself? Part of the issue there is that the four of us spent a few seasons dealing losses to each other (and you ). The algorithm doesn't really penalize us for having roughly a .500 record against each other, because an even record against someone with the same ranking doesn't hurt your ranking at all (nor should it).

In the later seasons, we haven't seen each other nearly as much, and our win percentages have all gone up. I was 16-10 in the early seasons, and I'm 29-10 since. Does that 29-10 look less out of place to you?

 

[Foudzing]

I think if new players continued to play and play a lot of games the majority of the players would reach easy Megasilver level.
And a lot of players would maintain a ~57% win ratio.
It's not a hard ratio to maintain. Over 66% it starts to be difficult to maintain yes but under this not really.

So they don't get to the top not because they're not good enough but because they don't play enough games.
And this is bad.

 

[vegietarian18]

Maybe some of the issue here is just the high rankings of Kinseth, Filthy, Bengi, and myself? Part of the issue there is that the four of us spent a few seasons dealing losses to each other (and you ). The algorithm doesn't really penalize us for having roughly a .500 record against each other, because an even record against someone with the same ranking doesn't hurt your ranking at all (nor should it).

In the later seasons, we haven't seen each other nearly as much, and our win percentages have all gone up. I was 16-10 in the early seasons, and I'm 29-10 since. Does that 29-10 look less out of place to you?

...especially since I played in the early seasons. Playing in the earlier seasons means that I was able to play against the people who are now the Top 25 a ton of times, and those wins are worth a lot now.

I think that this idea is basically correct. Other players will never have the advantage of getting a ton of games against quality opponents, so those who played in the first online seasons have a massive advantage there.

 

[dok]

Maybe some of the issue here is just the high rankings of Kinseth, Filthy, Bengi, and myself? Part of the issue there is that the four of us spent a few seasons dealing losses to each other (and you ). The algorithm doesn't really penalize us for having roughly a .500 record against each other, because an even record against someone with the same ranking doesn't hurt your ranking at all (nor should it).

In the later seasons, we haven't seen each other nearly as much, and our win percentages have all gone up. I was 16-10 in the early seasons, and I'm 29-10 since. Does that 29-10 look less out of place to you?

...especially since I played in the early seasons. Playing in the earlier seasons means that I was able to play against the people who are now the Top 25 a ton of times, and those wins are worth a lot now.

Those statements are not saying the same thing, really. The point is that my 16-10 record that's heavily weighted towards playing top players is not really any better or worse for my ranking than the subsequent 29-10 against a wider field.

I think that this idea is basically correct. Other players will never have the advantage of getting a ton of games against quality opponents, so those who played in the first online season have a massive advantage there.

No, that's not right at all. Playing good players and going .500 against them does not raise your ranking faster than playing average players and winning 60%-70% of your games. (Winning 60%-70% of your games against top players will raise your ranking faster, of course, but not any faster than winning 90% of your games against average players.) The algorithm does not reward simply losing to good players. (It will bring your variance down, but if that's all you do you will settle on a mediocre ranking.)

I think if new players continued to play and play a lot of games the majority of the players would reach easy Megasilver level.
And a lot of players would maintain a ~57% win ratio.
It's not a hard ratio to maintain. Over 66% it starts to be difficult to maintain yes but under this not really.

It seems odd to assert that most players could be above average.

So they don't get to the top not because they're not good enough but because they don't play enough games.
And this is bad.

How is this bad? The ranking algorithm exists to try to take a guess at player strength, to help us with pairings. These players have not proven themselves. If they stick around and continue to play well, they will rise in the rankings. If they don't, they won't. Either way, I don't see anything horrible about it.

 

[vegietarian18]

I don't think it was just trading wins with good players though; there were other people too. You, Kinseth, Filthy, and Bengi all had better than .500 records in the first seasons, IIRC. It's hard for me to have this discussion for sure, since you obviously know more about how the rankings work, and the rankings weren't released then (which is another debate entirely).

 

[dok]

The rankings didn't exist then, actually. We just compiled the records and plugged them in.

 

[Foudzing]

I think if new players continued to play and play a lot of games the majority of the players would reach easy Megasilver level.
And a lot of players would maintain a ~57% win ratio.
It's not a hard ratio to maintain. Over 66% it starts to be difficult to maintain yes but under this not really.

It seems odd to assert that most players could be above average.

Megasilver level is not average.

And for the 57% ratio thing I said "a lot" not "most".

 

[MegaSilver]

I think if new players continued to play and play a lot of games the majority of the players would reach easy Megasilver level.
And a lot of players would maintain a ~57% win ratio.
It's not a hard ratio to maintain. Over 66% it starts to be difficult to maintain yes but under this not really.

It seems odd to assert that most players could be above average.

Megasilver level is not average.

And for the 57% ratio thing I said "a lot" not "most".

Hey now!

My combined record is around 55%, it's lower on this one because for one season I lost every single game.

I want to get back to my normal win ratio soon...

 

[Kinseth]

How well your record is really doesn't have any relation with what your rating is. Not everyones schedules are equal, think of it like NCAA football. LSU could have a record of 9-2, but Miami of Ohio could be 11-0. I think all realize that LSU is the better team, but they usually have a brutal schedule. It is unrealistic for them to have as good of a record as a lesser team playing a lesser schedule.

The rankings system takes the data, and assumes based on the limited knowledge it has(Games you have played) how good you likely are. It can't for sure tell until you really start playing a good number of games. Just because you are 3-0 doesn't mean that you are the best player because you had a win% of 100%.

With most rankings systems it takes time for you to ascend the ranks as there are many more established players. The nice thing is that overtime, the system is able to see that the game you won against johnny when it was his first game ever, and he turns out to be a 15-2 player, you will get more credit for that win.

It eventually works itself out, the only explanation for why some players with worst win % are ranked higher than others is because some players may have played harder schedules.

 

[MegaSilver]

It eventually works itself out, the only explanation for why some players with worst win % are ranked higher than others is because some players may have played harder schedules.

 

[vegietarian18]

I realized that I've been articulating my main concern about the rankings pretty poorly. The rankings will give us an accurate picture of the top players eventually. The problem I have is that they are being used for the playoffs right now, and those players who have advantages, such as me, dok, bengi, kinseth, and filthy have those advantages right now. We performed well in seasons where these rankings and end of year playoffs weren't even conceived, yet this boosts our ranking and chance of making the playoffs.

 

[awesomeunleashed]

I realized that I've been articulating my main concern about the rankings pretty poorly. The rankings will give us an accurate picture of the top players eventually. The problem I have is that they are being used for the playoffs right now, and those players who have advantages, such as me, dok, bengi, kinseth, and filthy have those advantages right now. We performed well in seasons where these rankings and end of year playoffs weren't even conceived, yet this boosts our ranking and chance of making the playoffs.

If it's any consolation, we're probably going to use a leaderboard that includes only 2014 games to determine the top 16 for the 2014 EOY tournament.

 

[Kinseth]

I realized that I've been articulating my main concern about the rankings pretty poorly. The rankings will give us an accurate picture of the top players eventually. The problem I have is that they are being used for the playoffs right now, and those players who have advantages, such as me, dok, bengi, kinseth, and filthy have those advantages right now. We performed well in seasons where these rankings and end of year playoffs weren't even conceived, yet this boosts our ranking and chance of making the playoffs.

There is no perfect system for this EoY right now, as it the entirety of the online seasons to date. Naturally those who have played more, and done well are going to have a better chance. Just like in 2014 if someone only plays two seasons compared to someone who plays five(it appears we are going to have ~5 seasons a year.). People who have excelled in fives seasons compared to others who only played in two seasons and excelled in those are going to have a better chance.

Thus we move forward the best we can with what we have.

 

[vegietarian18]

Next year, I think the problem will be lessened because people will know the rules for the end of the year championship. People will know that if they want to maximize their chances of making it, they will have to play in all the seasons.

It's a bigger issue this time, since for the first four seasons, there was no knowledge that there was even going to be an end of the year playoff. The next four seasons (I believe the new end of the year qualifying system was revealed DURING season 8), we were told that it was going to be based on playoff performance. Really, the qualifiers for the EOY playoffs should not be taking data from anything besides Season 9, since that is the only season where the playoff system was set.

 

[greygnarl]

I think the rankings are fine as is, but the EOY would be better served by something like the tiered points for progressing into the playoffs that you had before.

 

[dok]

I realized that I've been articulating my main concern about the rankings pretty poorly. The rankings will give us an accurate picture of the top players eventually. The problem I have is that they are being used for the playoffs right now, and those players who have advantages, such as me, dok, bengi, kinseth, and filthy have those advantages right now. We performed well in seasons where these rankings and end of year playoffs weren't even conceived, yet this boosts our ranking and chance of making the playoffs.

It only boosts us in the sense that the algorithm has more data on us and is therefore more confident in its assertions about our strength. The early seasons were not easier or harder to build rankings on in any way. And they count less, anyway, since there is a small decay built into the rankings.

There have been more than enough seasons since we announced we were tracking records and recording rankings internally for any player to build up a ranking simply by playing well. And this is reflected in the rankings: Wanderer999 has only been playing since season 5, he missed at least one season, and he is #5. If someone didn't play enough to build their ranking, I can't really see this as a flaw: it's just a choice they made. It's the same in every individual sport/game that has invitational events (Tennis, NASCAR racing, chess, whatever).

It's hard to say who the people are who are negatively impacted by this, anyway. I expect we will have at least two and probably three or four people in the top 16 not accept their invite. So really, the top 20 or so will probably get in. There's only a few people below that threshold who are even 3 games above .500. Some of them (like dalu and Slipperyslope) have so few games and haven't played in so long that I don't think anyone can reasonably argue they should be in the top 16.

So what does that leave? I mean, I guess you could make an argument that, say, dhyanam deserves a spot over vegie. But it's really stuff on the margins we're talking about here.

 

[vegietarian18]

Not playing enough should not be held against anyone in terms of End of Year playoff qualification since there was no way to know that not playing enough was a negative until Season 9. In seasons before that, players did not know that their records were going towards the end of the year playoffs. It's just unfair to count anything before the exact system was announced.

 

[dok]

Not playing enough should not be held against anyone in terms of End of Year playoff qualification since there was no way to know that not playing enough was a negative until Season 9. In seasons before that, players did not know that their records were going towards the end of the year playoffs. It's just unfair to count anything before the exact system was announced against someone.

Really? You think there was no reasonable expectation that you needed to play games in order to qualify?

How did you think it was going to work?