Revisiting Average Score and Top Players

When we introduced the Top Players leaderboard a while back, the intention was to create a place that tracked each player’s average score and then ranked everyone based on that average score. Some folks have noticed that in the past 24 hours, this leaderboard just changed significantly, and stopped stagnating. Why?

Because, we realized that our computation of Average Score was naïve and wrong. So, we fixed it. Since we launched it, we’ve only been counting full games (for completeness, we define a “full game” as any game that you’ve played in for longer than 1:50). But, when we computed your average score, we did so by taking every complete game you’ve EVER played and doing the math. The problem with this, as observed by Puzzlex28, is that it doesn’t take “improvement over time” into consideration. For example, here’s an aggregated stats chart of one of our best players:

image

The black line demonstrates the average score over time, and also, the player’s trend. With our old averaging system, all of the early games were still holding down the newer, improved later games. With our new system, we are sampling a much smaller window of scores, so it much more accurately tells you where you stand in average score.

This change has brought a lot of dynamicism to the Top Players leaderboard (which was the original intent), and demonstrates who is consistently clocking in the highest scores over time.

Hope this helps explain what’s going on!

– Black Snapper

About these ads

About Wordament

Wordament is a fun and addictive word game from You vs. the Internet, a Microsoft Studio.
This entry was posted in Uncategorized. Bookmark the permalink.

15 Responses to Revisiting Average Score and Top Players

  1. Adam says:

    First, I want to say — for lack of being able to say it anywhere else as I’m not on Facebook or Twitter — is that Wordament is my absolute favorite game. Over many years, I diligently worked on the New York Times crossword puzzle every day until I cancelled my subscription not too long ago. Wordament has certainly filled that void in having intellectual fun.

    Thank you for the above change in computing the average scores. And now my own suggestion for another leaderboard category. The percentage ranking for each game, I believe, is the best indicator of well an individual is playing Wordament as it takes out many of the other factors when one plays. I thought that would be an interesting measure to look at across the universe of players on an ongoing basis.

    Anyhow, keep up the good work!!

    acoti

  2. What does the % rank mean when you see what rank you placed?

    • Wordament says:

      % rank indicates how you ranked relative to all the other players in that round. Simply put it’s “your rank” / “total ranking players in that round”. In many ways this is more interesting than your actual rank since you can compare it round over round.

  3. Ssmallfry says:

    Great job on the new stats calculations! Are there any odds of getting those newer leader boards online also?

  4. arilev says:

    Thanks for clarifying and honing the leaderboards. Does this apply to the other metrics (average # or words and average ranking) too? Will you be implementing some options to the “Me” section so that we can better track our improvement? That would be very helpful.

  5. Thanks for the clarification! A couple of questions:
    1. I was getting the feeling that the game was tracking my average score across games I joined midway as I noticed a drop of my overall average score – by 10 points. If this has been fixed, why do I still notice a drop in my both average word count as well as score when I consistently score above?
    2. What happened to the Digram games where you has us find words related to ‘fruit’ or ‘finance’? those were a lot of fun as they were really high scoring!

  6. mdominik13 says:

    Does the average take into account the fact that some games offer more point than others? In other words, if I play a level that only has 1200 points possible, and I score 400, that’s not bad (30% of the points). However, if I play a level that offers 4000 points and score 400, I didn’t do so well (10%). If it isn’t taken into consideration, a player could drop out of games where the possible points was lower in order to keep their average high.

    One off topic question… is there a way to multiple users tied to a number? My wife is screwing up my scores!!!!

    • Wordament says:

      Today, we are not normalizing, we are averaging. But, if you assume that you are playing a representative set of games, like everyone else, then you will have an average on part with your ability. The degenerate case if that someone only plays the highest scoring games and gets a better average than someone that plays only the lowest scoring games. Given that our point distributions are pretty regular, this would be deeply degenerate.

      We don’t have a way to support multiple account profiles on a single device today. We agree that’s a good and useful feature.

  7. Erik says:

    As a stopgap for multiple accounts, how about a “Guest” button which just doesn’t use the scores from that session to update your standings. My kids have discovered Wordament, which is great for them but not so much for my average score:-).

    I’d also like some kind of visual indication (could be subtle, e.g. italics) to indicate whether or not an in progress game will be counted — I’ve found that I’m reluctant to start mid-game for fear it’ll drag me down…

    • Wordament says:

      The Guest button is a good idea. We know that multiple accounts or a guest account is an important feature and it’s on our work list.

      Regarding whether an in progress game counts towards your stats: Only full games (games where you’ve played for 1:50 or more of the 2:00) contribute to your average score ranking.

      Thanks for playing!

  8. fuul says:

    I think averages and some of statistics that are tracked should be weighted based on the total possible score of each round. For instance, having a score of 500 in a round with a max of 1500 points is a lot better than in a round with s top score of 3000. Ranking should be based on the percent of possible points scored… So 500 out of 1000 is .50, and 500 out of 3000 is around .17, which average to .33. That sounds more directly comparable, especially as new types of rounds are added like Themes, where the total possible scores are increasing. Currently, it would be easy for me to only play rounds that have very high possible scores in order to get my averages up.

    of course I am not complaining! This game is awesome and I can’t even remember when I played any other game…. Keep up the good work!

    • Wordament says:

      Hi fuul,

      You make an interesting argument and we’ve considered “normalizing” the averages, but we don’t think at this point it’s worth it for a couple of reasons: First, gaming the system is really academic. You would have to be a really dedicated individual to only play the best boards. It’s not how people play. They jump in and stay in for a bunch of rounds and then bail. To achieve what you are talking about would mean shutting off your phone every time you didn’t do well or there weren’t enough points. I don’t know about you, but that doesn’t sound very fun to me! BTW, and without going into to much detail, if you only played single games, our code can detect that as an “awkward pattern” and you won’t get counted toward our Top Players leaderboard (though you will get counted against the others).

      The other problem with the normalization is: what do you compare to when you are done? Sure, people will be in the mathematically correct order, but the scores would be unintelligable compared to what you see in games. While we could probably do a bunch of work to convert back to an unnormalized range, but it’s probably not different enough to justify it vs. what we have.

      In the “normal” case of how people play, you will play a variety of boards with a variety of scores and the average of those boards will “work itself out” over time and the best players will gravitate to the top. We take cheating pretty seriously, but we also have to be careful. We have some really good and capable players out there that clock in scores that look in-human.

      Thanks for the discussion and encouragement! We appreciate you taking the time to give us your point of view! If you disagree with anything I’ve said, please feel free to push back and continue the discussion,

      Black Snapper

  9. kem says:

    Last night, my Samsung focus reset itself. And when I say reset, I mean everything. I lost my contact, pictures, etc, AND my connection or ID to wordament.
    How do I re-install it on my phone, AND have access to all of my stats?
    Ken johnston

Comments are closed.