#toy stats

[fbitennis]

Are These ATP Players Similar?

While researching something else that required extracting cumulative match stats for ATP players, I got sidetracked with wondering whether any two players are alike, or whether everyone has a unique set of skills, or at least unique ability in certain skills areas.  There probably are lots of ways to measure this, but I didn't start out with this goal, so I wanted to use the data I had already extracted.

I used the cumulative match stats of the Top 200 players (as of 10-7-2018), as they performed on hard courts over the last two and a half years against opponents ranked <=300.  I eliminated 11 players with fewer than 150 service games from the data set, since that's a fairly poor sample relative to the time period covered.

I placed each player in a "bin" with respect to each of these seven stats:  Ace%, DF%, 1stIn%, 1stWon%, 2dWon%, 1stRetWon% and 2dRetWon%.  I labeled them with letters, depending on the bin I put them in.  

Players greater than 1.5 standard deviations from the mean in a stat category got Tier A, 

Players between 1.5 standard deviations and 0.5 standard deviations got Tier B, 

Players between 0.5 standard deviations and -0.5 standard deviations got Tier C, 

Players between -0.5 standard deviations to -1.5 standard deviations got Tier D, and 

Players more than -1.5 standard deviations from the mean got Tier F.*   

In all categories but DF%, the highest numbers got A's and the lowest numbers got F's.

*Many of those stat categories are not normally distributed, so we don't have a nice bell shape.  In particular, Ace%, DF%, 1stIn% and 2dRetWon% are pretty warped.  Ace% is particularly whacked out, as there are a number of extraordinary ace artists, but only Yoshihito Nishioka is leaps and bounds worse than the mean.

In a very simplistic method, I simply strung the tier letters together to see if that would indicate similarities between the players in terms of style or performance. For example, John Isner is an A in all the serve categories, except a B on double faults, and an F in the two return categories.  Following the stat order above, he is coded as a ABAAAFF.  Are there any other players in the Top 200 who are ABAAAFF's on hard courts?  Nope.

In fact, out of the 189 players, only 10 have the same code as another player and no three players have the same code.  Here is a list of the players with the same code as another player, and their tier code if you are interested in matching the tier to the stat:

Taylor Fritz and Aljaz Bedene (BBDBCCC)

Frances Tiafoe and Leonardo Mayer (CCCCBCC)

Jaume Munar and Daniil Medvedev (CCCCCBB)

Nicolas Mahut and Pablo Cuevas (CCCCCCD)

John Millman and Kei Nishikori (DBCCBBB)

Philipp Kohlschreiber and Miomir Kecmanovic (DBCCBCC)

Yannick Maden and Gilles Simon (DBCDCBA)

Jason Jung and Tatsuma Ito (DCBDDBC)

Thomas Fabbiano and Radu Albot (DCBFCBB)

Diego Schwartzman and Kamil Majchrzak (DCCDCBA)

Eh, are those some weird pairings or what?  If I included height or age in the tiers, most of these guys would be unpaired immediately.  I don't think Millman and Nishikori are terrible comps in terms of style, although they clearly do not have the same talent (or at least performance).  The Munar and Medvedev pairing is ridiculous.

Part of the reason for the weird pairings is that there are only five bins.  You can be in the same bin with someone in a category, but be significantly better or worse than your mate in that category. The margins on the tour are small.  For instance Bedene and Fritz are tightly paired in every category but one:  1stRetWon%, where Bedene's mark is 1.3% higher (but still in the same tier).  Might not sound like much, but that's a huge difference in terms of results.  Yet it isn't that big a difference in terms of style, although they are in vastly different stages of their career arcs.

The more important reason for weird pairings is the difference in competition that these players face.  In other words, the match stats are a function not just of the player's abilities but his opponents' abilities.  Taking Bedene and Fritz again, while they are very alike in every category (usually very much alike), the average rank of Bedene's hard court opponents the last two and a half years is about 88, whereas Fritz's number is 114.  Rankings are not the end-all, be-all of opponent quality, but that's a big difference.  Except for Jung/Ito, the differences in opponent ranking for each of these pairings is huge.  The difference between average opponent rank is largest in the Schwartzman/Majchrzak and Kohlschreiber/Kecmanovic pairings.

It would be nice if we could normalize the stats to see what they would be against a common opponent, and then re-bin them and do the same exercise as above.  There are probably lots of ways to do this -- and none of them are particularly easy -- but here's what I did for purposes of this post.  Typically when you want to normalize sports statistics, you adjust for the context in which they play (in baseball for example, the ballparks they play in and the leagues they play in, particularly when comparing across eras).  But context is quite difficult in tennis.  We can start by limiting to a particular surface, but the wider "league" context is problematic.  Nominally, tennis players play against the universe of professional players, but the reality is that every tennis player faces a unique mix of opponents during any particular window.  Using a dataset that includes the Top 200's matches against only the Top 300 helps a bit, but in two and a half years on a hard court, each Top 200 player with a significant number of matches on this surface plays only about 50 unique opponents in the Top 300, often many fewer.  In other words, every tennis player is playing in a different league with its own quality.

So my approach to normalization (at least for now), is to determine a players' opponents during the two and a half years on hard courts, determine their collective (weighted) means in each stat category against the Top 300, and compare that to the overall mean for the Top 200 in those categories.  This should tell us how strong their own universe of opponents is, relative to the Top 200 average in each category.  We then use that universe's positive or negative deltas from the overall mean to adjust the subject player's stats in that category. 

So, for instance, if Taylor Fritz's universe of opponents is better than average on 1stRetWon%, we can boost Taylor Fritz's 1stWon% (the serve side) to simulate what he would achieve if he faced a merely average opponent.  I ignored for this purpose DF% and 1stIn%.  Both of those should be influenced by opponents' return ability, but unlike the other five categories, there isn't a directly opposing stat to make it relatively easy.  Ace% has OppAceAgainst%, 1stWon% has Opp1stRetWon%, 2dWon% has Opp2dRetWon%, 1stRetWon% has Opp1stWon% and 2dRetWon% has Opp2dWon%.  I don't know quantitatively how good returners affect DF% and 1stIn%.

After re-binning using the normalized stats, I got 11 groups of similar players, including one instance of three players being similar to one another.

Miomir Kecmanovic and Evgeny Donskoy (CBCCBDC)

Tim Smyczek and Bjorn Fratangelo (CBCDCCC)

Taylor Fritz, Lukas Lacko and Andreas Seppi (CBDBCCC)

Denis Kudla and Jaume Munar (CCCCCCB)

Frances Tiafoe and Leonardo Mayer (CCCCCCC)

Lukas Rosol and Gregoire Barrere (CCCCDDC)

Dennis Novak and Michael Mmoh (CDBDDCC)

Pablo Carreno Busta and Guido Pella (DBBDBCB)

Yannick Maden and Adrian Mannarino (DBCDCBB)

Damir Dzumhur and Radu Albot (DCBFCBB)

Nikoloz Basilashvili and Dominik Koepfer (DDCDCCC)

Admittedly I don't know all these names, e.g., whether Barrere is like persona non grata Rosol, but generally, this list based on the normalized stats looks a lot more reasonable than the first one.  In fact, there are a couple of really good comps here, including #2, #3, #4 and #7.  Nothing seems patently ridiculous to me, although Basilashvili may be in the process of breaking up the marriage in #11.

You will note that only one pairing from the unnormalized version survived normalization:  Frances Tiafoe and Leonardo Mayer.  It still seems strange to think of them as paired in this way, but actually their normalized stats are closer together than their unnormalized stats, so the pairing isn't just because they rate in the middle tier in every single category.

I've often wanted to do Bill James-style similarity scores for tennis players, but have found the task daunting.  James' sim scores for baseball players was pretty straightforward, because he did not use normalized stats.  Using the same method for tennis would require only an adjustment to the points associated with various stat differences between the players, but I don’t think we can get away with that in tennis.  I once created a spreadsheet (many years ago) that used baseball players' stats normalized for era and ballpark, and then applied sim scores.  That sounds appealing for tennis, but normalizing tennis is not quite as simple, for the reasons stated above:  every tennis player is essentially in his own league.  And the method I used here, which was pretty tedious despite its simplicity, was only for two and a half years, not a career span.  

Anyway, it's something to dream about.

[fbitennis]

Popcorn Score

A couple of days ago I posted something about toy stats, encouraging fans to come up with some on their own.  I can hardly ask someone else to do something without trying one myself.

As a reminder, here are the criteria I set for a good toy stat:

Be relatively easy to calculate, which means only addition, subtraction, multiplication and division, assuming use of a basic calculator (real or computer-based) with a memory function

Use stats that are readily available to anyone with an Internet connection

Be scaled so that someone seeing the toy stat will have a sense of how good, bad or average the measured performance is

Tell us something that we would not otherwise know using an already-existing stat

Bonus:  Have a clever name that is descriptive and easy to promote

The purpose of the Popcorn Score is to potentially identify good matches to watch that you wouldn’t necessarily recognize on your own.  Of course we don’t need a toy stat to tell us we should watch Roger play Rafa.  But setting aside your personal favorites, sometimes it isn’t so obvious.  Take this weekend’s grass semifinal matches, for example, in Antalya and Eastbourne respectively. 

Here is Saturday’s lineup:

1. Monfils vs Mannarino

2. M. Zverev vs Kukushkin

3. Lacko vs. Cecchinato

4. Vesely vs. Dzumhur

There’s no Fed vs Roger in there.  I put those in the order of my own personal preference, for subjective reasons.  Monfils is entertaining and has a contrasting style with Mannarino.  Zverev might serve and volley to make that match a little more interesting.  Cecchinato is having a great summer, but is basically new to the grass.  Not sure what might be appealing about the last match:  Dzumhur is fast and might slide under the net?

Notice that none of my reasons are related to how well the players play tennis, or how close the match is expected to be.  That doesn’t seem right.

Popcorn Score Concepts

Enter the Popcorn Score to quantify which matches maybe I should be watching that I otherwise wouldn’t.  Maybe the Popcorn Score will show me a hidden gem, or more appropriately, a hidden kernel.

The Popcorn Score is focused on two concepts:  (1) the quality of the players in the match and (2) how competitive the match ought to be.  But we have to keep things relatively simple, to stay within my toy stat principles.

We could use the official rankings (readily available) to determine the quality component, but rankings frequently are not an accurate summation of how good a player is, particularly on a surface-specific basis.  For more accurate portrayals, we have surface-specific ELO.  That would violate the ease of calculation test, except that Tennis Abstract now publishes surface-specific ELO ratings for about 175 men and 175 women, so that makes them readily available.  

Still, I don’t want to dismiss the rankings entirely, for two reasons:  (1) there will be matches for which the ELO rankings of both players are not yet published and (2) the public is conditioned to thinking of rankings as the standard of quality, and we can’t have a toy stat that purports to identify the most watchable matches but ignores public perception.

The measure of competitiveness is a bit easier.  You can use any forecast you want, whether its your own gut feel, a published forecast from your favorite website, or even just the derived forecast from the decimal odds published by your favorite bookmaker (but remember to take out the juice).

Calculating the Popcorn Score

So let’s get to the calculation.  First I decided that the score ought to be half-defined by the quality of the players and half-defined by the competitiveness of the match.  The weights could be different, of course, but 50/50 seems to work pretty well.  However, since I’m using two measures for the quality of player (ranking and ELO), I need to have separate weights for that half of the equation.  After experimenting a bit, I decided it works pretty well if player ranks count 15%, player ELO counts 35% and competitiveness counts 50%. 

We’ve got to get those all on the same scale, though.  Lower ranks should increase Popcorn Scores.  Yet higher ELO ratings should produce higher Popcorn Scores.  And bigger margins of forecasted win percentage should produce lower Popcorn Scores.  So you can’t just add this stuff together.

Ultimately, I want to get a Popcorn Score on a scale of 0 - 100, with 0 being “I’d rather mow the lawn,” 50 being “maybe if I don’t have something else to do” and 100 being “I will not hear anything my wife says for the next two-point-five hours, and I will have a popcorn stomach ache this evening.”  

It is possible to have a negative Popcorn Score, but I’d be shocked if that manifests in any main draw match. It’s also possible to exceed 100, which will happen occasionally, but that’s not something to worry about with a toy stat.  In fact, it kind of makes it sexier.

The Ranking Portion (15%)

(150 - Average Rank of the two players)/150 * 100

Why average the ranks?  Because watching a highly ranked opponent play a very low-ranked opponent does not make me want popcorn.  If you believe watching a great player play a horrible player still is a popcorn match, then your version of this stat might say to use the highest ranked player in the match.  That’s not my idea of popcorn, but I do sorta make an allowance for that situation at the very end.

Where did the 150s come from?  The first 150 is a very round approximation of the back-end player ranks that will be subjected to the calculation (e.g., decent qualifiers at a Grand Slam).  The second 150 represents the difference between the best player we could evaluate (#1) and the baseline player at #150, rounded up a bit.   

I am confident the 150s should not be higher, as higher baselines and divisors overinflate the quality of the players in the Popcorn Score.  The 150s may be too low, but in tinkering with it, it seemed to work best at 150, because any lower and you get too many negative scores.  

And yes, you can get a negative score for this component.  If #175 is playing #176, that’s a -17, as well it should be.  If #90 plays #98, you get 37.  Suppose we used 100 instead of 150 in the equation.  The #175/#176 match is -75.  The #90/#98 match is only a 6, and that’s before weighting it at 15%.  I don’t think the player quality is that low.   

The ELO Portion (35%)

(Average ELO of the two players - 1200)/800 *100

I’m averaging the two players’ surface-specific ELOs for the same reason I averaged the ranks.  

I fiddled around with the 1200 number as the baseline.  Intuitively it seems too low to me, because I don’t expect many players with an ELO of 1200 will be having a Popcorn Score calculated on them.  I originally conceived this with 1300 as the baseline, but Jeff Sackman recently re-did his ELO formula at Tennis Abstract (to account for player absences), and all the ELO scores dropped, so I dropped my baseline to 1200 to account for that. 

Why not raise it so it makes intuitive sense?  There is a lot of ELO clustering in the 1500s.  The ATP/WTA rankings are linear, but ELO ratings are not.  So if you raise the 1200 baseline to 1400, for example, you have a ton of matches between decent players that are treated as having very low quality players for the Popcorn Score, and that violates my sense of justice.

The 1100 divisor is the spread between the baseline of 1200 and the best players that will be in the equation with ELOs around 2300.  The greatest players have occasional peaks above 2300, but that’s not going to happen often.  (As an aside, my own Popcorn Calculations use a baseline of 1300 and a divisor of 1000, because my own ELO Ratings are higher in the middle than TA’s, although it doesn’t make much practical difference in the forecasting context. 

One more note about the ELO component.  If you don’t have a reliable (and by that, I mean surface-specific) ELO score for a player, then you should learn to calculate it yourself.  I’m kidding.  In that instance, just weight the ranking portion at 50% instead of 15%, and ignore the ELO portion.  

The Competitiveness Portion (50%)

(.99 - Difference between the two players expected win percentage)/.99 *100

This one’s a lot easier, because no one is going to have a 100% or 0% win percentage.  And, the narrower the difference between their forecasted win percentage, the more competitive the match is expected to be.

Putting it all together

We have the three components, so let’s put them all together.  Note that I multiplied each component by 100 so that we wouldn’t have any numbers below 1 when doing the individual components, but since we are adding this stuff together, we can just add the three components and multiply by 100 at the end.  So we’ve got:

Step One: (150-Avg Rank)/150 * .15, plus

Step Two: (Avg ELO - 1200)/800 * .35, plus

Step Three: (.99 - Difference in expected win percentage)/.99 * .50

Step Four: Multiply the sum by 100 to get it on the 0-100 scale.  

Examples

Let’s do each of the Antalya and Eastbourne semifinals.  For this I’ll use Tennis Abstract’s version of Grass ELO, and the implied odds from the average decimal odds listed on Oddsportal.com, with the juice extracted.  (No one drinks juice with popcorn, do they?)  

These are listed in the same order I originally ranked them.

Well maybe I picked a bad week to do this, because they are all kind of the same, and none have super high Popcorn Scores.  But that’s not surprising...look at the lineup.  The Popcorn Score isn’t going to turn Lacko vs. Cecchinato into a 100.  What they all having going for them is that they are reasonably competitive, and what works against them is that there are no stars, although Monfils has some separate entertainment appeal.  (Note:  The calculations above used a divisor of 1100 on the ELO portion, which I have since dropped to 800 if you use TA’s ELO ratings).

In any event, maybe I should be more interested in the Vesely-Dzumhur match than I originally was.  It features the highest ranked player of the four matches and is forecast to be the most competitive.  And apparently I overestimated Zverev-Kukushkin.

Wait, this Popcorn needs some spice

In running this formula on various random combos of players, I noticed all sorts of situations where I had minor disagreements between my subjective idea of popcorn matches and the Popcorn Score.  That’s okay, this isn’t science, and data isn’t going to fully account for the appeal of Gael Monfils.

BUT, in some instances, the differences really bothered me, and those had something in common:  The match had one superstar, but the Popcorn Score was dragged down too far by a weak opponent.  Think of it this way:  At some level, a match with Federer, Nadal or Serena always has appeal, even if the opponent is terrible.  It doesn’t usually hold all the way through the match, and I don’t think they automatically should get Popcorn Scores of 100 just because of who they are and regardless of opponent, but I do think that certain players should have enough weight to carry a poor opponent to a somewhat higher Popcorn Score.

Yet it won’t work to say “if Federer is playing, boost the Popcorn Score by X amount.”  Otherwise you have to constantly change your list of who gets a boost, and that’s too narrowly tailored.  

Initially, I decided that if a match has one player ranked #5 or better, you add 15 points to the Popcorn Score (15 points to Hufflepuff.  Y’know, Popcorn.  Puff.  Get it?)  You do NOT add that bonus if both of the players in the match are Top 5 or better, because the Popcorn Score already accounts for the high quality of that match.  

Nadal and Fed clearly qualify.  Sascha Zverev probably should.  Del Potro and Cilic are harder sells, but whatever.  On the women’s side you’ve got a nice Top 5 group that I’m comfortable with, but Serena matches don’t currently get a bonus, so that seems weird.  There are other situations where the Popcorn Score is off because of player layoffs, etc. (e.g., last week’s Murray-Wawrinka got a low Popcorn Score).   

Update: Jeff Sackman from Tennis Abstract sent me the idea of basing the bonus on peak rank, not current rank, so that you can pick up the appeal of someone like Serena or Murray.  He also suggested scaling the bonuses, for higher peaks, which nicely differentiates between the star power of a #1 (or former #1) vs the “star” power of #5 (or former #5).

Jeff’s suggestion was to take 6 minus the peak ranking (treat negative numbers as zero), raised to the power of 3, and then scale it appropriately.  I did that, and took the natural log of the results for purposes of scaling it to a top bonus of 20 points for peak #1s.  (I’m not going to pretend to know what the natural log of anything is, but I know you can sometimes use it for scaling so I tried it).  As it turns out, when you do that, you can get pretty close to the same result by taking 6 minus the peak ranking, and multiplying it by 4.  

So, a peak #1 would get a 20 point bonus, a peak #2 would get a 16 point bonus, and so on down to 4 points for a peak #5.  The log numbers would have been 19, 17, 13, 8 and 0.  I’d be fine with this scale and with a 0 for a peak #5 actually, but using logs doesn’t fit with the simplicity principle in the toy stat rules, and maybe using exponents doesn’t either. The exponent is just multiplication, which fits our rule, but it seems scarier to some people, and while a very basic calculator probably has x^2, it may not let you choose a different exponent.  Plus, you don’t really need to remember the 6 minus peak times 4 rule...you can just do 20 points for peak #1 and work your way down to 4 points for peak #5s in a linear fashion.  

I kind of like not giving a bonus for peak #5s, but let’s see who that involves.  I tried 50 searches for male players who might have peaked at #5, and had zero hits (who knew Grosjean peaked at #4?).  Try it, it’s harder than you think.  On the women’s side, I got to Ostapenko in five tries, and she deserves a 4 point bonus, so I’m leaving the #5 bonus in there.

There still will be anomalies with the scaled bonus structure, of course.  As Jeff pointed out, Zvonareva would get a good-sized bonus (16 points in my formulation), which is probably okay for the first comeback match at Wimbledon based purely on curiousity, but will oversell her future matches.  There are ways to address that (e.g., only take peak ranking in the last X years), but I don’t want to over-complicate it.

Because of this update, you will apply the bonus calculation to the highest peak between the two players, rather than applying it only if one player was Top 5.  So, if Federer (peak #1) plays Nishikori (peak #4), it’s a 20 point bonus.