Charting the importance of completion rates

A modest defence of our most common or garden variety stat

Welcome to Stats Drop, an inundation of rugby league numbers.

If you’re receiving this by email, a reminder that the Datawrapper embeds work best on desktop, then next best if you click through on mobile and least best as presented in the email.

What is the completion rate? A betting website that topped Google says:

The completion rate is the percentage by which a team 'completes' their set of six tackles, without an error on any of the plays.

Probably the most important NRL stat other than tries, the completion rate indicates everything from how many attacking opportunities a team presents to the opposition, to discipline, and how well coached the team is.

I have added the bold because that is a bold claim. Even without using any graphs, I think you’d be hard pressed to argue that completions are the second most important stat. Perhaps it is the one given the second most emphasis.

There is considerable disdain from the Amateur Rugby League Analytics Fraternity (or ARLAF¹) towards completion rates. To take a quote out of context from the Rugby League Eye Test:

This reinforces the theory I have that completion rate is a junk statistic…

This disdain arises from the gap between the importance collectively assigned to the completion rate versus its usefulness. Personally, I’ve occupied different perspectives on completion rates: from not thinking about it at all, to dismissing it as useless information for the uninformed, to coming back around to finding some value, as I try to process a dozen games a week for the newsletter.

Today, we will explore the junkiness of completion rates. We’ll be looking at the NRLM above the fold. Below, after the paywall - UPGRADE TO A PAID SUBSCRIPTION TODAY - we’ll look at the other competitions. The Dataset, a link to which is also after the paywall, only has completion rate data for the NRLM from 2021 to round 19, 2025² because I haven’t gone back and gotten the rest of the data yet. That’s a (very tedious) off-season project.

Over this period, rates have fluctuated little. I think, but will have to prove after I’ve recollected the data, the long run average is something like 75% from 2013, which is as far back as NRL dot com goes. It would not be surprising to discover the predominance of the Panthers’ style of play is a sustained deviation from long term historical trends but within the window we have available, rates are stable.

There isn’t a great deal of variation within the season either. Average rates start a little below average as players lose their off-season rustiness, rise through the middle of the season and then drop away again as teams stop caring. While this is a convenient narrative, most of this change could be within the bounds of statistical insignificance.

Given completions rates are tied to errors, and errors occur randomly at a given rate in accordance with a skewed Poisson distribution [citation needed], then it is less surprising that there’s no meaningful change over a limited timeframe (either of a season or several years) and all of the potential sources of variance, and explanatory power, wash out over a large enough volume of n.

Further to this point, we have to be careful with this chart. NRL dot com doesn’t note the weather or ground condition for every game and presumably this comes down to how the ground manager/stats recorder felt on the day. It’s the kind of thing that fell by the wayside during covid.

With that in mind, we acknowledge that the partly cloudy set was one (1) game of 408 games that we have recorded weather for of the 986 games that were played. 25 games were listed as ‘rain’, 20 as ‘showers’, three as ‘light rain’ and three as ‘cloudy’. The balance of the 408 were listed as ‘fine’.

To reinforce an old point, you will never know who deals with the wet the best, irrespective of how well search engine optimised the post might be. I am hesitant to draw too many conclusions about wet weather footy but it’s an interesting anecdote nonetheless.

There’s a plainly obvious correlation between completion rates and winning football games. The argument about whether this is a junk stat ends there. In two-thirds of games, the team with the superior completion rate won the game. You complete more sets, you win more games. The simplicity is why it is memetically effective.

If you’ve read this far, it’s likely that you’re smart enough to work out the underlying mechanism. If a team has the ball more, uses it and doesn’t turn it over to errors or penalties, then they’re going to score more points while denying the opposition the same opportunity. It’s not complex - even a LLM can figure it out - but the completion rate isn’t the driver - possession, errors and efficiency are the drivers to the outcome - but completion rates are narratively concise.

The correlation between a team’s completion rate and the margin has a low R-squared of just 0.13 - certainly well below expectations for “second most important stat” status - but putting the completion rate into buckets of 5% and comparing to winning percentage of those teams has an R-squared of 0.98, which is line with what we would expect to see for tries scored and line breaks.

The correlation between net completion rate (that is, the home team’s minus the away team’s completion rate) and margin improves the R-squared to 0.23. There is a useful relationship between playing better than your opponent and how badly you beat them by but it is not an especially strong correlation.

This provides important insight into how we should talk about completion rates. The exact percentage is barely relevant. As running for 1405 metres or 1415 metres won’t make a meaningful difference to the outcome of the game, whether the completion rate is 74% or 76% is similarly unimportant.

We should probably just look at the first number - is it a 5, 6, 7, 8 or 9? - and that contains all of the worthwhile information we’re going to get. If both teams have the same significant digit, then both teams are playing at roughly the same level.

To complete Rugby League Eye Test’s out of context quote:

…in a vacuum. All things equal, promoting completion rate above all else would see you value completing 24/27 sets (88%) above completing 33/38 (87%). Again, it’s not the rate that you need to focus on, it’s the relation of it to your opponent.

Completion rates, even in a vacuum still correlate with winning. However, the completion rate lacks precision, in a way that counting errors, line breaks, penalties and tries do not. As discussed under the last Stats Drop, there are better ways of measuring what completion rates attempt to achieve but it is worth remembering that even scoring more tries doesn’t guarantee victory: just ask Zac Lomax or the 2023 Burleigh Bears.

Further, most advanced metrics do not have perfect explanatory power. The team with the superior SCWP wins about 77% of games and the team with the superior FTy wins 85%. Taylor-based player ratings are around 60% successful at predicting the match winner.

Instead, let’s consider what is being communicated. The point of statistics is to take the chaos of a football match and reduce it down to a few comprehensible numbers. Who are our potential users of completion rates?

• Coaches should be operating in a more sophisticated analytical environment than completion rates offer, capable of digesting up to several numbers simultaneously to understand what’s going on.

• Commentators find completion rates useful. Big number better. Closer to 100% means closer to perfection. Easy stuff. Makes it sound like you know what you’re talking about. Great for getting across in halftime spot between ads for McDonalds.

• Players will be a little of column A and a little of column B.

• You, the reader, members of ARLAF and associated sickos are handsome and sophisticated enjoyers of rugby league, so completion rates are worse than useless: they are a de-humanising insult to the collective intelligence of the community.

• Most normal people don’t think about this at all³.

If you gave up on completion rate, could you explain your expected points model or that actually in this case six errors isn’t that many because they’ve had so much ball cogently and quickly enough to make a point that the general audience will actually understand? Even allowing for the low bar set by the mainstream coverage of the sport, this would be unlikely to improve clarity.

Completion rates can also serve a purpose for describing playing style. Famously, Trent Robinson’s teams eschew completions in favour of whatever it is Sam Walker happens to be delivering. Ivan Cleary’s sides play with a rigid discipline and commitment, reflected in the highest completion rate in the league. You might then count the rings over the last four seasons and draw a conclusion, even noting that Robinson’s rate is a whole 5% lower than Cleary’s but as we can see, it’s a much more mixed bag than that.

The comparison between Robinson and Cleary is about as extreme a comparison as could be reliably made⁴ to make something like a valid point but the rest of the league is there or thereabouts and it’s much of a muchness. Maybe the normal people are onto something.

---

Thanks for reading Stats Drop

If this your first time, you might like to subscribe to get future free versions of these newsletters below.

If you enjoyed it, consider sharing with the people you think it might interest.

If you really enjoyed it, considering upgrading to a paid subscription to access what's beyond this paywall or chuck us a few bucks on Ko-Fi.

After the paywall, we're looking at completions in the women’s and minor leagues, competition Futures, Mandates of Heaven, updated and historical Dashboards and a fresh link to The Dataset™.

---