Do killer Celtic stats actually match up with what the eyes see? - Alan Morrison

I track my own data for Celtic By Numbers and collect over 150 data points per player per game which allows me to calculate many more aggregated and/or derived metrics. There is no one 'magic bullet' stat that captures the essence of a player you’d think.

Many believe xG (Expected Goals) is the closest in terms of a step-change in describing a match outcome in a way that is more reflective of how that game played out. But even that is over-simplistic and, coming back to an individual player, how is that even possible?

Well, StatsBomb introduced a new set of derived metrics called On Ball Value (OBV) in 2021. You can read the detailed definition in the link. The essence of it is to use Possession Value and xG to train a model to apply a 'value add' to key skills – Goalkeeping, Defensive Actions, Passing, Dribbling & Carrying, Shooting and an overall OBV for all event types.

It is ambitious and I don’t agree with all of the methodology but no matter, this isn’t an exercise in critiquing the model method but to compare with some more familiar advanced stats (StatsBomb’s 'secret sauce' recipe for how these metrics are calculated is, err, secret... they are a commercial operation after all).

Neither is this an exercise to say 'my data is better than yours'. I want to investigate the extent to which we can reduce player performance down to one or two key KPIs and whether those meet the eye test. In other words, does it match what we see? That is a wholly subjective measure, the outcome of which depends on a whole load of personal biases and experiences and perspectives.

I’m going to split this exercise into two articles – one covering Celtic's defensive play and the other offensive play. We'll start with defensive...

Goalkeeping

I want to plot two metrics per position. For keepers, the Goalkeeping OBV and Passing OBV seem the most relevant.

In the 'Celtic By Numbers corner', my two key metrics are Goals Saved Over Expected (how many goals, per 90 minutes, were saved above the xG value of shots on target faced) and Pack Passes Completed (the volume of line-breaking forward passes per 90 minutes).

Here are the StatsBomb plots for the last three seasons:

According to this, the outstanding keeping performance over the last four years was Scott Bain's in 2018-19. Now he did deserve to displace Craig Gordon as I wrote about at the time, however...

Similarly, Joe Hart’s OBV numbers from last season were identical to Vasilis Barkas’s for 2020-21. Equally, that does not pass an eye test.

Here are the comparative metrics mentioned above:

This has some merits in that:

• Bain’s good 2018-19 season is reflected but his regression in 2020-21 is clear
• Gordon’s poor season in 2018-19 is evident
• Barkas has the worst rate of goals conceded under xG (which feels right)
• Hart is OK at shot-stopping but adapted well to the passing game
• Fraser Forster’s 2018-19 season now seems more reflective of the game-changing saves we witnessed at the time

There is a 'but' here and it is that it still feels a bit reductive in the sense it doesn't seem to really capture the essence of each player. Factors like cross-catching, positional play and speed of getting the ball back into play are missing.

In summary: Goalkeeper OBV looks like it needs a lot more work while my more conventional advanced stats are a bit superficial themselves.

Full-backs

For this, I’ll use Defensive Action OBV and Passing OBV and then compare those to Defensive Action Success Rate and Pack Passes.

StatsBomb first:

This is a pretty fascinating view:

• Some of the recent full-backs Celtic have had are defensively strong but not great ball progressors (for instance, Diego Laxalt, Jeremy Toljan and even Greg Taylor to some extent). This is a great representation of that
• Kieran Tierney comes out the most balanced, which feels right
• Mikael Lustig’s Defensive OBV is a surprise but this was an 'age-regressed' Lustig. His passing was underrated
• Jeremie Frimpong rating alongside Boli Bolingoli is... surprising
• Anthony Ralston (insert eye-fluttering emoji!)

More traditional advanced stats:

• The three current full-backs being top right of the chart suggests that team style is playing a big role in this result – that is to say, it’s Ange-ball over individual contribution
• Lustig’s defensive capabilities look better represented (to my eyes)
• Frimpong’s strengths are not shown at all in this view – he really is an outlier with elite ball-carrying (if I used Dribble & Carry OBV he’d be on an island of one)
• Toljan looks the weakest, which chimes with the eye test

Overall, I like both views for different reasons and would suggest the StatsBomb OBV is more nuanced.

Centre-backs

For this we’ll use the same as for the full-backs – defensive action and passing.

StatsBomb first (it's a bit weird):

• The big outlier is Stephen Welsh and his passing – it's fine that it's underrated but to that extent?
• By this Dedryck Boyata looks the most defensively dominant and his pass OBV is average - that's acceptable
• Christopher Jullien’s strengths don’t seem to be well represented
• Kristoffer Ajer doesn’t do well out of this – maybe the doubters were right all along! - but also, like Frimpong, by not using Dribble & Carry OBV we are missing his great strength
• Shane Duffy and Cameron Carter-Vickers being within the same narrow orbit feels... not right

What does Defensive Action Success Rate and Pack Passing say?

• With this view Carter-Vickers is in a pleasing top-right island
• Jullien’s defensive strengths seem better reflected
• I have to accept I may have overstated Ajer’s defensive numbers in the past
• Duffy and Jozo Simunovic are middle-class citizens
• Carl Starfelt and Welsh are good ball-progressors but relatively weak defensively

I’m going to have to be a bit bullish here and say the old-school metrics match my eye test but, as mentioned, that also means my biases. I also accept that the OBV numbers may be capturing subtleties about players that are, well, undervalued. This could be the case with Welsh.

Defensive midfield

We’ll use exactly the same metrics for the holding or defensive midfield position.

Once again this is a fascinating view:

• Is this neatly summarising Scott Brown’s decline or is his defensive work undervalued? Also, does this reflect the relative conservatism of his passing?
• Nir Bitton does really well from this view but his defensive action OBV will be complicated by appearances at centre-back
• Callum McGregor 2021-22 is a real positive view of his development in this position.

Using Defensive Action Success Rate and Packing:

• This is a very similar view using the same data that was very different for centre-backs
• Bitton looks good by this but his passing range was very much a sweet spot – if we plotted defensive errors it would be different again
• Brown was pretty consistent by these measures but passing limitations are clear
• McGregor’s improvement is also clear as well as his advantage in ball progression

I’ve used the same metrics for full-backs, centre-backs and defensive midfielders yet only for the latter population do the StatsBomb OBV metrics and my own seem to be telling the same story.

The centre-back narrative is admittedly hard to reconcile while the full-back one is very interesting as each view provides different nuances.

Completing the defensive comparisons, goalkeeping is quite hard to represent in this limited form and the stats back that up.

Most of all, I loved doing this and I think each view provides insight into each player. Reducing performance to two metrics is clearly over-simplistic but I’d be fascinated to hear from you as to how it met - or indeed didn’t meet - your own eye tests?

Leave a comment below and let us know - I will repeat this exercise for the attacking players next time out.