Celtic's Christopher Jullien and the dangers of data

As this sphere of football has evolved, it has followed a path that echoes other sports. The first generation of ‘advanced stats’ for football included what has become the almost ubiquitous expected goals (xG) metric. Various event-based metrics followed as nearly all on-ball events were translated into data: passes, tackles, fouls, shots... you get the picture.

Event data has continued to evolve, with various modelling techniques creating possession-value metrics like expected threat (xT). The idea behind those models is to try to ascertain the value of events that did not result in a shot, which is necessary for xG to be generated.

For example, in a scenario where a midfielder plays a killer through ball to a striker who proceeds to fumble and a shot is never registered, possession-value models aim to quantify the value of such a pass.

Ultimately, comprehensive tracking data for all players and the ball itself through the entirety of all games are likely to become ubiquitous as it has in some other professional sports leagues. But in the meantime, analysts are left to try to do their best to draw accurate conclusions from the tools available.

Enter On-ball Value (OBV) from StatsBomb, which is their version of a possession-value model in the neighbourhood of xT. My friend Alan Morrison completed a two-part series comparing the various OBV metrics with data he captures.

As with any model, there are nuances and assumptions which drive potential gaps or issues. For example, StatsBomb made a decision with the model not to quantify the receipt of passes, with that potential ‘value’ presumably flowing through to resulting actions such as subsequent passes and shots.

One of the reasons I believe possession-value models like OBV offer worthwhile information relative to football performance analysis is the ability to consider the ‘net value’ of various aspects of players.

This is something the ‘eye test’ has trouble with, as we all carry around our biases about players. For those for whom we are favourably biased, each positive action gets emblazoned into our memories while for those players for whom we are negatively biased the same occurs with every misplaced pass.  

With Christopher Jullien being in the news recently due to the aborted move to Schalke 04 and his garnering some pre-season minutes, let us use all his league games at Celtic to date as an example:

The radar is custom-built to try to focus upon various skills and related metrics which could be important to centre-backs asked to play in a high line.

There are various counting stats, such as fouls and ball recoveries in the opposition half, with Defensive Action OBV included. Defensive Action OBV offers a way to net positive actions with negative actions and tries to distil things down to whether the player’s total value helped or hurt their team relative to scoring goals. 

A pure ‘counting stat’ review of Jullien’s performances would show excellent metrics for aspects such as tackles and aerial duals, suggesting he was a good defender.

However, events such as fouls can offer opposing teams quality goal-scoring opportunities and Defensive Action OBV provides one way to quantify how these offset one another. In addition, not all fouls present the same level of opportunity for the opposition, as those around the defensive penalty area are obviously far more threatening than those in the opponent's half.  

As we can see from the radar, Jullien’s percentile for fouls was very low, meaning he fouled at a high rate. He was also quite error-prone and not much above average in terms of being dispossessed.

All these metrics were accrued playing in a system in which his Average Defensive Action was around average, meaning he wasn't really playing much of a ‘high line.’ One could imagine these potential weaknesses manifesting to an even greater magnitude if he had been asked to play in Ange Postecoglou’s system regularly.

Here was Jullien at Celtic compared to Carl Starfelt last season playing in Postecoglou’s system:

We can see from the comparison that Starfelt was comparable in many ways while playing in more of a high-line system, as the higher Average Defensive Action metric suggests. Fouls were similarly elevated (Starfelt is also at zero for Defensive Action OBV) and both suffered a material decline in their passing accuracy when put under pressure. 

This ensemble approach in which I consider Defensive Action OBV within a broader context of many metrics, along with the style of play a team is playing, is why I believe something like OBV offers considerable value.

The profile for Jullien suggests he could have significant issues playing in Postecoglou’s system even domestically. For Starfelt, his profile suggests areas of weakness in which most domestic opponents simply lack the quality to punish Celtic.

However, here is the same radar for Starfelt’s games in the Europa League group prior to his injury compared with his league performances:

Once again, we see he rated highly in various volume-related defensive metrics but some of those areas of potential weakness became amplified playing against higher-quality opposition. Those errors and being dispossessed against Dundee United - for instance - may result in an angled shot from 30 yards while against a team like Bayer Leverkusen it can mean Florian Wirtz on the ball with Patrik Schick making a darting run.

These potential player-specific issues can get stacked upon one another as we saw in the 2020-21 season, with the way Neil Lennon chose to deploy a 35-year-old Scott Brown while asking Shane Duffy to play as a ball-playing centre-back in a relatively high line. Alan Morrison brilliantly termed such issues ‘toxic combinations'.

Here is a custom-built radar named ‘Sweeper Keeper’ comparing Joe Hart’s metrics from the league with the Europa League group games:

Similar to Starfelt, we see areas of relative weakness become more exposed against higher-level competition. Having a player like Starfelt potentially miscast in front of him may have contributed to Hart facing close to the highest amount of xG and post-shot xG in the competition, which was not his ‘fault.’

However, what was an around expected level of shot-stopping in the league was worse over those games, with other issues like playing out from the back and claiming crosses also worse. 

Obviously, the Europa group games sample size is very small but the issues also ‘fit’ with broader analytical profiles of both players from preceding years' performances at prior clubs.

Hopefully this exercise offers a window into the OBV metric and the potential utility of using it as part of a broader analytical framework.