Could automated video analysis make rugby safer?

It all began with a cheetah’s tail. And robots.

Dr Amir Patel, a senior lecturer in the department of electrical engineering at the University of Cape Town (UCT), started looking at the movements of big cats as he tried to build a robot that could move at high speed. Then, as the technology used to study the animal was refined, it was realised that Patel’s manner of capturing movement on video could be applied to the human body too.

This is where rugby comes in.

“The robotics team had developed a way to study the movements of a cheetah and basically detect the movement automatically, just from a normal camera,” begins Dr Sharief Hendricks, from the exercise science and sports medicine department of UCT. “Dr Patel presented (on the system) at our department. And then we basically got chatting from there. It was asked ‘Why can’t we apply this to rugby?’ with the same system. They essentially had the code, we are just now applying it to videos of rugby.”

Hendricks has been studying rugby’s tackle since 2008. He has pored over injury epidemiology for South Africa’s Bok Smart programme, and since 2010 has focused on video analysis, mechanisms for injury and performance determinants.

Once Hendricks and his department realised they could harness the robotics work, a number of potential options presented themselves: one for rolling out an automated analysis system that ‘codes’ itself (tagging clips to catalogue rucks, tackles, passes and myriad other incidents, so analysts, athletes and coaches can study them faster) or to use it like cricket or tennis’s Hawk-Eye system, for match official to asses whether or not a tackle was safe or not, in real time.

At the moment there is not a huge amount of resources available – as you read this, a mechanical engineering student is likely feeding clips into the system. But the work is edging forward.

In Charlie Morgan’s recent Telegraph piece ‘How far behind football is rugby analysis?‘, analyst Michael Hughes explains: “Line-speed is probably the equivalent of the high-press in football, and (rugby) analysts are still having to manually tag clips of line-speed or kick-chase as event data because you don’t have the resolution of player tracking data to automate that for you.”

To a smaller degree, this is what UCT’s Auto Tackle Detection System hopes to combat. At the moment, the team in Cape Town are uploading examples of situations so the system can more easily identify the ball, who the carrier is and who the tackler is. At this early stage, work is needed so the system is sophisticated enough to automatically differentiate between individuals when there is a double tackle or a ball-carrier is being driven ahead by a bound team-mate, for example.

However, the goal is for a fully-automated system to consistently tag incidents as either high-risk or low-risk tackle situations. Essentially, whether the tackler and tackled player’s heads are in the same vicinity or near another body part, or not.

More from the system (Sherief Hendricks)

“Our work is safety driven and performance driven – and the two don’t have to be mutually exclusive,” says Dr Hendricks, who is also a visiting fellow at Leeds Beckett University. “My work focuses specifically on technique, and there is always an argument in there: it is good for safety and it is good for performance if you have good technique. So we measure technique quite a bit.

“You’re never going to get away from the subjectivity (side of judging a tackle incident). I feel that sometimes we argue against subjectivity, but a lot of sports are based on that. Go to the Olympics and you’ll see judges holding up scorecards.

“What you can do is keep subjectivity in check, with liability, with expertise, with validity.

“Today there are ways to check that subjectivity. So when they came and said, ‘Look, there’s a simple way to do this with the video that you give us’, great. Before, there were ways to do biomechanical analysis if you got guys in the lab (hitting bags), but it’s all out of context.”

Like everything in academics, more time and greater resources would help move things along. There is also potential to commercialise the technology down the line but there are papers to write first and no one wants to get ahead of themselves yet. These processes take time.

Back to the welfare side, Hendricks adds that “the ideal situation is that World Rugby takes this on”. We will see if the governing body beat a path to the senior lecturer and researcher’s door.

Related: Sam Underhill on how to make the perfect tackle

It’s likely unhelpful to predict what second, third or fourth generations of this technology could look like in years to come, but if it is realised in the first instance as it is evolving now, there are several options for how to use such a system.

The Hawk-Eye style review of a tackle is one way. There is also the warning sign system too – say it flags up that one player has put in a few shots approaching dangerous play, they can be made aware and if they reach a certain number of defensive challenges like that, it could be made clear to coaches that this athlete needs to address their technique.

Warning sign: Could a warning system work? (Getty Images)

“That’s a nice ’empowering’ concept,” Hendricks says. “One of the ways World Rugby loves to try change behaviour is ‘nudging.’ They want to lower the tackle height, so they may change laws because they need to get guys lower, essentially. Another idea is to empower the player and the coach, so they know why they need to go lower.

“The tackle warning system idea is a good one because the player gets to work on their technique, learn what they are doing wrong. So that’s one way it can be used, to tell the system to label high-risk, low-risk and medium-risk tackles and if a player does three high-risk tackles they need a talking-to.”

There is some way to travel with this, as more than 100 clips are fed in. Yet the options for commercialisation and welfare reviews are there. If the technology realises its potential, there is certainly something there to get tails wagging. And hopefully, ultimately, make elite rugby safer.

Can’t get to the shops? You can download the digital edition of Rugby World straight to your tablet or subscribe to the print edition to get the magazine delivered to your door.

Please follow Rugby World on Facebook, Instagram and Twitter.