Hoverboards were meant to be here by now guys, if Back To the Future II was to be believed. There’d be giant artificial lifeforms amongst us, Blade Runner had it, and you don’t wanna know what Total Recall suggested we’ll have by 2084. 

Hey, fiction needs dreamers. Perhaps sport needs those visionaries even more. Rugby certainly does. And what’s worth noting is that some new thinking and wondrous technologies aren’t that far away. So while we consider at what would, could and perhaps should be in our sport, let’s take a look at the advancements that might begin lacing through our game in the very near future. This is where tech in rugby could go… 

TECH IN RUGBY – WATCHING THE GAME

What do you think of when you hear the phrase ‘augmented reality’? For some, the next obvious thought is ‘lineouts’.

AR 51 out of Tel Aviv are a business which brings live 3D streaming of events direct to you. As co-founder Erez Tal tells us, “Imagine yourself sitting with the Apple Vision Pro (headset) in your living room and later what will be augmented reality glasses, which are going to be the same size as (a typical pair of) glasses. These understand surfaces and 3D and can provide you with holograms of the live game, as the stadium will be in front of you, from your coffee table to infinity. 

“And you can control the size and the rotation and the angle of viewing, and where you want to sit (in the virtual stadium), together with friends and family – seeing the same experience in 3D and seeing each other. Or maybe your whole living room becomes a Stade Français living room. 

“Our main technology is real-time, markerless motion capture. It means that we’re setting cameras around the (venue) with any size of (venue) and any number of players, and we’re able to capture it and stream the motion and the scene in real time, to viewers at home.”

Related: Real-time stats, one-pieces and tradition – Is this the future of rugby kit?

There’s a lot to take in there, but we’ll explain markerless motion capture. Think Avatar and actors in a Morphsuit, with ping-pong balls all over them, with a green screen behind. Those are ‘markers’ needed for the technology to animate the motions. This is markerless – it just captures the athletes in action. 

This technology is here, now. Tal shows Rugby World a video of its intended application. Of course, there is commercial potential here. You could also have a can of a sponsor’s beverage on that coffee table, beside the tiny rugby pitch showing the latest big game. But there are other possible uses for the system. And training is one. 

Paris-based Stade get a mention after AR 51 were recently at their tech summit, and in the recent past former France assistant coach Karim Ghezal, now coach at Stade, made use of the markerless motion capture tech as the national side ran lineouts surrounded by cameras. According to Tal, this can help individuals train with virtual reality or AR headsets, to see 3D clips of lineout training and position themselves in a life-sized hologram of it. No need for team-mates to run it over and over for you, just get in the middle of it.

An Apple headset will not be cheap. But with future innovations, the size will come down and costs will shrink, we are told. Broadcasts will start with traditional 2D pictures aided with mixed-reality offerings on table tops. Then it’s 3D. 

As for other streaming and broadcast innovations, artificial intelligence (AI) has made its presence felt. The company Pixellot uses end-to-end, modular video streaming service with what’s known as ‘automated production’. In short, you could set up a number of cameras that take in a whole pitch and with AI it is like a virtual camera operator, following play. Built into this is live production and streaming, live data, instant highlights, the whole shebang. And there’s a sliding scale of costs, for the more basic cameras and cloud storage subscription for an ambitious amateur club up to elite leagues in several sports. 

Brian Phillpotts, vice-president of business development, says the top end could see a body spend around £4,500 a camera, plus a sub of £250 a month, while a grass-roots side could opt for about £1,000 a camera, which is mobile and can be taken home and away, plus the £100 fee for storage of footage. 

Already in a number of sports, Pixellot say they broadcast some 15,000 sporting meets every weekend. South African schools rugby and the regional New Zealand game are getting involved. One of the selling points is the opportunity for sponsorship and ad placements around broadcasts, the implications for betting and eCommerce, but also let’s not ignore the chance to scale up coverage of a league. Then there are scouting implications.

What’s ace, Pixellot say, is that the more footage the AI is fed, the better the coverage will be. It’s learning. And if that is the case for a system in a sport desperately trying to market its stars, an instant highlights package focusing on just one star player – which is possible – becomes valuable on a few levels.  

It’s worth noting now, because we can jealously cast our eyes to how Amazon Prime are using machine learning for their ‘Next Gen stats’ in NFL coverage. American Football fans have the option to see data on location, speed and acceleration for every player. They can see real-time graphics of separation as players pull away from each other. They can pull up biographical info on any star there, and they can have it highlighted during the broadcast which defender is likely to make the next big play. 

The issue for rugby is that many teams covet their GPS data, fearful that some other team could unpick their secrets. Athlete ownership of numbers is a debating point. Finding interest around the edges is key.

ANALYSING THE GAME

Simon Chi can understand resistance against using AI to track individual player involvements in a game. A data scientist working in healthcare in Canada, he has worked with Canada women’s sevens and New Zealand sevens, as well as helping with player recruitment projects for a number of professional sides.

“One of the core competencies of a video analyst is to take game videos and tag/code relevant actions at both team (scrums, lineouts, etc) and individual (carries, tackles) levels,”
he tells us. “It’s a time-consuming, manual process which requires great attention to detail. In the near future this process could be carried out using AI which could free up analysts to allocate their attention to other aspects of performance analysis.  

“As with anything automated with AI, there will be resistance and questions about accuracy of the work. However, there are already many complex environments outside of sports where AI is having an impact, like the finance, medicine and automotive industries.” 

It is impossible for any person to be across all the available data, he says. Does that mean missing something? He can foresee a time when AI could be used to parse all of the available data points from game footage, game data from third-party providers, and GPS data, and using what is found to highlight unseen trends or threats, as well as helping to shape a team’s approach to decision-making. 

He can also see us potentially underusing what we already have. GPS is typically used to monitor player load, but he tells us “player location data can also be pulled from individual GPS units but this hasn’t been utilised to any great extent in rugby union. There is great potential for new insights, as it allows for analysis of off-the-ball activity”.

Related: Reimagining the future of rugby broadcasts

There are platforms like Matchtracker too, that uses varying data sources to gain new insights. It’s used in other sports like football and F1, but Chi does not believe it has properly hit rugby yet.  

Of course, video analysis doesn’t just have to be about performance. 

Back in 2020, Rugby World talked with rugby researcher Sharief Hendricks, Senior Lecturer at Cape Town University. Back then, he was looking at how automated video analysis could be used to ‘code’ a rugby tackle in real time and determine if it was ‘safe’ or not. While that project has moved on elsewhere, Hendricks is keen to call for more collaboration in the world of research and advancement in rugby. So in line with that, he brings up the introduction of instrumented mouthguards. 

This piece of tech, that World Rugby brought in for the WXV competitions and is meant for a wider roll-out, is used to monitor head motion and what they call a “head acceleration event”. The idea is to track the number and the magnitude of those acceleration events for medics to assess if a player needs an HIA. As Hendricks explains, there is now work going on to essentially tie the raw data the mouthguards feed back, with the context of live video. To get to a place where you are alerted about a head acceleration, but then on the video you can see what has led to that reading, whether it was a big hit or if they are just falling onto a ball, say.

It’s important to note, Hendricks adds, that everyone is still learning. They could, for example, find a large number of accelerations come from what we would call textbook legal hits. Tech in rugby needs to be improved continually as well. But in the future, the ideal is for pitchside medics to be alerted to an acceleration and instantly have a clip of the affected player. 

Which brings us back to tackle technique and monitoring it. Initially, Hendricks worked with the uni robotics lab in Cape Town. But as they moved onto other projects, the notion interested others. Notably Leinster and Ireland back-row Will Connors. 

Not just an abrasive openside, he studied computer science and data science at UCD. Whenever he had the chance, he enjoyed looking at sport through the prism of computer science. Then after a number of serious knee injuries and an Achilles repair, he had a year and a half on the sidelines. In that time he went to work for Kitman Labs, the data-driven performance company founded by former Leinster player Kevin McLaughlin. It got him thinking…

“I’ve always been semi-obsessed over the tackle,” Connors tells us. “For my own performance I have always tried to maximise what I can get out of it around tackle technique.”

He knew that a lot had been done to look at motion capture in lab settings, and he wondered if more could be done to use the computer vision side of things, to look at what’s called ‘pose estimation’ from live game play. Basically a system where a computer generates an image to show you the knee angle or torso angle of someone in a tackle situation; to draw you the shapes they make during the contact. 

There are issues to get around – with so many bodies on a pitch, occlusion or simply folk getting in the way of the picture is one to fight with. But Connors has worked hard, and he thanks Leinster and the IRFU for all their help. 

“Hopefully at the end of it, I’d love to have a tool that can detect negative tackle patterns from a player. So for example, if you run my hopeful model over me in a stretch of ten games you could say, ‘Look, Will in 80% of his tackles gets his head on the wrong side’ or ‘He doesn’t dip his body height’ or things like that. People like Sharief have shown that negative tackle patterns can lead to injuries, concussions. 

Related: Shaking up where Lions tours should go

“Then from there you’d be able to be like, ‘Let’s highlight Will as someone who’s constantly getting his head on the wrong side. How, as a coach, do I enable him to practise the best habits in training’ and little things like that.”

The big dream is for this to eventually be automated. And then Connors would love for coaches to buy in and maybe they can develop a programme.  

TECH IN RUGBY – RESEARCHING THE GAME

Researching the contact area is a big undertaking. But it feels like Kathryn Dane likes to be busy. The scrum-half is not only contracted with Ireland and a physiotherapist by trade, but she is now completing a PhD on tackle safety in women’s rugby. Trying to get that done in the final year is “crazy”, she says.

On the driver, she explains: “I was kind of aware of different playing contexts between the men’s game and women’s game. I was noticing some trends in skill development, particularly in the contact area. I quickly discovered that there really is a research gap.”

The need for more research in women’s rugby has been evident for some time, and Dane wanted to use a mix of qualitative interviews, surveys and video analysis to get a better picture of tackle safety performance from the perspective of players, coaches and objective video data. With some talking about noticing differences in the patterns and mechanisms of injury for women rugby players when compared to male counterparts, particularly in terms of brain injury, Dane talks well on the importance of being specific and nuanced in how we look at complex problems in rugby. 

Sometimes, where there is a need, we can get bogged down in the negative. But Dane is excited about the opportunities out there for research in the women’s game, and shouts out the diverse cast of researchers operating in rugby right now. In the next five years, Dane feels we will have a much better understanding across disciplines as women’s rugby advances. 

Changing approach to how we teach tackle technique could be one outcome and the number of female-specific education resources should keep going up. That should provide safer and more enjoyable contact participation for female players. And with the World Cup in England in 2025, with more eyes on the women’s game, the product could see a real boost.      

TECH IN RUGBY FOR PROTECTING THE GAME

There are perils of intense attention, of course. Already this season we have seen England captain Owen Farrell and World Cup final TMO Tom Foley opt to step away from the Test stage. Social media can be savage and some on there can send out sickening abuse. Others spread real fear for safety. 

In order to fight back, World Rugby have recruited the help of Signify.

“We’re an artificial intelligence firm,” begins Jake Marsh, head of Signify Sport and someone with a background in private investigations and combating match fixing. “In terms of sport, we’ve worked across lots of major events now.

“The whole service is set up to be a proactive one. It’s not an off-the-shelf software that organisations and sports bodies just plug in and run themselves. We are not a moderation firm. 

“So if a player or a match official has a social media account, all we need to know is their handle. Whether that’s for Twitter, Instagram, Facebook, etc. If we’ve identified and verified their account, they’re automatically covered under the ‘threat matrix’ service. What that service does is use AI to go out across social media at scale. 

“So for example, for Twitter we get an enterprise API (application programming interface). That means we get everything Twitter has ever had posted. It means we can also go backwards if we need to, into someone’s history; if we need to prove they’ve posted something historically that has since been taken down. The system is looking for certain types of abuse. We’ve got about 25 categories of abuse but within that there are various different words, phrases and terminology that come under that.”

All of the data they are going through is open-source. It’s all out there. Then when the AI identifies what it feels is abuse – putting to work it’s constantly evolving understanding of languages, slang, context, and factoring typos or related terminology and also use of emojis – the example is then served up for another two layers of human analysis. You cannot rely just on the AI, of course. But it’s spotting danger signs.

As Marsh adds: “The system picks up that violent threat piece and that’s quite key, particularly when it comes to match officials in rugby and also in football. We’ve seen they’re getting people taking photos of them when they’re walking their dog and saying, ‘We can’t believe you did that last night’.”

When a threat is picked up, it’s then investigated, the account is studied, and if needed the threat is passed onto the relevant governing bodies and/or legal authorities. The laws around online abuse vary in different territories, but legal action has already been taken against online abusers in differing jurisdictions. Marsh adds of one case in Australia: “The Australian police are fantastic when it comes to issues in sport and they’ve got relevant laws in place. That means the threats that were made against a particular match official, they contravened those laws, and they can use it to go after these people.”

The days of hiding behind an alias could be nearing extinction if systems like this flourish. The deterrent already seen in football, where some fans have been stripped of season tickets, could also be impactful in this fight. 

The aim isn’t to shut down debate or respectful critique but to go after genuine threats and hate speech. There is an education piece to this too, with what information athletes and officials allow to get out there about their lives and how to handle the intensity of the spotlight. But the role of groups like Signify is to have their backs online. 

Would you like to see from tech in rugby in the future? Maybe you’d like less tech in rugby? Let us know at rugbyworldletters@futurenet.com or on social media

Download the digital edition of Rugby World straight to your tablet or subscribe to the print edition to get the magazine delivered to your door.

Follow Rugby World on Facebook, Instagram and Twitter/X.