When it comes to ball sports, machine-vision techniques have begun to revolutionize the way analysts study the game and how umpires and referees make decisions. In cricket and tennis, for example, these systems routinely record ball movement in three dimensions and then generate a virtual replay that shows exactly where a ball hit the ground and even predicts its future trajectory (to determine whether it would have hit the wicket, for example).

But this kind of ball tracking is notably absent in other ball sports, such as basketball, volleyball, soccer, and so on. In these sports, the ball is often hidden from view behind players, its movement is significantly different when it is in a player’s possession compared to when it is flying through the air, and players’ interactions with the ball can be rapid and unpredictable.

These factors, as well as the small size of the ball in a frame and the sometimes low quality of video images, make ball-tracking much harder in these sports.

Today, Andrii Maksai and pals at the Ecole Polytechnique Federale de Lausanne in Switzerland outline a new way to track balls in these sports that outperforms other state-of-the-art approaches.

Most ball tracking systems rely on two different approaches. The first looks to follow the movement of the ball in three dimensions and then predicts various likely trajectories in the future. This “tree” of possible trajectories can then be pruned as more ball-tracking data becomes available.

The advantage of this approach is that the laws of physics are built in to the trajectory predictions so unphysical solutions can be avoided. However, it is hugely sensitive to the quality of the ball tracking data and so tends to fail when the ball is occluded or when players interact with the ball in unpredictable ways.

Another method is to track the players and note when they are in possession of the ball. The movement of the ball is then assumed to follow the player and when possession transfers from one player to another. The advantage here is that the system does not get so confused by rapid or unpredictable passes—indeed, this approach works well in basketball, where dribbling and occlusion can make life difficult for ball trackers. However, without physics-based constraints on the motion of the ball, these systems can produce inaccurate tracks.

Maksai and co have come up with an obvious solution. They track both the ball and the players accurately. They then use one of several different ways of solving the ball-tracking problem that depend on how the players are interacting with the ball.

For example, a basketball shot towards the basket follows a ballistic trajectory. But a rolling ball follows a different path. Both of these require different ball tracking solutions to a spike in volleyball which causes a sharp change in trajectory. And a ball being dribbled by a soccer player follows yet another type of irregular trajectory, requiring another solution. “We explicitly model the interaction between the ball and the players as well as the physical constraints the ball obeys when far away from the players,” say Maksai and co.

The team has tested its algorithm on a number of video sequences of various volleyball, basketball, and soccer games. The data comes from several cameras that record the same action from different angles to create a 3-D model of what’s going on. However, the data is far from perfect with many instances of occlusion, unpredictable passes and irregular trajectories.

The results show some improvement existing techniques. “We show that our approach is more robust and more accurate than several state-of-the-art approaches on real-life volleyball, basketball, and soccer sequences,” they say.

It’s not perfect, however. A crucial performance milestone for these systems is the ability to produce a virtual replay of a ball’s movement quickly and accurately enough for a TV audience.

That’s a tough ask, not least because this new system becomes better at tracking players as the length of the video sequence increases. But this dramatically increases processing time.

But greater processing time severely limits the system’s utility for live-broadcasts of sporting events when the virtual replay has to be available almost immediately.

However, the accuracy of certain types of ball tracking—ballistic shots, for example—becomes easier on shorter sequences since there is less unpredictability. So some kind of optimization process should help here.

With work like this, ball tracking in games like soccer, basketball, and volleyball is coming closer. But it is not yet fast enough to be commercially viable for a sports broadcaster.

That may require a step change in the way researchers approach this problem. One possibility could be to employ deep-learning techniques, where an AI system learns to predict future ball movement using its learned knowledge of previous games. That could significantly simplify some of the tasks that ball tracking involves.

Either way, still more work to be done here.

Ref: http://arxiv.org/abs/1511.06181: What Players Do With the Ball: A Physically Constrained Interaction Modeling