It’s been a brutal winter for those of us in the northeastern U.S. As a longtime New Yorker and now a Jersey City resident, I can’t remember a winter with this much snow and cold. To make matters worse, I’ve been sick on and off for the past few months—“thanks” to the bugs my son keeps bringing home from daycare. But it’s March now, and I’m glad that season is finally behind us.

Let's get down to business.

For those who aren’t familiar with the prototype I’m working on, please check out my previous post. In short, I’m building a prototype of an autonomous drone designed to rally with a human tennis player. The ambition is to create an always-available tennis practice partner powered by robotics and AI.

My partner CJ and I started with a long list of technical questions we needed to answer to validate the idea. At a high level, they fall into three categories:

1. Perception — How well can the drone see what’s happening on the court?

2. Decision-making — How well can it decide or predict what to do based on what it sees?

3. Reaction — How well can it physically execute the response?

We believe the biggest uncertainties are in perception and physical reaction, so we decided to prioritize those first. Here’s a quick overview of what we’ve been testing:

Detecting a tennis player on the court is probably the easiest part. We use a YOLO computer vision model to identify players in the frame. For court detection, we trained our own model using publicly available datasets. That part turned out to be relatively straightforward as well.

As shown in the images above, the tennis ball is by far the hardest object to track. When the camera is positioned near the baseline, the ball becomes extremely small in the image, making it difficult for the model to detect reliably. Another issue—one that’s not obvious in the images—is frame rate. With a low FPS camera, fast-moving balls can disappear between frames, which makes continuous tracking much harder.

We had some initial success detecting the ball at around 10 feet using a standard webcam, but tracking a fast-moving ball on a full tennis court is a completely different challenge.

Now that the public tennis courts in my neighborhood are open again, my next step is to test the system on a real court using a 4K camera. I suspect there will be a trade-off between higher video quality and the computing speed required to process those frames in real time.

Next, let’s talk about the drone.

This is the part that tends to generate the most skepticism from people I’ve discussed the idea with—and honestly, that skepticism is fair. I have my doubts as well, and I’m open to exploring a more traditional robotic approach if the drone ultimately proves impractical.

For now, I’ve chosen to start with a quadcopter design, which offers a good balance between stability and maneuverability.

The first major challenge for any drone system is battery life. As a potential user myself, I think it would be reasonable to swap batteries every 30 minutes. Anything shorter than that would likely become frustrating during practice.

Based on typical FPV drone setups, a drone capable of hovering for 30 minutes would likely require at least a 6S LiPo battery. However, if we account for the extra power needed to accelerate, reposition, or generate counterforce when returning a fast incoming ball, we would probably need something closer to an 8S battery, which often weighs over 1 kg.

The problem is that larger batteries increase the weight of the drone, which then requires even more power to stay airborne. At some point, you hit diminishing returns.

One alternative we will consider is a tethered drone—a drone connected to a ground station via a power cable. This setup could provide much larger and more consistent power, eliminating many of the battery limitations. For now, we’re keeping that option open.

The winter is over and the best time to build and play tennis is now! I am looking forward to testing these technologies in the filed. Building a startup is a process of minimizing the risks and maximizing the potentials. We are on the right track.

Stay tuned until next time.

Disclaimer: This article is proof read by AI but all the content is 100% original.

It’s been a brutal winter for those of us in the northeastern U.S. As a longtime New Yorker and now a Jersey City resident, I can’t remember a winter with this much snow and cold. To make matters worse, I’ve been sick on and off for the past few months—“thanks” to the bugs my son keeps bringing home from daycare. But it’s March now, and I’m glad that season is finally behind us.

Let's get down to business.

For those who aren’t familiar with the prototype I’m working on, please check out my previous post. In short, I’m building a prototype of an autonomous drone designed to rally with a human tennis player. The ambition is to create an always-available tennis practice partner powered by robotics and AI.

My partner CJ and I started with a long list of technical questions we needed to answer to validate the idea. At a high level, they fall into three categories:

1. Perception — How well can the drone see what’s happening on the court?

2. Decision-making — How well can it decide or predict what to do based on what it sees?

3. Reaction — How well can it physically execute the response?

We believe the biggest uncertainties are in perception and physical reaction, so we decided to prioritize those first. Here’s a quick overview of what we’ve been testing:

Detecting a tennis player on the court is probably the easiest part. We use a YOLO computer vision model to identify players in the frame. For court detection, we trained our own model using publicly available datasets. That part turned out to be relatively straightforward as well.

As shown in the images above, the tennis ball is by far the hardest object to track. When the camera is positioned near the baseline, the ball becomes extremely small in the image, making it difficult for the model to detect reliably. Another issue—one that’s not obvious in the images—is frame rate. With a low FPS camera, fast-moving balls can disappear between frames, which makes continuous tracking much harder.

We had some initial success detecting the ball at around 10 feet using a standard webcam, but tracking a fast-moving ball on a full tennis court is a completely different challenge.

Now that the public tennis courts in my neighborhood are open again, my next step is to test the system on a real court using a 4K camera. I suspect there will be a trade-off between higher video quality and the computing speed required to process those frames in real time.

Next, let’s talk about the drone.

This is the part that tends to generate the most skepticism from people I’ve discussed the idea with—and honestly, that skepticism is fair. I have my doubts as well, and I’m open to exploring a more traditional robotic approach if the drone ultimately proves impractical.

For now, I’ve chosen to start with a quadcopter design, which offers a good balance between stability and maneuverability.

The first major challenge for any drone system is battery life. As a potential user myself, I think it would be reasonable to swap batteries every 30 minutes. Anything shorter than that would likely become frustrating during practice.

Based on typical FPV drone setups, a drone capable of hovering for 30 minutes would likely require at least a 6S LiPo battery. However, if we account for the extra power needed to accelerate, reposition, or generate counterforce when returning a fast incoming ball, we would probably need something closer to an 8S battery, which often weighs over 1 kg.

The problem is that larger batteries increase the weight of the drone, which then requires even more power to stay airborne. At some point, you hit diminishing returns.

One alternative we will consider is a tethered drone—a drone connected to a ground station via a power cable. This setup could provide much larger and more consistent power, eliminating many of the battery limitations. For now, we’re keeping that option open.

The winter is over and the best time to build and play tennis is now! I am looking forward to testing these technologies in the filed. Building a startup is a process of minimizing the risks and maximizing the potentials. We are on the right track.

Stay tuned until next time.

Disclaimer: This article is proof read by AI but all the content is 100% original.