Enes Mentese went snowboarding with friends a few years ago, bringing along a GoPro camera and looking forward to capturing some "awesome video" of his friends on the slopes. But when he got home and looked at the footage, it was not what he'd expected.
"I was really careful and thought I was doing a great job, but the video was super shaky, with shots of the sky and the ground—nothing like the GoPro advertisements," he said. "And I had sacrificed my own snowboarding experience to get this disappointing video."
Mentese, a graduate student in electrical engineering at UC Santa Cruz's Baskin School of Engineering, figured there must be a technological solution. He recruited a team—most of them either current students or alumni of UC Santa Cruz—and went to work. The device they developed, called Raven, is a wearable, robotic camera stabilization system, powered by artificial intelligence, that can lock onto a moving target and automatically track it to capture smooth, continuous video footage of the selected target. The company they founded, Reflex Robotics, is preparing to launch a Kickstarter campaign to raise funds so they can start manufacturing the Raven.
Hannover Messe
The team took first place in the UC Santa Cruz Business Design Showcase on April 12. In late April, Mentese is joining a UC delegation to the Hannover Messe international trade show in Germany, where the project will be featured in the UC exhibit booth. Hannover Messe is the world's leading trade fair for industrial technology. Sue Carter, director of UCSC's Center for Innovation and Entrepreneurial Development (CIED), nominated Mentese for this opportunity.
"When I first met with Enes, he was very poised, and I thought he had a great idea and was the type of person who could execute it and make it happen," Carter said. "I don't often sit down with a grad student and think they'd be a good entrepreneur."
Carter, a physicist who has launched three start-up companies herself, said she was impressed by what the team had been able to accomplish on a shoestring budget. Their prototypes look like marketable products, not clunky mock-ups with loose wires hanging out.
"Hardware technology is really hard to do," she said. "If you can actually build a product and have an idea how to get it manufactured, that requires special talent. The image processing software is also quite sophisticated, so those are two difficult challenges they were able to pull off with almost no support."
Computer vision
Seongdo Kim, a graduate student in computer engineering, is a cofounder of Reflex Robotics and one of the first people Mentese asked to help with the project. For his Ph.D., Kim is working on computer vision technology with Roberto Manduchi, professor of computer engineering and head of the Computer Vision Lab at UC Santa Cruz.
"Seongdo is our computer vision and machine learning expert," Mentese said. "After taking a class together and seeing his research firsthand, I knew he'd be the perfect addition to the team."
Cofounder Abdo Babukr (B.S. from UC Santa Cruz, M.S. from San Jose State University, both in electrical engineering) is the team's embedded systems and digital signal processing expert. Mentese's brother Zubeyir, who has a degree in business management economics from UC Santa Cruz, brings business experience as the founder of CopterOptics, a drone videography company. The fifth current member of the team is Brent Bucci, a UCSC alumnus with experience in technology communications and startup management.
Their product is designed for action sports videography and other extreme environments where traditional camera control is not possible. It is small enough to be mounted on a helmet or a drone, using a standard GoPro mount. A GoPro camera fits into the device, which has its own camera feeding data to the embedded computer system, as well as accelerometers, magnetometers, and gyroscopes for stabilization.
Through an app on a smart phone or tablet, the user sees what the Raven sees and can select the object or person to be tracked. After that, the intelligent system takes over and the camera automatically follows the target while both camera and target are on the go, even if the view is momentarily obstructed.
"It's very computationally intensive, but we wanted to keep the system small and compact, so it was a complex project in terms of both the hardware development and the software development," Mentese said.
Machine learning
The Raven uses Nvidia's Jetson platform for embedded visual computing, and the custom software code enables the system to "learn" to recognize the object being tracked as it moves and changes position. This active machine learning during tracking is one of the keys to making the system robust and reliable, Mentese said.
"Robotics is the new thing, but a robot is only as good as its ability to perceive the outside world," he said. "Computer vision and machine learning is the ultimate sensor for robotics. We have a whole list of other project ideas we plan to pursue after Raven."
For Mentese and Kim, however, balancing their work on a startup company with their work as graduate students has been a challenge. "It's been hard. We don't get a lot of weekends off," Kim said.
Mentese said he is excited to be going to the Hannover Messe trade show as one of only two student entrepreneurs in the UC delegation. "It's a huge honor to be invited, and a great opportunity to spread the word about Raven," he said.
