Home > Blog > Build Logs > Portal Sentry > Software Shenanigans

Software Shenanigans

Posted by qwindelzorf on March 12, 2013

So, some time ago it became apparent that this whole thing would need a fair bit of processing power behind it to actually do anything cool, like track and fire upon a target.  Various options were evaluated, and eventually it was decided that using an old Android phone would be a good idea.  It has a camera to see the target, a sound system to play amusing Portal sounds, and it has a reasonably powerful CPU to do the image processing and such. This all sounded grand.

First, I procured a IOIO board to allow the phone to control the turret's pan and tilt axes, along with firing the guns and turning on and off the targeting laser.  Again, an evening later, I had the IOIO running and talking via I2C to the controller that runs the robot's mechanical bits.

Next, I decided to jump right into what I expected to be the hardest part, I started working on target tracking. I started by looking at the OpenCV library, which has a convenient Android port. This seemed ideal, since I could do the major algorithm development on a computer with a camera, and then just convert that to run on Android.

Algorithm development went smoothly, and by the end of an evening, I had a working target tracker running on my laptop. This was just way too easy.

Then came the part where it all had to run at the same time. That's where everything fell apart.  Converting the OpenCV code from the laptop to an Android native library proved to be a huge pain. And even once that was done, it was really slow, achieving only 1 or 2 frames per second. That was further exacerbated by the fact that it leaked memory so profusely that the application would crash after just a few seconds. And that was without any sound subprocesses running, and without the IOIO stuff.

So, that's basically where that lies. I have given up in frustration for the moment, and gone back to working on various remaining hardware issues. I built a new rear knee out of solid wood to improve the strength of that leg. 

Other than that though, the project is pretty much dead for the foreseeable future. We got pretty close, but motivation is lacking. Here's where the project stands, and here is where it will probably remain.