Using gyroscope data for a crosshair GUI

Question

Using my phone held in my hand to represent a toy gun, I move it about ("aim", if you would) and transfer the orientation data (pitch, yaw, roll) to my laptop where I render a moving crosshair relative to a webcam feed pointed forwards.

I start the application by getting the user to press enter on the laptop while they are holding the phone straight in front of them - this is the initial calibration step, and I use the initial yaw/pitch as a reference to aiming at the center.

Then during the camera feed loop, where I draw the crosshair, I measure changes in the pitch/yaw relative to those initial calibrated values, and I use them to redraw the crosshair left/right/up/down.

This is my current code:

  pitchDiff = initPitch - newPitch   #corresponds to Y axis
  yawDiff = 0-(initYaw - newYaw)     #corresponds to X axis
  pitchChangeFactor = 10
  yawChangeFactor = 10

  xC = int((imgWidth/2) + yawDiff*yawChangeFactor) 
  yC = int((imgHeight/2) - pitchDiff*pitchChangeFactor)

  ## THE TARGETING GUI
  cR = 40 #circle radius
  cv2.circle(img, (xC,yC), cR, (20, 20, 255), 3)

What I'm asking for in this question is how I can do this more accurately and smoothly. The gyro data is noisy, so when I sample every 30 frames-per-second, I actually take the median of about 60 gyro readings for pitch/yaw.

Also, I believe my dynamical model is wrong: I'm simply moving the targeting crosshair a constant amount across the screen according to changes in angles. I would think trigonometric functions are needed, but it's not clear to me what I should try. Using only 1 camera, I clearly lack depth data. However, I am OK for assuming that the target I would like to aim at is, say, 3-4 meters in front of me.

Thank you for any help

Answer 1

how can I do this more accurately and smoothly?

Gyroscopes measure yaw rate, roll rate, and pitch rate but can't measure roll, pitch, and yaw directly. When you request the pitch and yaw angles from your phone it combines gyro and accelerometer data (and possibly magnetometer data) to give you an estimate of pitch and yaw. By gyro data I assume that you mean the estimated yaw and pitch that your phone provides. Make sure that you are using the yaw and pitch angles and not the yaw rate, pitch rate etc.

If you are using the estimation angles then you can look at different methods of filtering those signals before you you do your calculations. You mentioned a 60 sample median filter. Have you tried other filters? A simple low pass filter may perform better for your situation. Median filters are good if you get large and sudden spikes in your signals but a low pass filter or moving average may perform better for your situation. The proper way to do this would be to fix the position of your phone while acquiring data for some time then analyze the frequency components of the noise and choose a filter with the appropriate cut-off frequency to remove as much noise as you can, though this may be overkill for what you're trying to do.

I would suggest experimenting with a simple moving average or low pass filter to start. There is a lot of resources on the web showing how to implement filters in software.( http://en.wikipedia.org/wiki/Low-pass_filter )

Dynamical Model

As far as your calculations, they seem fine to me as long as your phone is providing angles and not rates as discussed above.

You do not need trigonometric functions. If you are just trying to map the angle to the screen position then a simple linear mapping like you've done is all you need so that a change of angle of x degrees corresponds to a change in screen position of M*x pixels.

Another thing that could help is if your phone can provide data faster than 30 Hz. You can still update the screen at 30 Hz but your filtering may be more effective when sampling faster. This all depends on the nature of the noise though, you should experiment with the sampling rate if you can.

Good luck with your project.