TalkMiner - Sensor Fusion on Android Devices: A Revolution in Motion Processing

Title: Sensor Fusion on Android Devices: A Revolution in Motion Processing

sensor fusion on android devices: a revolution in motion processing david sachs august 2, 2010 go o9je’:”

google techtalks are designed to disseminate a wide spectrum of views on topics including current affairs, science, medicine, engineering, business, humanities, law, entertainment, and the arts.

sensor fusion on android devices a revolution in motion processing davd sachs advrcd appiict1on dvdopmcnt m ariaçcr invensensé

introduction _____ 111k accelerometers, gyroscopes, and compasses 2 inveflseflsb

introduction o accelerometers, gyroscopes, and compasses ) 2 invensense

introduction accelerometers, gyroscopes, and compasses • honorable mention: invensensi i

introduction i accelerometers, gyroscopes, and compasses ‘ honorable mention: ‘ six axis accelerometer 2 invensensi

introduction 1 1 accelerometers, gyroscopes, and compasses ‘ honorable mention: • six axis accelerometer • six axis gyroscope • gyroscopic accelerometer • gyrocompass? slid 2 invensensi

introduction • gyrocompass • notsupportedby android 3 invensense

introduction 4 • application examples i slid 4 invensense

introduction application examples • sensors • sensor fusion • system integration • using sensor fusion slid 4 invensensi

application examples invensense

gaming c. .4. 4— —c • —— — 1 slid 6 invensense

virtual/augmented reality invensensi side7 — ez ---

mouse -

ira. to ______ ____ uoo.u tjcont,dv,iw [ow. v wt jrmrt b o,rn.$ rj ;1] i;0]

u t ___“øtoao.ta &iconuavww om,cn. ‘i dtla.*q4t ia.w _____ _____ _____________________ a ztnccvs..,, ;1] 4’l___ ,n;0] — n ‘ __

user interface ,

user interface sird. 11

user interface t .). ‘dc

image stabilization slid 13 invensense

navigation ‘ ___ a ;1] i;0] — rme gps sl,al was - lost 1.05 — mm ago 23:22 —1• — 14 invensensi

handset user interface [ii .gaming • image stabilization i 7_ ii eu1tb0n augmented reality invensensi sltd.15

accelerometer mass on spring 1 17 invensense

accelerometer mass on spring a gravity free fall slide 17

accelerometer mass on spring 0 ravity free fail linear acceleration

accelerometer mass on spring linear aceieration -i gravity free fall lz slide 17

accelerometer ) invensense

i i c, c, cd cd -i 0 3 cd cd 4

compass magnetic field sensor -.. sildeig

compass magnetic field sensor - z 3-axis compass’?

compass magnetic field sensor 3-axis compass? 19 invensensi z

compass magnetic field sensor 3-axis compass? slid invensensi z y

compass magnetic field sensor 3-axis compass? magnetic declinatkon slid inverisensè z y

compass magnetic field sensor 3-axis compass? 0 magnetic declination horizont • magnetic field ravity vector magnetic inclination ) invensense z y

compass hall effect -) sud 20 invensense

compass hall effect 20 invensensi

gyroscope • angularvelocity sensor ) slid 21 invensense

gyroscope • angularvelocity sensor ;1] ‘1s ____;0] 21 invensensi

gyroscope angular velocity sensor • conchs effect • hurricanes / [ax27xd , i — slid 21 invensensi

gyroscope • angular velocity sensor • coriolis effect • hurricanes • toilets x ii1.y invensensi

gyroscope • angularvelocity sensor • coriolis effect • hurricanes • toilets x i —v slid 21 invensensi

gyroscope • angular velocity sensor • conchs effect • hurricanes • toilets if x ) slid 21 invensense [at2xd j

‘gyroscope s11d l invei7,7efls

accelerometers and gyroscope invensensê’

accelerometers and gyroscope a linear c) 0 -1 accelerometer as a tilt sensor poor diiamic response amplified hand jitter gravity •g. knr slid 24 invensensi

accelerometers and gyroscope poor damic accelerometer as a tilt sensor on ampilli ed hand jifler elayed signal low pass filter? linear c-) —i 24 invensenso

accelerometers and gyroscope linear c) —i angular velocity -) slide 24 invensense accelerometer as a tilt sensor delayed signal re:pone low pass filter? gyroscope as a rotation sensor

sensor fusion __ sbde 25 invensensi

sensor fusion @-.tatio]-_ gravity 1 -) sird 25 invensense

single integration sled, 26 invensense

single integration p11 1 ii i ‘‘i on turns noise into drift slid 26 invensense 11111l rij

single integration integration turns noise into drift = 2( sin(2fl) ) slid 26 invensense

single integration • frequency and time requirements 6=6 slid 27 invensense

single integration frequency and time requirements 6=6 • high rate hardware sensor fusion +wat 27 invensensè’

single integration • frequency and time requirements 6=6+wat • high rate hardware sensor fusion ) slid 27 invensense

single integration • frequency and time requirements 8=6+wzv • high rate hardware sensor fusion 27 invensense

compass and gyroscope • tilt compensation invensensi

compass and gyroscope • tilt compensation • accelerometers don’t measure gravity well... invensensi

sensor fusion &tatio_-’ 29 invensensi

sensor fusion @-tatio]-_ gravity 1 ____________ fljl/ ‘ sled. 29 invensensi

sensor fusion orientation — slid 29 invensenso

— i. as —fl s l) v giroscope s z gyroscope e1i j litoocata 1conbdv4w [onrs lc’ i 1acd1lerometers artpbqiinihv ftsvfrvklfli q)mduin s compass coirfli kaixiw forward 1 /4 ___ -- 11 p.m ,nt, kn’n . 1 ___

double integration sltde 30 invensens

double integration& ___ linear acceleration = accelerometer data — gravity v=j’adt x=fvdt invensens

double integration ___ ___ linear acceleration = accelerometer data — gravity v=j’adt x=fvdi single and double integration of noise invensensé

double integration linear acceleration = accelerometer data — gravity v=fadt x=fvdt a=gsin(o) single and double integration of noise invensens

double integration linear acceleration = acceieroneter data — gravity v=j’adt x=ivdt a=gsin(o) x=at2 o 2q ao 00:0 :: single and double integration of noise double integration of noise and bias invensensi i -

sensor fusion onentat gravity 1 i linear movement sild.fl iflveflseflso

sensor fusion 4idi 1 gravity 1 linear movement :han high pass filter -) slid 1 invensense

sensor fusion orientation north tfigravity 1 linear movement • betterthan high pass filter • model dynamics with kalman filter i slid 31 invensense

coordinate systems z body coordinates world coordinates slid 32 invensensi y x

sensor coordinate systems • accelerometer and compass • measure angle relative to the earth 0 33 invensense

sensor coordinate systems • accelerometer and compass • measure angle relative to the earth • gyroscope • measures angular velocity relative to the body slid 33 invensense

degrees of freedom v slid 4 invensense yaw

degrees of freedom v 6dof vs 6 axis? ) 34 invensense yaw

system integration invensensê’

system integration 4no $4. k.o.irii t.s1, • flob. 4m6f *ta. (irabl./c,ibi.) • h.id mat.i /d.v/ivrt t.dp £apibr .rii.i 14.w ?a%ks • m.n..mplat. • oki,eb..i. mpl dala . tiia.i appiieatia. sensors fmpu hw slid ) invensense

system integration jfle a sled 37 invensense1

slid 37 invensense system integration i s & i’- f t ii o !‘f e

i (il sensor manager raw gyro accel magnetometer (d 6-ac’s rnrrc (j’q (&c’m3)) 0) gyroacc1 4 meg) quatemion ercri gravity

sensor manager rawgyro accel ma9netometef 6-a’s run fflf( (&. m3) 51 cgroaccai4 ?.eg) quatem ion unac’aiacn graity gesture glyph pedometer motion (yin) orientation composite \

sensor tvlanager rawgyro accel magnetometer 6-ags r&axi rnrr’c (accd’ m asrc1ionn (gyro•acca. ?.eç) quatem ton u race4cri gravity gesture glyph pedo meter motion (yin) orientation composite (ii (d 3 (d a) 0

using sensor fusion slid e invensensi

sensor fusion output • gravity • linear acceleration invensense

sensor fusion output • gravity • linear acceleration • orientation euler angles • rotation matrices • axis and angle • q.uaternions • angle change invensensi

sensor fusion output • gravity • linear acceleration • orientation ‘ euler angles • rotation matrices • axis and angle • quaternions • angle change • calibrated raw data • bias tracking 39 invensensi

gravity ) slid 40 invensense

gravity sltd.40 invensensi easy to port existing accelerometer co( i i i

gravity • easy to port existing accelerometer code replace accelerometer data with gravity data ) 40 invensense

linear acceleration ) slid 41 invensense

linear acceleration • replace accelerometer data with linear acceleration data -7 41 invensense

expressing rotation 42 invensense

expressing rotation slid 42 invensensi

rotation — ) 43 invensense

rotation 7 43 invensenso

rotation v ? x z ) 43 invensense

angle change silde 44 invensense

angle change • change in angle between two rotation matrices ) 45 invensense

angle change • change in angle between two rotation matrices • not accurate • easy to map to a user interface panx + angiechange[1)wpanxsf; pany + angiechange[o)*panysf; gitrns1ae(panx, pany, 0); slid 45 invensensi

euler angles 45 invensensi i

euler angles yaw 47 invensense

euler angles yaw pitch roll 47 invensense

pitch euler angles yaw romryaw pitch .ii i invensensi slide 47

using euler angles • pitch can’t go to 90 degrees • change euler angle definitions on the fly invensensë’

using euler angles • pitch can’t go to 90 degrees change euler angle definitions on the fly gi.gj.rotqtet(—pstc ..2t, o.o, o.0f); g1.g1rotate(yos, o.of, o.o, 1.o); gc->sctcorapaxcm (1@t, jon, 0.0, aboiutcaititudege, range, pitch, yaw, 11.0); 4, invensensè’

v.peuypii. imthtalkry

rotation matrix ) slid 49 invensense

rotation matrix 1 i2 i3 1 v’ ‘fl p3 v_v r31 !32 733 v: ) 49 invensense

using a rotation matrix • use entire matrix to rotate objects in opengi i/get rotation natrix into float buffer rotationmatrix gl.glmultmatrixf (rotationmatrix); //drat openol object 1 i2 ?i3 ,“21 r22 i”23 0 1230 0001 ) 50 invensense

using a rotation matrix • use entire matrix to rotate objects in opengi i/get rotation matrix into float buffer rotationmatrix gl.gimultmatrixf(rotationmatrix); i/draw opengl object i2 ?‘i3 ,‘21 r22 i”23 0 1t32t330 0001 • map interaction to numbers from rotation matrix directly 50 invensens

axis and angle <x, y, z> i invensense

axis and angle <x, y, z> ) slide si invensense

quaternion slid. 52 invensensi

quaternion four dimensional vector that lies on a hypersphere ) slid. 52 invensense

4 quaternion _____ four dimensional vector that lies on a hypersphere <x, y, z> q = kcosj,xsin}ysin,zsin.j) slid $2 invensense

using a quaternion • use quaternion in girotatef gi.g1rotatet((1oat) (2.of ‘ htth.acos(quat[o]) 180.of / lttb.pi), quat(1) quat(23, quat(3j); //drau openg!, obecc slid 5 invensense

using a quaternion 43 use quaternion in girotatef g1.girotatet((1oat) (z.ot ‘ ?!th.acos(quat(oj) ieo.o / flth.pt), quat(130 qut[2], quat(3fl; i/draw opengt. object • interpolation/extrapolation (slerp) ) slid 53 invensense

using a quaternion use quaternion in girotatef gi.g1rotatf((1oat) (2.of ‘ ?!ith.acos(quat(oj) 18o.o / !th.pi), quat(1) qut[z], quat(33); //dru opengl object • interpolation/extrapolation (slerp) (7n slid 53 invensense

Original on youtube.com

You need Flash player 8+ and JavaScript enabled to view this video.

Link:
Published: 2010-08-08
Channel: GoogleTechTalks
NEW: Explore related talks
Abstract: Google Tech Talk
August 2, 2010

ABSTRACT

Presented by David Sachs.

Gyroscopes, accelerometers, and compasses are increasingly prevalent in mainstream consumer electronics. Applications of these sensors include user interface, augmented reality, gaming, image stabilization, and navigation. This talk will demonstrate how all three sensor types work separately and in conjunction on a modified Android handset running a modified sensor API, then explain how algorithms are used to enable a multitude of applications.

Application developers who wish to make sense of rotational motion must master Euler angles, rotation matrices, and quaternions. Under the hood, sensor fusion algorithms must be used in order to create responsive, accurate, and low noise descriptions of motion. Reducing sensing errors involves compensating for temperature changes, magnetic disturbances, and sharp accelerations. Some of these algorithms must run at a very high rate and with very precise timing, which makes them difficult to implement within low-power real-time operating systems. Within Android specifically, this involves modifying the sensor manager, introducing new APIs, and partitioning motion processing tasks.

David Sachs began developing motion processing systems as a graduate student at the MIT Media Lab. His research there led him to InvenSense, where he continues this work with MEMS inertial sensors used in products such as the Nintendo Wii Motion Plus. David's designs incorporate gyroscopes, accelerometers, and compasses in various combinations and contexts including handset user interfaces, image stabilizers, navigation systems, game controllers, novel Braille displays, and musical instruments.