wiimotes - 3D User Interfaces for Games 3D User Interfaces...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3D User Interfaces for Games 3D User Interfaces for Games and and Virtual Reality Lecture #4: The Wii Remote Spring 2011 Sprin Joseph J. LaViola Jr. Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Motivation Wiimote controller provides 3D UI in the home 3DUI in mobile environments Makes games accessible to casual users Need to understand the device Spring 2011 advantages and disadvantages how to develop 3DUIs CAP6121 – 3D User Interfaces for Games and Virtual Reality CAP6121 ©Joseph J. LaViola Jr. 1 The Wiimote Device Wiimote features Supports two handed interaction uses Bluetooth for communication senses acceleration along 3 axes acceleration along axes optical sensor for pointing (uses sensor bar) provides audio and rumble feedback standard buttons and trigger uses 2 AA batteries can use 2 Wiimotes simultaneously Wii Easily expandable Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Wiimote Attachments Nunchuk Wii Helm Spring 2011 Steering Wheel Boxing Gloves Zapper Sports Pack CAP6121 – 3D User Interfaces for Games and Virtual Reality Fishing Reel ©Joseph J. LaViola Jr. 2 The Wiimote – Coordinates Wiimote Coordinates Image courtesy of www.osculator.net Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. The Wiimote – Optical Data Data from optical sensor uses sensor bar triangulation to determine depth distance between two points on image sensor (variable) distance between LEDs on sensor bar (fixed) roll (with respect to ground) angle can be calculated from angle of two image sensor points Advantages 10 LED lights (5 of each side) accurate up to 5 meters provides a pointing tool gives approximate depth approximate depth Disadvantages line of sight, infrared light problems only constrained rotation understanding Sensor Bar Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3 The Wiimote – Motion Data Data from 3-axis accelerometer senses instantaneous acceleration on device (i.e., force) along each axis arbitrary units (+/- 3g) always sensing gravity finding position and orientation at rest – roll and pitch can be calculated easily in motion – math gets more complex error accumulation causes problems often not needed – gestures sufficient Advantages at rest acceleration is g (upward) rest acceleration is (upward) freefall acceleration is 0 easily detect course motions mimic many natural actions many natural actions Disadvantages ambiguity issues player cheating not precise (not a 6 DOF tracker) Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. The Wii Motion Plus Current Wiimote device gives user a lot of useful data not perfect perfect Wii Motion Plus Spring 2011 ambiguities poor range constrained input moving toward better device finer control uses dual axis “tuning fork” angular rate gyroscope true linear motion and orientation CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 4 Visualizing Wiimote Data Important to see data to understand device Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Programming with the Wiimote Connect to computer does not work for every bluetooth device Obtain Wiimote software Wii many variations and APIs (C,C++, C#, Java, Flash) Brian Peek’s API (www.coding4fun.com) Paul Varcholik’s XNA 3DUI Framework (www.bespokesoftware.org) low level API contained within larger framework include gesture recognizer gesture recognizer Unity 3D Write code and enjoy Spring 2011 heuristics gesture analysis CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 5 Case Studies Wiimote used in many different ways Two main approaches wear sensor bar, use Wiimote as camera hold/wear Wiimote Games most famous – Johnny Chung Lee music dance navigation Robotic control Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. One Man Band Goal provide musical interfaces using a single device using single device easy to use, powerful, and expressive mimic real instruments Features guitar, violin/bass, drums, trombone theremin trombone, theremin MIMI – Multi-Instrument MultiMusical Interface Spring 2011 Bott et al., 2009 heuristic recognition exponential smoothing CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 6 Real Dance Goal full body experience that feels like real dancing untethered: untethered: no need to stand in one place or position position increasing amount of recognizable movements preventing cheating by inaccurate movements Approach attach Wiimotes to arms and ankles heuristic evaluation of motion signals Visual Interface timeline of icons score feedback with avatars animated instructor figure user based figure Spring 2011 Charbonneau et al., 2009 2009 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Navigation in World of Warcraft Work at V. Tech – Silva and Bowman Goal reduce interaction complexity in WoW improve navigation help player concentrate on other tasks Approach Approach Spring 2011 leaning metaphor use Wiimote as camera wear IR sensor bar CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 7 Navigation in Sports Games – I Goal more natural interface in American football maneuvering navigation evasion Approach Williamson et al., 2010 IR head tracker Wiimote/gyroscope Wiimote/gyroscope gesture recognition integration Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Navigation in Sports Games – II Work at Brown University – Wilson, Reddy, and Jenkins Goal explore exergaming natural locomotion wiisoccer Approach track players foot motion sensor bar attached to leg Wiimote used as camera as camera kick, pass and player speed detected http://www.cs.brown.edu/people/awilson/exergaming-home.html Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 8 Robot Control Goal explore robot control explore robot control using using Wiimote Approach use rest orientation for driving robot (tilt sensing from accelerometers) gestures with simple classifier Spring 2011 forward, back, turn left, turn right, stop Varcholik et al., 2008 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Next Class Visual displays displays Readings Spring 2011 Siggraph 2010 course notes on 3D UI and Video Game Hardware CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 9 ...
View Full Document

Ask a homework question - tutors are online