games3DUI - Games and 3D User Interfaces: Games and 3D User...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Games and 3D User Interfaces: Games and 3D User Interfaces: Past, Past, Present, and Future Lecture #2: Games and 3DUI* Spring 2011 Sprin Joseph J. LaViola Jr. *Special thanks to Ivan Poupyrev Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI and Video Games – Why? Video games Driving force in technological innovation graphics is not enough anymore complex spatial, 3D user interfaces are coming to home (example: Nintendo Wii) Nintendo Wii) Why 3D user interfaces for games? graphics algorithms and hardware, sound, AI, etc. technological transfer to healthcare, biomedical research defence, education (example: Folding@Home) Recent innovations in 3D user interfaces multimulti-billion dollar industry: $10.5 billion in 2005 in US major driving force in home entertainment: average gamer today is 33 years old advanced 3D graphics in HOME rather then universities or movies studios natural motion and gestures reduce complexity more immersive and engaging Research in 3D UI for games is exciting Spring 2011 will transfer 3DUI to other practical applications, e.g. education and medicine CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 1 3DUI and Video Games – What? Goal of 3DUI in games desiging input devices and interaction techniques to effectively control highly dynamic 3D computer generated contentent generated contentent three are basic approaches Mapping 2D input to interact with 3D world keyboard and mouse, joysticks, game controllers traditional form of gaming user interfaces: e.g. Flight Simulator, Second life, Halo 3 Simulating real world tools or using physicial props simulation: steering wheels, light guns, musical instruments physical props: dance pads True spatial tracking of user gestures camera, e.g. Microsoft Kinect acceleration/infrared tracking: Wii controllers, Sony Move Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Lecture Outline Historical Perspectives 3DUI in the home today new generation of game UI The Future of UI in games UI early consoles arcades early 3D/VR game interfaces AR/VR/mobile games working towards the future Conclusions Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 2 Historical Notes on Game UIs Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Early Video Games 1947: CathodeCathode-ray tube amusement device probably the earliest proposal for electronic gaming device not known if it was implemented Proposed Spring 2011 interface interface knobs and buttons CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3 Early Video Games Tennis Spring 2011 for two: Second ever video game: 1958 by William Higinbotham @ Brookhaven National Laboratory Laboratory display: oscilloscope, Input: dial and a button first ever computer game was invented by Douglas, A. at Cambridge in 1952 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Early Video Games Spacewar! first (?) computer game: Spring 2011 1961 by Russell, S., Graetz, M. and Wiitanen, W. at Graetz, Wiitanen, MIT used DEC PDP-1 DPinterface: mostly buttons, but also joysticks and light pen CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 4 Early video games 1971: “Computer Space” is a first ever arcade game Spacewar! clone created by Nolan Bushnell interface is mostly buttons has not become very popular since its rules were too complex Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Early Video Games 1972: Magnavox “Odyssey” is a first ever home game console invented by Ralph Baer could play Ping-Pong Pinggame collaborative: two people first game controllers: button and dials: 1D battery operated 1975: 1975: Atari creates Pong Atari creates Pong for for home and arcades Spring 2011 game industry is born CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 5 Early Video Games 1977: Atari 2600 console Spring 2011 cartridge based system, i.e. allows to change software 2D controllers: Joystick as well as peripherals devices 2D controllers: Joystick as well as peripherals devices, i.e. trackball introduce quality sound hardware: still popular today CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Early Video Games 1978: Magnavox Odyssey Odyssey2 Spring 2011 includes full-sized fullkeyboard used for educational software and programming first home electronics device with speech synthesis CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 6 Modern Consoles 1983: Nintendo Famicon modern controller layout: controls for both hands, directional buttons increasingly complex controllers and interfaces: games are still increasingly complex controllers and interfaces: games are still 2D, 2D, but interaction is becoming more complex and rich. 1994: Nintendo 64 Spring 2011 first “true” 3D console adds joystick to controller, game pad gets more controls CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Modern Consoles 1996: Sony dual-shock controller dual adds second joystick and shoulder buttons standard controller for PS, PS2, PS3 Some observations Spring 2011 gradually increasing complexity of game interfaces to allow more expression in games; difficult to master focuses more and more on “hard-core” hardgamers; casual gamers often find games difficult similar situation was in early arcades games CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 7 Arcade Games Arcade games: “Easy to learn, but difficult to master master" has to be learned immediately can not have complex interfaces specialized interfaces for particular games many innovative and original interfaces often based on simulation activities shooting driving snowboarding shooting, driving, snowboarding, fishing, sliding etc. many innovative and original interfaces has been developed: 3D, haptic response, realistic. Spring 2011 www.afterpicture.com CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Arcade Games Video Arcades began in the mid 1970s (2D games only) Pong Breakout Space Invaders First game with 3D graphics – Battlezone (1980) vector graphics very simple interaction Spring 2011 move and rotate on 2D plane used two joysticks CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 8 Arcade Games – UI Innovation BeachHead Spring 2011 Football Power Aliens Extermination CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Arcade Games – UI Innovation Manx TT Spring 2011 Dance Dance Revolution CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 9 Virtual Reality Arcade Games Arcades were first to introduce introduce Virtual Reality and 3DUI in games head/body tracking stereoscopic vision immersive displays 3D spatial interaction Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Virtual Reality Arcade Games Dactyl Nightmare: one of the one first VR games part of several other VR games: games: Legend Quest, Hero, Grid Busters Quest Hero Grid Busters 1-4 players basic shoot-em-up game shoot-em- Developed by W Industries/Virtuality in early 1990s 1990s Spring 2011 system sold as the 1000CS used Amiga 3000 computer HMD with tracked 3D joystick CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 10 Virtual Reality Arcade Games VR entertainment centers multimulti-user combat simulation in BattleTech universe fighting robots first opened in 1990 opened in 1990 provided an immersive experience very little in the way of 3D user interface Can still play in Houston, Texas MechCorps (www.mechcorps.com) Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Virtual Reality Arcade Games DisneyQuest: Indoor interactive theme park (opened in 1998) Several VR games Pirates of the Caribbean: Battle for Buccaneer's Gold uses motion platform, shoot cannons, navi ate with steering wheel ig surround screen display, users wear stereo glasses Virtual Jungle Cruise Aladdin's Magic Carpet Ride Spring 2011 users sit in raft, steer and paddle users wear HMD, sit on motorcycle-like device to steer motorcycle- CAP6121 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 11 3D and VR on Game Consoles Several attempts to introduce 3D/VR for game consoles Nintendo U Force Mattel Power glove Sega 3D glasses Nintendo Virtual Boy Not successful Spring 2011 low quality, did not work well did not really necessary since games were simple enough considered to be a gimmick CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3D and VR on Game Consoles 1986: Sega Master System Spring 2011 3D glasses used active LCD shutters few games were supported CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 12 3D and VR on Game Consoles 1995: Nintendo Virtual Boy Spring 2011 Virtual reality goggles, monochrome, stereo l CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Some Conclusions From History Games complexity increases Spring 2011 1970: Pong 1980: Donkey Kong 2000: Halo interaction complexity increases CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 13 Some Conclusions From History The complexity of controllers increased use sa use same interface components co as in 60s Buttons Joysticks Keyboard / mouse combined together / increased number more difficult to learn and master less accessible to casual user 3D spatial controllers / 3DUI Spring 2011 very successful in arcades failed in home devices inaccurate/low quality CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI in the Home Today Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 14 3DUI in the Home Today Revival and rapid growth of 3D spatial interfaces for games today The first 3D UIs in people hands Spring 2011 cheaper and higher quality of sensors h fast game hardware can perform complex tracking/recognition need for simpler and more intuitive interaction with games games has become mainstream culture, more casual not only hard-core gamers hardoften based on previous research results and ideas simplified for price CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI in the Home Today 2003: Sony PS2 Eye Toy Spring 2011 video camera interface for PS2 PS2 casual/party games significant success in Europe/US based on several decades of research on visual tracking in robotics on visual tracking in robotics and and computer vision developed by Richard Marks CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 15 3DUI in the Home Today Nintendo Wii latest game console from Nintendo Key innovation – Wiimote controller Makes games accessible to casual users Spring 2011 provides 3D UI in the home CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI in the Home Today Wiimote features uses Bluetooth for communication senses acceleration along 3 axes acceleration along axes optical sensor for pointing (uses (uses sensor bar) provides audio and rumble feedback standard buttons and trigger uses 2 AA batteries Supports two handed interaction Easily expandable expandable Spring 2011 can use 2 Wiimotes simultaneously use Wiimotes simultaneously CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 16 3DUI in the Home Today Nunchuk Steering Wheel Wii Helm Spring 2011 Boxing Gloves Zapper Sports Pack CAP6121 – 3D User Interfaces for Games and Virtual Reality Fishing Reel ©Joseph J. LaViola Jr. 3DUI in the Home Today Playstation Move features 3 axis accelerometer 3 axis gyroscope magnetometer coupled with Eye color changing sphere standard buttons rumble bluetooth TwoTwo-handed interaction Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 17 3DUI in the Home Today Microsoft Kinect features RGB camera depth sensing camera uses infrared structured light multimulti-array microphone No buttons buttons Full body interaction Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI in the Home Today PS3 AR Game / Eye of Judgement Spring 2011 first 3D AR games on the first 3D AR games on the market market 3D interaction and manipulation of 3D graphics images possible based on Cybercode: tec technology for tracking 2D square markers invented in 1990s at Sony CSL CAP6121 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 18 3DUI in the Home Today 3D HDTV high resolution 1920x1080 high definition 1080p no special graphics card needed runs at 240Hz (120Hz each eye) Play games in 3D as high as 480Hz requires shutter glasses 3D stereoscopic content sent to TV via DVI or HDMI port Available from most TV Available from most TV manufactures manufactures DVDs as well, TV channels DLP, LCD, Plasma Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3DUI in the Home Today Some observations/conclusions Spring 2011 renaissance of 3D / spatial user interfaces in gaming interfaces in gaming for the first time very successful with public attracts casual gamers allows for easier introduction of new 3D user interfaces in the future still very simplistic when compared with 3DUI developed compare 3DUI in in research labs great possibilities for the future growth! CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 19 Future of Game UI Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Future of 3DUI in Gaming What are the technologies that will influence future that will influence future game game 3DUIs? 1. 2. 3. 4. Transfer the body of VR research into games Development of complex Augmented Reality games Outdoor games with complex 3DUI complex 3DUI Mobile 3D games Some Spring 2011 Toshiba Bubble Helmet (360 degree view) examples follow CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 20 Future of Gaming: VR Port of Quake II to the CAVE developed by Paul Rajlich (NCSA) fully immersive experience immersive experience uses 6DOF wand as gun proxy head tracking allows for peering around walls players can physically jump and duck Quake III Area ported to CAVE as well Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Future of Gaming: VR SwordPlay: explore what 3D UIs are appropriate in gaming developed as part of course in “Innovating Game Innovating Game developement developement“ (Brown U. 2006) user has sword and shield/bow and arrow uses Mine‘s Over-the-Shoulder Over-thedeletion technique to invoke bow and arrow user can draw spells in 3D with sword http://www.cs.brown.edu/courses/cs196-2/groups/swordplay/ Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 21 Future of Gaming: AR AquaGuantlet developed at Mixed Reality Systems Laboratory, Japan (Tamura et al. 2001) collaborative AR environment players wear see-through seeHMDs shoot creatures superimposed into real superimposed into real scene scene guns have vibration feedback www.jello.net Spring 2011 CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Future of Gaming: AR Markerless AR technology technology Spring 2011 tracking natural features (SLAM techniques) no visual markers: works in any works in any unprepared environment future of AR gaming CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 22 Future of Gaming: Outdoor Games AR Quake where monsters are superimposed into real world (i world (i.e., Quake in the Quake in the physical physical world) developed by Thomas, Piekarski et al. in 2000 (South Australia) can walk around in both indoor and outdoor indoor and outdoor environments environments equipment is somewhat cumbersome Spring 2011 getting smaller and cheaper CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Future of Gaming: Mobile Games Today mobile gaming platforms platforms PSP and Nintendo DS interaction is still 2D Future mobile platform Spring 2011 true spatial interaction location-based based interaction interaction AR tracking and interaction CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 23 Moving Towards the Future of 3DUI and Games Body of knowledge on 3D user interfaces of knowledge on 3D user interfaces interaction technique interaction metaphors and styles input devices usability studies Want to transfer to the video game domain to transfer to the video game domain Spring 2011 reduce interaction complexity provide more realistic experiences exercise!!! CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Conclusions Spring 2011 3D UI for games is important and interesting research area interesting research area Its real and possible to create new user interface culture Transfer to other areas of everyday human activity You can start developing 3D game user interfaces yourself CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 24 Next Class Unity 3D Bootcamp Unity 3D Bootcamp begins The Wiimote and You Readings Spring 2011 LaViola and Marks’ SIGGRAPH 2010 course notes Unity 3D Game Development by Example (Creighton) CAP6121 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 25 ...
View Full Document

This note was uploaded on 08/25/2011 for the course CAP 6121 taught by Professor Lavoilajr. during the Spring '11 term at University of Central Florida.

Ask a homework question - tutors are online