3DUIevalIII(1) - 3D User Interface Evaluation III 3D...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3D User Interface Evaluation III 3D Lecture #18: Example Evaluations Spring 2009 Joseph J. LaViola Jr. Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Usability Evaluation in 3DUIs Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 1 Example Evaluations Non-isomorphic rotation (3DUI 07) Non Interaction Offset (3DUI 07) Interaction Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. IEEE Symposium on 3D User Interfaces 2007 An Exploration of Non-Isomorphic 3D Rotation in An Surround Screen Virtual Environments Joseph J. LaViola Jr.* Michael Katzourin Brown University March 10, 2007 * Now at the University of Central Florida Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 2 Talk Outline Motivation and Goals Motivation Non-Isomorphic Rotation Non Related Work Related Experiment Experiment Results Results Discussion Discussion Conclusion Conclusion Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Motivation and Goals Rotating objects in 3D space is a Rotating fundamental task Want to understand how 3D rotation Want techniques perform Isomorphic and non-isomorphic Isomorphic approaches Explore these approaches in SSVE Explore extend and augment existing knowledge extend does existing knowledge transfer? does Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 3 Non-Isomorphic 3D Rotation Human-Machine interaction Human input device input display device display transfer function (control to display mapping) transfer Non-isomorphic – scaled linear/non-linear Non mapping manual control constrained by human anatomy manual more effective use of limited tracking range (i.e more (i.e vision-based tracking) visionadditional tools for fine tuning interaction techniques additional Isomorphic – one-to-one mapping Isomorphic more natural more Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Non-Isomorphic Rotation Technique Quaternion – four-dimensional vector (v,w) where Quaternion v is a 3D vector and w is a real number Let q c be the orientation of the input device qd be the Let displayed orientation, and orientation then qo be the reference − qq = qck , qd = (qc qo 1 ) k qo , k = CD gain coefficient Using relative mapping Using qdi = (qci qc−i11 ) k qd i−1 − Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 4 Related Work User performance with different 3D rotation User techniques (Chen 1988, Hinckley 1997) Rotating real and virtual objects (Ware 1999) Rotating Framework, design guidelines, non-isomorphic Framework, effectiveness (Poupyrev 2000) Non-isomorphic head rotations (LaViola 2001, Non Jay 2003) GlobeFish and Globe Mouse (Froehlich 2006) GlobeFish Hybrid haptic rotations (Dominjon 2006) Hybrid Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Experimental Study Further explore non-isomorphic rotation of Further virtual objects Systematic evaluation of different rotation Systematic amplifications Understand benefits of non-isomorphic in Understand SSVE head tracking head stereoscopic vision stereoscopic Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 5 Experimental Design 16 subjects (13 male, 3 female) 16 Conducted in Brown “Cave” Conducted Based on Poupyrev 2000 → Hinckley 1997 → Based Chen 1988 4 x 2 x 2 balanced, within-subjects design (16 balanced, conditions) Independent variables Independent amplification (1,2,3,4) amplification rotation amplitude (20-60, 70-180 degrees) rotation (2070Error threshold (6, 18 degrees) Error Dependent variables Dependent completion time completion orientation error orientation Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Experimental Procedure Task – rotate house from Task random to target orientation Pre-questionnaire Pre 16 practice trials 16 16 sets of 10 trials each 16 Ordering was randomized Ordering Post-questionnaire Post Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 6 Results - ANOVA Repeated measures, three way ANOVA Repeated Effect Error S F3,13=3.26, p=0.056 F3,13=4.8, p<0.05 T F1,15=13.66, p<0.05 F1,15=22.96, p<0.05 A F1,15=55.46, p<0.05 F1,15=0.001, p=0.98 SxT F3,13=0.29, p=0.83 F3,13=1.575, p=0.243 SxA F3,13=0.87, p=0.523 F3,13=0.562, p=0.649 TxA F1,15=5.03,p<0.05 F1,15=0.573, p=0.46 SxTxA Spring 2009 Time F3,13=0.73, p=0.55 F3,13=0.97, p=0.436 S = scaling factor T = error threshold A = angle CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results - Post Hoc Analysis Pairwise comparisons on scaling factor using Pairwise Holm’s sequential Bonferroni adjustment Significant differences between S1 and S2 and S1 and S3 Spring 2009 Significant difference between S1 and S4 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 7 Results – Subject Preferences Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results - Summary Subjects performed 11.5% faster with S2 Subjects and 15.0% faster with S3 with no statistically significant loss in accuracy Appears to be correlation between subject Appears preferences and mean completion time scaling factor of 3 is preferable amplification scaling coefficent Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 8 Discussion - Error Interesting differences with previous Interesting studies Poupyrev – 6.8 degrees Poupyrev Hinckley – 6.7 degrees Hinckley Ware (physical objects) -- 4.4 degrees Ware Our study – 3.9 degrees Our threshold of 6 – 3.41, threshold of 18 – 4.4 threshold Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Discussion – Completion Time Poupyrev Poupyrev 5.15 seconds for isomorphic 5.15 ≈4.75 seconds for non-isomorphic Hinckley Hinckley 17.8 seconds for isomorphic (no training, 17.8 accuracy focus) Our study Our 2.2 seconds for isomorphic 2.2 1.96 seconds for non-isomporhic 1.96 Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 9 Discussion – Implications Differences attributed to Differences different hardware configurations different previous studies on desktop previous our study in SSVE our Poupyrev’s amplification factor (1.8) Poupyrev Hinckley – “… accuracy of rotation less affected Hinckley by interface then by difficulties in perception of error…” head tracking head stereoscopic vision stereoscopic Others – display size, refresh rate, video game Others proficiency, tracking lag Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Conclusion Presented experiment exploring non-isomorphic Presented rotation in SSVE Rotation task completed 15% faster with Rotation amplification factor of 3 than with isomorphic rotation no statistically significant loss in accuracy no subjects preferred this amplification factor subjects Faster and more accurate performance in SSVE Faster in general perception of objects closely matched with physical perception reality many other factors could contribute many Further work needed Further Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality CAP6938 ©Joseph J. LaViola Jr. 10 IEEE Symposium on 3D User Interfaces 2007 An Exploration of Interaction-Display Offset in An Surround Screen Virtual Environments Dmitri Lemmerman Joseph J. LaViola Jr.* Brown University March 10, 2007 * Now at the University of Central Florida Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Talk Outline Motivation and Goals Motivation Display-Interaction Offset Display Related Work Related Experiment Experiment Results Results Discussion Discussion Conclusion Conclusion Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 11 Motivation and Goals Want to build effective interfaces in VE Want applications Many different interaction technique Many choices set parameters to optimize performance set guidelines needed guidelines Techniques centered around user’s body Techniques are common in VEs Where should virtual object be placed with Where respect to user? Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Display-Interaction Offset What is display-interaction offset? What two frames of reference two display display interaction interaction Display frame of reference Display perceived location of rendered graphical perceived feedback Interaction frame of reference Interaction location of physical interaction location Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 12 Display-Interaction Offset (HMD) Courtesy of www.5dt.com Projection surface occludes physical hand Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Display-Interaction Offset (SSVE) Interaction in a SSVE Physical hand occludes projection screen Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 13 Related Work Virtual object manipulation for docking Virtual task With HMD – Mine(1997) With found users performed faster with collocation found over positional offset With Responsive Workbench – With Paljic(2002) found collocation or minimal offset minimized found time to completion Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Experimental Study Hypothesis – Translational offset between Hypothesis interaction and display frames of reference would improve user performance for 3D widget-based task Chose color matching with HSV widget Chose Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 14 Experimental Design 24 subjects (12 male, 12 female) 24 Conducted in Brown “Cave” Conducted 3 conditions of display-interaction offset conditions collocated collocated 3 inches inches 2 feet feet 15 trials per condition 15 Within-subjects design Within Target color used as second factor to Target control for color difficulty Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Experimental Procedure Pre-questionnaire & Pre color vision screening 6 practice trials practice For each trial For centering task centering color matching color Post-questionnaire Post 15 more trials with 15 subject chosen offset Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 15 Performance Metrics Measurements Measurements Time to center hand Time Time to chose matching color Time Chosen color Chosen Derived Derived Distance between target and chosen Distance Accuracy per time Accuracy Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results – Time to Match Color Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 16 Results – Centering Time Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results – Distance Between Target and Chosen Color Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 17 Results – Accuracy per Time Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results - ANOVA Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 18 Results – Paired Sample T-Tests Bonferroni correction Bonferroni Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Results – Second Phase Chosen offset Chosen mean – 1.34 mean feet SD – 1.026 SD Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 19 Post – Questionnaire Results Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Discussion Both offset conditions were significantly better Both than collocation distance distance user preference user No significant difference between offset No conditions For centering task For most similar to Mine and Paljic most collocation and minimal offset significantly faster collocation Results agree and disagree with previous work Results Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 20 Conclusion Compared effect of positional offsets on Compared user performance in SSVE color matching color centering centering Centering task performance in line with Centering previous work Color matching performance shows Color technique does not fit within established guidelines Further studies needed Further Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. Next Class Student Presentations Begin Student Spring 2009 CAP6938 – 3D User Interfaces for Games and Virtual Reality ©Joseph J. LaViola Jr. 21 ...
View Full Document

This note was uploaded on 06/13/2011 for the course CAP 6938 taught by Professor Staff during the Spring '08 term at University of Central Florida.

Ask a homework question - tutors are online