Test 1
Automated Oracles for GUIs
Atif M. Me on m Martha E. Pollack Mary Lou S offa De of Com r S nce pt. pute cie Unive rsity of Pittsburgh
GUI Test Cases
2
GUI
st ase GUI Te C
e nce ve S que of e nts
A Test Case for WordPad
3
This is the text. S le xt e ctTe ("This") S le xt e ctTe ("te xt")
Form at
Font
18
OK
Form at
Font
Unde rline
OK
This is the text.
What Is Correct Behavior
4
This is the text.
?!
Font 18 OK
SelectText ("This")
Format
This is the text.
SelectText ("text")
Format
Font
Underline
OK
Check State, not only Output !!
This is the text.
Research Focus
Goal
ck r ach ve To che theGUI 's stateafte e e nt
5
s Approache
Manual ate Autom d
Actual State Expected State Automated Verifier Verdict
C nge halle Challe s
ne xpe d Ge rating e cte state Extracting actual state om xpe d s C paring e cte & actual state
Outline
rvie Ove w of GUI Oracle ne cte tate Ge rating Expe d S
ling tate Mode theGUI 's S
cts Obje rtie Prope s
6
ling nts Mode theEve
Obtaining Actual GUI's S tate C paring Actual & Expe d S s om cte tate C S ase tudy: MSWordPad C oncluding Re arks m
Overview of GUI Oracle
Test Case Formal GUI Model Run-time information from executing GUI
7
Expected-state Generator
Expected State
Verifier
Actual State
Oracle
Verdict
Execution Monitor
Modeling the GUI
A GUI consists of Objects Form
8
W indow St at e wsN ormal W idt h 1088 Aut oScroll TRUE
Align Capt ion Color Font
Label
Button
alN one Files of t ype: clBt nFace (t Font ) Capt ion Cancel Enabled TRUE Visible TRUE Height 65
9
All Properties of Cancel
Determining Properties
ination of GUI Manual Exam pe duce e S cifications (Re d S t)
ing ste GUI be te d
10
ple e Toolkit/Language(Com teS t)
rtie All availableprope s
Now we know how to represent the GUI's state
Modeling Events
nts tate rs Eve areS Transduce
i
11
S :S tate
This is the text.
SelectText ("This")
Eve e nt:
j i
This This is the text.
j
S :S tate
Representing Events
fine ve Wede an e nt as:
j i
12
State = [State , event] xam : For e ple
j i
State = [State , cut] e pact re se pre ntation Ne d a com
Operators
File Edit View Ins Me nu1 Effects: FORALL Obj in Obje cts S le d(Obj) e cte ADD inClipboard(Obj) DEL onS e cre n(Obj) DEL S le d(Obj) e cte ADD isCurre nt(Me nu1) DEL isC nt(Me urre nu2).
Obtaining next state
13
Operator :: C UT Preconditions: isC nt(Me urre nu2).
Cut
Me nu2
Deriving Expected State
0
14
Give S , theinitial state n , que e nts A se e nceof eve e
1 1 0 2 1 3
1
2
3
n
e
n
S0 e S1 = e , e ] S2 Obtain S i-1 [S
i i
e
S3
e
Sn
And S = [S , e]
Obtaining Actual GUI's State
cution Monitor Exe
cre n craping S e S rie Que s om cte tate C patiblewith Expe d S turns <Obje Prope Value ct, rty, > Re
15
<Button1, "Caption", "Cance l">
Automated Execution
Test Executor GUI Under Test
16
Test Cases Execution Monitor Verifier
AC TUAL S TATE: (isC nt ROOT) urre (C ontains ROOT D (C ontains ROOT D
Expected State
Comparing Actual and Expected States
rifie Ve r e ve sting Thre Le ls of Te
hange Prope S t (Operators) d rty e C le rty e GUI Re vant Prope S t (Specifications) om te rty e C ple Prope S t (Toolkit/Language)
17
Hybrid Approach
Useall 3
Case Study
: te ine Purpose De rm
e rive cte tate Tim to De Expe d S e cute rifie Tim to Exe Monitor and Ve r
18
rim ntal sign Expe e De
rsion of MSWordPad (36 Modal Windows, 362 GUI: Our Ve e nts) ve ne d st ase ve Test Cases: Ge rate 290 Te C s (6-56 e nts) using an AI Planne r ntiumbase Machine 256 d , Hardware Platform: 350 MHz Pe MB RAM duce e Properties: Re d S t le rty e Level of Testing: GUI Re vant Prope S t
The Big Picture
GUI Specifications
Test/Oracle Designer
19
GUI Implementation Tools (Languages/Toolkits)
GUI Implementer
GUI Model
Oracle
Expected-state Generator
Expected-state sequence
GUI Test-Case Generator
Test Case
GUI State (run-time)
Execution Monitor
Actual State Information Our implementation was based on AI Planning
Verifier
Verdict
Deriving Expected State
Generating Test Cases and Deriving Expected State 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0....