4932 CIS Special Topics Software Testing Fall 2001 Integration Testing
2001, Dr. E.L. Jones
Purpose
This course prepares a developer to practice unit testing on a project by applying the concepts covered in the Software Unit Test Concepts course.
11/2001
Integration Testing
2
Purpose
This course prepares the development team to plan and carry out an orderly process of assembling the system from tested units, and certifying the system ready for formal system test. Use of a controlled test environment ensures the validity of test results and facilitates transition to formal test.
11/2001 Integration Testing 3
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 4
What is Integration?
The process of combining individually developed components into a system.
C
System A AB
B
ABC
11/2001 Integration Testing 5
What is A Component?
A component is a system building block. The smallest component is a (composite) unit A component may be formed by combining other components Top level components are identified in the software architecture
11/2001
Integration Testing
6
The Software Architecture
Architecture defines components and connections Architecture depends on technology Standard architectures exist The software architecture is a design product
11/2001
Integration Testing
7
Software Architecture
Building blocks and their connections
System Interface
A
B
C
Shared component
11/2001
D
Integration Testing
8
What is An Interface?
An interface is a set of connections between components
Activated by a software call Involves the flow of data in either direction Connection may be transient or persistent A connection is called an interface operation
Each interface (operation) is a unit
11/2001 Integration Testing 9
Interface Operations
One interface may have many operations.
ftp System email NetComm telnet
A
B
D
11/2001
Integration Testing
10
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 11
An Example of Integration -the aboutproduct.com Website
Simple site Two action buttons
Refer a Friend -- send an email to a third pary Click to Order -- accept responses to a survey and store in a database
Accesses the corporate global database Uses communication services
11/2001 Integration Testing 12
Integration -- Connecting The Parts
Net Comm Form X email
Bus Logic SoftLogin GDBAccess
Applic DB Global DB
11/2001
Integration Testing
13
Interfaces Between Components
Forms
Form BusinessLogic NetComm
Business Logic
Form SoftLogin AppDB NetComm
SoftLogin
BusinessLogic GDB GDBAccess
11/2001
GDBAccess
SoftLogin GDB
Integration Testing 14
Form Action Buttons
Action buttons connect paths through the system.
Click to Order
Form (Survey) BusinessLogic SoftLogin AppDB GDBAccess GDB
11/2001
Refer a Friend
Form (Refer) SoftLogin BusinessLogic AppDB NetComm
Integration Testing
15
Interfaces -- CLICK TO ORDER
Home page
click Field Fixes
NetComm
Survey Data
email
Survey Data
Survey Form
Submit
BusLogic
Applic DB
click
N/E (Name, email)
Info page
SoftLogin
N/E
N/E
Global DB
GDBAccess N/E
11/2001 Integration Testing 16
What Can Go Wrong During Integration?
Form accepts erroneous data Form does not accept valid survey data Business logic accepts invalid data SoftLogin can not access global database Business logic does not store survey data etc.
Integration Testing 17
11/2001
Getting to the Bottom of Failures
BRUTE FORCE: Individually check each connection in the system
Set up in-flows to interface operation Activate the interface operation Verify out-flows from interface operation
BETTER: Incrementally integrate and test, progressively expanding the number of interfaces that have been tested.
11/2001 Integration Testing 18
Making the Connection I
Diagnosing integration problems is hard
many interface operations many possible invocation sequences difficulty reproducing the problem
An orderly, managed approach avoids chaos and reduces the overall effort Integration Testing approach should be based on the architecture.
11/2001 Integration Testing 19
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 20
What is Integration Testing?
Testing all connections within the software in a controlled physical environment? A Series of Unit Tests!
Enough to ensure each interface has been tested These tests should already exist as unit tests Selected ones should be applied (e.g., user operation action buttons)
11/2001 Integration Testing 21
Integration Test Approaches
Interface Driven -- black-box test of each interface, focusing on data-flow
Simple, but many test set-ups: unit test
Assembly Driven -- Add new component to assembly, then test new assembly
Orderly test process (per integration plan)
Usage Driven -- end-to-end user operations
Difficult with unreliable components
11/2001 Integration Testing 22
Interface Driven Integration Test
Point-to-Point test focused on data flow
Black-box unit test of interface operation Verify that correct calls made to interfaces Early testing or inspection
form Survey Data
BusLogic
Name, email calls to SoftLogin
Survey Data
calls to ApplicDB
11/2001
Integration Testing
23
Assembly Driven Integration Test
Each stage adds one new component
Sequence of integrating the components based on system architecture Test each new interface to new component Regression test existing interfaces (as needed)
Advantages -- test/development overlap Disadvantages -- regression testing
11/2001
Integration Testing
24
Usage Driven Integration Test
Test end-to-end operations -- use case or user operation Advantages
Higher level test, fewer tests to run Allows incremental development for high priority user operations
Disadvantages
Difficult with unreliable components
11/2001 Integration Testing 25
Our Turn -- Usage Test Design
Discuss a process for identifying interfaces used during a usage driven test. Consider how the concept of coverage used in white-box testing can be applied to minimize the number of usage-driven test cases.
11/2001
Integration Testing
26
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 27
Architecture May Reflect a Division of Labor
Components are work assignments
System All of Us
A Larry Moe
11/2001
B Curly
C
Joe
D
Integration Testing
28
Object-Oriented Architecture Sequence UML Diagrams
Sequence of methods required to implement (test) a use case Methods are explicit, but not every method is an interface May be incomplete if interfaces to non-OO portions of system are not included Sequence Diagram good for internal integration based on the OO model.
11/2001 Integration Testing 29
A Layered System Architecture Model
User operation triggers thread of activity across many levels. 2 Components vary in distance from user. 3
User-Visible -- GUI Business Logic
1 Service -- Database, Comm
11/2001
Integration Testing
30
Making the Connection II
User operations correspond to end-toend execution threads through system A carefully designed unit test for an operation usable as integration tests
Usage-Driven Interface Driven
11/2001
Integration Testing
31
Our Turn -- Architectural Models
From your experience, discuss how often:
a system architecture is hierarchical a system can be viewed in layers a system contains service components a system has little architectural structure at all
11/2001
Integration Testing
32
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 33
Rationale
Integration testing is driven by project size and architecture. Simple projects can be tested using a subset of the end-to-end tests used for system testing. More complex systems may require a combination of topdown, bottom-up and sandwich integration, tested by selected unit tests.
11/2001
Integration Testing
34
Integration Strategy
form tests button tests
User-Visible Components
Calls to business logic ...
Business Logic Test Drivers
Services Components
11/2001
Integration Testing
35
Proposed Integration Test Process
0. 1. 2. 3. 4. 5. 6.
11/2001
Define system architecture Define the integration test environment Associate components with layers Identify interfaces to layer components Bottom-up test services layer interfaces Top-down test user layer components Sandwich test business logic components
Integration Testing 36
0. Define System Architecture
Architecture should already exist Refine to reflect components built, bought or reused for the system Each component can be viewed as collection of callable functions and shared data, along with internal functions and private data Specify the physical environment in which testing will occur.
11/2001 Integration Testing 37
1. Integration Test Environment
Test results depend on test environment Different tests need different set-ups Repeatability of results requires control of test environment
Hardware platforms Native software suite and versions Data sets Version of software under test
11/2001 Integration Testing 38
2. Map Components To Layers
Logically associate software with layers
user-visible -- UI, navigation, user operations business logic -- data flow, processing, output services -- standard, reusable (comm, database)
Each layer contains one or more components
composite units like classes groups of units/components
11/2001
Integration Testing
39
3. Identify Layer Interfaces
Interface provides access to layer
calls with data passing global or shared data (database)
Identify interfaces actually used by system Record these in the IT Worksheet
Each interface must be covered during testing
11/2001
Integration Testing
40
4. Bottom-Up Test Service Interfaces
Service component provides set of related functions/data Refine/Develop test drivers for service component
This should have been done at unit test time Driver depends on the physical test environment Manual or automated set-up may be required
Run tests and verify expected results
11/2001 Integration Testing 41
5. Top-Down Test User Components
Verify presentation, navigation
Accomplished during unit test of forms/UI Repeat tests within integration environment
Verify calls to business logic, services triggered by action buttons
(Development): Verify interface calls compile (Testing): Usage-driven test
Can be done in parallel with services testing
11/2001 Integration Testing 42
6. Sandwich Test Business Logic
User-layer functions call business logic, which calls tested services The sandwich test is an end-to-end test of system This test reveals business logic errors
Top-down test verified that the right calls made Services have been tested Errors caused by business logic
11/2001 Integration Testing 43
Our Turn -- Test Practice
From your experience, discuss:
how often an explicit system architecture exists how often "components" are cleanly grouped things that may complicate mapping components to layers where most of the system test errors come from
11/2001 Integration Testing 44
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 45
RECOMMENDED PRACTICE
Architecture Diagram
Analysis Design
IT Strategy Interface List
IT Test Cases
Implementation Execution
Test Worksheet
Services Drivers UserOp Scripts IT Environment #Incidents, Incidents List
Evaluation
11/2001
Integration Testing
46
Products vs Deliverables
A Product is an informal artifact from applying a testing process.
belongs to the developer a physical or electronic document
A Deliverable is an artifact that attests to the application of the testing process
belongs to the team an attesting signature required
11/2001 Integration Testing 47
Deliverables
System/Software Architecture Diagram IT Environment Description
Repeatable test environment, set-up scripts, version control.
IT Test Worksheet (template)
Strategy, tests applied and interfaces covered, results summary
IT Incidents List / Count Signatures on the above
11/2001 Integration Testing 48
Products
Services Test Driver Services Test Results User Operation Test Script* User Operation Test Results*
*Incorporated into system (use case) test
11/2001
Integration Testing
49
Agenda
What is Integration? An Example of Integration Integration Test Approaches More on Architecture Proposed Integration Test Process Products & Deliverables Putting it All Together
11/2001 Integration Testing 50
Putting It All Together
Nail down the physical test environment and institute change control Integrate trusted components Devise strategy based on architecture Exploit unit testing for integration testing Automate testing of service components
11/2001 Integration Testing 51
Summary
The key to integration testing is the use of unit tested components. The architecture of the system drives the sequence of integration and test activities. With proper planning, integration testing leverages unit testing, resulting in a smooth and efficient integration test effort.
11/2001 Integration Testing 52
SOLUTIONS TO EXERCISES