Design
Brian Experimental Tissot Washington State University Vancouver
Lecture overview
1. 2. 3. 4. Objectives & Challenges Design Execution Management
Experimental Design
Scientific study of the natural world Major components:
1. Scientific method 2. Experimental design 3. Statistical analysis
Objectives
How can I design a study that will be able to detect meaningful biological changes with a reasonable amount of effort?
Major Challenges
1. Poor understanding of question 2. Poor design
Lack of proper replication Lack of statistical power to detect changes
Major Components 1) Design 2) Execution 3) Management
Define the Problem
Be able to concisely state to someone else the question you are asking. Your results will only be as coherent and comprehensible as your initial conception of the problem.
Define the Population
Clearly define the statistical population to sample. Define the statistical population precisely on a local scale.
Statistical Population
The study area of interest Limits the scope of inference Is large relative to our ability to obtain information
Census vs. Sample
Population
Census vs. Sample
Transect & Quadrats
Census vs. Sample
Census vs. Sample
Census 100% population
Census vs. Sample
Sample <1 % population
Census vs. Sample
Statistical analysis:
Populations are large; we are small Our inferences are limited and imperfect Need to use rigorous design Answer questions with degree of uncertainty (e.g., probabilities)
Replication
Take independent, replicate samples within each combination of time, location, and any other controlled variable. Lack of independence = Pseudoreplication
Replication
Repeated independent sampling of environmental responses
e.g., Quadrats along a transect
1
2
3
Replication
Replication must occur in each sampling subunit:
Site 1
Shallow
Site 2 3
Shallow
1
2
1
2
3
1
Deep
2
3
Deep
1
2
3
QUEST Sampling Design
West
East
Power & Sample Size
Calculate appropriate sample size based on power analysis Power: ability of statistical test to reject a false null hypothesis H 0 : 1 = 2 H1 : 1 2
Statistical Errors
Reality H0 True Accept H0 H0 False Type II Error
OK
Results of test
Reject H0
Type I error
OK
Consequences of errors
Experiment Monitoring
Type I Type II
False Effects
False Alarm
Undetected results
Undetected impacts
Factors Influencing Power
1. Alpha: as - 2. Variance: as 2 - 3. Sample size: as n -
Set Minimize by design Power analysis
X
Power Curves
T-Test
X1 X 2 T= s n
T=.05 2
Minimum n
2s n X1 X 2
2
X
Power Curves
Power Analysis
Average coral cover Kapoho, Hawai'i
300 300
Power Analysis
Average fish abundance Molokini, Maui = 0.05 = 0.10 Minimum sample size
200
= 0.05 = 0.10 Minimum sample size
200
100
100
0 0 10 20 30 40 50
0 0 10 20 30 40 50
Minimum detectable difference
% difference between means
Minimum detectable difference
% difference between means
X
Power Analysis
Total coral cover
300
Kapoho Puako Molokini
Minimum sample size
200
= 0.05 = 0.10
Optimal sample size given accurate vs. effort
100
0 0 10 20 30 40 50
Minimum detectable difference
% difference between means
Control vs. Impact
To test whether a condition has an effect, collect samples both where the condition is present and where the condition absent is but where all else is the same. An effect can only be demonstrated by comparison to a control.
Control vs. Impact
Coral reef
Control
Impact
Before vs. After
To test whether a condition has an effect, collect samples both before the condition is present and after the condition is absent but where all else is the same. An effect can only be rigorously demonstrated by before vs. after comparisons.
Before vs. After
Coral reef
Sample Before
Sample After
Impact
BACI: Before-After Control-Impact Comparison Procedure
Before Control After
Density
Perturbation Impact
Time
Diablo Canyon NPP
Diablo Canyon NPP
Mazzaella flaccida Iridescent seaweed
75 Mean % Cover
Control
50 25 0 1978 1981 1984 1987 1991 1995
Time
75 Mean % Cover
Control
50 25 0 1978 1981 1984 1987 1991 1995
Time
75 Mean % Cover
Impact
50 25 0 1978 1981 1984 1987 1991 1995
Time
75 Mean % Cover
Control
50 25 0 1978 1981 1984 1987 1991 1995
Time
75 Mean % Cover
Impact
50 25 0 1978 1981 1984 1987 1991 1995
Time
BACI
Two-sample T-test: Before vs. After DF = 6, T = 5.55, P < 0.01
50 Difference in Mean % Cover
There is a greater difference between control and impact sites after the Plant began than before. However, there is <1% possibility that this difference is due to chance
25
Difference
0 1978 1981 1984 1987 1991 1995
Time
Mill Creek
On WSU Vancouver campus New culvert installed in summer 1999 SEPA Mitigated DNS: Erosion minimized through silt fencing Maintained during construction Disturbed slopes reseeded within 24h
Mill Creek
YSI Multi-probe Sonde Measured temperature, [O2], turbidity Before, during and after construction
Mill Creek, WA Water quality during WSU culvert construction
1000
Pre-Construction Baseline August, 1999
Rain Event
100
10
Turbidity (NTU)
1 8/10/2099
8/12/2099
8/14/2099
8/16/2099
Mill Creek, WA Water quality during WSU culvert construction
1000
Pre-Construction Baseline August, 1999
Rain Event
100
10
Turbidity ...