107: Stat Introduction to Business and Financial Statistics
Class 3: Statistics Review, Part I
1
Homework Discussion
Due Midnight Friday night via email
[e-mail]
Idea of Momentumpapers on website
etfreplay.com
200 day moving average (more today on this)
Excel spreadsheet 200-ma.xls
2
UAPIX vs FISMX
Which fund
would
would you
prefer to own?
FISMX
UAPIX
3
Review of Basic Statistics
Measures of location (mean and median)
Measures of spread (variance and standard
deviation)
Measures of association (covariance and
correlation)
Random Variables and Distributions
Hypothesis Testing (next lecture)
Simple Regression (in a few lectures)
4
Textbook Note
The Estrada book, The FT Guide to
Understanding Finance, has three wonderful
appendices on basic statistics which would
be
be a good refresher.
FYI, as far as I can tell, the first edition,
Finance in a Nutshell, is about the same
book, and available used for $22 on Amazon.
5
Mean (Arithmetic Average)
The (sample) Mean is the arithmetic average
of data values
n = Sample Size
n
x
x=
i =1
n
i
x1 + x 2 + L + x n
=
n
6
Mean (Arithmetic Average)
The most common measure of central tendency
Mean = sum of values divided by the number of values
Hmmm.Does
Affected by extreme values (outliers)
financial data ever
have outliers?
0 1 2 3 4 5 6 7 8 9 10
Mean = 3
1 + 2 + 3 + 4 + 5 15
=
=3
5
5
0 1 2 3 4 5 6 7 8 9 10
Mean = 4
1 + 2 + 3 + 4 + 10 20
=
=4
5
5
7
Finance Application: Moving Averages
If you follow data collected over time (called
time series data), a useful statistic to
compute is called a moving average
A moving average smooths the time series
data, helping you observe trends.
8
Advertisement
9
GE stock price : 10 day moving average
10
What happens as more days are used in the MA ?
Moving averages are lagging
indicators, telling you what
happened, not what will happen
The terminology of a fast versus slow moving average
11
Doing this in R
> getSymbols("GE",from="2010-01-01")
[1] "GE
plot(Cl(GE),main="GE Closing Prices")
lines(SMA(Cl(GE),200),col="blue")
> lines(SMA(Cl(GE),50),col="red")
12
Resulting Graph
13
The 200 Day SMA System
Buy when the price closes above the 200 day
moving average.
Sell when the price closes below the 200 day
moving
moving average.
Does this work? Do your homework this
week.
14
StrategyDesk
From TD Ameritrade
Very easy to test trading systems
15
StrategyDesk
16
What a Frickin Mess
17
Another Basic Stock Trading System
Calculate two moving averages, a slow one (say
100 days) and a fast one (say 10 or 20 days).
The fast MA reflects the state of the short term and
the
the slow MA the state of the longer term trend.
Trading Signals are generated when the two MAs
cross, and the direction of the signal is the same
direction as the direction that the short term crosses
the longer term.
18
Does this work ?? Maybe..
S
S
B
B
S
B
19
StrategyDesk Output
20
More StrategyDesk Output
Dont bet the farm
21
Other Moving Averages
There are a ton of different moving averages.
If you examine the simple moving average,
you see that each observation is weighted by
n
1/n
1/n
xi n 1
x = i =1 = xi
n
i =1 n
People love to play with these weights to try
to obtain better moving averages.
22
Weighted Moving Averages
The weighted moving average method consists
of computing a weighted average of the most
recent n data values for the series.
The more recent observations are typically given
more weight than older observations.
For convenience, the weights usually sum to 1.
n
w x
ii
i =1
23
Weighted Moving Averages
Well revisit this topic when we cover
technical analysis.
help(SMA) in R:
24
The Median
Not affected by extreme values
0 1 2 3 4 5 6 7 8 9 10
Median = 3
Median = 3
In an ordered array, the median is the
middle number
If n is odd, the median is the middle number
If n is even, the median is the average of the
two middle numbers
The median isnt used really in technical analysis..
25
Computer Science Type Question
Computationally, which summary measure of
location is more expensive to compute, the
mean or the median?
26
Shape of a Distribution
Describes how data is distributed
Symmetric or skewed
Left-Skewed
Symmetric
Right-Skewed
Mean < Median < Mode Mean = Median =Mode Mode < Median < Mean
(Longer tail extends to left)
We will revisit skewness in a few slides.
(Longer tail extends to right)
27
Quartiles-another descriptive measure
Quartiles split the ranked data into 4
equal groups
25% 25%
Q1
Q2
25%
25%
Q3
28
The summary command in R
29
Measure of Dispersion
The mean and median give us information about the
central tendency of a set of observations, but these
numbers shed no light on the dispersion, or spread
of the data.
Example: Which data set is more variable ??
5,5,5,5,5
Mean = 5
1,3,5,8,8
Mean = 5
30
Which Mutual Fund is Riskier
FDVLX = Fidelity Value
FCNTX = Fidelity Contra
31
Which one is riskier?
Black = Value
Red = Contra
32
Aside
There is a new package in R I am playing
with that is great for comparing several
stocks at once.
Library(PerformanceAnalytics)
33
The Variance
The variance of a set of data is defined as
n
( x i x )2
2
sX =
Benninga discusses this n-1
crap a little bit in the book.
i =1
n 1
What are the units of this formula?
34
Standard Deviation
Most commonly used measure of variation
Shows variation about the mean
Has the same units as the original data
n
(x i x )2
s=
i=1
n -1
:
35
Which is a Riskier Fund?
36
Simple ways to compare funds
(based on monthly returns)
http://seekingalpha.com/article/274266-building-a-permanent-portfolio-using-etfs
37
R code for last graph (just a fyi)
fnames=c("FDVLX","FDEGX","FCNTX","PRPFX","MUHLX","TPINX,
"PTTAX","PUBAX","DODIX","SPHIX")
mike=function(ticker) {
x <- getSymbols(ticker, auto.assign=FALSE,from="2008-01-01")
foo=monthlyReturn(x)
return(c(mean(foo),sqrt(var(foo))))
}
results=matrix(nrow=10,ncol=2)
for(i in 1:10)
results[i,]=mike(fnames[i])
plot(results[,2],results[,1],cex=1.5,xlab="Risk",ylab="Return",
ylim=c(-.0025,.0075))
text(results[,2],results[,1]+.0005,cex=.7,fnames)
38
Standard Deviation has issues
There are many performance measures we
will eventually look at.
One issue with standard deviation is upside
and downside deviations are treated equally,
but in finance downside deviations are more
serious.
Semi standard deviation is a measure of
downside risk, which we will eventually
discuss.
39
Trampoline Example
Of the two, wouldnt this be the
preferred investment manager?
#1
#2
Semi standard deviation or downside risk
Here is the Trampoline Math
5
-5
5
-5
5
-5
Standard Deviation =
9
-1
9
-1
9
-1
5.48
5.48
Implication-we probably need something a little bit more sophisticated than
standard deviation to compare investments. Details in a few lectures.
The empirical rule will help us understand sx and
relate the summaries back to the dotplot (or histogram).
Empirical rule:
no
For mound shaped data:
Approximately 68% of the data is in the interval
( x s x, x + s x ) = x s x
Approximately 95% of the data is in the interval
( x 2s x , x + 2s x ) = x 2s x
42
Tons of information is
available on the web:
finance.yahoo.com
Want this positive!
Want this small!
What does the empirical rule
say about possible gains and
losses ?
x 2 sx
(what are we assuming? Very Important!)
43
You find xbar +/- 2s in Many Places
Bollinger bands are x 2 s x
(based on a moving window of
20 time periods)
See http://en.wikipedia.org/wiki/Bollinger_Bands
44
Bollinger Bands Example Chart
Sell signal
Buy signals
Sometimes, the buy
signals just keep coming and
you can go broke!
45
Dont fall in love with xbar +/- 2s
Standard deviation is a good measure of
spread if your data is symmetric; if your data
is not symmetric it really isnt interpretable.
Look at the results trading a basic bband
system:
46
Further Moments of the Distribution
There are further statistics that describe the
shape of the distribution, using formulae that are
similar to those of the mean and variance
1st moment - Mean (describes central value)
2nd moment - Variance (describes dispersion)
3rd moment - Skewness (describes asymmetry)
4th moment - Kurtosis (describes peakedness)
Further Moments Skewness
Skewness measures the degree of asymmetry
exhibited by the data
n
(x x)
3
i
skewness =
i =1
ns
3
If skewness equals zero, the histogram is
symmetric about the mean
Positive skewness vs negative skewness
Further Moments Skewness
Further Moments Kurtosis
Kurtosis measures how peaked the histogram is
n
(x x)
4
i
kurtosis =
i
ns
4
3
The kurtosis of a normal distribution is 0
Kurtosis characterizes the relative peakedness
or flatness of a distribution compared to the
normal distribution.
The R command kurtosis() does not subtract 3
Further Moments Kurtosis
Platykurtic When the kurtosis < 0, the
frequencies throughout the curve are closer to be
equal (i.e., the curve is more flat and wide)
Thus, negative kurtosis indicates a relatively flat
distribution
Leptokurtic When the kurtosis > 0, there are
high frequencies in only a small part of the curve
(i.e, the curve is more peaked)
Thus, positive kurtosis indicates a relatively
peaked distribution
Positive Kurtosis Issues
This is a little premature since we havent covered
the Normal distribution yet but.
Kurtosis measures the "fatness" of the tails of a
distribution. Positive kurtosis means that
distribution has fatter tails than a normal
distribution (more outliers than normal).
Fat tails means is there a higher than normal
probability of big positive and negative returns
realizations.
Many stocks exhibit positive Kurtosis (heavy tails)
52
Mandelbrot Knew This
Benot B. Mandelbrot[note 1][note 2] (20 November 1924 14
October 2010) was a French American mathematician. Born
in Poland, he moved to France with his family when he was
a child. Mandelbrot spent much of his life living and working
in the United States, and he acquired
dual French and American citizenship.
Mandelbrot worked on a wide range of mathematical
problems, including mathematical physics and quantitative
finance, but is best known as the father of fractal geometry.
He coined the term fractal and described the Mandelbrot set.
Mandelbrot extensively popularized his work, writing books
and giving lectures aimed at the general public
53
Reality vs. Models
The standard statistical approach to risk management is based on a bell
curve or normal distribution, in which most results are in the middle and
extremes are rare.
It is the bell curve to which investors are referring when they talk about a
nine standard deviation event.
But financial history is littered with bubbles and crashes, demonstrating that
extreme events or so-called fat tails occur far more often than the bell curve
predicts.
Spooking investors
Oct 25th 2007
From The Economist print edition
Deal or No Deal? Normal or Not Normal?
Histogram of Monthly Returns
May 1986 - April 2006
14
Percentage of Months (%)
15
12
10
8
6
4
2
0
< -10 -10 to -9 -9 to -8 -8 to -7 -7 to -6 -6 to -5 -5 to -4 -4 to -3 -3 to -2 -2 to -1
-1 to 0
0 to 1
1 to 2
2 to 3
3 to 4
4 to 5
5 to 6
6 to 7
7 to 8
8 to 9
>9
Returns Range (%)
Russell 1000
Russell 2000
Russell Midcap
Russell 3000
Twenty Year Histogram of Monthly Index Returns
If it doesnt fit, you must acquit.
- Johnnie Cochran
The Fatter the distribution tails, the less
reliable the statistics!
If the population of price changes is strictly normal, on average for any stock..an
observation more than five standard deviations from the mean should be observed
about once every 7,000 years. In fact such observations seem to occur about once
every
every three or four years Eugene Fama, Journal of Business, January 1965
Under the assumption of normal return distributions, the probability of the October
1987 crash was so remote that according to efficient market theory it would have
been virtually impossible Jackwerth and Rubinstein, Journal of Finance, Vol 51 1996
The problem for traders is that it is much more complicated to create models for a
world of fat tails than for a world of bell curves. As a result, traders repeatedly get
caught out by unprecedented market movements. The collapse of two hedge funds,
Long-Term Capital Management in 1998 and Amaranth Advisors in 2006, were cases in
point The Economist, October 18th 2007.
Well come back to these ideas later in the course
Calculating Moments in R
library(moments)
> getSymbols("JPM",from="2009-01-29")
[1] "JPM"
> hist(dailyReturn(Ad(JPM)),breaks=20)
> kurtosis(dailyReturn(Ad(JPM)))
daily.returns
13.74744
57
Excel Sheet: kurtosis.xls
58
AIG-fat tails
59
JPM-fat tails
60
Covariance and Correlation
Both are measures of linear association
Covariance measures direction of association
Correlation measures direction and strength
Correlation is always between -1 and 1.
61
The Covariance Formula
The sample covariance between x and y is:
s xy
1n
=
( xi x)( yi y)
n 1 i =1
62
The Variance-Covariance Matrix
Note that Cov(X,X)=Var(X)
63
Making Size Matter
The correlation coefficient
rxy =
Standard deviation of x
s xy
covariance
s xs y
Standard deviation of y
64
Using R
65
Rule of Thumb
Magnitude of r
Interpretation
.00-.20
Very weak
.20-.40
Weak to moderate
.40-.60
Medium to substantial
.60-.80
Very Strong
.80-1.00
Extremely Strong
66
Some Correlation Issues
DIVERSIFICATION is supposed to be one
of the rare free lunches in finance. Spread
your assets geographically (or by asset
class) and the chances are that your
investments will not rise and fall together.
Investors should be able to get the same
reward with less risk.
But, as the chart shows, global
stockmarkets have steadily become more
correlated over the past few decades.
Wake to the financial headlines on any
given morning and you will find that a selloff in Asia has spread to Europe and that,
all too often, both continents are reacting to
a late plunge on Wall Street. It is rare for
individual markets to go against the trend.
http://www.economist.com/node/21528640
67
Nowadays, everything is correlated
http://www.zerohedge.com/news/stock-correlations-soar-972-heres-why
68
What time period? It matters
> getSymbols("GLD",from="2011-01-01",to="2011-02-01")
[1] "GLD"
> getSymbols("SPY",from="2011-01-01",to="2011-02-01")
[1] "SPY"
> cor(dailyReturn(Ad(SPY)),dailyReturn(Ad(GLD)))
daily.returns
daily.returns
-0.1397383
> getSymbols("GLD",from="2011-03-01",to="2011-04-01")
[1] "GLD"
> getSymbols("SPY",from="2011-03-01",to="2011-04-01")
[1] "SPY"
> cor(dailyReturn(Ad(SPY)),dailyReturn(Ad(GLD)))
daily.returns
daily.returns
0.1452962
69
Low Correlation Portfolios
There is something to be said for holding low
or no correlation assets.
More on this later in the course.
See PRPFX (remember the risk/return chart)
Also from ripetrade.com using different asset
classes for momentum trading:
70
Random Variables
X is a random variable if it represents a
random draw from some population
A discrete random variable can take on
only
only selected values
A continuous random variable can take on
any value in a real interval
Associated with each random variable is a
probability distribution
71
A summary before we begin
Sample (Data)
Histogram
X Random Variable
Distribution
Mean x
Expectation E(X)
Variance s 2
Variance Var(X)
We will focus on discrete random variables today, and get to
continuous random variables in the next lecture.
72
The Probability Function
The probability mass function PX(x) of a discrete
random variable expresses the probability that X
takes the value x:
PX ( x) = P( X = x)
Example: X = outcome when we roll a fair die.
PX (1) = 1 / 6,PX (2) = 1 / 6,etc ...
What is PX(17) ??
73
We assign probabilities (chances) to all the
values that the random variable can take on.
There are certain requirements about the
values we assign:
If a random variable X can take values xi, then
the
the following must be true:
(1) 0 PX (x i ) 1
(2 )
PX (x i ) = 1
(exhaustive)
all x i
PX (x ) is sometimes called the probability distribution function
74
Cumulative Distribution Function
Another concept we sometimes use is called
the cumulative distribution function (CDF):
The cumulative distribution function FX(x0) of
a random variable X expresses the probability
random
that X does not exceed the value x0:
FX ( x 0 ) = P ( X x 0 ) =
P
X
(x )
x x 0
Probability X is less than or equal to some value
75
Expectation and its Applications
For a random variable, the analogy to the sample
mean is called the expectation or expected value.
The letter E usually denotes an expected value,
and this symbol is usually followed by brackets
enclosing the random variable of interest.
Definition: Given a random variable X
with values xi, the expected value of X is
mu? moo!
X = E( X ) =
x P( X = x )
i
i
all xi
=
x P( x )
i
i
all xi
The expected value is simply a weighted average of the possible values X can
assume. where the weights are the probabilities of occurrence of those values.
76
Let W = winnings. W = 499 (cost $1 to play) or -1
1
999
E (W ) = 499
+ ( 1)
= 0.50
1000
1000
77
So in the long run, we lose 50 cents each time we play.
Note that on any one play we never lose 50 cents (we
either win 499 or lose 1); rather, this is saying that if you
play the game 10000 times, you can expect to be roughly
down $5000 at the end.
An even better way to look at it is that if 10 million people
play the game every day, the state can expect to only
have to give back about $5 million, a daily profit of a cool
$5 million (this is why states run lotteries!)
78
One more betting example
Find the expected value of each of the following bets:
a) you get $5 with probability 1.0.
b) you get $10 with probability 0.5, or $0 with probability 0.5.
c) you get $5 with probability 0.5, $10 with probability 0.25
and $0 with probability 0.25.
d) you get $5 with probability 0.5, $105 with probability 0.25
or lose $95 with probability 0.25.
Which bet is the best bet ?
79
Variance of a Random Variable
The mean of a random variable is a very useful
and informative quantity, but we are often
interested in other measures of a distribution.
The variance and standard deviation are measures
of the dispersion of a random variable around its
mean.
80
Technically, the variance of a random variable is
the expected value of the squared deviation of a
random variable from its mean. Phew.
The general mathematical formula is
2 = Var ( X ) = E [(X X )2 ] = E (X 2 ) 2
X
X
For discrete random variables, this simplifies to
2 = Var (X ) = E [(X X )2 ] =
X
The standard deviation is
X =
of course
2
X
( x i )2 P ( X = x i )
all x i
weighted average of deviations from the mean81
Example : Pick 3 Game
2 = Var (X ) = E [(X X )2 ] =
X
( x i )2 P ( X = x i )
all x i
Var = [499 ( .5)]2 x
1
999
+ [ 1 ( .5)]2 x
= 249.75
1000
1000
The standard deviation of this is sqrt(249.75)=$15.8
What does it imply that the std deviation is so much
larger than the mean ?
82