235, MS&E Internet Commerce
Stanford University, Winter 2007-08
Instructor: Prof. Ashish Goel, Notes Scribed by Shrikrishna Shrin
Lecture 7a: Long Tails
Imagine a market where some class of products is being sold. We plot the ith largest product on the X axis and the market size (Revenue/No of units sold/No of customers) on the Y axis. A long tail typically looks like the graph in the image 1 below.
The graph is said to have a long tail if it can be represented by an equation of the form m(i) a(b + i)e where e is called the exponent. Notice that the market size only decreases polynomially with increase in i. For simplicity, let us assume that m = ie . We can plot another curve with market size on the X axis and frequency (number of items with the corresponding market size) on the Y axis. We nd that as market size increases, frequency decreases. di dm It is observed that long tails are created as a consequence of human endeavor or manufactured (ie. not natural) activities of human beings. Natural traits however do not exhibit the long tail phenomenon and tend to decay exponentially (of the form bi , notice that i is in the exponent in this case) instead. A few quotes on how long tails are created: f = Long tails are a signature of human activity.
1 Original
image obtained from http://www.entrepreneurs-journey.com/images/long-tail-
graph.gif
1
Long tails are a result of the tension between human individualism and social behavior. For example, if customers were to walk into a shoe store, some customers would buy a Nike shoe because it is a popular shoe and many of his friends may have bought that shoe (social behavior). However, some people would prefer to buy a shoe that is not very common and it is this tension between social behavior and individualism that leads to the long tail. If all customers bought a shoe only based on social behavior, then everyone would buy the same shoe. On the other hand, if everyone bought a shoe only based on individualism, everyone would buy a dierent shoe (assuming there are enough choices).
Modeling the long tail phenomenon
Let us assume that customers arrive one by one at a music website. Every customer does two things. 1. Choose a previously heard song with probability proportional to the number of times it has been heard in the past (Social behavior) 2. Find a new obscure song and listen to it. After t customers arrive, we have 2t hearings and t songs that have been heard. Let mt (1), mt (2), .....mt (t) = the number of times song i has been heard after t steps. Probability that a customer at time t + 1 listens to the rst song is mt (1)/2t. Probability that t + 1th customer song chooses i(i t) = mt (i)/2t. Somewhat informally, mt (i)/2t can be thought of as the expected increase in volume of product i (song i) at time t + 1. Therefore, mt+1 (i) = mt (i) + mt (i)/2t. Rewriting the above equation we get, mt (i) mt+1 (i) mt (i) = t+1t 2t On taking the limit, the above equation becomes: dmt (i) = mt (i)/2t dt On rearranging the above equation we get: dmt (i) dt = mt (i) 2t On integrating with limits i to t ( mt (i) is not dened for t < i) we get: ln(mt (i)) ln(mi (i)) = 1 (ln(t) ln(i)) 2
2
Now, ln(mi (i)) is 0 as at t = i, mi (i) = 1. Therefore, we get: t i This equation is the power law. Suppose customers were to choose a new product with probability p and an old product with probability 1 p then the only change in the power law will be the value of the exponent. Now we can obtain the frequency as: mt (i) = f = 1 di = dm dm di
2
= 3 1 t 3
i2
i Substituting for i we get,
3 2
t
( mt )3 t t m3 As m increases, f of selling the item decreases. However, the volume of sale can still be signicant.
Determining the median
The median is that point at which half the market lies in the tail. Now, we can derive that market size = 2t as follows: t mi (t) i Integrating from limits i = 1 to i = t we get,
t
mi (t) di =
1
t
t
1
1 di i
= 2 t( t 1) 2t mi (t) di = Let M be the total market size and med be a value such that 1 M 2 = t (M = 2t). (Half the market lies below this product and half the market lies after this product (ie. in the tail)). 3
med
Therefore, 2 t med t. t Which implies med 4 . The top 25% of the pro...