Unformatted text preview: Improvements to APriori
ParkChenYu Algorithm Multistage Algorithm Approximate Algorithms Compacting Results
1 PCY Algorithm
Hashbased improvement to APriori. During Pass 1 of Apriori, most memory is idle. Use that memory to keep counts of buckets into which pairs of items are hashed.
Just the count, not the pairs themselves. Gives extra condition that candidate pairs must satisfy on Pass 2.
2 Picture of PCY
Item counts Hash table Frequent items Bitmap Counts of candidate pairs Pass 1 Pass 2
3 PCY Algorithm  Before Pass 1 Organize Main Memory
Space to count each item.
One (typically) 4byte integer per item. Use the rest of the space for as many integers, representing buckets, as we can. 4 PCY Algorithm  Pass 1
FOR (each basket) { FOR (each item) add 1 to item’s count; FOR (each pair of items) { hash the pair to a bucket; add 1 to the count for that bucket } } 5 Observations About Buckets
1. If a bucket contains a frequent pair, then the bucket is surely frequent.
We cannot use the hash table to eliminate any member of this bucket. 2. Even without any frequent pair, a bucket can be frequent.
Again, nothing in the bucket can be eliminated.
6 Observations  (2)
3. But in the best case, the count for a bucket is less than the support s.
Now, all pairs that hash to this bucket can be eliminated as candidates, even if the pair consists of two frequent items. 7 PCY Algorithm  Between Passes
Replace the buckets by a bitvector:
1 means the bucket count exceeds the support s (frequent bucket); 0 means it did not. Integers are replaced by bits, so the bitvector requires little secondpass space. Also, decide which items are frequent and list them for the second pass.
8 PCY Algorithm  Pass 2
Count all pairs {i,j } that meet the conditions:
1. Both i and j are frequent items. 2. The pair {i,j }, hashes to a bucket number whose bit in the bit vector is 1. Notice all these conditions are necessary for the pair to have a chance of being frequent.
9 Memory Details
Hash table requires buckets of 24 bytes.
Number of buckets thus almost 1/41/2 of the number of bytes of main memory. On second pass, a table of (item, item, count) triples is essential.
Thus, hash table must eliminate 2/3 of the candidate pairs to beat apriori.
10 Multistage Algorithm
Key idea: After Pass 1 of PCY, rehash only those pairs that qualify for Pass 2 of PCY. On middle pass, fewer pairs contribute to buckets, so fewer false positives frequent buckets with no frequent pair. 11 Multistage Picture
Item counts First hash table Freq. items Bitmap 1 Second hash table Freq. items Bitmap 1 Bitmap 2 Counts of Candidate pairs 12 Multistage  Pass 3
Count only those pairs {i,j } that satisfy:
1. Both i and j are frequent items. 2. Using the first hash function, the pair hashes to a bucket whose bit in the first bitvector is 1. 3. Using the second hash function, the pair hashes to a bucket whose bit in the second bitvector is 1.
13 Important Points
1. The two hash functions have to be independent. 2. We need to check both hashes on the third pass.
If not, we would wind up counting pairs of frequent items that hashed first to an infrequent bucket but happened to hash second to a frequent bucket.
14 Multihash
Key idea: use several independent hash tables on the first pass. Risk: halving the number of buckets doubles the average count. We have to be sure most buckets will still not reach count s. If so, we can get a benefit like multistage, but in only 2 passes.
15 Multihash Picture
Item counts First hash table Second hash table Freq. items Bitmap 1 Bitmap 2 Counts of Candidate pairs 16 Extensions
Either multistage or multihash can use more than two hash functions. In multistage, there is a point of diminishing returns, since the bitvectors eventually consume all of main memory. For multihash, the bitvectors total exactly what one PCY bitmap does, but too many hash functions makes all counts > s.
17 All (Or Most) Frequent Itemsets In < 2 Passes
Simple algorithm. SON (Savasere, Omiecinski, and Navathe). Toivonen. 18 Simple Algorithm  (1)
Take a mainmemorysized random sample of the market baskets. Run apriori or one of its improvements (for sets of all sizes, not just pairs) in main memory, so you don’t pay for disk I/O each time you increase the size of itemsets.
Be sure you leave enough space for counts.
19 The Picture
Copy of sample baskets Space for counts 20 Simple Algorithm  (2)
Use as your support threshold a suitable, scaledback number.
E.g., if your sample is 1/100 of the baskets, use s /100 as your support threshold instead of s . 21 Simple Algorithm  Option
Optionally, verify that your guesses are truly frequent in the entire data set by a second pass. But you don’t catch sets frequent in the whole but not in the sample.
Smaller threshold, e.g., s /125, helps. 22 SON Algorithm  (1)
Repeatedly read small subsets of the baskets into main memory and perform the first pass of the simple algorithm on each subset. An itemset becomes a candidate if it is found to be frequent in any one or more subsets of the baskets.
23 SON Algorithm  (2)
On a second pass, count all the candidate itemsets and determine which are frequent in the entire set. Key “monotonicity” idea: an itemset cannot be frequent in the entire set of baskets unless it is frequent in at least one subset.
24 Toivonen’s Algorithm  (1)
Start as in the simple algorithm, but lower the threshold slightly for the sample.
Example: if the sample is 1% of the baskets, use s /125 as the support threshold rather than s /100. Goal is to avoid missing any itemset that is frequent in the full set of baskets.
25 Toivonen’s Algorithm  (2)
Add to the itemsets that are frequent in the sample the negative border of these itemsets. An itemset is in the negative border if it is not deemed frequent in the sample, but all its immediate subsets are. 26 Example: Negative Border
ABCD is in the negative border if and only if it is not frequent, but all of ABC, BCD, ACD, and ABD are. 27 Toivonen’s Algorithm  (3)
In a second pass, count all candidate frequent itemsets from the first pass, and also count the negative border. If no itemset from the negative border turns out to be frequent, then the candidates found to be frequent in the whole data are exactly the frequent itemsets.
28 Toivonen’s Algorithm  (4)
What if we find something in the negative border is actually frequent? We must start over again! Try to choose the support threshold so the probability of failure is low, while the number of itemsets checked on the second pass fits in mainmemory.
29 Theorem:
If there is an itemset frequent in the whole, but not frequent in the sample, then there is a member of the negative border frequent in the whole. 30 Proof:
Suppose not; i.e., there is an itemset S frequent in the whole, but not frequent or in the negative border in the sample. Let T be a smallest subset of S that is not frequent in the sample. T is frequent in the whole (monotonicity). T is in the negative border (else not “smallest”).
31 Compacting the Output
1. Maximal Frequent itemsets : no
immediate superset is frequent. 2. Closed itemsets : no immediate superset has the same count.
Stores not only frequent information, but exact counts. 32 Example: Maximal/Closed
Count A 4 B 5 C 3 AB 4 AC 2 BC 3 ABC 2 Maximal s=3 No No No Yes No Yes No Closed No Yes No Yes No Yes Yes
33 ...
View
Full Document
 Fall '09
 Algorithms, hash function, main memory, Cuckoo hashing, Bloom filter, negative border

Click to edit the document details