{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lec19 - ohdatamineRandForests

Lec19 - ohdatamineRandForests - DATA MINING Susan Holmes...

Info icon This preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
. . . . . . DATA MINING Susan Holmes © Stats202 Lecture 19 Fall 2010 A B a b c d f g h i e j kl
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
. . . . . . Special Announcements I Do not update your version of R before the end of the quarter. I All requests should be sent to [email protected] . I A new homework will be put up Wednesday. I Kaggle: data mining competition, details on Wednesday.
Image of page 2
. . . . . . Last Time: SVMs and Ensemble Methods Examples I Support Vector Machines. I Bootstrap: multiples (with replacement). I Bagging: Bootstrap Aggregation. I Boosting : combining weak learners. I Random Forests.
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
. . . . . . Bootstrapping table(sample(30,30,replace=T)) 1 3 4 5 6 7 10 11 12 16 17 20 21 23 24 25 26 27 28 30 2 2 1 1 1 2 2 1 3 1 2 3 1 1 1 1 1 1 2 1 > rep30=rep(0,500) > for (i in 1: 500){rep30[i]=length(table(sample(30,30,replace=T))) > mean(rep30) [1] 19.042 mean(rep30)/30 [1] 0.6347333 ######Limit by simulation > 1-(999/1000)^1000 [1] 0.6323046 P ( x 1 is in the bootstrap resample ) = 1 - (1 - 1 n ) n (1 - 1 n ) n = exp ( nlog (1 - 1 n )) expn ( - 1 n ) = exp ( - 1) = 1 e = 0 . 36788 OOB=out of the bag (not included in the Bootstrap resample). OOB prediction is determined by a majority rule vote of all trees whose training set did not contain that observation.
Image of page 4
. . . . . . Example of Boosting library(rpart) library(mlbench) data(BreastCancer) l <- length(BreastCancer[,1]) sub <- sample(1:l,2*l/3) train=BreastCancer[sub,-1] BC.rpart <- rpart(Class~.,data=BreastCancer[sub,-1], maxdepth=3) BC.rpart.pred <- predict(BC.rpart,newdata=BreastCancer[-sub,-1],ty tb <-table(BC.rpart.pred,BreastCancer$Class[-sub]) error.rpart <- 1-(sum(diag(tb))/sum(tb))
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
. . . . . . Example of Boosting tb : BC.rpart.pred benign malignant benign 131 3 malignant 13 86 error.rpart [1] 0.06866953 train2=train[which(!is.na(train$Class)),] BC.adaboost =adaboost.M1(Class~.,data=train2, mfinal=25, maxdepth=3) BC.adaboost.pred =predict.boosting(BC.adaboost, newdata=BreastCancer[-sub,-1]) BC.adaboost.pred[-1] $confusion Observed Class Predicted Class benign malignant benign 147 4 malignant 3 80 $error[1] 0.02991453 BC.adaboost$importance Cl.thickness Cell.size Cell.shape Marg.adhesion Epith.c 15.789474 10.526316 6.140351 5.263158 5.26315 Bare.nuclei Bl.cromatin Normal.nucleoli Mitoses 23.684211 7.017544 20.175439 6.140351
Image of page 6
. . . . . .
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern