{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# trees - Classification/Decision Trees(I...

This preview shows pages 1–10. Sign up to view the full content.

Classification/Decision Trees (I) Classification/Decision Trees (I) Jia Li Department of Statistics The Pennsylvania State University Email: [email protected] http://www.stat.psu.edu/ jiali Jia Li http://www.stat.psu.edu/ jiali

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Classification/Decision Trees (I) Tree Structured Classifier Reference: Classification and Regression Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Chapman & Hall, 1984. A Medical Example ( CART ): Predict high risk patients who will not survive at least 30 days on the basis of the initial 24-hour data. 19 variables are measured during the first 24 hours. These include blood pressure, age, etc. Jia Li http://www.stat.psu.edu/ jiali
Classification/Decision Trees (I) A tree structure classification rule for the medical example Jia Li http://www.stat.psu.edu/ jiali

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Classification/Decision Trees (I) Denote the feature space by X . The input vector X X contains p features X 1 , X 2 , ..., X p , some of which may be categorical. Tree structured classifiers are constructed by repeated splits of subsets of X into two descendant subsets, beginning with X itself. Definitions: node , terminal node (leaf node) , parent node , child node . The union of the regions occupied by two child nodes is the region occupied by their parent node. Every leaf node is assigned with a class. A query is associated with class of the leaf node it lands in. Jia Li http://www.stat.psu.edu/ jiali
Classification/Decision Trees (I) Notation A node is denoted by t . Its left child node is denoted by t L and right by t R . The collection of all the nodes is denoted by T ; and the collection of all the leaf nodes by ˜ T . A split is denoted by s . The set of splits is denoted by S . Jia Li http://www.stat.psu.edu/ jiali

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Classification/Decision Trees (I) Jia Li http://www.stat.psu.edu/ jiali
Classification/Decision Trees (I) The Three Elements The construction of a tree involves the following three elements: 1. The selection of the splits. 2. The decisions when to declare a node terminal or to continue splitting it. 3. The assignment of each terminal node to a class. Jia Li http://www.stat.psu.edu/ jiali

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Classification/Decision Trees (I) In particular, we need to decide the following: 1. A set Q of binary questions of the form { Is X A ? } , A X . 2. A goodness of split criterion Φ ( s , t ) that can be evaluated for any split s of any node t . 3. A stop-splitting rule. 4. A rule for assigning every terminal node to a class. Jia Li http://www.stat.psu.edu/ jiali
Classification/Decision Trees (I) Standard Set of Questions The input vector X = ( X 1 , X 2 , ..., X p ) contains features of both categorical and ordered types. Each split depends on the value of only a unique variable. For each ordered variable X j , Q includes all questions of the form { Is X j c ? } for all real-valued c .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern