#running classification tree

[shihab1992]

Week 1 - Lab: Running Classification Tree

Week 1 python decision trees As part of peer review assigment I am working on decision trees in Python. This assigment is intended for Coursera course “Machine Learning for Data Analysis by Wesleyan University”. 

Installation in Linux Ubuntu.

sudo chmod +x Anaconda3-2022.10-Linux-x86_64.sh

./Anaconda3-2022.10-Linux-x86_64.sh

conda install scikit-learn

conda install -n my_environment scikit-learn

pip install sklearn

pip install -U scikit-learn scipy matplotlib

sudo apt-get install graphviz

sudo apt-get install pydotplus

conda create -c conda-forge -n spyder-env spyder numpy scipy pandas matplotlib sympy cython

conda create -c conda-forge -n spyder-env spyder

conda activate spyder-env

conda config --env --add channels conda-forge

conda config --env --set channel_priority strict

python -m pip install pydotplus

dot -Tpng tree.dot -o tree5.png

I have to perform a decision tree analysis to test nonlinear relationships among a series of explanatory variables and a binary, categorical response variable. Data set is  provided by The National Longitudinal Study of Adolescent Health (AddHealth).

I will not complicate things here, therefore I am focusing on regular smoking (TREG1 variable).

I choosed few of vars to determine if they can predict regular smoking. predictor = dc[['HISPANIC','WHITE','BLACK','NAMERICAN','ASIAN']]

Therefore I did change predictor variables to just two: gender and age. I got this tree.

my code added below by pyhton

-- coding: utf-8 --

""" Created on Wed Dec 28 11:06:15 2022

@author: rfernandez """ import numpy as np import pandas as pd import matplotlib.pylab as plt

from sklearn.cross_validation import train_test_split

from sklearn.model_selection import train_test_split from sklearn.tree import DecisionTreeClassifier from sklearn.metrics import classification_report from sklearn import tree import pydotplus import sklearn.metrics

#Load the dataset

data = pd.read_csv("tree_addhealth.csv")

dc = data.dropna() dc.dtypes dc.describe() """ Modeling and Prediction """

#Split into training and testing sets

predictor = dc[['HISPANIC','WHITE','BLACK','NAMERICAN','ASIAN']] target = dc.TREG1 pr_train,pr_test,t_train,t_test= train_test_split(predictor,target, test_size=0.4) pr_train.shape pr_test.shape t_train.shape t_test.shape

#Build model on training data

classif = DecisionTreeClassifier() classif = classif.fit(pr_train,t_train) pred=classif.predict(pr_test) sklearn.metrics.confusion_matrix(t_test,pred) sklearn.metrics.accuracy_score(t_test,pred)

#Running Classification Tree

tree.export_graphviz(classif, out_file='tree_race.dot')

[briefexpertfun-blog]

Running a Classification Tree (Week 1 Machine Learning)

LIBNAME mydata "/courses/d1406ae5ba27fe300 " access=readonly;

DATA new; set mydata.addhealth_pds;

/* Datamanagement for Being overweight */

IF H1TS4=8 THEN H1TS4=.;

IF H1TS4=6 THEN H1TS4=.;

IF H1TS4=. THEN H1TS4=9;

if H1TS4=1 then H1TS4=1;

else H1TS4=2;

ods graphics on;

proc hpsplit assignmissing=popular seed=13039;

class H1TS4 H1GH1 H1GH1A H1TS1 H1TS3 H1TS5 H1TS6

  H1DA3 H1DA4 H1DA5  H1DA6  H1GH2  H1GH10;

model H1TS4 =AGE H1GH1 H1GH1A H1TS1 H1TS3 H1TS5 H1TS6  H1GH17

  H1DA3 H1DA4 H1DA5 H1DA6 H1GH2 H1GH10

  H1GH12 H1GH13 H1GH14  H1GH17;

grow entropy;

prune costcomplexity;

RUN;

In the example above/ below was performed Decision tree analysis in order to test nonlinear relationships among a series of explanatory variables and a binary, categorical response variable. All possible separations (categorical) or cut points (quantitative) are tested. For the analyses, from a few I chose an option entropy for the split criterion/ to grow the tree. And a cost complexity algorithm was used for pruning the full tree into a final subtree.

The following explanatory variables were included as possible contributors to a classification tree model evaluating The problems of being overweight ( H1TS4- for a response variable), age, How is your health, Difficulty using your hands, Foods you shouldn’t eat, Smoking, Drinking, and Drug abuse. Then Poor appetite, Watching television or videos, or play video games, Going roller-blading, roller-skating, skate-boarding, or bicycling, play an active sport, such as baseball, softball, basketball, Doing exercise, such as jogging, walking, karate, Having a headache and Feeling sick.

The HP SPLIT Procedure:

Number of Leaves Before Pruning: 231 leaves

Number of Leaves After Pruning: 12 leaves

Model Event Level: 1 which means that is yes for the target variable             The problems of being overweight.

Number of Observations Read: 6504

Number of Observations Used: 6504

Proc HPSLIT creates a plot of the cross validated Average standard Error/ASE or Average missclasification rate based on number of leaves each of the trees generated on the training sample. Vertical reference line represents the tree with the number of leaves that has the lowest cross validated ASE. It is 8 for this line. Horizontal reference line represents the average standard error plus one standard error for the complexity parameter. 1-SE is used for pruning the cost complexity. Minimal average missclasification rate is 0.315. Parameter is 0.0010.

The pruning plot for a smaller tree with the subtree started at node =0 is presented with splits for these tables: H1TS1/ The foods you should and shouldn’t eat, H1TS3/ Smoking and Age. Model-based confusion matrix shows the performance for the final classification tree. The model classified correctly 87% : one minus error rate (1- 0.13) for the people who have some problems in overweight. And 43% (1- 0.57) is for those who did not. People who eat non healthy food, are more likely prone to smoking in age >= 15 ( exactly in the age >=15.732).

From the Model-Based Fit Statistics for Selected Tree also Sensitivity with 0.870 value and Specificity with 0.43 value is clearly right the statement above:

 The total model classified 31% of the sample correctly, 87% experimenters (Sensitivity) and 43% of age above 15/ smokers (Specificity). Moreover it was graphically plotted in the last plot ROC Curve for H1TS4.