DecisionTree
Decision tree analysis was performed to test nonlinear relationships among a series of explanatory variables and a binary, categorical response variable. All possible separations (categorical) or cut points (quantitative) are tested. For the present analyses, “the entropy- goodness of split” criterion was used to grow the tree and 5% min_samples_split was used to decide whether to split further branch or not.
The following explanatory variables along with their importance mostly contributed to a classification tree model evaluating smoking experimentation (my response variable):- age(.029), White(.077), Black(.013), Alcohol use(.115), marijuana use(.500), availability of cigarettes in the home(.012), alcohol problems(.010), deviance(.043), depression(.017), parental presence(.014), parental activities(.031), family connectedness(.022), school connectedness(.047) and grade point average(.052).
The most important feature for marijuana use with 50% contribution. The total model classified 85% of the sample correctly, 90% of experimenters (sensitivity) and 55% of nonsmokers (specificity).
Note:- The Code given below can be run directly in Google Colab. Test data size is set to 915 and training data 3660.
------------------------------------------Code----------------------------------
Imports:-
from google.colab import drive,files import pandas as pd import matplotlib.pyplot as plt import os import pydotplus import io from IPython.display import Image from sklearn import tree from sklearn.metrics import confusion_matrix from sklearn.metrics import accuracy_score from sklearn.model_selection import train_test_split drive.mount('/content/gdrive') corpusName ="_d21b2085472fd467f689f21cd421b13b_tree_addhealth.csv" folderPath = "/content/gdrive/My Drive/MachineLearningForDataAnalysis" corpus = os.path.join(folderPath, corpusName) df_train = pd.read_csv(corpus) print("Dataset shape = ") print(df_train.shape)
Output: Dataset shape = (6504, 25)
df_train.dropna(inplace=True) x_train,x_test = train_test_split(df_train,train_size=0.8,random_state=42) y_train,y_test = x_train.TREG1,x_test.TREG1 model=tree.DecisionTreeClassifier(random_state=42,criterion="entropy",min_samples_split=(int)(x_train.shape[0]*0.05),) model.fit(x_train.drop("TREG1",axis=1),y_train) y_pred=model.predict(x_test.drop("TREG1",axis=1)) print("Confusion matrix = ") print(confusion_matrix(y_test,y_pred))
Output: Confusion matrix = [[714 56] [ 77 68]]
print("Accuracy = ") print(accuracy_score(y_test,y_pred))
Output: Accuracy = 0.8546448087431694
out = io.StringIO() tree.export_graphviz(model, out_file=out) graph=pydotplus.graph_from_dot_data(out.getvalue()) img = Image(graph.create_png()) filePath = os.path.join(folderPath, "DecisionTree.png") with open(filePath, "wb") as png: png.write(img.data) png.close()
dict_ = dict() for i,col in enumerate(x_train.drop("TREG1",axis=1).columns): dict_[col]=model.feature_importances_[i] print(dict_)
Output: {'BIO_SEX': 0.0, 'HISPANIC': 0.0, 'WHITE': 0.07689800188375509, 'BLACK': 0.013329278083582821, 'NAMERICAN': 0.0, 'ASIAN': 0.0, 'age': 0.028676800427653523, 'ALCEVR1': 0.11531998044836722, 'ALCPROBS1': 0.010359831147465094, 'marever1': 0.5003916650012903, 'cocever1': 0.0, 'inhever1': 0.0, 'cigavail': 0.011564724963814329, 'DEP1': 0.017293445565594052, 'ESTEEM1': 0.003972967897391851, 'VIOL1': 0.008243966063649775, 'PASSIST': 0.005773466454035425, 'DEVIANT1': 0.042695886446359736, 'SCHCONN1': 0.046592301917146584, 'GPA1': 0.05240826234607083, 'EXPEL1': 0.0, 'FAMCONCT': 0.022239864016842078, 'PARACTV': 0.030525880193369718, 'PARPRES': 0.013713677143611642}