The Curious Data Scientist @curiousds - Tumblr Blog

Data Management and Visualization: Week 4 Assignment

Creating your own graphs :SAS and Python

SAS: Code

Output:

Python: Code

#importing Pandas and NumPy libraries import pandas import numpy import seaborn import matplotlib.pyplot as plt #Loading the Mars Crater dataset DATA = pandas.read_csv('marscrater_pds.csv',low_memory=False) pandas.set_option('display.max_columns',None) pandas.set_option('display.max_rows',None) #Printing the number of rows and coloums in the dataset print(len(DATA)) print(len(DATA.columns)) #Converting the value of the object into numeric DATA['LATITUDE_CIRCLE_IMAGE']= pandas.to_numeric(DATA['LATITUDE_CIRCLE_IMAGE']) #defining a new variable in order to divide the latitude into three groups, viz >25, <-25, and -25<0<25 def LAT (row): if row['LATITUDE_CIRCLE_IMAGE'] >25: return 1 if row['LATITUDE_CIRCLE_IMAGE'] < -25: return 2 else: return 0 DATA['LAT']= DATA.apply(lambda row:LAT(row),axis=1) #Printing the number of rows and coloums in the dataset print(len(DATA)) print(len(DATA.columns)) #subsetting to obtain the specific data required ie the data coming in the range of latitude -25 to +25 EQTR=DATA[(DATA['LAT']==0)] EQTR2=EQTR.copy() #Printing the number of rows and coloums in the dataset print(len(EQTR2)) print(len(EQTR2.columns)) #Converting the value of the object into numeric EQTR2['DIAM_CIRCLE_IMAGE']= pandas.to_numeric(EQTR2['DIAM_CIRCLE_IMAGE']) EQTR2['DEPTH_RIMFLOOR_TOPOG']= pandas.to_numeric(EQTR2['DEPTH_RIMFLOOR_TOPOG']) EQTR2['NUMBER_LAYERS']= pandas.to_numeric(EQTR2['NUMBER_LAYERS']) #Omitting missing data EQTR2['DEPTH_RIMFLOOR_TOPOG']=EQTR['DEPTH_RIMFLOOR_TOPOG'].replace(0, numpy.nan) EQTR2['NUMBER_LAYERS']=EQTR['NUMBER_LAYERS'].replace(0, numpy.nan) #Data management of latitude variable along with frequency and precentages EQTR2['EQR']= pandas.cut(EQTR2.LATITUDE_CIRCLE_IMAGE,[-25,-15,-5,5,15,25]) EQTR2['EQR']=EQTR2['EQR'].astype('category') print("Latitude grouped") FREQ1=EQTR2['EQR'].value_counts(sort=True) print(FREQ1) PER1=EQTR2['EQR'].value_counts(sort=True, normalize=True)*100 print(PER1) #Data management of diameter variable along with frequency and precentages EQTR2['DIA']= pandas.cut(EQTR2.DIAM_CIRCLE_IMAGE,[1,1.5,2.5,3.5,4.5,1096.65]) EQTR2['DIA']=EQTR2['DIA'].astype('category') print("Diameter of craters in equatorial region grouped") FREQ2=EQTR2['DIA'].value_counts(sort=True) print(FREQ2) PER2=EQTR2['DIA'].value_counts(sort=True, normalize=True)*100 print(PER2) #Data management of depth variable along with frequency and precentages EQTR2['DPTH']= pandas.cut(EQTR2.DEPTH_RIMFLOOR_TOPOG,[0.1,0.2,0.4,0.6,0.8,4.95]) EQTR2['DPTH']=EQTR2['DPTH'].astype('category') print("Depth of craters in equatorial region grouped") FREQ3=EQTR2['DPTH'].value_counts(sort=True) print(FREQ3) PER3=EQTR2['DPTH'].value_counts(sort=True, normalize=True)*100 print(PER3) #Summary statisstics of variables print("SUMMARY STATISTICS OF VARIABLES") print("Diameter of craters at equatorial region") stat1=EQTR2['DIAM_CIRCLE_IMAGE'].describe() print(stat1) print("median") mdn1=EQTR2['DIAM_CIRCLE_IMAGE'].median() print(mdn1) print("mode") md1=EQTR2['DIAM_CIRCLE_IMAGE'].mode() print(md1) print("Depths of craters in the equatorial region") stat2=EQTR2['DEPTH_RIMFLOOR_TOPOG'].describe() print(stat2) print("median") mdn2=EQTR2['DEPTH_RIMFLOOR_TOPOG'].median() print(mdn2) print("mode") md2=EQTR2['DEPTH_RIMFLOOR_TOPOG'].mode() print(md2) print("Number of ejecta layers on craters in the eqiatorial region") stat3=EQTR2['NUMBER_LAYERS'].describe() print(stat3) print("median") mdn3=EQTR2['NUMBER_LAYERS'].median() print(mdn3) print("mode") md3=EQTR2['NUMBER_LAYERS'].mode() print(md3) #Univariate graphs EQTR2['NUMBER_LAYERS']=EQTR2['NUMBER_LAYERS'].astype('category') seaborn.countplot(x="NUMBER_LAYERS", data=EQTR2) plt.xlabel('Number of ejecta layers') plt.title('Number of ejecta layers on craters in the Equatorial region of Mars surface') plt.show() seaborn.distplot(EQTR2['DIAM_CIRCLE_IMAGE'].dropna(), kde=False) plt.xlabel('Diameter of the craters in kms') plt.title('Distribution plot of the diamaters of the craters in the equtorial region') plt.show() seaborn.countplot(x="DIA", data=EQTR2) plt.xlabel('Diameter of the craters in kms') plt.title('Histogram of the diamaters of the craters grouped in the equtorial region') plt.show() seaborn.distplot(EQTR2['DEPTH_RIMFLOOR_TOPOG'].dropna(), kde=False) plt.xlabel('Depths of the craters in kms') plt.title('Distribution plot of the depths of the craters in the equtorial region') plt.show() seaborn.countplot(x="DPTH", data=EQTR2) plt.xlabel('Depths of the craters in kms') plt.title('Histogram of the depths of the craters grouped in the equtorial region') plt.show() #Bivariate graphs scat1=seaborn.regplot(x="LATITUDE_CIRCLE_IMAGE", y="DIAM_CIRCLE_IMAGE", fit_reg=False, data=EQTR2) plt.xlabel('Latitude') plt.ylabel('Diameter of craters in equatorial region') plt.title('scatter Plot: Latitude vs Diameters of craters') plt.show() scat2=seaborn.regplot(x="LATITUDE_CIRCLE_IMAGE", y="DEPTH_RIMFLOOR_TOPOG", fit_reg=False, data=EQTR2) plt.xlabel('Latitude') plt.ylabel('Depth of craters in equatorial region') plt.title('scatter Plot: Latitude vs Depth of craters') plt.show() seaborn.factorplot(x="EQR", y="DIAM_CIRCLE_IMAGE", data=EQTR2, kind="bar",ci=None) plt.xlabel('Latitude') plt.ylabel('Diameter of craters in equatorial region') plt.title('Bar Plot: Latitude vs Diameters of craters') plt.show() seaborn.factorplot(x="EQR", y="DEPTH_RIMFLOOR_TOPOG", data=EQTR2, kind="bar",ci=None) plt.xlabel('Latitude') plt.ylabel('Depth of craters in equatorial region') plt.title('Bar Plot: Latitude vs Depth of craters') plt.show()

Output:

The variables are grouped for better data management and plotting purposes. The summary statistics of the three variables of interest are found. It can be seen that the mean diameter of the craters at the equatorial region is 3.35 kms. This is in agreement with the hypothesis proposed at the beginning of the course that is the diameter of craters at the equatorial region are all small and is less than 8 kms. Similarly the depth (elevation) of the craters are also a low value that is 0.45 km. This can be connected to fact that most of the craters are having only 1 layer of ejecta.

It can be clearly seen from the histograms that all variables have right skewed distribution with the depth variable having a bi-modal right skewed nature. The variables are grouped into categories to find this.

Scatter plots are created with diameter and depth of craters as the response variables(y) and latitude as explanatory variables(x). While examining the scatter plot a relationship cannot be determined between the variable. So the variables are grouped and it can be concluded that the Diameter and latitude have a zigzag relationship. At the equatorial region its showing a bi-modal trend. The depth and latitude have a negative linear relationship in this region of mars.

Data Management and Visualization: Week 3 Assignment

Making Data Management Decisions: SAS and Python

SAS: Code

/*Reading in the Mars crater dataset*/ LIBNAME data "/courses/d1406ae5ba27fe300" ACCESS = readonly; /*Setting the Mars Crater dataset to new name DATA*/ DATA new; SET data.marscrater_pds; /*Labelling the variables into readable names*/ LABEL LAT = "Division of latitude: 0 (-25<0<+25), 1 (> 25), 2 (<-25)" DIAM_CIRCLE_IMAGE = "Diameter of the Craters at equatorial region" DEPTH_RIMFLOOR_TOPOG = "Depth of the Craters at the equatorial region" NUMBER_LAYERS ="Number of ejecta layers on the craters at the equatorial region"; /*Dividing Latitude variable into three group*/ IF LATITUDE_CIRCLE_IMAGE > 25 THEN LAT = 1;/*90 to 25 latitude */ ELSE IF LATITUDE_CIRCLE_IMAGE < -25 THEN LAT = 2;/*-25 to -90 latitude*/ ELSE LAT= 0;/* +25 to -25 latitude*/ /*Omitting missing data*/ IF DEPTH_RIMFLOOR_TOPOG = 0 THEN DEPTH_RIMFLOOR_TOPOG =.; IF NUMBER_LAYERS=0 THEN NUMBER_LAYERS=.; /*Subsetting data*/ IF LAT = 0; /*Sorting according to the unique Crater id*/ PROC SORT; BY CRATER_ID; /*Finding and Printing the frequency and percentages of the four variables*/ PROC FREQ; TABLES LAT NUMBER_LAYERS DEPTH_RIMFLOOR_TOPOG DIAM_CIRCLE_IMAGE ; RUN;

Output

Python:Code

#importing Pandas and NumPy libraries import pandas import numpy #Loading the Mars Crater dataset DATA = pandas.read_csv('marscrater_pds.csv',low_memory=False) #Printing the number of rows and coloums in the dataset print(len(DATA)) print(len(DATA.columns)) #Converting the value of the object into numeric DATA['LATITUDE_CIRCLE_IMAGE']= pandas.to_numeric(DATA['LATITUDE_CIRCLE_IMAGE']) #defining a new variable in order to divide the latitude into three groups, viz >25, <-25, and -25<0<25 def LAT (row): if row['LATITUDE_CIRCLE_IMAGE'] >25: return 1 if row['LATITUDE_CIRCLE_IMAGE'] < -25: return 2 else: return 0 DATA['LAT']= DATA.apply(lambda row:LAT(row),axis=1) #Printing the number of rows and coloums in the dataset print(len(DATA)) print(len(DATA.columns)) #Converting the value of the object into numeric DATA['LAT']= pandas.to_numeric(DATA['LAT']) C=DATA['LAT'].value_counts(sort=True) print(C) #subsetting to obtain the specific data required ie the data coming in the range of latitude -25 to +25 EQTR=DATA[(DATA['LAT']==0)] EQTR2=EQTR.copy() #Printing the number of rows and coloums in the dataset print(len(EQTR2)) print(len(EQTR2.columns)) #Converting the value of the object into numeric EQTR2['DIAM_CIRCLE_IMAGE']= pandas.to_numeric(EQTR2['DIAM_CIRCLE_IMAGE']) EQTR2['DEPTH_RIMFLOOR_TOPOG']= pandas.to_numeric(EQTR2['DEPTH_RIMFLOOR_TOPOG']) EQTR2['NUMBER_LAYERS']= pandas.to_numeric(EQTR2['NUMBER_LAYERS']) #Omitting missing data EQTR2['DEPTH_RIMFLOOR_TOPOG']=EQTR['DEPTH_RIMFLOOR_TOPOG'].replace(0, numpy.nan) EQTR2['NUMBER_LAYERS']=EQTR['NUMBER_LAYERS'].replace(0, numpy.nan) #Labelling, finding printing the frequency and percentage of the Diameters of the craters in the equatorial region print("Frequency of Diameters of craters in the equatorial region in kms") FREQ1= EQTR2["DIAM_CIRCLE_IMAGE"].value_counts(sort=True) print (FREQ1) print("Percentages of Diameters of craters in the equatorial region") PRE1=EQTR2["DIAM_CIRCLE_IMAGE"].value_counts(sort=True, normalize=True) print (PRE1) #Labelling, finding printing the frequency and percentage of the Depth of the craters in the equatorial region print("Frequency of Depth of craters in the equatorial region in kms") FREQ2= EQTR2["DEPTH_RIMFLOOR_TOPOG"].value_counts(sort=True, dropna=False) print (FREQ2) print("Percentages of Depth of craters in the equatorial region") PRE2=EQTR2["DEPTH_RIMFLOOR_TOPOG"].value_counts(sort=True, normalize=True) print (PRE2) #Labelling, finding and printing the frequency and percentage of the number of layers of ejecta in the equatorial region print("Frequency of number of ejecta layers of craters in the equatorial region") FREQ3= EQTR2["NUMBER_LAYERS"].value_counts(sort=True,dropna=False) print (FREQ3) print("Percentages of number of ejecta layers of craters in the equatorial region") PRE3=EQTR2["NUMBER_LAYERS"].value_counts(sort=True, normalize=True) print (PRE3)

Output:

1. For data management a new variable is introduced in both called “LAT” which denoted the three divisions of Latitude. From the frequency distribution of this its clear that there are only 189672 observations in the equatorial region ie 49.3% craters over Mars surface

2. All the unknown or missing values in the variables of interest have been omitted and correspondingly the data have been managed.

3. From the new frequency distribution we can see that about 1.67% of craters at the equatorial region have diameter 1.01km

4. About 2.13% craters at the equatorial region have 0.1 km depth.

5. About 81.7% craters at the equatorial region have only single layer of ejecta

Data Management and Visualization: Week 2 Assignment

Running the Program:Python and SAS

Python: Code

import pandas #importing Pandas and NumPy libraries import numpy #Loading the Mars Crater dataset DATA = pandas.read_csv('marscrater_pds.csv',low_memory=False) #Printing the number of rows and coloums in the dataset print(len(DATA)) print(len(DATA.columns)) #Converting the value of the object into numeric DATA['LATITUDE_CIRCLE_IMAGE']= pandas.to_numeric(DATA['LATITUDE_CIRCLE_IMAGE']) #subsetting to obtain the specific data required ie the data coming in the range of latitude -25 to +25 EQTR=DATA[(DATA['LATITUDE_CIRCLE_IMAGE']>= -25)& (DATA['LATITUDE_CIRCLE_IMAGE']<=25)] EQTR2=EQTR.copy() #Converting the value of the object into numeric EQTR2['DIAM_CIRCLE_IMAGE']= pandas.to_numeric(EQTR2['DIAM_CIRCLE_IMAGE']) EQTR2['DEPTH_RIMFLOOR_TOPOG']= pandas.to_numeric(EQTR2['DEPTH_RIMFLOOR_TOPOG']) EQTR2['NUMBER_LAYERS']= pandas.to_numeric(EQTR2['NUMBER_LAYERS']) #Labelling, finding printing the frequency and percentage of the Diameters of the craters in the equatorial region print("Frequency of Diameters of craters in the equatorial region in kms") FREQ1= EQTR2["DIAM_CIRCLE_IMAGE"].value_counts(sort=True) print (FREQ1) print("Percentages of Diameters of craters in the equatorial region") PRE1=EQTR2["DIAM_CIRCLE_IMAGE"].value_counts(sort=True, normalize=True) print (PRE1) #Labelling, finding printing the frequency and percentage of the Depth of the craters in the equatorial region print("Frequency of Depth of craters in the equatorial region in kms") FREQ2= EQTR2["DEPTH_RIMFLOOR_TOPOG"].value_counts(sort=True) print (FREQ2) print("Percentages of Depth of craters in the equatorial region") PRE2=EQTR2["DEPTH_RIMFLOOR_TOPOG"].value_counts(sort=True, normalize=True) print (PRE2) #Labelling, finding and printing the frequency and percentage of the number of layers of ejecta in the equatorial region print("Frequency of number of ejecta layers of craters in the equatorial region") FREQ3= EQTR2["NUMBER_LAYERS"].value_counts(sort=True) print (FREQ3) print("Percentages of number of ejecta layers of craters in the equatorial region") PRE3=EQTR2["NUMBER_LAYERS"].value_counts(sort=True, normalize=True) print (PRE3)

Output

SAS:Code

/Reading in the Mars crater dataset/ LIBNAME data "/courses/d1406ae5ba27fe300" ACCESS = readonly; /Setting the Mars Crater dataset to new name DATA/ DATA new; SET data.marscrater_pds; /Labelling the variables into readable names/ LABEL DIAM_CIRCLE_IMAGE = "Diameter of the Craters at equatorial region" DEPTH_RIMFLOOR_TOPOG = "Depth of the Craters at the equatorial region" NUMBER_LAYERS ="Number of ejecta layers on the craters at the equatorial region"; /Subsetting to obtain the specific data required/ IF LATITUDE_CIRCLE_IMAGE LE 25; IF LATITUDE_CIRCLE_IMAGE GE -25; /Sorting according to the unique Crater id/ PROC SORT; BY CRATER_ID; /Finding and Printing the frequency and percentages of the three variables/ PROC FREQ; TABLES DIAM_CIRCLE_IMAGE DEPTH_RIMFLOOR_TOPOG NUMBER_LAYERS; RUN;

Output

The following conclusions can be arrived at from the above program results:

Out of 384343 observations 189672 of them comes within the latitude range of = -25 and +25. i.e about 49.3% craters are at the equatorial region.

About 1.67% of craters have 1.01 km diameter, which is the highest at the equatorial region.

About 80% of craters in this region have unknown depth, denoted by the value 0. This can be considered as missing value in the dataset.

About 94% of craters in this region have unknown number of ejecta_ denoted by value 0. Again this can also be considered as missing value in the dataset.

#data science #data analysis #data #assignment #data management and visualisation #coursera

Data Management and Visualization : Week 1 Assignment

Being a Physics majors from college, as soon as I saw that there is an astrophysics related dataset available for research I was very much excited as I decided I’m going to select the MARS CRATER dataset.

A quick overview of the dataset gave a brief idea and an interesting topic popped in my head and it is the morphology of craters. I don’t know whether the dataset can answer the question related to this or not. Whatever the case is I selected some variables of my interest and made my own personal codebook.

The variables of interest are:-

• CRATER_ID – crater ID for internal sue, based upon the region of the planet (1/16ths), the “pass” under which the crate was identified, ad the order in which it was identified

• DIAM_CIRCLE_IMAGE – diameter from a non-linear least squares circle fit to the vertices selected to manually identify the crater rim (units are km)

• DEPTH_RIMFLOOR_TOPOG – average elevation of each of the manually determined N points along (or inside) the crater rim(units are km) o Depth Rim - Points are selected as relative topographic highs under the assumption they are the least eroded so most original points along the rim o Depth Floor – Points were chosen as the lowest elevation that did not include visible embedded craters

To be more specific about the research question, I selected one more topic after going through the codebook so that I can narrow down my research question. The topic is the location of the craters formed. I added a couple of more variables to my personal codebook

LATITUDE_CIRCLE_IMAGE – latitude from the derived center of a non-linear least-squares circle fit to the vertices selected to manually identify the crater rim (units are decimal degrees North)

LONGITUDE_CIRCLE_IMAGE – longitude from the derived center of a non-linear least-squares circle fit to the vertices selected to manually identify the crater rim (units are decimal degrees East)

NUMBER_LAYERS – the maximum number of cohesive layers in any azimuthal direction that could be reliably identified

The research question I arrived at is “whether there is any difference in morphologies of craters (larger or smaller or no. of layers of ejecta) formed at the equatorial region as compared to the rest of the mars surface?”

I did an elaborate literature review to find out whether anyone had ventured out to find the answer to the same question as I asked above and arrived at a hypothesis; the first stage of any scientific research. The search terms used for this are : impact craters at the equator of mars, Mars craters at equator, equatorial craters on mars.

Article 1: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2005GL022758

Small rampart craters in an equatorial region on Mars: Implications for near‐surface water or ice

D. Reiss , E. Hauber, G. Michael, R. Jaumann , G. Neukum and the HRSC Co‐Investigator Team

First published: 24 May 2005

Abstract: Small onset diameters (1 km) of rampart craters were identified in High Resolution Stereo Camera (HRSC) imagery in the equatorial region of the Valles Marineris plateaus. This is in contrast to previous global studies based on Viking imagery, which showed onset diameters in the range of 4 km to 7 km in equatorial regions. The observed small rampart craters show morphologically eroded ejecta blankets. This indicates a formation in the early Martian history, when erosion rates were higher. Volatile rich target material might have been available shortly after the formation of the Hesperian–aged plateaus surrounding Valles Marineris, and this is in agreement with the reported Hesperian fluvial activity in this region. The lack of fresh small rampart craters indicates a lowering of the ground ice table with time.

Article 2: Martian dichotomy: From Wikipedia, the free encyclopedia

The most conspicuous feature of Mars is a sharp contrast, known as the Martian dichotomy, between the Southern hemisphere and the Northern. The two hemispheres' geography differ in elevation by 1 to 3 km. The average thickness of the Martian crust is 45 km, with 32 km in the northern lowlands region, and 58 km in the southern highlands.

The boundary between the two regions is quite complex in places. One distinctive type of topography is called fretted terrain. It contains mesas, knobs, and flat-floored valleys having walls about a mile high. Around many of the mesas and knobs are lobate debris aprons that have been shown to be rock glaciers.

Many large valleys formed by the lava erupted from the volcanoes of Mars cut through the dichotomy.

The Martian dichotomy boundary includes the regions called Deuteronilus Mensae, Protonilus Mensae, and Nilosyrtis Mensae. All three regions have been studied extensively because they contain landforms believed to have been produced by the movement of ice or paleoshorelines questioned as formed by volcanic erosion.

The northern lowlands comprise about one-third of the surface of Mars and are relatively flat, with as many impact craters as the southern hemisphere.The other two-thirds of the Martian surface are the highlands of the southern hemisphere. The difference in elevation between the hemispheres is dramatic. Three major hypotheses have been proposed for the origin of the crustal dichotomy: endogenic (by mantle processes), single impact, or multiple impacts. Both impact-related hypotheses involve processes that could have occurred before the end of the primordial bombardment, implying that the crustal dichotomy has its origins early in the history of Mars.

Hypothesis: The craters formed at the equatorial region (LATITUDE_CIRCLE_IMAGE = -250 to +250) have smaller morphologies. Diameters ( DIAM_CIRCLE_IMAGE ) of the craters are smaller and most of them have unknown depth ( DEPTH_RIMFLOOR_TOPOG ) and also most of the craters have unknown number of ejecta ( NUMBER_LAYERS ) layers.

#coursera #data management and visualisation #assignment #data analysis and interpretaion specialization

With a baby of only 3.5 months old i m thinking how the hell did i finished my final project

#data #data analysis #data science

Exploring the BRFSS Data: A Statistical study of eating habits, influence of income over drinking and influence of education over income of the US population

Summary The following work is done as the peer-reviewed assignment for the completion of the first course “Introduction to Data and Probability” of the Specilization “Statistics with R” by Duke university in Coursera. Following below is not a complete analysis since its only based of the introductory concepts and so the conclusions drawn from them are pretty vague and are open for furthur analysis, which may result in a contradictory outcome.

Any part of the following write up is not copied and is solely done by myself

Setup

Load packages

library(ggplot2) library(dplyr)

Load data

load("brfss2013.RData")

Part 1: Data

The Dataset used for the following analysis is from “The Behavioral Risk Factor Surveillance System (BRFSS)”, which is “a collaborative project between all of the states in the United States (US) and participating US territories and the Centers for Disease Control and Prevention (CDC)” as quoted from their website. The survey is conducted amoung the US population who is 18+ through telephone and cellular phones. Hence the sampling is random. The participants are not asked to be part of an experiment since this is an observational study. So there is no random assignment. Hence the results can be generalized but not causal.

Several queries are self rated and also requires recollection from past which can lead to enormous errors in data analysis. Because of the above two reasons this analysis cannot be the ultimate answer to several questions. This analysis best answers the question with the available information. The results are prone to changes if done in a even more scientific way.

Part 2: Research questions

Research quesion 1: Healthy diet always leads to a healthy lifestyle and thus people who follow this tends to maintain healthy weight. From the survey's available information I'm trying to figure out, who is more health conscious amoung the US Population;Men of Women?. The analysis is based of the total consumption of fruits and vegetables by participants coming under each BMI category and also under each general health rating.

Research quesion 2: Several people undergoes stress and depression in their daily life due to work related issues, tension in relationships, financial situation etc. Some of these people rely on drinking to get a relief from all this for a brief time. Overtime this becomes an addiction. Through this data i'm trying to find whether there is any correlation between the income category and drinks consumed in past month by US citizens?

Research quesion 3: Its a general notion that graduating with a higher educational qualification can make sure that you receive a hefty paycheck. From the survey's information about the general population of US ,educational levels and their respective place in the income category, a pictorial analysis will shed light on how many of them belonging to each income category have completed their specific educational level. The analysis is done gender wise to see whether they also follow the same trend and who is better?

Part 3: Exploratory data analysis

Research quesion 1: Who is more health conscious MEN or WOMEN?

#Selecting the necessary data required from brfss dataframe.Filtering out missing values and adding a new coloumn to the new dataframe for furthur analysis rq1<- select(brfss2013, sex, X_vegesum, X_frutsum, X_bmi5cat, genhlth) %>% na.omit() %>% mutate (hlthdt= X_vegesum + X_frutsum)

Two new dataframes rq12 and rq13 are created with mean consumption of fruits and vegetables for both BMI category and General health ratings and plots are made gender based for easier analysis

#create new data frame for gender based BMI category with mean consumption of fruits and vegetable rq12<- rq1%>% group_by(X_bmi5cat,sex) %>% summarise (mn_hlth =mean(hlthdt)) rq12

## # A tibble: 8 x 3 ## # Groups: X_bmi5cat [?] ## X_bmi5cat sex mn_hlth ## <fct> <fct> <dbl> ## 1 Underweight Male 275. ## 2 Underweight Female 356. ## 3 Normal weight Male 317. ## 4 Normal weight Female 372. ## 5 Overweight Male 305. ## 6 Overweight Female 348. ## 7 Obese Male 283. ## 8 Obese Female 323.

q1.1<- ggplot(rq12, aes(x=sex, y=mn_hlth)) +geom_point(color="blue")+facet_grid(~X_bmi5cat) q1.1+ ylab("Mean consumption of fruits and vegetables")+ ggtitle("Gender based mean consumption of fruits and vegetables for each BMI Category")

From the above plot its clear that people who maintain healthy weight do include more fruits and vegetables in their diet. Its crystal clear that women in US eat more healthy food than men and so are more health conscious. Another important point is this that, Even the healthy men in US eats less fruits and vegetables than women who eats least as compared to rest of the women . Additionally it can be seen that Underweight men in US eat very less fruits and vegetables as compared to that of underweight women and the rest of the categories have somewhat a similar trend

This can again be analysed using the general health ratings. Following the same method as above the analysis is done.

#create new data frame for gender based general health ratings with mean consumption of fruits and vegetables rq13<- rq1%>% group_by(genhlth,sex) %>% summarise (mn_hlth =mean(hlthdt)) rq13

## # A tibble: 10 x 3 ## # Groups: genhlth [?] ## genhlth sex mn_hlth ## <fct> <fct> <dbl> ## 1 Excellent Male 350. ## 2 Excellent Female 409. ## 3 Very good Male 310. ## 4 Very good Female 364. ## 5 Good Male 285. ## 6 Good Female 332. ## 7 Fair Male 269. ## 8 Fair Female 305. ## 9 Poor Male 268. ## 10 Poor Female 297.

q1.2<-ggplot(rq13, aes(x=sex, y=mn_hlth)) +geom_point(color="blue")+facet_grid(~genhlth) q1.2+ylab("Mean consumption of fruits and vegetables")+ ggtitle("Gender based mean consumption of fruits and vegetables for each General health rating")

From above plot its again clear that women in US who self rated themselves to have excellant health condition did infact had the most intake of fruits and vegetables than men who came under the same category.Similar conclusions arrived from the first plot can be derived from this plot too. Hence it can be generalized that Women in US are more health conscious than men

Research quesion 2: Correlation between income category and drinking habit

#selecting the necessary data from the main dataframe and removing entries with NA rq2<- select( brfss2013,X_drnkmo4, X_incomg) %>% na.omit()

#Modifying dataframe by grouping each income category with the mean no.of drinks in past 30 days rq2<- rq2%>% group_by(X_incomg) %>% summarise(mn_drnk=mean(X_drnkmo4)) #Print modifies dataframe rq2

## # A tibble: 5 x 2 ## X_incomg mn_drnk ## <fct> <dbl> ## 1 Less than $15,000 8.09 ## 2 $15,000 to less than $25,000 8.84 ## 3 $25,000 to less than $35,000 10.4 ## 4 $35,000 to less than $50,000 11.8 ## 5 $50,000 or more 14.4

#Plot of Mean no.of drinks in past month for each income category q2<- ggplot(rq2, aes(X_incomg, mn_drnk)) +geom_point(color="red")+xlab("Income Category")+ ylab("Mean no.of drinks in past month")+ggtitle("Mean no.of drinks in past month for Income category")

#x labels had to be renamed since the labels were overlapping q2+scale_x_discrete(breaks= c("Less than $15,000","$15,000 to less than $25,000","$25,000 to less than $35,000","$35,000 to less than $50,000","$50,000 or more" ), labels=c("< 15k$","15k$-25k$","25k$-35k$","35k$-50k$",">50k$"))

From the plot it can be said that as the income of the Us citizens increases more the no.of drinks they have in a month.. This is an example to the fact that correlation is not causation. It cannot be generalized that the increase in drinking is due to more income. The plot is nearly linear and thus it can be said that The consumption of drinks by the general population of US is directly proportional to their income

Research quesion 3 Gender based income category and completed educational levels

#Selecting data from main data and exculding data entries with "NA" rq3<- select(brfss2013, X_educag, X_incomg,sex) %>% na.omit()

There is not much calculation to do rather than a simple bar plot to see how many US citizens in each income category have completed their respective educational levels

q3<-ggplot(rq3, aes(x=sex, fill=X_educag)) +geom_bar(width=0.7)+ scale_fill_discrete("Educaional level completed")+ggtitle("Gender based Income category and educational level completed") q3<- facet_wrap(~X_incomg, ncol=3,scales="free")

#renaming facet titles since its half hidden and not readable l<- c("Less than $15,000" = "< 15k$","$15,000 to less than $25,000" = "15k$-25k$", "$25,000 to less than $35,000" = "25k$-35k$","$35,000 to less than $50,000" = "35k$-50k$", "$50,000 or more" = ">50k$" ) q3<-ggplot(rq3, aes(x=sex, fill=X_educag)) +geom_bar(width=0.7)+ scale_fill_discrete("Educaional level completed")+ggtitle("Gender based Income category and educational level completed") q3+ facet_wrap(~X_incomg, ncol=3,scales="free", labeller= labeller(X_incomg=l))

A solid conclusion cannot be arrived at, but a general statement can be made that most of the US citizens who graduated from a college or a technical school did earn the highest.Most of them have graduated from high school and have tried to contine with higher education. Majority of general population of US are High school graduates in all categories except the highest earning group. This is true for both genders. Keeping in mind the above conclusion it can be said that In United states, Women comprises the major workforce of every Income category and Women in Us earn better than men

#introduction to data and probability #data analysis #exploratory data analysis #data #data science #r programming #course project #coursera #brfss 2013 dataset

Trending Blogs

Recently Viewed Blogs

The Curious Data Scientist