#@statistics

[jeromyanglim]

Advice for psychology students wanting to implement hierarchical longitudinal models

I am quite interested in Bayesian longitudinal hierarchical models. They provide a flexible approach for testing a range of psychologically interesting hypotheses:

flexible modelling of individual differences in the effect of time

within subject covariates

predicting between person parameters from between person covariates

flexibility in modelling distributions of level 1 and level 2 variables

and more

I tend to use a Bayesian approach to estimating parameters in such models. This offers several advantages. In particular:

it's simple to incorporate non-normal distributions

it's easy to include non-linear effects

inference can readily be performed on parameters tailored to a particular hypothesis

the whole supporting framework of posterior predictive checks, model recovery simulations, plotting fits, model comparison, and so on is a nice way to do model development.

I have a post discussing getting started with Bayesian approaches.

Nonetheless, for many students that I supervise, it would take too long to acquire the statistical and programming expertise necessary to implement such models. Thus, this post explores alternative options for students interested in implementing a multilevel model.

Other options

A good book for getting started with multilevel approaches to repeated measures data is Applied Longitudinal Data Analysis by Singer and Willett. UCLA has implemented many of the exercises in a range of software packages.

In general, many of my students have existing training with SPSS. The SPSS mixed-effects program MIXED is thus often a good starting point. "Multilevel and longitudinal modeling with IBM SPSS" (available as an ebook at Deakin) provides a tutorial style introduction with examples of longitudinal psychology applications.

Other common options that involve dedicated multilelvel software include MlWin and HLM.

R has the nlme and the lme4 packages. If you're familiar with R, these are good options.

[jeromyanglim]

What does it mean to correlate factors in a confirmatory factor analysis?

I received the following question via email:

With the CFA task I have produced a five-factor uncorrelated model and a five-factor correlated model with only a slight improvement seen in the correlated model. What does correlating the five latent factors do exactly? It seems to help the fit of the model in some way but how does it do this?

Answer

In an uncorrelated model, the model forces the correlation between factors to be zero. In a correlated model, factors are allowed to correlate. As a result in the correlated model, the correlations between factors may be large or they may be still be close to zero. The algorithm will choose the correlations which maximises fit in some sense.

Conceptual explanation: There are oten theoretical questions about whether factors correlate and if so, by how much. For example, in the case of Big 5 personality, there is a lot of discussion about whether the Big 5 factors of personality are correlated or not. For example, see some of these papers on correlations between the Big 5 and whether such correlations imply an even higher level personality factor. Thus, a model with uncorrelated factors is in some sense simpler, but often not a realistic model of the structure of a set of psychological factors.

Once you see that a correlated factor model fits better, you may want to then interpret the pattern of factor correlations in terms of their general size and in terms of which correlations are larger or smaller. So for example, you might find that the correlations are mostly in the .2, .3 range (i.e., fairly small), but two factors correlate .5 (i.e., fairly large). Thus, you have now learnt something about the test and possibly the underlying psychological constructs.

Statistical explanation: Model fits in SEM are related to the degree to which the observed covariance (or correlation) matrix is reproduced by the model. So in the context of a typical CFA of a psychological test, your observed covariance correlation matrix is the set of all pairwise correlations between the items of the test.

So for example imagine a stnadard uncorrelated factor model with with uncorrelated factors and that only has loadings from factors to items belonging to that factor (i.e., no cross-loadings). Such a model can not reproduce any correlations between items across scales. However, with correlated factors, such correlations could be reproduced. Thus, by better modelling the observed correlations, the model is improved (i.e., less discrepancy between model and data).

Many other modifications can be understood in a similar way. For example, cross-loadings help to reproduce correlations between a particular item and items that belong to another factor. Similarly, correlated residuals for items help to reproduce particularly high correlations between items that are greater than would be implied simply by their belonging to a common factor and the relevant factor loadings.

[jeromyanglim]

What causes negative item total correlations and low alpha for psychological scale in SPSS?

Problem: I recently received this problem via email:

I have been looking for the reliabilities for each of my subscales, 2 scales have been fine but the other two have come out with low alpha values and several items have negative correlations with each other in the inter-item correlation matrix. They appear to be the values I have recoded. I am really stuck with this and want to get it sorted.

Solution:

Psychological scales commonly contain a mix of positively and negatively worded items. In order to form a scale score for each individual, negatively worded items need to be reversed before the mean or the sum of items can be obtained.

I explain the process in great detail using SPSS in this post. In partiuclar, my approach for SPSS is to create a copy of all items reversing only those that need reversing and making a copy for the others. Then use these when creating scale scores.

In the problem above you are aware about recoding (i.e., reversing items) but you are getting low reliabilities and negative item-total correlations. The most common causes of this:

You have reversed items twice. If you use syntax where you override the existing variables, then you can easily double reverse and take items back to what they were originally. When reversing items you should also create new variables.

You have reversed the wrong items or have failed to reverse items that needed to be reversed. There are several strategies for diagnosing this. (1) Look at the item means along with the wording for the items and see whether the patern of means make sense. In psychology you typically have expectations about which items people will endorse more or less. Relate this to items that you are getting negative item total correlations. (2) Run a factor analysis on the supposedly reversed items for the subscale and extract one factor and see which loadings are positive and negative. This should highlight any reversal errors.

You have not put the reversed items into the reliability analysis (i.e., you've used the original unreversed items). You need to used the reversed items where items need reversing in the reliability analysis in SPSS. If you have used the originals then your results will be invalid and this will be expressed in terms of low or negative item total correlations and low alphas.

A general point with all this is it is essential that you save your syntax for creating scale scores (i.e., item reversal, means, and sums for scales). You can then check re-run the code and also use the code to understand where you have gone wrong.

[jeromyanglim]

What do negative path loadings mean in SEM?

I recently received this email

Having read through your SEM paths loading analysis ans reports. I wish to inquire from you whether negative paths loading (rages from -0.2 to -0.9) can be consider as good loading or significant. Sir, i have an example in my attachment. In the attachment, can i report the paths loading between ACC to ST (having -.042) as statistically significant?

Answer

A negative path loading is basically the same as a negative regression coefficient. I.e., For a path loading from X to Y it is the predicted increase in Y for a one unit increase on X holding all other variables constant. So a negative coefficient just means that as X increases, Y is predicted to decrease.

To determine whether a negative coefficient is statistically significantly different from zero, you'd have to look at further output in your SEM program. In general for a given x-y path, the statistical significance will generally not be influenced by whether the path is positive or negative. So, for example, 0.2 will have the same p-value as -0.2.

[jeromyanglim]

How to calculate Cohen's d from an F test?

I was recently asked about how to get Cohen's d from a study which only reports an F test and no sample size.

Assuming the study also reports group sample sizes, then you can derive d from F and d.

Formula 5 in this document shows how to do it.

[jeromyanglim]

Explaining maximum likelihood estimation to students with minimal background in mathematics or probability

I was recently asked about how to intuitively explain maximum likelihood estimation to students, most of whom would not have done a class on calculus or probability. The context was explaining what maximum likelihood estimation meant in the context of factor analysis.

This was my quick proposed approach:

Draw a normal density and take one value and show the probability density of that point.

Explain how the probability density of a set of independent points is just the product of the density of the individual points (this is the likelihood assuming certain parameter values).

If we assume the parameters are unknown, but the data is fixed, then for different values of the parameters the likelihood would change. At this point you could draw a plot with an example parameter like the mean on the x axis and the likelihood on the y-axis (it might look like a normal distribution; in fact in many cases it does).

Explain how the likelihood varies with the value of this unknown parameter, but how there is a mode to the likelihood. And say how it seems reasonable that if we don't know what the parameter is, perhaps our best guess would be the value that maximises the likelihood of the data... and that's what maximum likelihood estimation does. It defines a likelihood for the data given parameter values, and estimates the values of the parameters to be those which maximise that likelihood.

Then explain how with factor analysis there are more parameters than just a mean, but that the idea of maximising the likelihood extends to these more complex models.

For a relatively accessible introduction to this topic, I quite liked Ben Bolker's book which is free online: http://www.math.mcmaster.ca/~bolker/emdbook/ Chapter 6 has a discussion of likelihood.=

[jeromyanglim]

How to get a count of univariate outliers in SPSS?

In an SPSS lab class, students were asked to calculate the number of outliers for a given variable in a dataset. An outlier was defined as an observation with a z-score $> |2.58|$.

One strategy for doing this is to create a z score for the variable using "Save standardised variables" in analyze - descriptives - descriptives dialog. This will add a z-score version of the variable supplied in the dialog box to the end of the data file. You can then sort this ascending (by right clicking the variable in the data view) and count the observations less than -2.58, and then sort descending and the observations greater than 2.58. This count can then be converted to a proportion by dividing by the sample size.

I was asked about how this process can be completed in SPSS with out the manual sorting and counting.

One strategy involves creating an outlier indicator variable (1 if an outlier and 0 if not an outlier). You can then run analyze - descriptives - frequencies on the indicator variable to get a count and percentage of the outliers for that variable.

To create the indicator variable you can use the transform - compute menu. Specifically, if you have a variable called x, then the expression is x > 2.58 | x < -2.58. It could also be written x > 2.58 | x < -2.58.

Here is some SPSS syntax to perform the task (see here for tips on SPSS syntax.

* create a variable to demonstrate the example. COMPUTE x=RV.NORMAL(0,1). EXECUTE. * standardise the variable and thereby create a variable called zx. * could use analyze - descriptives - descriptives. DESCRIPTIVES VARIABLES=x / SAVE. * compute whether is an outlier. * could use transform - compute. COMPUTE outlierx = zx > 2.58 or zx < -2.58. EXECUTE. * run frequencies on the outlier indicator variable. * could use analyze - descriptives - frequencies. FREQUENCIES outlierx.