Fortitude, gratitude, and grad school @revaclav - Tumblr Blog

#365Papers Days 28-31

2/14: Making Sense of Students’ Actions in an Open-Ended Virtual Laboratory Environment from Gal, Uzan, Belford, Karabinos, and Yaron.

This is another paper from our visitor this week! We had an enlightening conversation, especially focused on t he use of cognitive science research techniques and principles in the design of studies and curricula in CER. This paper was of interest to me because I can see the trend as I read more that faculty members want to be able to evaluate information about students’ process skills and development, however the data needed from each student in order to do this is too substantial for assessment in large course (i.e., over 100 students). This technique of training a computer to read student behaviors interacting with an online assessment tool and determine the potential problem-solving strategies used seems useful for future work in this area.

2/15: A meta-analytic review of the Motivated Strategies for Learning Questionnaire by Crede and Phillips

We use the MSLQ in one of the tools that we use in our ongoing investigations of the effect of affect on organic chemistry achievement. We are fortunate to have obtained good psychometric properties for our population, but I’ve noticed more often in the CER literature an ignorance of the importance of psychometric quality in the use of instruments. This is surprising in a population like that of chemists, who know that if we can’t trust the tool we use, we can’t trust the results that we obtain. Here’s to a new wave of CER students who will keep the function of instruments in mind when evaluating the relationship of a variety of constructs with achievement!

2/16: Psychometric Assessment of the Short Grit Scale Among Chinese Adolescents by J. Li, Y. Zhao, F. Kong, S. Du, S. Yang

The grit scale has been used in such a wide variety of populations that I wonder how its applicability can be so broad. I am for familiar with instruments designed to measure constructs in the population of undergraduate students, which are often designed with a single group of individuals in mind, and personality evaluations like grit are more often broad (as far as I understand), so maybe this is an artifact of that design. However, I hope to do a similar evaluation as this in the population of organic chemistry students using the long grit scale.

2/17: Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence by Arjoon, Xu, and Lewis

Since reading about psychometric properties was a theme for this week, I thought re-reading this paper would be a good reminder. This comes from one of the groups at my own institution, and I think serves as a sounding call to CER to consider psychometric properties of the instrument which they use. This indicates: There is no reason to ignore the importance of these things in future research- and I hope that this paper, despite being from 2013, will remain in the memory of researchers to come.

#365Papers Day 27

2/13: Evidence-Based Approaches to Improving Chemical Equilibrium Instruction by Davenport, Leinhardt, Greeno, Koedinger, Klahr, Karabinos, and Yaron.

The corresponding author on this report is visiting us to give a seminar this week. This is interesting to me because it is so applied. This is a direct alternative to the method of instruction. I find this interesting because usually in chem. ed. we read about reformed classrooms in which the method of instruction is totally changed, while this paper demonstrates another way of introducing a topic without altering the course structure. From my research on instructional change, this seems like it would be more appealing to a faculty member who doesn't have to digest a full shift in their instructional methodologies. Looking forward to a lively discussion with all of the grad students when he visits!

You cannot resolve a problem in the conditions that created it

Elaine Seymour

#365Papers Days 23-26

2/9: Enhancing the role of assessment in curriculum reform in chemistry from Holme, Bretz, Cooper, Lewis, Paek, Pienta, Stacy, Stevens, and Towns.

The moral of the story is that assessment is the key to understanding if reform is really working, and that assessment is not just about seeking to understand if students have gained the ability to regurgitate content knowledge. I return to this paper often when I think about assessment, and it makes me wonder what I’ll do when I have my own classroom....

2/10: Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology by Seymour.

Coming from the author of Talking about Leaving, the classical work on departure from STEM degrees, you know that this piece is going to give some great perspective on research directions. I think that even though this paper is from 2002, it is essential to take a step back and grasp the full scale of everything occurring including initiatives, funding directions, and other attentions which are targeted at reforming STEM education and look at successes and failures. Recall, the primary finding from Talking about Leaving is that for both those who left STEM majors and those who completed them, the quality of instruction was their primary concern. This should be of high concern to all DBER researchers, especially because of the potential for increased active learning initiatives are recognized to potentially assist students from all backgrounds rather than only benefiting those who are traditionally higher represented in STEM.

2/11: Longitudinal assessment of trait emotional intelligence: Measurement invariance and construct continuity from late childhood to adolescence by Keefer, Holden, and Parker

Once again I’m having a methods Sunday. I stumbled upon longitudinal invariance while evaluating a longitudinal reciprocal relationship with affect measures and achievement in organic chemistry. The fact that an instrument might not function the way we hope it does over time is essential in this type of research. We care about changes in individuals’ and groups’ changes in mean values of a construct over time. Hence, the construct-relevant variance that we seek must be established to be true through the use of longitudinal invariance. This is not present in the current state of CER literature, but I hope it will in the future.

2/12: Assessing assessment: in pursuit of meaningful learning by Grange and Blackie

This is a largely theoretical piece from the CER literature that his on the currently popular concept of meaningful learning. Using Legitimation Code Theory’s semantics differential, these authors operationalized an axis upon which to identify the extent to which we ask students to think in the abstract or concrete world along with the level of symbolism in the language we use. This is an interesting thought along the line with some other publications of late, most notably the 3d-LAP, designed to elicit the extent to which the NRC’s Framework for K-12 Science Education’s 3 dimensions of core ideas, crosscutting concepts, and science practices are measured by a particular assessment tool.

academicssay

#365Papers Day 22

2/8: Using Student Writing and Lexical Analysis to Reveal Student Thinking about the Role of Stop Codons in the Central Dogma by Prevost, Smith, and Knight

Today’s paper is from biology education! One of my groupmates is conducting a lexical analysis study using data collected from our organic chemistry students. This system uses the terms or phrases to determine if a student would correctly or incorrectly answer an item, and therefore enables these researchers to highlight conceptual difficulties in responding to an item.

#365Papers Day 21

2/7 MY SECOND PUBLICATION CAME OUT TODAY! Beliefs about learning and enacted instructional practices: An investigation in postsecondary chemistry education by Gibbons, Villafane, Stains, Murphy, and Raker

It’s easy to get bogged down in the routine of the semester. This brightens my spirits today.

#365Papers Day 20

2/6: Organizational Environments and Doctoral Degrees Awarded to Women in Science and Engineering Departments by Fox

I’ve been reading a bit from this group lately in my search for a theory of institutional factors influencing the growth in degrees awarded in the sciences. Based on a qualitative study, this report highlights differences in departmental leadership attitudes toward increasing female participation in graduate school. Ultimately, those departments which were higher-producing in female PhDs were more engaged in a mentoring system, however, the individual nature of the PhD degree based on the opinion of the research advisor and their relationship to the student prevails at all institutions as a cultural barrier to innovative programs for increasing female participation.

#365Papers Days 16, 17, 18, 19

Day 16 2/2: Trends in Ph.D. Productivity and Diversity in Top-50 U.S. Chemistry Departments: An Institutional Analysis by Laursen and Weston

Diversity! A term that gets thrown around with abandon. This study explores the top 50-producing chemistry PhD programs in the US and evaluates their growth in the number of PhDs awarded as related to percent composition of sex and race/ethnicity/citizenship of the graduates as well as of the faculty in the department. While there are some of the top 50 which both were in the high growth and high representation of women and URM/international students, a general trend is not evident. I think this is an interesting route which should be studied to a much greater extent if we are to grow the chemical enterprise.

Day 17 2/3: Undergraduate Women in Science and Engineering: Effects of Faculty, Fields, and Institutions Over Time by Sonnert, Fox, and Adkins

I think that the findings of this study are so important to gleaning the meaning of the study of growth in undergraduate awarded degrees: STEM is not a single entity. The impact of departmental culture is more important than institutional culture.

Day 18 2/4: Testing for Nonuniform Differential Item Functioning With Multiple Indicator Multiple Cause Models by Woods and Grimm

This isn’t Chemistry Sunday, but it is a methods piece. This is a reading for my measurement course which I’m interested in applying in future research designs. MIMIC is a statistical procedure used within the structural equation modeling framework to evaluate constructs which are measured using multiple means. This enables the entrance of higher reliability for measurement! I believe that increased methodological rigor as associated with measurement instruments is essential for improved research quality in CER in future years. This report highlights one of the ways in which we can increase this rigor; digestible simulation studies are as essential for our progress as it is for those writing them. Discerning items within measurement instruments, be they affective measures or academic ones, can help identify potential bias which can be improved in future iterations of our measurement.

Day 19 2/5: Successful Programs for Undergraduate Women in Science and Engineering: Adapting versus Adopting the Institutional Environment by Fox, Sonnert, and Nikiforova

I’m excited that this report is about an intervention system. They highlight 410 programs which are designed to increase the likelihood of women to remain in science and found a distinction between program directions who see solutions as within their control and those who see it as out of their control. It also appears that successful programs work with both faculty and student life, while unsuccessful programs are more deeply involved internally and with recruitment programs from K-12 institutions. Simultaneously, the successful programs worked against the institutional culture and structure which had existed and (arguably) contributed to the disparity of women achieving STEM degrees, while those which we unsuccessful were unwilling to resist these norms.

#365Papers Days 14 and 15

1/31: Assessing students' understanding of inquiry: What do prospective science teachers notice? by Talanquer, Tomanek, and Novodvorsky

The construct of noticing is completely new to me. This study evaluated student teacher’s interpretations of a biology classroom and focused on their assessment of student learning. They ultimately assert that teachers look at and notice the alignment of student output with the objectives or expected outputs, rather than try to understand the student’s thinking. While I believe the results, I expect that prospective teachers, like those in this study, would have wildly different patterns of noticing than experienced teachers.

2/1: Do Science Teachers Distinguish Between Their own Learning and the Learning of Their Students? Brauer and Wilde

The title of this is provocative! I’m reading this as part of a piece of advice I received at a Research Boot Camp in my first year: before you finalize a report for publication, read 20 recent articles from that journal to ensure that you have proportioned the sections properly to reflect common practice. This is challenging, but it’s worked out for me so far!

The findings of this report are less provocative, but interesting. Namely, does a teacher think that the way that they learned is the best way, or that others learn differently? The results are probably biased toward the fact that the teachers that they surveyed are taking courses which inform them about constructivism, however, I still think it’s an interesting point. The impact of personal context and beliefs about how teaching and learning happen are essential to the way that teachers enact instruction, so, gaining of understanding of how they change can inform reform efforts.

#365Papers Day 13

1/30: Faculty and Students Conceptions of Assessment in Higher Education by Fletcher, Meyer, Anderson, Johnston, and Rees

This is a fascinating string of research. I hope I don’t end up too far down the rabbit hole reading for this particular project (generally about assessment in postsecondary chemistry education). That’s my biggest time-drainer: Reading deep in to a literature with which I’m not familiar and don’t need to be to accomplish the task of the day/week/moth/PhD. However, all reading helps me improve my writing!

Anyway, this article is about an evaluation of New Zealand postsecondary education and the differences between student and faculty member perceptions of assessment. I think it’s important to know that despite DBER’s high expectations, a reality check of what real faculty members think is essential to making any progress. It seems that faculty members perceive assessment as an accountability tool that has the potential to assist in learning, while students find it to be an unfair process in which they can’t win. While each individual’s experience is different, understanding the ways that faculty members use their assessments in practice can only help create better assessments!

#365Papers Day 12

1/29: Predicting Critical Thinking Skills of University Students through Metacognitive Self-Regulation Skills and Chemistry Self-Efficacy by Uzuntiryaki and Capa-Aydin

The dogs woke me up early today, so I’m trying to get some reading done and be ready for a big day ahead. We got galley proofs back for a new publication this weekend (YAY!) so I’ll be buried in those for most of the day.

This paper was a short one, but it utilizes one of the scales for chemistry self-efficacy that I’m interested in. I’m intrigued by the model used by the researchers here; there is a structure that I wonder if theory had informed or intuition. In the end, they find no significant relationship between self-efficacy and critical thinking, but they did find a significant relationship between metacognitive self-regulation (a hot topic right now) and critical thinking. I’m glad that they explored a few definitions of the peskily nebulous “critical thinking;” this term is so often tossed around and not defined of late.

professionalstudentthings

Reading my own writing like

#365Papers Days 10 and 11

Day 10 1/27 Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry by Villafane, Garcia, and Lewis

This paper measures something that I consider to be incredibly important when we are considering the effect of affect on chemistry achievement (plus, it’s from my institution and a former postdoc in my group). Coming from 2014, I think that this paper was really helping the field move forward and embrace psychometric methods in applied research. This study utilized a part of the Chemistry Attitudes and Experiences Questionnaire subscale on Self-Efficacy between groups of different race/ethnicity and sex. Their findings are surprising in the direction of changes over the semester for different race/sex combinations especially a decrease for Black and Hispanic mails and an increase for everyone else. Even though this study was conducted in a preparatory chem class, limiting the range of initial skills as determined by SAT math, I think the results point to some important directions for interventions.

Day 11 1/28 Chemistry Sunday! Emission Rates of Multiple Air Pollutants Generated from Chinese Residential Cooking by Chen, Zhao, and Zhao.

This was an interesting read! The researchers conducted monitored cooking experiences in China using traditional family recipes and measured fine particles and volatile organic compounds released in to the kitchen environment. While I don’t know what the differences between Chinese cooking methods and others cited by the authors can cause differences in the pollutants emitted, I learned a lot from this read. I appreciated that the authors surveyed a variety of families before conducted the experiments- this added authenticity to the results. I also didn’t know about time-monitoring techniques such as those used here. Also, I’m always leaving my kitchen’s fans and hood on from now on!

#365Papers Day 9

1/26: Student Affect and Conceptual Understanding in Learning Chemistry by Nieswandt

This is the first in a series of readings that I’m going to conduct about self-efficacy and self-concept in chemistry education. The important difference in this study from most studies of affect in chemistry is the measure of “meaningful conceptual understanding” rather than student achievement as compared to affect (although I would argue that they measured some form of chemical literacy rather than conceptual understanding). While not reaching true methodologically sound techniques- the sample size is far too small for the statistical procedures applied to have any appreciable degree of power for evaluating type one error- this was a good read and a great start to this literature review. This paper gives me some hope for my future work, because it utilizes structural equation modeling, as I often do, and reports on the relationship between affect and understanding rather than on an educational intervention, which my work has been criticized for doing in the past.

#365Papers Day 8

1/25: More on Grit! Much ado about grit: A meta-analytic synthesis of the grit literature. by Crede, Tynan, and Harms.

This meta-analysis makes me feel validated for my instinctual reaction to yesterday’s paper! This study evaluates the variety of studies which have been conducted on grit from the initial publication in 2007 until 2017. What I hope to do in my study is explore the factor structure of the grit scale, which this meta-analysis indicates is not strongly supported in the current literature. It also finds that the correlations between grit and other common non cognitive factors related to success such as conscientiousness and self-control are much higher than would normally be accepted as reasonable for two separate constructs (i.e., grit exhibits poor discriminant validity with these constructs). This is going to provide a strong context for my future work. This study calls for more intensive psychometric evaluations of the grit scale, particularly through item response theory methods. Sounds like a challenge!

#365Papers Day 8

1/24 Grit: perseverance and passion for long-term goals. by Duckworth, Peterson, Matthews, and Kelly

It’s time to get pop psychology literate! This paper is a part of the work referenced in this TED talk. I have a confession, I can’t stand most TED Talks. I feel that after the idea got so popular, they became a setting for the spouting of pseudoscience and soapbox declarations. However, one of my major research interests is the effect of affect on achievement in organic chemistry at the undergraduate level. Also, grit is becoming a more popular effective measure used to understand learning, so we plan on assessing grit in the context of organic chemistry. While most of the discipline-based education research looks at self-efficacy and other attitudinal data, I think that the high-interest area of grit can help inform future research to explore more psychological constructs.

When it comes to the paper itself, I was surprised to find that the first publication on grit was not in a journal that I recognized due to its high popularity. However, before the TED talk, I suppose it wasn’t a topic of much interest. While I respect the extent to which this study has been popularized, I was a little disappointed in this publication. I don’t know if I think that the participants who visited the recruitment website, authentichappiness.org, are an unbiased sampling of the population to measure a construct which is meant to vary widely across individuals. Another note is that the initial confirmatory factor analysis solution was not within the realm of acceptable fit, something that I wonder if reviewers noticed or if I’m looking to the measurement statistics because that it my area of expertise. In the end, the methodology used in the determination of factor structure with 2 factors, followed by the use of the scale score as a single factor make the results challenging to accept. I think that the construct became so popular despite these methodological challenges for two reasons: One, because it’s a sexy idea that appeals to the masses and two, because I don’t think (and this is true of the journals I write for as well) that the reviewer pool has a high enough number of methods experts to ensure that the basic qualifications for publication are met.

I’m going to be reading a lot about grit on my way to doing my own data analysis, so I look forward to seeing what else is out there.

Trending Blogs

Recently Viewed Blogs

Fortitude, gratitude, and grad school