My Machine Learning Blog

[nhandyjr]

Detecting Oscillating Submission Lags in HBO Therapy Claims: A Variance‑Based Fraud Detection Metric for Medicare Program Integrity

Definition of Key Term – Submission Lag

Throughout submission lag is defined as:

 Submission Lag = Submission Date – Claim‑Line From Date (in days)

Where:

Claim‑Line From Date = the date the service was actually provided (e.g., date of HBO therapy session).

Submission Date = the date the provider submitted the claim to the payer (e.g., Medicare).

Examples:

Service on Jan 1, submitted on Jan 1 → lag = 0 days.

Service on Jan 1, submitted on Feb 20 → lag = 50 days.

This lag is not inherently suspicious; providers may batch claims weekly or monthly. However, certain patterns of lags can indicate manipulation.

Executive Summary

The Issue 

Medicare limits Hyperbaric Oxygen Therapy (HBO) to 60 sessions per 365 days based on service dates. Some providers manipulate submission dates – not service dates – to evade prepayment edits and hide excessive sessions. They create an alternating pattern of submission lags (e.g., 0 days, then 100 days, then 0, then 100…). Both the mean lag and simple batching rules miss this pattern.

The Solution 

Two variance‑based metrics calculated from sorted claim sequences: 

1. Variance of submission lag – measures the amplitude of oscillation. 

2. Variance of reordering – measures how often claims are submitted out of service order. 

Together, they flag providers who are “gaming” the submission timing.

Impact 

In a pilot review of 45 HBO providers, two had extreme values on both metrics. Audit confirmed one case of backdating (90+ days) and one case of exceeding the 60‑session limit (85 sessions). Both were referred for recovery.

Personal note – earning the endorsement 

I had asked Dr. Zhenhua Huang (PhD in biostatistics) for a LinkedIn endorsement for SAS and Statistics for nearly a year. He never responded to my request. After showing him this variance‑based approach – which he himself had been trying to figure out how others used variance in similar models – he finally gave the endorsement. This paper is dedicated to that principle: earn it.

1. The Scam – “Submission Lag Offsetting”

The rule: No more than 60 HBO sessions in any rolling 365‑day period (by service date). 

The cheat: 

Deliver 60 sessions legitimately (service dates Jan–Jun). 

Submit half on time (lag=0), half with a long lag (e.g., 100 days). 

Deliver a second block of sessions later in the year, but submit those with the opposite pattern. 

Why?

Many payers run prepayment edits only on claims submitted within 90 days. The alternating pattern ensures half the claims skip prepayment checks. Also, when sorted by submission date, the two blocks interleave, hiding the true service date density.

The clue: Normal providers have low‑variance lags (e.g., all 45 days). Alternating schemes produce high variance and scrambled submission order.

2. Metrics – Technical Definition

Let a provider have n claims for a given patient (or aggregated at provider level). Sort claims by service date (oldest to newest). Assign service_order = 1,2,…,n.

Define submission lag for claim i :

Metric 1 – Variance of lag

Metric 2 – Variance of reordering 

Sort claims by submission date (ties broken by service date). Assign submission_order = 1,2,…,n. For each claim, compute the absolute difference:

Then calculate:

Rule of thumb (based on simulation, n≥15): 

Low risk:  below peer median AND  ≈0

Medium risk: either metric above 75th percentile 

High risk: both metrics above 90th percentile (flag for audit)

3. Results from Pilot Data

Using simulated data that mirrored real patterns (n=10 per provider, as in the attached Excel file):

Provider B would be flagged for high var_lag alone. Provider D (random, chaotic submission) would be flagged for both. In real data, high var_reorder without high var_lag might indicate a different issue (e.g., frequent resubmissions). The two‑metric approach reduces false positives.

4. Discussion

Why variance beats mean: 

Mean lag is blind to oscillation. Variance captures the amplitude. This is what distinguishes suspicious alternating patterns from normal batch billing.

Why reordering matters: 

A provider who batches every 45 days will have zero reordering variance. A provider who alternates will scramble submission order, producing positive reordering variance. The combination is powerful.

Limitations: 

Small claim counts (<15) give unstable variances. 

Trends (e.g., linearly increasing lags) also increase variance; detrending may be required. 

Not diagnostic – flags only indicate need for audit.

Extensions:  

Add autocorrelation at lag 1 to explicitly test for alternation. 

Use peer‑group benchmarking (specialty, region) instead of fixed percentiles. 

Integrate into automated monthly monitoring dashboard.

5. Conclusion

A simple, explainable metric – variance – can uncover a sophisticated submission timing scam that mean‑based statistics miss. The dual‑metric approach (lag variance + reordering variance) is easy to implement in SAS, requires no machine learning, and has already led to real recoveries. For program integrity analysts, it’s a new tool in the toolkit.

Acknowledgments

My esteemed and dear friend, Dr. Zhenhua Huang, who made me earn every bit of praise, and whose honesty and rigor I deeply respect.

Code and methodology are open for reuse. Contact me for collaboration or questions.

SAS Implementation

*** Step 1: Sort by provider and service date;

proc sort data=claims out=step1;

   by provider_id service_date submission_date; run;

*** Step 2: Create service_order;

data step2;

   set step1;

   by provider_id;

   if first.provider_id then service_order = 0;

   service_order + 1; run;

*** Step 3: Sort by provider and submission date to get submission_order;

proc sort data=step2 out=step3;

   by provider_id submission_date service_date; run;

data step4;

   set step3;

   by provider_id;

   if first.provider_id then submission_order = 0;

   submission_order + 1; run;

*** Step 4: Sort back into service order for variance calculation;

proc sort data=step4 out=final_aligned;

   by provider_id service_order; run;

*** Step 5: Compute metrics using PROC SQL;

proc sql;

   create table provider_metrics as

   select provider_id,

          count(*) as claim_count,

          mean(lag_days) as mean_lag,

          var(lag_days) as var_lag,

          var(abs(service_order - submission_order)) as var_reorder

   from final_aligned

   where calculated claim_count >= 15

   group by provider_id; quit;

***Step 6: Flag outliers (example: top 10% by var_lag);

proc univariate data=provider_metrics noprint;

   var var_lag var_reorder;

   output out=pctl pctlpre=P_ pctlpts=90 75; run;

data flagged;

   if _n_=1 then set pctl;

   set provider_metrics;

   flag_lag_high = (var_lag > P_var_lag_90);

   flag_reorder_high = (var_reorder > P_var_reorder_90);

   flag_audit = (flag_lag_high and flag_reorder_high); run;

Notes on the code:

var() in PROC SQL returns sample variance (denominator n-1). 

Ties in submission date are broken by service_date in the second sort, matching the ROW_NUMBER behavior. 

Minimum claim count (15) ensures stability.

[nhandyjr]

Longitudinal Feature Stability Across Election Cycles (2012-2024) using Decision Tree, Random Forest, and XGBoost

Norman A. Handy, Jr. - Independent Researcher

May 2026

Abstract

Using a decision tree on the 2012 Outlook‑on‑Life (OOL) Survey (where other algorithms failed due to extreme class imbalance), this study identifies the top survey questions predicting Tea Party membership. The current scholarship extends the analysis to 2016‑2024 ANES data using Decision Tree, Random Forest, and XGBoost to predict strong Trump support (MAGA). The results show that the Black Lives Matter thermometer (therm_blm) is the direct successor to BWEqulJust_7 (criminal justice fairness), while work_way_up ("Blacks should work their way up without special favors") is the direct successor to BWEqulOppty (racial equality of achievement opportunity). This one-to-one mapping provides a precise empirical test of Dr. Ronald W. Walters' theory that racial attitudes adapt to new political symbols.

Introduction

How do researchers measure racial attitudes when explicit racism is socially unacceptable? Political strategists have long used coded language. Lee Atwater famously explained that by the 1980s, overt racial slurs were replaced by abstract economic policies, with the understanding that “Blacks get hurt worse than Whites” as a byproduct. Dr. Ronald Walters systematized this insight, coining the term proxy issues – seemingly neutral survey questions or policy positions that serve as vehicles for underlying racial resentment.

This study applies machine learning to two distinct datasets:

2012 Outlook‑on‑Life (OOL) Survey, targeting Tea Party membership. Only a decision tree produced stable results due to the very small minority class (≈2% Tea Party members).

2016‑2024 ANES Surveys, targeting strong Trump support (feeling thermometer ≥ 70). Here, Decision Tree, Random Forest, and XGBoost all perform well.

The goal is to identify which survey questions are the strongest predictors and to track how those proxies change from the Tea Party era to the MAGA era.

Results

The feature importance tables below show the top predictors for 2016, 2020, and 2024 after dropping the low‑importance features (police_treat, police_howmuch, discrimination_blacks). The BLM thermometer (therm_blm) is the dominant variable in every year and model, while work_way_up is consistently second or third.

The table shows Decision Tree results (the most interpretable model). Full tables for all three models are provided in the appendix.

 Footnote: Feature importances are normalized to sum to 1.0. A value of 0.46 means that variable accounted for 46% of the total predictive power (impurity reduction) of the decision tree; 0.75 means 75%, etc. The higher the number, the more the model relies on that survey question to separate Trump supporters from non‑supporters.

Interpretation: Walters & Atwater

Atwater’s Blueprint → Walters’s Theory → Empirical Test

This represents a key evolution from the 2012 model. In 2012, BWEqulOppty (racial equality of achievement opportunity) was the top predictor and BWEqulJust_7 (criminal justice fairness) was second. By 2016-2024, the focus shifted. The therm_blm (BLM) thermometer is now the dominant variable in every year, while work_way_up, the successor to BWEqulOppty, is consistently second or third.

Explanation for the evolution of the 2012 model

The emergence of therm_blm variable to the top of the feature importance rankings is not accidental.

2014‑2016 - Police body‑worn cameras expand rapidly. In August 2014, President Obama proposes federal funding to reimburse half the cost of body‑camera programs. By 2016, major cities (Washington, D.C., New York, Los Angeles) have launched pilot programs.

2016 – therm_blm already dominates the Decision Tree (0.46 importance). By this point, the public had seen videos of the deaths of Eric Garner, Michael Brown, Tamir Rice, Walter Scott, Freddie Gray, Alton Sterling, and Philando Castile. Body‑worn cameras were becoming standard.

2018‑2020 - Body‑worn cameras become nearly universal in large urban police departments. By 2020, footage is routinely released within days of a shooting, shaping public perception almost immediately.

2020 – therm_blm reaches 0.75 importance. This is the year of Breonna Taylor’s killing, George Floyd’s murder (viral video), and the largest racial justice protests in a generation. The BLM thermometer captured the raw, visceral reaction to what people had watched on their phones, televisions, and computers.

2024 – therm_blm remains the top predictor (0.48 importance). The baseline awareness of police violence had permanently shifted. BLM was no longer a hashtag. It is a firmly established political symbol.

Conclusion

The one-to-one mapping provides a clear empirical test of Walters’ theory: proxy issues evolve in lockstep with the political discourse they reflect. This study demonstrates that machine learning can uncover the evolving proxy issues that express racial and social resentment in American politics. From the Tea Party to MAGA, the undocumented immigrant thermometer remains a consistent strong predictor, while the BLM thermometer rises to dominance as the direct successor to the criminal justice fairness question. The work_way_up variable persists as the successor to the racial equality of achievement opportunity question.

Do Proxies Add Value Beyond Demographics?

To test whether the proxy questions capture attitudes independent of standard demographic and partisan controls, I re‑trained the Decision Tree models on an expanded set of features that included age, education, income (where available), gender, race, rural/urban, and party identification (7‑point scale). All models were evaluated on a 20% test holdout (stratified).

The results show that adding demographics – especially party ID – improves predictive performance, but the original proxies retain meaningful importance.

In 2016, the demographic‑augmented model improved AUC from 0.754 to 0.822. The decision tree feature importances show that party_id (party identification) accounted for 60% of the predictive power, yet therm_undoc (undocumented immigrants) still contributed 14%, indicating that xenophobia adds information beyond partisanship.

In 2020, the improvement was even clearer: AUC rose from 0.895 to 0.935. party_id dominated with 86% importance, but therm_blm (4%) and blacks_gotten_less (2.6%) remained non‑trivial.

In 2024, only limited demographics were available (age, gender, race, rural – party ID was missing for most respondents). Adding these did not improve performance (AUC dropped slightly from 0.839 to 0.806). In this year, the proxies themselves dominated: work_way_up (44%), therm_undoc (17%), and therm_blm (18%) were the top three features.

SHAP Interaction Finding

Across all years, the SHAP dependence plots for therm_blm (BLM thermometer) revealed that the feature with the strongest interaction was blacks_gotten_less (“Blacks have gotten less than they deserve”). The correlation between blacks_gotten_less and the SHAP values of therm_blm was consistently moderate to strong:

This means that the effect of BLM thermometer on Trump support is not uniform – it is amplified among respondents who believe Blacks have gotten less than they deserve, pointing to an interaction between anti‑Black grievance and attitudes toward the BLM movement

SHAP Bar Plots - Global Feature Importance

Figure 1: SHAP bar plots for 2016, 2020, and 2024. The bars show the mean absolute SHAP value for each feature – the average impact of that feature on the model’s prediction (positive or negative). Higher bars indicate greater overall importance.

In 2016, therm_blm is the most important feature, followed by work_way_up and therm_undoc. In 2020, therm_blm dominates with a large margin. In 2024, the most important feature is work_way_up, with therm_blm as a close second. This shift suggests that while BLM remains a powerful proxy, other attitudes (e.g., toward undocumented immigrants) may have gained relative importance by 2024.

Figure 2: SHAP summary dot plots. Each dot represents one survey respondent. The x‑axis shows the SHAP value (positive → pushes prediction toward Trump support; negative → pushes away). Color indicates the feature value (red = high, blue = low).

For therm_blm, red dots (warm toward BLM) consistently appear on the left (negative SHAP) across all years, meaning warmer BLM feelings decrease Trump support probability. Blue dots (cold toward BLM) appear on the right (positive SHAP). This pattern is strongest in 2020, where most therm_blm dots are concentrated on the far left. Other top features, such as work_way_up and therm_undoc, show the opposite direction (higher values → positive SHAP).

Figure 3: Dependence plots for therm_blm. The x‑axis shows the BLM thermometer values as coded in the ANES data (range 0–1000, where 1000 corresponds to the warmest possible feeling). The y‑axis is the SHAP value for that feature (positive → pushes toward Trump support; negative → pushes away). Points are colored by the interacting feature with the highest correlation – here, blacks_gotten_less (“Blacks have gotten less than they deserve”). In 2020, the downward slope is steepest: higher BLM warmth (toward 1000) sharply lowers Trump support probability. The color reveals that respondents who also agree with blacks_gotten_less (high values – i.e., believe Blacks got less than they deserve) show even more negative SHAP, indicating an interaction between anti‑Black grievance and BLM attitudes. Note: SHAP (SHapley Additive exPlanations) values measure how much each feature pushes a prediction away from the baseline (average Trump support rate). Positive SHAP → increases Trump support probability; negative → decreases. The bar plot shows mean absolute SHAP (overall importance). The summary dot plot shows direction and distribution. The dependence plot shows how the effect of therm_blm changes with its value and interacts with other features

Appendix: Full Feature Importance Tables 

Figure 4: Full feature importances by year and model (Decision Tree, Random Forest, XGBoost).

Ethical note

Identifying proxies is not about labeling individuals. It is about understanding how systems of racial resentment operate beneath the surface – a necessary step for those who wish to counteract the politics of division.

Interactive Dashboard

Explore the decision tree models live: Proxy Politics Dashboard

References 

Atwater, L. (1981). Interview (excerpts).   

Walters, R. W. (2003). White Nationalism, Black Interests. Wayne State University Press.    

ANES (2016, 2020, 2024). American National Election Studies.   

Outlook‑on‑Life Survey (2012). ICPSR. 

[nhandyjr]

Why Three More “Advanced” Algorithms Failed and a Simple Decision Tree Succeeded

In my analysis, "Predicting Political Party Membership: A Validated Decision Tree Approach," I set out to predict a rare event: Tea Party membership (only 2% of the unweighted sample). A decision tree was built that achieved validation AUC = 0.862 and caught 79% of actual Tea Party members (sensitivity).

But before settling on that tree, I tested three other widely used algorithms. All of them failed – some, in spectacular fashion – for reasons that teach an important lesson about small, imbalanced data.

1. Logistic Regression (SAS, stepwise, validated on 30% holdout)

A stepwise logistic regression with the same weighting (weight=34 for Tea Party members) and the same validation split. I even added the MISSING option to handle missing values like the decision tree.

Results (validation):

AUC = 0.653 (barely better than a coin flip)

Sensitivity = 0.31 (caught only 31% of true Tea Party members)

Specificity = 0.88

Why it failed: Logistic regression assumes linear, additive relationships – no interactions. The true patterns in the data are non‑linear and interactive (e.g., “strongly agree with equal opportunity” and “rate undocumented immigrants very low” → high Tea Party probability). A linear model cannot capture that, no matter how many variables you throw at it.

2. XGBoost (Python, single tree mode, tuned)

I used the same features, one‑hot encoding, missing indicators, and class weight (scale_pos_weight=34) with depths from 4 to 10 and weights from 20 to 47.

Best result (validation):

AUC = 0.718

Sensitivity = 0.071 (only 1 out of 14 true positives caught)

Specificity = 0.99 (almost never predicted Tea Party)

Why it failed: XGBoost’s gradient‑based splitting is dominated by the massive majority class, even with scale_pos_weight. It became extremely‑conservative, preferring to predict “non‑member” almost always. The rare class signal was too weak for its boosting mechanism.

3. Random Forest (Python, 100 trees, max depth=5, sample weights)

A random forest with the same depth and leaf size as the decision tree, using sample_weight to replicate SAS’s WEIGHT statement.

Results (validation):

AUC = 0.806 (slightly higher than the single tree)

Sensitivity = 0.214 (only 3 of 14 true positives caught)

Specificity = 0.973

Why it failed (for my purpose): Although the AUC was respectable, the forest missed 79% of actual Tea Party members. It was too conservative – it only predicted “Tea Party” when the signal was overwhelming. For my goal (identifying who is likely to be a Tea Party member), that low sensitivity makes the model useless. The single tree was far better at actually finding the rare cases.

The Lesson: Fancy is not always Better

With only 2,294 observations and a 2% target rate, complex algorithms often:

Over‑fit to the majority class

Become overly conservative

Fail to learn rare patterns

A well‑pruned, weighted decision tree proved to be the best – because it:

Captures non‑linear interactions naturally

Handles missing values by treating them as a separate category

Gives interpretable rules (e.g., “If BWEqulOppty = 1 or 2 and RateUnDoc_100 < 15.16 → 75% Tea Party probability”)

Achieves high sensitivity: it actually finds the people we care about.

So when someone tells you “you should use XGBoost or random forest, (or some other, more powerful algorithm)” remember: the simplest model that fits your data and your goal is often the right one.

*** Final Note: Why SVM and Neural Networks Were Not Attempted ***

Again, with only 2,294 observations and a 2% target rate, two other popular algorithms – Support Vector Machines (SVM) and Neural Networks – were not even attempted. Here’s why:

SVM – Relies on finding support vectors from both classes. With only ~45 positive cases, the minority‑class support vectors would be too few to define a stable decision boundary. SVMs also cannot handle missing values natively and do not produce interpretable rules.

Neural Networks – Require large amounts of data to learn meaningful weights. With 2,294 rows and a rare target, any neural network would either memorize the training set or never converge to a useful pattern. Moreover, they lack the interpretability of the simple decision tree which is the antithesis of my goal to understand why people are Tea Party members.

Given the small, imbalanced dataset, these algorithms were doomed from the start. My decision tree succeeded because of its simplicity, transparency, and design for data with the aforementioned characteristics.

[nhandyjr]

*** Upcoming Research ***

Estimating the Causal Effects of Covert Actions on Institutional Quality in Sub-Saharan Africa

1. Motivation:

Sub-Saharan Africa has experienced repeated cycles of political instability since independence, with military coups d'état representing one of the most persistent threats to democratic consolidation and economic development. In the 1960s alone, over 30 coups occurred across the African continent, a pattern that has dangerously reemerged in the 2020s with successful military takeovers in Mali, Burkina Faso, Niger, Gabon, and Sudan. Beyond the tragic local consequences, these events are often shadowed by deeper historical questions: What role did foreign powers play in engineering these regime changes? And more fundamentally, do these interventions—whether foreign-engineered or purely domestic—systematically alter the quality of governance in ways that hinder long-term development? 

This research seeks to answer this question by bridging two previously separate domains of inquiry: institutional economics, which measures the quality of governance, and the historiography of covert operations. By constructing a novel dataset that leverages the historic 2025 declassification of records related to the assassinations of President John F. Kennedy and others, this project will, for the first time, quantitatively estimate the causal effect of CIA-involved military coups on long-term institutional quality, a cornerstone of economic development. 

2. Research Questions:

The primary research question is: 

"What is the causal effect of military coups on institutional quality in Sub-Saharan Africa, and does documented CIA involvement systematically moderate these effects?"

This overarching question will be addressed through three secondary lines of inquiry: 

1.  General Effect: What is the average causal effect of all coups on institutional quality metrics in Sub-Saharan Africa from 1996 to 2024? 

2.  Heterogeneity by Foreign Involvement: Do coups with documented prior CIA involvement or support have a measurably different effect on subsequent institutional quality compared to coups with no such foreign involvement? 

3.  Effect Moderation by Colonial Heritage: Does the effect of a coup (particularly those with foreign involvement) differ systematically between countries with a history of "extractive" colonial institutions versus "residential/settler" colonial institutions? 

3. Literature Review:

This project is situated at the intersection of several mature but distinct literatures, drawing key insights from each. 

A. Colonial Origins of Comparative Development

The foundational work by Acemoglu, Johnson and Robinson (2001) established that the type of colonial strategy—extractive versus settler—has a profound and persistent effect on modern institutions. Countries where European colonizers faced high mortality rates, such as Ghana, were more likely to extract resources and establish weak governing structures that persisted after independence. Conversely, settler colonies with lower mortality rates, such as Kenya, developed more robust institutions. This "colonial heritage" variable will serve as a key moderator in my causal models. 

B. Coup Literature

The study of coups has largely focused on their domestic determinants, predicting which nations are most at risk based on economic grievances, political exclusion, and military dissatisfaction. While research has noted the role of foreign powers in these events, it has struggled to systematically account for covert foreign support due to the classified nature of the evidence. This gap is precisely where this dissertation intends to make its most significant contribution. 

C. Institutional Quality and Development

A large body of research, operationalized by the World Bank's Worldwide Governance Indicators (WGI), has demonstrated that high-quality institutions—characterized by rule of law, control of corruption, and government effectiveness—are central drivers of long-term economic growth. However, most research treats governance as a static or slowly evolving domestic variable and does not adequately account for sudden, externally influenced shocks like a military coup. 

4. Data and Novel Contribution:

The project's core contribution is the creation of a multi-level panel dataset. 

A. Institutional Quality (Outcome Variable) 

The primary outcome will be the six dimensions of the Worldwide Governance Indicators (WGI). The WGI 2.0 update provides a complete, recalculated time series from 1996 across over 200 economies. 

B. Coup Events (Primary Treatment Variable) 

The treatment variable for each country-year will be derived from the coups dataset maintained by the Center for Systemic Peace (CSP). This dataset provides detailed information on the date and outcome (e.g., successful vs. attempted) of coup events globally from 1946 to the present. 

C. Colonial Heritage (Moderator Variable) 

Countries will be coded based on the classification used by Acemoglu, Johnson & Robinson (2001), distinguishing "extractive" colonies from "settler/residential" colonies. This serves as a key moderating variable in my analysis. 

D. Foreign Involvement in Coups (Novel Instrumental Variable) 

The project's unique and causally crucial contribution is the creation of a novel binary indicator, CIA_Involvement. Coded at the country-year level, a value of 1 will indicate that declassified records confirm the existence of a CIA station or documented covert political action (e.g., surveillance, financial support for opposition groups, plotting) targeted at that country in the early 1960s. 

This indicator serves two purposes: 

1.  As a key variable to test if foreign-involved coups have different effects. 

2.  As a powerful instrument to isolate the causal effect, as a foreign power's geopolitical interest (e.g., a CIA base) is a strong predictor of future coup involvement but is external and plausibly exogenous to a country's internal governance dynamics. 

5. Methodology and Machine Learning Integration:

This project will pioneer a platform integrating traditional causal econometrics with advanced machine learning. 

Phase 1: Feature Importance with ML 

The initial analysis will train a Random Forest model to predict Institutional_Quality_{t+1} using lagged coup events, historical CIA involvement data, economic controls, and a wide array of other political and social indicators as features. The feature importance rankings will reveal which factors—specifically, the CIA_involvement indicator—are the most powerful predictors of future governance trajectories. The results will be compared using XGBoost. 

Phase 2: Causal Modeling 

Building on the insights from Phase 1, the core causal analysis will follow a two-stage strategy: 

1.  Baseline Models: The fundamental relationship will be estimated using Panel Fixed Effects Models to control for all time-invariant country-specific heterogeneity. This will be refined with an Instrumental Variables (IV) approach, using CIA_Involvement as a powerful instrument for coup events. 

2.  Advanced Causal ML: To estimate the heterogeneous effects of coups, the project will use Causal Forests, an ML method from the econml library. This powerful technique will model how the effect of a coup varies across nations based on their unique characteristics (e.g., colonial heritage, resource wealth). 

3.  Comparative Case Studies: For countries with clear CIA involvement, the project will implement Synthetic Control Methods (SCM). SCM builds a data-driven counterfactual to estimate what would have happened to a country's institutional trajectory had the foreign-influenced coup not occurred. This provides a powerful, intuitive validation of the broader statistical results. 

 

6. Ethical Considerations and Feasibility:

The proposed use of declassified records to examine past foreign interventions does not involve the collection of new, sensitive data on living individuals. The analysis is focused on historical documents and publicly available aggregate indicators. As such, this project is not expected to require formal Institutional Review Board (IRB) approval, but consultation with the IRB will be sought to ensure compliance with all university standards. Methodologically, the project is highly feasible; the WGI and CSP datasets are public, the AJR colonial data is well-documented, and the JFK records are now accessible online. 

7. References:

Acemoglu, D., Johnson, S., & Robinson, J. A. (2000). The colonial origins of comparative development: an empirical investigation (No. w7771). National Bureau of Economic Research.  

Acemoglu, D., Johnson, S., & Robinson, J. A. (2001). The colonial origins of comparative development: An empirical investigation. American Economic Review, 91(5), 1369-1401.  

Adekera, D. (2025). Predicting coups in real time: A 4IR-based Early Warning Framework for the African Union. Journal of African Union Studies, 14(3).  

Dube, A., Kaplan, E., & Naidu, S. (2008). Coups, Corporations, and Classified Information. UC Berkeley.  

Kaufmann, D., & Kraay, A. (2024). The Worldwide Governance Indicators: Methodology and 2024 Update. World Bank Policy Research Working Paper.  

Romano, T. P., et al. (2025). Reproducibility Package for 2025 Update of Worldwide Governance Indicators. World Bank.  

Trump, D. J. (2025). Executive Order 14176: Declassification of Records Concerning the Assassinations of President John F. Kennedy, Senator Robert F. Kennedy, and the Reverend Dr. Martin Luther King, Jr.

World Bank. (2025). Worldwide Governance Indicators, 2025 Revision.  

[nhandyjr]

Predicting Political Party Membership: A Validated Decision Tree Approach

This decision tree analysis was conducted to test nonlinear relationships between a non-political subset of Outlook-On-Life (OOL) survey questions and the binary, target/predictor variable “Tea Party Membership” in 2012.  In upcoming research, we look at subsequent election cycles (2016, 2020,2024) to verify alignment with MAGA/Trump support and stability of features.

Theoretical Context - The Insights of Dr. Ronald W. Walters:

The decision tree analysis that follows is grounded in the scholarly framework of the late Dr. Ronald W. Walters, a preeminent political scientist, and tenured faculty member in the Department of Government and Politics at the University of Maryland.  Walters argued that a strong, often unstated, hostility fuels the modern conservative movement, generating policies that protect the interests of whites at the expense of others.  Dr. Walters understood that the Tea Party’s power did not come from its stated commitments to fiscal conservatism nor limited government.  Instead, he saw a movement animated by proxy issues – attitudes about race, immigration, and social change that could be expressed in seemingly race‑neutral language.

The decision tree analysis provides empirical evidence for this proxy structure. By modeling survey responses from nearly 2,300 Americans, the tree reveals a clear hierarchy of belief: what sorts of questions best predict Tea Party membership, and how those questions chain together. The results show that the most powerful predictor is not "lower taxes”, nor "smaller government,” but whether one believes blacks and whites have equal opportunity for acheivement. From there, views of undocumented immigrants becomes the decisive second split. This is not anecdote; it is the structure of the electorate captured in data. What follows is a detailed look at how the model was built, why rarity of the target required a weighting strategy, and the full decision tree that emerges.

MODEL APPROACH:

Exactly 29 OOL questions served as possible explanatory variables in classification of responses to the question, “Tea Party Membership (TPM).”  Care was taken to exclude questions related to political parties, groups, individuals, or topics that were also related to the target variable TeaPartyMem_b:

1. How optimistic are you that you will develop a serious and/or marital relationship? (Optmsm_Rlshp)  

2. Are you extremely [optimistic/pessimistic], moderately [optimistic/pessimistic], or slightly [optimistic/pessimistic] (that you will develop a serious and/or marital relationship)? (Optmsm_Rlshp_ntr)  

3. When you think about your future, are you generally optimistic, pessimistic, or neither optimistic nor pessimistic? (Optmsm_Futr)  

4. [To own a home] For yourself and people like you, how easy or hard is it to reach these goals? (HardToOwnHome)  

5. [To have a financially secure retirement] For yourself and people like you, how easy or hard is it to reach these goals? (HardToRetire)  

6. [To send one's children to college] For yourself and people like you, how easy or hard is it to reach these goals? (HardToCollege)  

7. [To become wealthy] For yourself and people like you, how easy or hard is it to reach these goals? (HardToWealth)  

8. [To do better than one's parents did] For yourself and people like you, how easy or hard is it to reach these goals? (HardToBetterParents)  

9. Society has reached the point where Blacks and Whites have equal opportunities for achievement. (BWEqulOppty)  

10. Over the past few years, Blacks have gotten less than they deserve. (BlksLTDeserve)  

11. Irish, Italians, Jews, and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.   (BlksNoFvrIIJ0)  

12. It's really a matter of some people not trying hard enough; if Blacks would only try harder they could be just as well off as Whites. (BlksTryHarder)  

13. Generations of slavery and discrimination have created conditions that make it difficult for Blacks to work their way out of the lower class.   (SlavDiscrmHardForBlks)  

14. Discrimination against Blacks is no longer a problem in the U.S.   (NoBlkDiscrm)  

15. [Black people should teach their children to fight against racial discrimination as much as possible] How much emphasis or de-emphasis should Black people place on each statement in the education of their children? (BlkTchFightDisc)  

16. What about Black women? Do you think that generally what happens in this country to Black women will have something to do with what happens in your life? (BlkwomEffect_b)  

17. [Black people should teach their children to avoid behaviors that are characteristic of Black stereotypes] How much emphasis or de-emphasis should Black people place on each statement in the education of their children? (BlkTchAvdstyp)  

18. [Black people should teach their children to be careful around the police] How much emphasis or de-emphasis should Black people place on each statement in the education of their children? (BlkTchCrfPpI)  

19. [The police] How much do you think you can trust the following institutions? (TrustPolice)  

20. Which of these classes would you say most members of your family belong to? (ClassFam)  

21. Are you a citizen of the United States? (USCit_b)  

22. Is anyone in your household on active duty in the U.S. Armed Forces, Military Reserves, or National Guard? (MilitaryHHAct_b)  

23. How far along the road to your American Dream do you think you will ultimately get on a 10-point scale where 1 is not far at all and 10 nearly there? (RateDrmPath_10)  

24. How would you rate people on welfare? (RatePplwelf_100)  

25. How would you rate Latinos? (RateLatino_100)  

26. How would you rate Blacks? (RateBlk_100)  

27. How would you rate Asians? (RateAsian_100)  

28. How would you rate Undocumented Immigrants? (RateUnDoc_100)  

29. On a seven-point scale, do you think that Blacks and other minorities are treated the same as Whites in the criminal justice system or do not receive equal treatment? (BWEqulJust_7)

THE CHALLENGE - A Rare Target Variable:

Only 45 out of 2,294 respondents (≈1.96%) identified as Tea Party members (TeaPartyMem_b = 1). Without special handling, a classification model would simply predict “non‑member” for everyone – achieving 98% accuracy but 0% sensitivity. To avoid this, I used case weighting: observations with TeaPartyMem_b = 1 were given a weight of 34, while non‑members kept weight = 1. This effectively balanced the two classes during training, forcing the algorithm to pay attention to the rare but interesting group.  Model design & validation

Split criterion: Gini impurity (Entropy produced identical results)

Pruning method: reduced‑error pruning  

Validation: 30% holdout sample (698 observations) 

Final tree size: 8 leaves  

All code was run in SAS Studio, using a custom macro that applies weighting and repeats the process for robustness.

THE DECISION TREE (step by step):

The final tree uses only four survey questions – a remarkably parsimonious set that still achieves strong predictive power.

**Root split**: belief in equal opportunity  

Full question: "Society has reached a point where blacks and whites have equal opportunity for achievement" (BWEqulOppty)

--------------------------------------------------------------------------------------------------------

Left branch (BWEqulOppty = -1,3,4,5 or missing): respondents who do not strongly agree.  

Training TPM = 15.7% (baseline is 1.96%, so this group is already more likely to be members).  

Right branch (BWEqulOppty = 1,2): respondents who strongly/agree.  

Training TPM = 62.4% – a dramatic jump.

Right‑branch subtree (believers in equal opportunity)

Split on RateUnDoc_100 – "How do you rate undocumented immigrants?" (0‑100 scale).  

< 15.16 or missing (n=905): TPM = 75.1% (training) / 78.9%(validation).  

Final split on RateLatino_100 – "How do you rate Latinos?" (0‑100 scale):  

≥ 95.96 → TPM = 91.4% training / 94.4% validation. This leaf is almost pure Tea Party.  

< 95.96 → TPM = 70.9% training / 75.8% validation. Still high.  

≥ 15.16 (n=348): TPM drops to 29.3% training / 24.3% validation → predicts non‑member.

Left‑branch subtree (weak agreement on equal opportunity)

Split on BWEqulJust_7 – “On a seven-point scale, do you think that Blacks and other minorities are treated the same as Whites in the criminal justice system or do not receive equal treatment?” (values 1,5,9(refused to answer) vs. others).  

Values 1,5,9 (n=462): TPM = 36.8% training / 43.6% validation.  

Further split on RateUnDoc_100:  

< 10.11 → TPM = 66.7% training / 69.4% validation → positive leaf.  

≥ 10.11 → TPM = 13.2% training / 25.0% validation → negative leaf.  

Other values (n=838): TPM = 4.1% training / 16.2% validation → negative leaf.

The tree reveals a clear interaction: even strong belief in equal opportunity is not enough by itself – it must be combined with negative views on undocumented immigrants (very low RateUnDoc_100) to produce high TPM. And among that group, living in an area with ≥96% Latino population makes Tea Party membership almost certain (94% in validation).

PERFORMANCE & VALIDATION:

The training‑to‑validation AUC gap (0.907 vs. 0.862) is moderate but acceptable – well within the range of a generalizable model. The validation AUC of 0.86 is strong for this type of data, and the model does not exhibit the extreme overfitting (e.g., training AUC near 1.0) that would make it unreliable.

VARIABLE IMPORTANCE:

Conclusion:

This analysis shows that belief in racial equality for achievement, attitudes toward undocumented immigrants, perceptions of justice, and local demographic context work together to predict Tea Party membership. The model is simple enough to explain and robust enough to trust. The use of validation and case weighting makes these results far more credible than a naïve model that merely maximizes training accuracy.

SAS CODE:

filename reffile '/home/nhandyjr0/ool_pds.csv';

proc import datafile=reffile dbms=csv out=import; getnames=yes; run;

proc contents data=import; run;

proc sql; create table home.ool_rename as select     W1_A12 as ObamaApprove_b    ,w1_l2_4 as TeaPartyMem_b

/* Socio-Econ Information */    ,w1_p2 as ClassCat    ,w1_p3 as ClassFam    ,w1_p13 as USCit_b    ,w1_p13a as USCitNat_b    ,w1_p4 as Sex_Orient    ,w1_p5 as LGBTRel_b    ,w1_p6 as MilitaryHHAct_b    ,w1_p8 as UnionHH_b    ,w1_p9 as ArrestHH_b    ,w1_p10 as ConvictFAM_b    ,w1_p11 as UnEeHH_b

   ,w1_j1_b as IncSHPnlty_10    ,w1_p14 as HealthIns_b    ,w1_p15 as StockInv_b    ,w1_p20 as IncomeGrp    ,w1_p21 as DigiCable_b    ,w1_q1_a as neighcablehard

/* Optimism: Relationships */    ,w1_e2 as Optmsm_Rlshp    ,w1_e2a as Optmsm_Rlshp_ntr

/* Optimism: Personal Future */    ,w1_f1 as Optmsm_Futr    ,w1_f1a as Optmsm_Futr_ntr    ,w1_f2 as OptmsmUS_Futr    ,w1_f2a as OptmsmUS_Futr_ntr

/* Optimism: Life Goals */    ,w1_f3 as WorkHardGetAhead    ,w1_f4_a as HardToOwnHome    ,w1_f4_b as HardToRetire    ,w1_f4_c as HardToCollege    ,w1_f4_d as HardToWealth    ,w1_f4_e as HardToBetterParents    ,w1_f5_a as HardChildToOwnHome    ,w1_f5_b as HardChildToRetire    ,w1_f5_c as HardChildColl    ,w1_f5_d as HardChildWealth    ,w1_f5_e as HardChildBetPar    ,w1_f6    as RateDrmPath_10

/* View on Economy */    ,w1_j3a_b as Change2008Pov

/* View on Institutions */    ,w1_k1_a as TrustWash    ,w1_k1_b as TrustPolice    ,w1_k1_c as TrustLegal    ,w1_n1h as RateUnion_100    ,w1_n1n as RatePubTch_100

/* View on Groups*/    ,w1_n1c as RateLatino_100    ,w1_n1f as RateAsian_100    ,w1_n1d as RateWhite_100    ,w1_n1e as RateBlk_100    ,w1_n1b as RateNatAm_100    ,w1_h2 as NatAmWellProg    ,w1_h3 as GovtRespNatAmTdy

   ,w1_n1g as RateLGBT_100    ,w1_n1k as RateUnEe_100    ,w1_n1l as RateUnWed_100    ,w1_n1a as RatePplWelf_100    ,w1_n1m as RateUnDoc_100

/* Inter-racial dating */    ,w1_e3 as DatedOSRace_b    ,w1_e4 as WillDateOSRace_b    ,w1_e7 as SexOSRace_b    ,w1_e8 as SexOSRace_ntr

/* Views on black progress */    ,w1_h1 as BWEqulOppty    ,w1_k4 as BWEqulJust_7    ,w1_QB1 as BlkProg20_b    ,w1_h4 as BlksLTDeserve    ,w1_h5 as BlksNoFvrIIJO    ,w1_h6 as BlksTryHarder    ,w1_h7 as SlavDiscrmHardForBlks    ,w1_h8 as NoBlkDiscrm

/* Views on Black Child-Rearing */    ,w1_o1 as BlkTchFightDisc    ,w1_o3 as BlkTchNotWhite    ,w1_o4 as BlkTchAvdStyp    ,w1_o5 as BlkTchCrflPol  

/* Effects of Blacks */    ,w1_qa2 as BlkEffect_b    ,w1_qa2a as BlkEffect_ntr    ,w1_QB2 as BlkMenEffect_b    ,w1_QB3 as BlkWomEffect_b

/* Views on Black Self-Determination */    ,w1_qa4c as BlackOwned    ,w1_qa5d as BlackNoWhite

   ,* from work.import order by w1_caseid ;quit;

proc contents data=home.ool_rename(drop=w1_: w2_: PP:) varnum;run; proc freq data=home.ool_rename(drop=w1_: w2_: PP:);run; proc means data=home.ool_rename(drop=w1_: w2_: PP:); run;

data home.ool_clean; set home.ool_rename(drop=w1_: w2_: PP:); * clean the num values; array nums _numeric_; do over nums; if nums=998 then nums=.; end; * clean the character values; array chars _character_; do over chars; if chars = '-' then chars=""; end; * convert dep var 0->2(Not TPM) otherwise 1 (IS TPM); if TeaPartyMem_b =0 then TeaPartyMem_b=2; /*Not TPM*/ else TeaPartyMem_b=1; /*Is TPM*/ run;

proc contents data=home.ool_clean(drop=caseid) varnum;run; proc freq data=home.ool_clean(drop=caseid);run; proc means data=home.ool_clean(drop=caseid); run;

* Delete any existing macro;  %symdel classtree_final; 

* Define the macro; %macro classtree_final(ds=, msopt=, depvar=, growparm=, pruneparm=, weight_factor=);    %local i wf wf_str;    %do i=1 %to %sysfunc(countw(&weight_factor, %str( )));        %let wf = %scan(&weight_factor, &i, %str( ));        %let wf_str= &wf;        %if %sysevalf(&wf > 0) %then %do;            data _temp_weighted_&i;                set &ds;                if &depvar = 1 then weight = &wf;                else weight = 1;            run;            %let data_in = _temp_weighted_&i;            %let weight_stmt = weight weight;        %end;        %else %do;            %let data_in = &ds;            %let weight_stmt = ;        %end; ods output TreePerformance=fit_&i NodeTable=treenodes; ods trace on;

title "Weight factor = &wf_str";        title2 "Missing Option = &msopt";        title3 "Growth Parameter = &growparm";        title4 "Prune Parameter = &pruneparm";        proc hpsplit data=&data_in assignmissing=&msopt seed=15531 nodes=detail plots=all;            &weight_stmt;            class &depvar                                 BWEqulOppty                 BWEqulJust_7 /*                 Optmsm_Rlshp */ /* ClassFam */ /* USCit_b */ /* MilitaryHHAct_b */ /* Optmsm_Rlshp_ntr */ Optmsm_Futr /* HardToOwnHome */ /* HardToRetire */ /* HardToCollege */ /* HardToWealth */ HardToBetterParents /* TrustPolice */ /* BlksLTDeserve */ /* BlksNoFvrIIJO */ /* BlksTryHarder */ /* SlavDiscrmHardForBlks */ /* NoBlkDiscrm */ /* BlkTchFightDisc */ /* BlkTchAvdStyp */ /* BlkTchCrflPol */ /* BlkWomEffect_b */

;

           model &depvar =                 BWEqulOppty                 BWEqulJust_7                 RateLatino_100                 RateUnDoc_100 /*              RateUnion_100 */ /* Optmsm_Rlshp */ /* RateDrmPath_10 */ /* RateBlk_100 */ /*RatePplWelf_100*/ /* ClassFam */ /* USCit_b */ /* MilitaryHHAct_b */ /* Optmsm_Rlshp_ntr */                  Optmsm_Futr  *this factor gives lift, but has no variable importance; /* HardToOwnHome */ /* HardToRetire */ /* HardToCollege */ /* HardToWealth */ /*HardToBetterParents*/ /* TrustPolice */ /* BlksLTDeserve */ /* BlksNoFvrIIJO */ /* BlksTryHarder */ /* SlavDiscrmHardForBlks */ /* NoBlkDiscrm */ /* BlkTchFightDisc */ /* BlkTchAvdStyp */ /* BlkTchCrflPol */ /* BlkWomEffect_b */ ;            partition fraction(validate=.3);            grow &growparm;            prune &pruneparm;        run;

ods output close;        %if &wf > 0 %then %do;            proc datasets library=work nolist;                delete _temp_weighted_&i;            run;        %end;

   %end; %mend classtree_final;

* Call the macro with a list of weights (use %str to mask spaces); %classtree_final(ds=home.ool_clean, msopt=similar, depvar=TeaPartyMem_b,                 growparm=gini, pruneparm=reducederror,                 weight_factor=%str( 34 ));

%classtree_final(ds=home.ool_clean, msopt=similar, depvar=TeaPartyMem_b,                 growparm=entropy, pruneparm=reducederror,                 weight_factor=%str( 34 ));