Random John

Random thoughts on statistics, skepticism, and stuff in general.

[johndjohnson]

I set up a new data analysis blog

Well, I tried to write a blog post using the RStudio Rmarkdown system, and utterly failed. Thus, I set up a system where I could write from RStudio. So I set up a Github pages blog at randomjohn.github.io. There I can easily write and publish posts involving data analysis. via Blogger http://ift.tt/2gBDugt

[johndjohnson]

Windows 10 anniversary updates includes a whole Linux layer - this is good news for data scientists

If you are on Windows 10, no doubt you have heard that Microsoft included the bash shell in its 2016 Windows 10 anniversary update. What you may not know is that this is much, much more than just the bash shell. This is a whole Linux layer that enables you to use Linux tools, and does away with a further layer like Cygwin (which requires a special dll). However, you will only get the bash shell out of the box. To enable the whole Linux layer, follow instructions here. Basically, this involves enabling developer mode then enabling the Linux layer feature. In the process, you will download some further software from the Windows store. Why is this big news? To me, this installs a lot of the Linux tools that have proven useful over the years, such as wc, sed, awk, grep, and so forth. In some cases, these tools work much better than software packages such as R or SAS, and their power comes in combining these tools through pipes. You also get apt-get, which enables you to install and manage packages such as SQLite, octave, and gnuplot. You can even install R through this method, though I don't know if RStudio works with R installed in this way. If you're a Linux buff who uses Windows, you can probably think of many more things  you can do with this. The only drawback is that I haven't tried using any sort of X Windows or other graphical interfaces. via Blogger http://ift.tt/2cRpaOD

[johndjohnson]

Which countries have Regrexit?

This doesn't have a lot to do with bio part of biostatistics, but is an interesting data analysis that I just started. In the wake of the Brexit vote, there is a petition for a redo. The data for the petition is here, in JSON format. Fortunately, in R, working with JSON data is pretty easy. You can easily download the data from the link and put it into a data frame. I start on that here, with the RJSONIO package, ggplot2, and a version of the petition I downloaded on 6/26/16. One question I had was whether all the signers are British. Fortunately, the petition collects the place of residence of the signer, assuming no fraud. I came up with the following top 9 non-UK countries of origin of signers.

There are a couple of things to remember when interpreting this graph:

I left off the UK. The number of signatures is over 3 million, and contains by far the largest percentage of signatories.

5 of the 9 top countries are neighbors, including the top 2. The other 4 are Australia, the US, Canada, and New Zealand, who are all countries that have strong ties to the UK.

This assumes no petition fraud, which I can't guarantee. I saw at least one Twitter posting telling people to use her (if the profile pic is to be believed) residence code. There is a section of the petition data showing constituency, so I'm wondering if it would be possible to analyze the petition for fraud. I'm not as familiar with British census data as I am with US, but I imagine a mashup of the two would be useful.

via Blogger http://ift.tt/29cUGU0

[johndjohnson]

Little Debate: defining baseline

In an April 30, 2015 note in Nature (vol 520, p. 612), Jeffrey Leek and Roger Peng note that p-values get intense scrutiny, while all the decisions that lead up to the p-values get little debate. I wholeheartedly agree, and so I'm creating a Little Debate series to shine some light on these tiny decisions that may not get a lot of press. Yet these tiny decisions can have a big influence on statistical analysis. Because my focus here is mainly biostatistics, most of these ideas will be placed in the setting of clinical trials. Defining baseline seems like an easy thing to do, and conceptually it is. Baseline is where you start before some intervention (e.g. treatment, or randomization to treatment or placebo). However, the details of the definition of baseline in a biostatistics setting can get tricky very quickly.

The missing baseline

Baseline is often defined as the value at a randomization or baseline visit, i.e. the last measurement before the beginning of some treatment or intervention. However, a lot of times things happen - a needle breaks, a machine stops working, or study staff just forget to do procedures or record times. (These are not just hypothetical cases ... these have all happened!) In these cases, we end up with a missing baseline. A missing baseline will make it impossible to determine the effect of an intervention for a given subject.

In this case, we have accepted that we can use previous values, such as those taken during the screening of a subject, as baseline values. This is probably the best we can do under the circumstances. However, I'm unaware of any research on what effect this has on statistical analysis.

To make matters worse, a lot of times people without statistical training or expertise will make these decisions, such as putting down a post-dose value as baseline. Even with good documentation, these sorts of mistakes are not easy to find, and, when they are, they are often found near the end of the study, right when data management and statisticians are trying to produce results, and sometimes after interim analyses.

The average baseline

Some protocols specify that baseline consists of the average of three repeated measurements. Again, this decision is often made before any statisticians are consulted. The issue with such a statistical analysis is that averages are not easily comparable to raw values. Let's say that a baseline QTc (a measure of how fast the heart charge recovers from a pump, corrected for heart rate) is defined based on 3 electrocardiogram (ECG) measurements. The standard deviation of a raw QTc measurement (i.e. based on one ECG), let's say, is s. The standard deviation of the average of those three (assuming independence) is s/√3, or just above half the standard deviation of the raw ECG. Thus, a change of 1 unit in the average of 3 ECGs is a lot more noteworthy than a change of 1 unit in a single ECG measurement. And yet we compare that to single measurements for the rest of the study.

To make matters worse, if the ECG machine screws up one measurement, then the baseline becomes the average of two. A lot of times we lose that kind of information, and yet analyze the data as if the mystery average is a raw measurement.

The extreme baseline

In one observational study, the sponsor wanted to use the maximum value over the last 12 months as a baseline. This was problematic for several reasons. Like the average, the extreme baseline (here the maximum) is on a different scale, and even has a different distribution, than the raw measurement. The Fisher-Tippett (extreme value) theorem states that the maximum of n values converges to one of three extreme value distributions (Gumbel, Frechet, or Weibull). These distributions are then being compared to, again, single measurements taken after baseline. What's worse, any number of measurements could have been taken for those subjects within 12 months, leading to a major case of shifting sands regarding the distribution of baseline.

Comparing an extreme value with a later singular measurement will lead to an unavoidable case of regression to the mean, thus creating an apparent trend in the data where none may exist. Without proper context, this may lead to overly optimistic interpretations of the effect of an intervention, and overly small p-values. (Note that a Bayesian analysis is not immune to the misleading conclusions that might arise from this terrible definition of baseline.)

Conclusion

The definition of baseline is a "tiny decision" that can have major consequences in a statistical analysis. Yet, the impact of this decision has not been well studied, especially in the context of a clinical trial where a wide range of definitions may be written into a protocol without the expert advice of a statistician. Even a definition that has been well-accepted -- that baseline is the last singular pre-dose value before intervention -- has not been well-studied in the scenario of missing baseline day measurement. Other decisions are often made without considering the impact on analysis, including some that may lead to wrong interpretations.

via Blogger http://ift.tt/1PxB5Rd

[johndjohnson]

Simulating a Weibull conditional on time-to-event is greater than a given time

Recently, I had to simulate a time-to-event of subjects who have been on a study, are still ongoing at the time of a data cut, but who are still at risk of an event (e.g. progressive disease, cardiac event, death). This requires the simulation of a conditional Weibull. To do this, I created the following function: # simulate conditional Weibull conditional on survival > T --------------- # reliability function is exp{-(T+t/b)^a} / exp{-(T/b)^a} = 1-F(t) # n = number of points to return # shape = shape parm of weibull # scale = scale parm of weibull (default 1) # t is minimum (default is 0, which makes the function act like rweibull) my_rweibull <- function(n,shape,scale=1,t=0) {   if (length(t)!=1 && length(t)!=n) {     stop("length(t) is not 1 or n")   }   return(shape*(-log(runif(n))+(t/shape)^scale)^(1/scale)) }

You use this function just like rweibull, with the exception that you pass in another vector t of minimum times or a scalar representing the minimum time of all simulated values. The idea is that the probability that the random variable will be at least T is given by exp{-(T+t/b)^a} / exp{-(T/b)^a}, so you can simulate this with a uniform random variate. I use the inversion method on the reliability function (just like using the inversion method on the distribution function, with the insight that if U is uniform(0,1), so is 1-U). Truth be told, I ought to buckle down and learn how to do packages in R, but for now I'll just pass on some code on my blog if I think it will be helpful (or if I need to find it while doing a Google search later). via Blogger http://ift.tt/1U3xR6y

[johndjohnson]

The American Statistical Association makes a statement on p-values

This deserves to be read, re-read, re-re-read, and taken to heart. The ASA makes their statement on p-values. Some key points:

p-values by themselves offer very little information on the utility of a model, the truth of a statement, or what is behind a data process

Full documentation and transparency of data collection, handling, and analysis are essential to interpret results

This full transparency has not taken place historically for many reasons

Multiple comparisons, even if done informally, affect the interpretation of a p-value

Replicability and reproducibility are essential for scientific progress, and this full transparency is essential to replicability and reproducibility

Hopefully this statement will drive field journals, professors, and regulators to consider more than the p-value (the FDA in fact already does consider more than the p-value, but they still want to see that 0.05 threshold met twice in most cases).

via Blogger http://ift.tt/1U1r7Is

[johndjohnson]

Talk to Upstate Data Science Group on Caret

Last night I gave an introduction and demo of the caret R package to the Upstate Data Science group, meeting at Furman University. It was fairly well attended (around 20 people), and well received. It was great to get out of my own comfort zone a bit (since graduate school, I've only really given talks on some topic in biostatistics) and meeting statisticians, computer scientists, and other sorts of data scientists from many different fields. This is a relatively new group, and given the interest over the last couple of months or so I think this has been sorely needed in the Upstate South Carolina region. We'll be participating in Open Data day in March of this year, so if you are in the Upstate SC region, or don't mind making the trek from Columbia or Western NC, find us on Meetup. Our next meeting is a data hack night which promises to be interesting. via Blogger http://ift.tt/1ZnnzP6

[johndjohnson]

Even the tiniest error messages can indicate an invalid statistical analysis

The other day, I was reading in a data set in R, and the function indicated that there was a warning about a parsing error on one line. I went ahead with the analysis anyway, but that small parsing error kept bothering me. I thought it was just one line of goofed up data, or perhaps a quote in the wrong place. I finally opened up the CSV file in a text editor, and found that the reason for the parsing error was that the data set was duplicated within the CSV file. The parsing error resulted from the reading of the header twice. As a result, anything I did afterward was suspect. Word to the wise: track down the reasons for even the most innocuous-seeming warnings. Every stage of a statistical analysis is important, and small errors anywhere along the way and have huge consequences downstream. Perhaps this is obvious, but you still have to slow down and take care of the details. via Blogger http://ift.tt/1N7FvGR

[johndjohnson]

Statisticians ruin the day again, this time with a retraction

Authors retract second study about medical uses of honey - Retraction Watch at Retraction Watch:

For the second time, authors of manuscripts have had to retract their papers because of serious data analysis errors. While the details of the actual errors are scant, we do know that the article was published, a company tried to replicate the results but failed, the journal editor employed a third-party statistician who found serious errors in the data analysis, and the errors were serious enough that the paper, to stay accepted, would have had to go through a major revision and further peer review. Better to get a statistician to ruin your day before publication (and require another revision) than to have to eat crow because a third party did it. Other thoughts:

Did the authors have a statistician review this before submitting?

Did the journal have this go through a statistical peer review before accepting?

Come to think of it, was a statistician involved in the planning of this study?

via Blogger http://ift.tt/1GWF4NK

[johndjohnson]

The thirty-day trial

Steve Pavlina wrote about a self-help technique called the thirty-day trial. To perform the technique, you commit 30 days of some new habit, such as quitting smoking or writing in a journal. The idea is that it’s psychologically easier to commit to something for 30 days than to make a permanent change, but after the 30 days you break addiction to old habits and have the perspective of whether to continue on with the new habit, go back, or go a completely different direction. For activities like journaling or quitting smoking, this technique might work. After all, psychologist Jeremy Dean announced that it takes 21 days to form a new habit. (This has been treated with some due skepticism, which it should.) However, if you try a 30 day quit-smoking trial and it doesn’t work, try again. And if that doesn’t work, try it again. Until you succeed. However, for activities such as a new diet, treatments, or any sort of healthcare advice, the 30 day trial should not be used. For cases where one might try a 30 day trial just to see if it feels better, such as an elimination diet, confirmation bias is likely to be at play. This is especially true in the case of an elimination diet, where one eliminates some aspect of a diet, like dairy or gluten, and see if some kinds of symptoms, such as bloating or fatigue, go away. In such trials, the results may be due to the eliminated item in question, placebo/nocebo effect, or some third confounded eliminated item. For instance, bloating from gluten-containing foods probably comes from medium-length carbohydrates called FODMAPs. Just because you feel better for 30 days after discontinuing gluten-containing foods doesn’t mean that gluten is the culprit. In fact, it probably isn’t, as determined by a well-designed clinical trial. Likewise, eliminating milk from a diet isn’t likely to do too much unless lactose intolerance is the culprit, and there are tests for that. Returning to Pavlina’s advice, he recommends a 30 day trial over the results of a clinical trial. This is sloppy and irresponsible advice. First, it is very unlikely that an individual will have access to screening assays, animal models, or the millions of dollars needed to discover and develop a drug. That is, unless said individual is up for trying millions of compounds, many potentially toxic and will probably tragically shorten the trial. Instead, a pharmaceutical should be used under the supervision of a doctor, who should be aware of the totality of literature (ideally consisting of randomized controlled trials if possible and a systematic review) and can navigate the benefits and possible adverse effects. A 30-day trial of a pharmaceutical or medical device may be completely inappropriate to realize benefits or assess risks. Here is the primary case where the plural of anecdote is not data. The bottom line is that the 30 day trial is one of several ways (and perhaps not the best one) to change habits that you know from independent confirmation need to be done, like quitting smoking. It’s a terrible way to adjust a diet or determine if a course of treatment is the right one. Treatments should be based on science and under the supervision of a physician who can objectively determine whether a treatment is working correctly. via Blogger http://ift.tt/1Qvru9e

[rudescience]

The Average of Two Square Roots is Less than or Equal to the Square Root of the Avarage

File this under simple math things that make me unreasonably angry but are absolutely fascinating.

I mean… look at that… why???

[johndjohnson]

Ah concave functions ...

[johndjohnson]

Statistics: P values are just the tip of the iceberg : Nature News & Comment

Statistics: P values are just the tip of the iceberg : Nature News & Comment: This article is very important. Yes, p-values reported in the literature (or in your own research) need scrutiny, but so does every step in the analysis process, starting with the design of the study. via Blogger http://ift.tt/1EV9ssK

[spacepetals]

Mobius transformation as rigid motion of the Riemann sphere. Animation made in Mathematica.

code can be found here

[visualizingmath]

Check out Möbius Transformations Revealed by Douglas Arnold and Jonathan Rogness. This video was one of the winners of the 2007 Science and Engineering Visualization Challenge. So awesome. :D 

[johndjohnson]

The many faces of the placebo response

This week, a study was published that claimed that the placebo response is mediated by genetics. Though I need to dig a little deeper and figure out exactly what this article is saying, I do think we need to take a step back and remember what can constitute a placebo response before we start talking about what this means for medical treatment and clinical trials.

In clinical trials, the placebo response can refer to a number of apparent responses to sham treatment:

The actual placebo response, i.e. a body’s physiological response to something perceived to be a treatment

Natural course of a disease, including fluctuations, adaptations

Investigator and/or subject bias on subjective instruments (hopefully mitigated by blinding/masking treatment arms in a study)

Regression to the mean (an apparent time-based relationship caused by one measurement that varies markedly from the average measurement)

… and many, many other sources

This week’s discovery does suggest that there is something physiological to the actual placebo response, and certainly genetics can influence this response. This may be useful in enriching clinical trials where a large placebo response is undesirable, e.g. by removing those subjects who are likely to response well to anything active or inert. After all, you don’t want an estimate of a treatment effect contaminated by a placebo response, nor do you want an impossibly high bar for showing an effect.

But we still need to remember the mundane sources of “placebo response” and lower those before we get too excited about genetic tests for clinical trials.

via Blogger http://ift.tt/1OmvmqT

[koryos]

Can spiders fart?

You know what? Okay. You ask a question, you’re going to damn well get a serious answer. You want to learn about spider farts, punk? You’re going to learn. You’re going to learn a lot more than you bargained for.

Arthropods obviously have very different digestive systems than vertebrates do, and spider digestive systems are unique even for arthropods. All but one species of spider are strictly predatory, and they take advantage of this diet by actually performing most of their digestion outside the body. Their formidable-looking fangs act like hypodermic needles to inject venom that immobilizes their prey. They then spit a cocktail of enzymes into the holes their fangs have created with their mouths (the venom and digestive enzymes are produced in different parts of the spider’s body!). These enzymes act like the ones in our saliva and stomach: they begin to break down the meat. It just happens to still be on the inside of the prey’s skin.

Some spider species, rather than keep everything neatly contained, just tear their prey apart and spit the enzymes onto the pieces. To each his own.

Once the prey’s insides have become a pre-digested slurry (and yeah, the prey is usually dead by this point), the spider slurps it up. This is actually a part of a larger process of spitting and slurping until everything is sufficiently broken down; hairs around the spider’s mouth block particles that are too large from being ingested. This is because the spider’s internal digestive system is shit and can’t handle anything but a liquid diet.

The spider’s stomach is actually a specialized sucking organ (called, appropriately, a ‘sucking stomach’) that flexes in order to facilitate all that slurping and spitting. It’s basically a muscular pump.

The spider digestive tract up to the sucking stomach is actually lined with cuticle- the analogue to our external skin. If having regular skin growing through your mouth and down to your stomach sounds odd, at least you don’t have to shed yours in one large piece. When spiders shed their exoskeletons, they actually have to shed the interior of their sucking stomach, too, and they pull this cuticle out through their brains. You cannot make this shit up.

While spiders don’t have much in the way of internal digestion hardware, they do have excellent storage units. These would be the caeca, located in the midgut past the sucking stomach. Since the spider doesn’t have space taken up by digestive organs, the caeca have a lot more room and even extend down some pairs of legs and even up towards the eyes in some species. Some species can even expand their caeca thanks to their soft abdominal cuticles- most arthropods have hardened exoskeletons and would explode if their organs expanded. So now you know why spider species are soft compared to other arthropods!

This storage capacity means that spiders can generally go a long time without eating, and when they do strike a big windfall, they can store much more than other arthropods could.

So now you know all about spider digestion- except for the end part. Even spiders have to poop. As you can see on the above diagram, they have an anus. Once the spider has extracted all it can get from its prey, the remains move from the caeca to the stercoral sac, which does what our colon does: it compacts and dehydrates everything into poop. And then the spider poops.

Spider poop is actually rather similar to bird poop- it’s usually whitish and semiliquid. This is due to the fact that it is full of concentrated uric acid. (Those of you familiar with the study of poop in all its forms will infer from this that spiders do not, in fact, pee.)

Spider poop, ladies and gentlemen.

Back to the original question: do spiders fart? And how will all that information about the spider’s digestive system (while quite fascinating) help you understand it? The answers are maybe and it really won’t. We fart because the bacteria in our colons produce air during the fermentation of our food. The actual smell comes from volatile sulfur compounds, including hydrogen sulfide, which make up less than one percent of the released gas. So 99% of the gas released when you fart doesn’t smell.

Spiders, too, have bacteria involved in fermentation in their stercoral sac, though they are very different bacteria than ours. But theoretically, that means that gas is probably produced as a byproduct of that fermentation. Though I don’t know of any recordings of spider farts out there.

I hope that answers your goddamn question, spider fart anon.

[biomedicalephemera]

THINGS YOU MUST KNOW

[fuckyeahreligionpigeon]

[johndjohnson]

Fascinating insight into the thoughts of our nation's founders.

[johndjohnson]

Heaven is hotter than hell, proven with math and chemistry

http://www.patheos.com/blogs/exploringourmatrix/2012/10/heaven-is-hotter-than-hell-a-cautionary-example.html

Basically, using the Boltzmann-Stefan equation and the boiling point of sulfur (and biblical references), the author proves that heaven is 100C hotter than hell.