P-value abuse

Statisticians, gather!

In a first of its kind public statement, the American Statistical Association (ASA), has addressed the worrying misuse and misunderstanding of the famous p-value. Some niches in science mistreat it while others ignore it completely.

Here is the link to the full statement, and a link to an abbreviated message, it is separated in two parts. The first, explains the reasons and process that led the ASA to release the statement, including a reference to the ‘reproducibility crisis’ reported in a paper last year [1]. The second is the statement itself regarding the meaning of the p-value and recommended use.

It doesn’t address hard questions, like multiple comparisons or potential comparisons, but an interesting read altogether, especially for us who need to infer x>>0 times a day. They provide a long list of related papers if you are interested in reading them. I suggest to read Yoav Benjamini’s and Tal Gallili’s joint post on the matter.

In the 177 years since it’s foundation, the ASA has not publicly addressed such a fundamental scientific subject. This is tantamount to the Archmaesters of Hightower commenting to Westeros on how to ‘count coppers’.

[1] Peng, R. (2015), The reproducibility crisis in science: A statistical counterattack. Significance, 12: 30–32. doi: 10.1111/j.1740-9713.2015.00827.x


Coalesing in R

The coalesce function is a recursive null filler very common in every database software, however R seems to be missing this simple function. Here is my suggestion :

coalesce <- function(x,...) {

  fillerList <- list(...)
    y <- try(y <- unlist(..1))
    if(class(y)=="try-error" | length(y)==0L) {
        x <- x 
    else if(length(y)==1L) {
     x[is.na(x)] <- y
    else {
     x[is.na(x)] <- y[is.na(x)]
    # recursion
    if(length(fillerList)-1L<=0L) {return(x)}
    else {return(coalesce(x,fillerList[-1]))}