Cecil Gibb Seminar: - Special Panel

Date & time

4–5pm 28 April 2015


Peter Baume Building 42A Room 2.01 


Associate Professor Martin Sellbom,
Professor Michael Smithson,
Professor Elinor McKone,
Dr Dirk Van Rooy,

Is null hypothesis testing a dark art that should be banned?"


Associate Professor Martin Sellbom
Effect size estimates must be reported in the context of NHST
Abstract: Null-Hypothesis Significance Testing (NHST) is fraught with limitations and abuse, and the meaning associated with p less than a priori alpha is not associated with much meaning.  Effect sizes estimates, however, can bring about better meaning to significant (and non-significant) results. The current paper will illustrate how effect size estimates matter in the interpretation of results, and how NHST can be abused when effect size is ignored. If time permits, some underlying assumptions in the interpretation of effect size is discussed.

Professor Michael Smithson
"Doubt is an Uncomfortable Position, but Certainty is a Ridiculous One".
Abstract: Null hypothesis significance testing (NHST) has several flaws and limitations. However, none of these justify banning it. NHST also is subject to commonplace serious misinterpretations and misuses. But these don’t justify banning it either. Estimation alone (e.g., via confidence intervals) will not suffice. We cannot achieve a healthy science without both hypothesis-testing and estimation, but are there alternatives to NHST?  Chief among these is the Bayesian framework, which does have important advantages over NHST.  However, it has some drawbacks as well, and it isn't immune to misinterpretations or misuses. Our main enemy in statistical inference isn't NHST, its fallacious certainty.

Professor Elinor McKone
"Limitations on effect size as a useful measure.”
Abstract: Effect size measures (percentage of variance explained) have been suggested as a replacement for NHST, or made compulsory to report by some journals. "I will discuss how issues of experimental design (within- versus between- participants) and number of items in the test (i.e., number of trials) influence effect size measures, and thus why these often do not in fact provide a useful measure of how big an effect is, nor a replacement for NHST. I will argue that this arises because the variance is not a fixed thing that we need to explain."

Dr Dirk Van Rooy
“Banning NHST is not the solution to issues arising from what are essentially questionable research practices.”

Abstract: Critiques of the use of NHST in psychology are not new, but have gathered some momentum because of the recent association with the "replicability crisis" (and not just in psychology). Although the issues that are typically raised in this regard are real, I don’t think a credible argument has been made that abandoning NHST will somehow solve or even address them.  My main argument is therefore that, although NHST is often misunderstood and easily misused, when applied with good judgment it can be an effective aid to the interpretation of experimental data.  The bottom line is that there is no statistical approach or technique that can compensate for questionable research practices.

Updated:  25 June 2018/Responsible Officer:  Director, RSP/Page Contact:  Web Admin, RSP