Movatterモバイル変換

statcheck

What is statcheck?

statcheck is a “spellchecker” for statistics. It checkswhether yourp-values match their accompanying test statisticand degrees of freedom.

statcheck searches for null-hypothesis significance test(NHST) in APA style (e.g.,t(28) = 2.2,p < .05).It recalculates the p-value using the reported test statistic anddegrees of freedom. If the reported and computed p-values don’t match,statcheck will flag the result as an error.

What can I use statcheck for?

statcheck is mainly useful for:

1. Self-checks: you can usestatcheck tomake sure your manuscript doesn’t contain copy-paste errors or otherinconsistencies before you submit it to a journal.
2. Peer review: editors and reviewers can usestatcheck to check submitted manuscripts for statisticalinconsistencies. They can ask authors for a correction or clarificationbefore publishing a manuscript.
3. Research:statcheck can be used toautomatically extract statistical test results from articles that canthen be analyzed. You can for instance investigate whether you canpredict statistical inconsistencies (see e.g., Nuijten et al., 2017doi:10.1525/collabra.102), or use it to analyze p-valuedistributions (see e.g., Hartgerink et al., 2016doi:10.7717/peerj.1935).

How does statcheck work?

The algorithm behindstatcheck consists of four basicsteps:

1. Convert pdf and html articles to plain textfiles.
2. Search the text for instances of NHST results.Specifically,statcheck can recognizet-tests,F-tests, correlations,z-tests,\(\chi^2\) -tests, and Q-tests (frommeta-analyses) if they are reported completely (test statistic, degreesof freedom, andp-value) and in APA style.
3. Recompute thep-value using the reportedtest statistic and degrees of freedom.
4. Compare the reported and recomputedp-value. If the reportedp-value does not match thecomputed one, the result is marked as aninconsistency(error in the output). If the reportedp-value issignificant and the computed is not, or vice versa, the result is markedas agross inconsistency (decision_error in theoutput).

statcheck takes into account correct rounding of thetest statistic, and has the option to take into account one-tailedtesting. See themanual fordetails.

Installation and use

For detailed information about installing and usingstatcheck, see themanual onRPubs.

Also seestatcheck.io, a web-basedinterface for statcheck.