Ekologiya cheloveka (Human Ecology)Ekologiya cheloveka (Human Ecology)Экология человека1728-08692949-1444Eco-Vector9724910.17816/humeco97249ArticlesСтатьиReview ArticleInterpretation of and alternatives to p-values in biomedical sciencesИнтерпретация величины р и альтернативы её использованию в биомедицинских исследованияхhttps://orcid.org/0000-0002-5464-04985118-0081GrjibovskiAndrej M.ГржибовскийАндрей Мечиславович
Russian Federation

MD, MPhil, PhD

доктор медицины (Швеция), профессор

andrej.grjibovski@gmail.comhttps://orcid.org/0000-0001-8045-12204430-6841GvozdeckiiAnton N.ГвоздецкийАнтон Николаевич
Russian Federation

MD, Cand. Sci. (Med.)

к.м.н.

gvozdetskiy_an@outlook.comNorthern state medical universityСеверный государственный медицинский университетAl-Farabi Kazakh national universityКазахский национальный университет имени Аль-ФарабиWest Kazakhstan Marat Ospanov medical universityЗападно-Казахстанский медицинский университет имени Марата ОспановаNorth-Eastern federal universityСеверо-восточный федеральный университет имени М.К. АммосоваMechnikov North-Western state medical universityСеверо-Западный государственный медицинский университет имени И.И. Мечникова Минздрава России15062022010720222932092182401202225012022Copyright ©; 2022, Grjibovski A.M., Gvozdeckii A.N.Copyright ©; 2022, Гржибовский А.M., Гвоздецкий А.Н.2022Grjibovski A.M., Gvozdeckii A.N.Гржибовский А.M., Гвоздецкий А.Н.https://creativecommons.org/licenses/by-nc-nd/4.0https://hum-ecol.ru/1728-0869/article/view/97249

Existing difficulties in interpretation of the results of statistical analysis have been repeatedly mentioned as one of the factors behind poor reproducibility of research findings in biomedical sciences followed by a series of publications presenting alternatives to improve the situation including a abandonment of p-values and significance testing. In this paper we briefly present the scope of the problem as well as Fischer and Neyman–Pearson approaches to hypothesis testing. Moreover, we present confidence intervals and effect size calculation as alternatives to dichotomization of the results as significant or not significant using a certain cut-off level. In addition, we summarize the pros and cons of suggestion to change the cut-off value from traditional 0.05 to 0.005. We also present a list of the most common misunderstandings of p-values discussed in international statistical literature.

We conclude the paper with brief recommendations on careful interpretation of the results of statistical analysis to prevent misinterpretation and misuse of p-values in biomedical studies.

Существенные проблемы с интерпретацией результатов статистического анализа в биомедицинских исследованиях часто упоминают в литературе в качестве одной из причин кризиса воспроизводимости научных результатов. Закономерно появились предложения по улучшению ситуации, в том числе за счёт полного отказа от представления величины p в публикациях.

В настоящей работе рассмотрены причины сложившейся ситуации в контексте исторически различных подходов к проверке статистических гипотез и представлены альтернативы использованию значения р — доверительные интервалы и величина эффекта. Приведены аргументы «за» и «против» высказываемого в зарубежных источниках литературы предложения по изменению критического уровня значимости с 0,05 до 0,005. Для профилактики ошибочной интерпретации результатов статистического анализа сформирован список наиболее популярных заблуждений о смысле величины р, которые разбираются в ведущих журналах по статистике.

В статье предложены практические рекомендации для молодых учёных, следование которым может существенно сократить случаи некорректной интерпретации результатов статистического анализа в биомедицинских исследованиях.

p-valuesignificance leveleffect sizeconfidence intervalbiomedical researchstatistical analysisвеличина руровень значимостивеличина эффектадоверительный интервалбиомедицинские исследованиястатистический анализPolonioli A, Vega-Mendoza M, Blankinship B, Carmel D. Reporting in experimental philosophy: current standards and recommendations for future practice. Rev Philos Psychol. 2021;12(1):49–73. doi: 10.1007/s13164-018-0414-3Polonioli A., Vega-Mendoza M., Blankinship B., Carmel D. Reporting in experimental philosophy: current standards and recommendations for future practice // Rev Philos Psychol. 2021. Vol. 12, N 1. P. 49–73. doi: 10.1007/s13164-018-0414-3Amrhein V, Trafimow D, Greenland S. Inferential statistics as descriptive statistics: there is no replication crisis if we don’t expect replication. The American statistician. 2019;73(supl. 1):262–270. doi: 10.1080/00031305.2018.1543137Amrhein V., Trafimow D., Greenland S. Inferential statistics as descriptive statistics: there is no replication crisis if we don’t expect replication // The American statistician. 2019. Vol. 73. Suppl. 1. P. 262–270. doi: 10.1080/00031305.2018.1543137Amrhein V, Korner-Nievergelt F, Roth T. The earth is flat (p >0.05): significance thresholds and the crisis of unreplicable research. PeerJ. 2017;5:e3544. doi: 10.7717/peerj.3544Amrhein V., Korner-Nievergelt F., Roth Т. The earth is flat (p >0.05): significance thresholds and the crisis of unreplicable research // PeerJ. 2017. Vol. 5. P. e3544. doi: 10.7717/peerj.3544Szucs D, Ioannidis J.P.A. When null hypothesis significance testing is unsuitable for research: a reassessment. Front Hum Neurosci. 2017;11:390. doi: 10.3389/fnhum.2017.00390Szucs D., Ioannidis J. When null hypothesis significance testing is unsuitable for research: a reassessment // Front Hum Neurosci. 2017. Vol. 11. P. 390. doi: 10.3389/fnhum.2017.00390Akanov A, Turdaliyeva BS, Izekenova AK, et al. Assessment of use of statistical methods in scientific articles of the Kazakhstan’s medical journals. Ekologiya cheloveka (Human Ecology). 2013;20(5):61–64. (In Russ).Аканов А.А., Турдалиева Б.С., Изекенова А.К., и др. Оценка использования статистических методов в научных статьях медицинских журналов Казахстана. // Экология человека. 2013. Т. 20, № 5. С. 61–64.Dorey F. The p value: what is it and what does it tell you? Clin Orthop Relat Res. 2010;468(8):2297–2298. doi: 10.1007/s11999-010-1402-9Dorey F. The p value: what is it and what does it tell you? // Clin Orthop Relat Res. 2010. Vol. 468, N 8. P. 2297–2298. doi: 10.1007/s11999-010-1402-9Haller H. Misinterpretations of significance: a problem students share with their teachers? Methods of psychological research. 2002;7(1):1–20.Haller H., Krauss S. Misinterpretations of significance: a problem students share with their teachers? // Methods of psychological research. 2002. Vol. 7, N 1. P. 1–20.Palesch YY. Some common misperceptions about p-values. Stroke. 2014;45(12):e244–e246. doi: 10.1161/STROKEAHA.114.006138Palesch Y.Y. Some common misperceptions about p-values // Stroke. 2014. Vol. 45, N 12. P. e244–e246. doi: 10.1161/STROKEAHA.114.006138Zorin NA. «Validity» or «significance» — 12 years later. Pediatric Pharmacology. 2011;8(5):13–19. (In Russ).Зорин Н.А. «Достоверность» или «статистическая значимость» — 12 лет спустя // Педиатрическая фармакология. 2011. Т. 8, № 5. С. 13–19.Kmetz JL. Correcting corrupt research: recommendations for the profession to stop misuse of p-values. The American statistician. 2019;73(supl. 1):36–45. doi: 10.1080/00031305.2018.1518271Kmetz J.L. Correcting corrupt research: recommendations for the profession to stop misuse of p-values // The American statistician. 2019. Vol. 73. Suppl. 1. P. 36–45. doi: 10.1080/00031305.2018.1518271McShane BB. Abandon statistical significance. The American statistician. 2019;73(supl 1):235–245. doi: 10.1080/00031305.2018.1527253McShane B.B., Gal D., Gelman A., Robert C., Tackett J.L. Abandon statistical significance //The American statistician. 2019. Vol. 73. Suppl. 1. P. 235–245. doi: 10.1080/00031305.2018.1527253Perezgonzalez JD. Fisher, Neyman–Pearson or NHST? A tutorial for teaching data testing. Front Psychol. 2015;6:223. doi: 10.3389/fpsyg.2015.00223Perezgonzalez J.D. Fisher, Neyman–Pearson or NHST? A tutorial for teaching data testing // Front Psychol. 2015. Vol. 6. P. 223. doi: 10.3389/fpsyg.2015.00223Lew MJ. Bad statistical practice in pharmacology (and other basic biomedical disciplines): you probably don’t know p: statistical inference using p-values. Br J Pharmacol. 2012;166(5):1559–1567. doi: 10.1111/j.1476-5381.2012.01931.xLew M.J. Bad statistical practice in pharmacology (and other basic biomedical disciplines): you probably don’t know p: statistical inference using p-values // Br J Pharmacol. 2012. Vol. 166, N 5. P. 1559–1567. doi: 10.1111/j.1476-5381.2012.01931.xPernet C. Null hypothesis significance testing: a guide to commonly misunderstood concepts and recommendations for good practice. F1000Research. 2017;4:621. doi: 10.12688/f1000research.6963.5Pernet C. Null hypothesis significance testing: a guide to commonly misunderstood concepts and recommendations for good practice // F1000Research. 2017. Vol. 4. P. 621. doi: 10.12688/f1000research.6963.5Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021;31(1)010502. doi: 10.11613/BM.2021.010502Serdar C.C., Cihan M., Yücel D., Serdar M.A. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies // Biochem Med (Zagreb). 2021. Vol. 31. N 1. P. 010502. doi: 10.11613/BM.2021.010502Lee DK. Alternatives to p value: confidence interval and effect size. Korean J Anesthesiol. 2016;69(6):555–562. doi: 10.4097/kjae.2016.69.6.555Lee D.K. Alternatives to p value: confidence interval and effect size // Korean J Anesthesiol. 2016. Vol. 69, N 6. P. 555–562. doi: 10.4097/kjae.2016.69.6.555Grissom RJ, Kim JJ. Effect sizes for research. 2nd ed. New York: Routledge; 2012. doi: 10.4324/9780203803233Grissom R.J., Kim J.J. Effect sizes for research. 2nd ed. New York : Routledge; 2012. doi: 10.4324/9780203803233Sullivan GM, Feinn R. using effect size — or why the p value is not enough. J Grad Med Educ. 2012;4(3):279–282. doi: 10.4300/JGME-D-12-00156.1Sullivan G.M., Feinn R. Using effect size — or why the p value is not enough // J Grad Med Educ. 2012. Vol. 4, N 3. P. 279–282. doi: 10.4300/JGME-D-12-00156.1Colquhoun D. An investigation of the false discovery rate and the misinterpretation of p-values. R Soc Open Sci. 2014;1(3):140216. doi: 10.1098/rsos.140216Colquhoun D. An investigation of the false discovery rate and the misinterpretation of p-values // R Soc Open Sci. 2014. Vol. 1, N 3. P. 140216. doi: 10.1098/rsos.140216Stahel WA. New relevance and significance measures to replace p-values. PLoS One. 2021;16(6):e0252991. doi: 10.1371/journal.pone.0252991Stahel W.A. New relevance and significance measures to replace p-values // PLoS One. 2021. Vol. 16, N 6. P. e0252991. doi: 10.1371/journal.pone.0252991Anderson N.D. Teaching signal detection theory with pseudoscience. Front Psychol. 2015;6:762. doi: 10.3389/fpsyg.2015.00762Anderson N.D. Teaching signal detection theory with pseudoscience // Front Psychol. 2015. Vol. 6. P. 762. doi: 10.3389/fpsyg.2015.00762Benjamin DJ, Berger JO, Johannesson M, et al. Redefine statistical significance. Nat Hum Behav. 2018;2(1):6–10. doi: 10.1038/s41562-017-0189-zBenjamin D.J., Berger J.O., Johannesson M., et al. Redefine statistical significance // Nat Hum Behav. 2018. Vol. 2, N 1. P. 6–10. doi: 10.1038/s41562-017-0189-zRubanovich AV. Redefining the critical value of significance level (0.005 instead of 0.05): the bayes trace. Radiation biology. Radioecology. 2018;58(5):453–462. (In Russ). doi: 10.1134/S0869803118050156Рубанович А.В. Пересмотр критического уровня значимости (0.005 вместо 0.05): байесовский след // Радиационная биология. Радиоэкология. 2018. Т. 58, № 5. С. 453–462. doi: 10.1134/S0869803118050156Betensky RA. The p-value requires context, not a threshold. The American statistician. 2019;73(supl. 1):115–117. doi: 10.1080/00031305.2018.1529624Betensky R.A. The p-value requires context, not a threshold // The American statistician. 2019. Vol. 73. Supl. 1. P. 115–117. doi: 10.1080/00031305.2018.1529624Lakens D, Adolfi, FG, Albers CJ, et al. Justify your alpha. Nature human behaviour. 2018;2(3):168–171. doi: 10.1038/s41562-018-0311-xLakens D., Adolfi F.G., Albers C.J., et al. Justify your alpha // Nature human behaviour. 2018. Vol. 2, N 3. P. 168–171. doi: 10.1038/s41562-018-0311-xDi Leo G, Sardanelli F. Statistical significance: p value, 0.05 threshold, and applications to radiomics — reasons for a conservative approach. Eur Radiol Exp. 2020;4(1):1–8. doi: 10.1186/s41747-020-0145-yDi Leo G., Sardanelli F. Statistical significance: p value, 0.05 threshold, and applications to radiomics — reasons for a conservative approach // Eur Radiol Exp. 2020. Vol. 4, N 1. P. 1–8. doi: 10.1186/s41747-020-0145-yVexler A. Valid p-values and expectations of p-values revisited // Ann Inst Stat Math. 2021;73:227–248. doi: 10.1007/s10463-021-00800-8Vexler A. Valid p-values and expectations of p-values revisited // Ann Inst Stat Math. 2021. Vol. 73. P. 227–248. doi: 10.1007/s10463-021-00800-8