教育量化研究中的p值操弄現象初探：定義、影響與避免之道

A Preliminary Study on p-Hacking in Quantitative Educational Research: Definitions, Impacts, and Preventions

1920年代統計學家Fisher之統計理論體系為機率計算和評估的概念，意指在自變項不受任何影響或操弄的前提下，計算出觀察到的結果之機率，稱為p值，並且於該研究脈絡下評估該機率值的意涵；而後Neyman與Pearson則提出了虛無假設及對立假設的概念，認為假設檢定應包含此兩種假設。兩個派別對於檢定方法雖有所不同，但後世將其整合而形成虛無假設顯著性檢定，並以p值作為統計資料分析的一個標準設定。然而，近幾年來，學界開始意識到p-hacking現象，本文稱為「p值操弄」，意指研究者誤用或濫用資料分析方法，以便得到統計顯著結果，並據此宣稱得到成功的實驗，撰寫研究結果文章投稿至期刊發表。本文針對教育量化研究，首先說明p值的由來與學理，其次深入探討p值操弄方式、對教育學領域之學術研究的影響，以及期刊如何偵測與避免p值操弄。最後，本文提出研究者對p值應有的正確認知及避免操弄的實務作法，包括在提出統計顯著性的同時，也應提供實務顯著性的證據，確保研究結果的可重製性，詳細陳述分析架構與細節，學習並理性地選擇合適分析技術等。

In the 1920s, the statistician Ronald Fisher introduced a statistical theory system based on probability calculations and evaluations. Fisher coined the term“p-value”to denote the probability of observing the obtained results, assuming that the independent variable is not influenced or manipulated. This calculation helps evaluate the likelihood of such results within the research context. Later, Neyman and Pearson proposed the concepts of null hypothesis and alternative hypothesis, advocating for the inclusion of both in the hypothesis testing. While the two approaches to testing statistical hypotheses were different, these approaches were later integrated into what is now known as null hypothesis significance testing, with the p-value as a standardized measure for statistical analysis. In recent years, however, the academic community has identified a concerning trend known as“p-hacking,”where researchers misuse or abuse data analysis to achieve misleading statistical significance. Then, researchers claim to have speciously successful experiments and publish the implausible research results in journals. This paper focuses on p-hacking within quantitative educational research, exploring its origins, definition, technique, impact, and how journals prevent the manipulation of p-hacking. Finally, this paper reinstates the correct perception of p-value with recommendations for preventing p-hacking, including providing evidence of practical significance while presenting statistical significance, ensuring replicability of research results, stating the structure and details of statistical analysis, and judicious selection of appropriate analytical techniques.