HARKing (Hypothesizing After the Results are Known), reporting a hypothesis generated after seeing the results as if it had been formulated in advance, is widely counted among questionable research practices, yet its problems are sometimes held to be confined to quantitative, especially statistical, research. On the logical/objective Bayesian view of Fairfield and Charman (2019, 2022)—same information, same probability—readers need only check the priors and likelihood ratios, not trace how the i…
Read moreHARKing (Hypothesizing After the Results are Known), reporting a hypothesis generated after seeing the results as if it had been formulated in advance, is widely counted among questionable research practices, yet its problems are sometimes held to be confined to quantitative, especially statistical, research. On the logical/objective Bayesian view of Fairfield and Charman (2019, 2022)—same information, same probability—readers need only check the priors and likelihood ratios, not trace how the inquiry unfolded; building on this, Igashira (2024) urges caution about HARKing criticisms directed at qualitative research. Granting that the order in which evidence is incorporated does not change the result, I argue that how a hypothesis was constructed and selected must still be reported. First, when the best-fitting of many candidates is selected, its likelihood ratio against a specified rival can be large while its posterior odds and the Bayes factor for the candidate set as a whole remain small. Reporting that ratio instead can make an explanation look better supported than it is; concealing the selection deprives readers of what they need to check the candidates and priors. Second, being warranted by the evidence at hand differs from being likely to hold elsewhere. Dependence on conditions that stayed constant surfaces only when the explanation, with the hypothesis space assumed in building it, is exposed to settings where those conditions may vary—what I call open-world exposure. Presenting a condition added after a failure as foreseen turns failure and revision into an apparent hit: the revised explanation can appear to have passed a test it never faced, and confidence that the method anticipates unexamined conditions can be inflated. Independently of these evidential concerns, misrepresenting provenance is a failure of honesty. None of this is specific to frequentist testing or to quantitative research. Transparent reporting of the iterative process is a shared requirement, and those who teach research practice must explain its rationale and limits and answer objections.
====
HARKing(Hypothesizing After the Results are Known)—結果を見た後で生み出した仮説を、あたかも事前の仮説であったかのように報告すること—は問題のある研究実践の一つとされてきたが、その問題は量的、とりわけ統計的な研究に限られるとする見方もある。Fairfield and Charman(2019, 2022)の論理/客観ベイズ主義(同じ情報には同じ確率を)によれば、読者は事前確率と尤度比を吟味すればよく、研究の展開をたどる必要はない。井頭(2024)はこれを踏まえ、質的研究への HARKing 批判に慎重さを求める。本稿は、証拠を取り込む順序が結果を変えないことを認めたうえで、仮説をどう構成し選んだかはなお報告すべきだと論じる。第一に、多くの候補から最もよく合うものを選べば、指定した対立仮説との尤度比は大きくても、その事後オッズや候補群全体のベイズファクターは小さいままでありうる。その尤度比を代わりに示せば、説明は実際より強く支持されて見えうる。選択を隠せば、読者は候補と事前配分を点検する材料を失う。第二に、手元の証拠で説明に根拠があることと、別の状況でも当てはまりそうなことは別である。一定だった条件への依存は、説明を、それを作るときに想定した仮説空間ごと、条件が異なりうる場にさらして初めて露呈する。本稿はこれを開世界への曝露と呼ぶ。失敗の後に加えた条件を最初から見通していたと示せば、失敗と修正は見かけの的中に変わる。修正した説明は受けていないテストを通ったように見え、未検討の条件を見通せるという方法への信頼も過大になりうる。これらの認識論的な問題とは独立に、来歴を偽ることは正直性の問題である。以上は頻度主義の検定にも量的研究にも固有ではない。反復過程の透明な報告は共通の要請であり、研究の進め方を教える側には、その理由と限界を説明し、異論に答える責任がある。