统计 BH 假发现率控制

Stats False Discovery Rate Bh

专题: Statistics / 统计
难度: L3
来源: MyntBit

个人笔记

题目详情

回测 100 个算法交易策略，在显著性水平 α=0.05 下检验每个策略。假设实际上 100 个策略中无一真正盈利。预期多少策略会通过检验？如何使用 Benjamini-Hochberg (BH) 方法控制错误发现率？

任务：预期通过检验数 = 100 × 0.05 = 5 个假阳性。BH 方法：将 p 值从小到大排序，对第 k 个 p 值，若 p_k ≤ k × α/N 则拒绝。此法控制 FDR ≤ α 而非 FWER。

英文原题

You are backtesting 100 different algorithmic trading strategies on historical data. You decide to test each strategy for statistical significance at a significance level of $ $\alpha$ = 0.05$. Assume that, in reality, none of the 100 strategies are actually profitable; their apparent profitability is purely due to random chance.
On average, how many strategies would you expect to incorrectly show up as statistically significant? Which multiple hypothesis testing correction procedure directly contr

解析

问题分析

解法

根据题目要求实现相应功能。核心逻辑需要：

// 核心数据结构和方法——根据题目 API 约定实现
// 1. 确定状态表示——选择支持所需操作的数据结构
// 2. 实现核心算法——确保 O(·) 时间复杂度和正确性
// 3. 处理边界条件——空输入、极值参数、并发访问

验证

用具体输入验证：构造已知输入的测试用例，确认输出匹配预期结果。

复杂度与边界

时间复杂度：取决于选用的算法
空间复杂度：取决于数据规模
关键边界条件：空输入、极值参数、并发场景下的正确性保证

英文解析

Analysis

You are backtesting 100 different algorithmic trading strategies on historical data. You decide to test each strategy for statistical significance at a significance level of alpha = 0.05. Assume that, in reality, none of the 100 strategies are actually profitable; their apparent profitability is entirely due to random variation. With 100 tests at alpha=0.05, you expect approximately 5 false discoveries (100 * 0.05). The Benjamini-Hochberg procedure controls the False Discovery Rate (FDR) - the expected proportion of false discoveries among all discoveries - rather than the Family-Wise Error Rate (FWER).

Solution

std::vector<int> bhProcedure(const std::vector<double>& p_values, double alpha = 0.05) {
    // Sort p-values ascending with indices
    std::vector<std::pair<double,int>> sorted;
    for (int i = 0; i < (int)p_values.size(); ++i)
        sorted.push_back({p_values[i], i});
    std::sort(sorted.begin(), sorted.end());
    int m = sorted.size();
    int k = 0;  // Last p-value below threshold
    for (int i = 0; i < m; ++i) {
        double threshold = alpha * (i + 1) / m;  // BH threshold: i*alpha/m
        if (sorted[i].first <= threshold) k = i + 1;
    }
    std::vector<int> discoveries;
    for (int i = 0; i < k; ++i) discoveries.push_back(sorted[i].second);
    return discoveries;  // Indices of significant tests
}

Complexity & Edge Cases

Time complexity: O(m * log m) for sorting, O(m) for scan
Space complexity: O(m)
Edge cases: (1) No discoveries when all p-values exceed threshold (2) With 100 null tests, expect ~5 false discoveries at alpha=0.05 (3) BH is less conservative than Bonferroni (using alpha/m threshold)

Verification

Generate 100 random p-values (all from null hypothesis). Apply BH procedure - expect ~5 discoveries on average. Benchmark against Bonferroni finding ~0. Assess FDR control rate via simulation.

Key Considerations

BH-FDR control is essential in quantitative research where hundreds of strategies are tested simultaneously. Unlike Bonferroni (controlling probability of ANY false discovery), BH controls the proportion of false discoveries. This is more practical: with 100 tests, finding 10 significant strategies where 2 are false (20% FDR) is more useful than finding 0 strategies because Bonferroni threshold is too strict.