Статистика в Python 6. Оценка мощности тестов// Александр Вихорев, НГУ

In this video, we'll explore one of the most important and often overlooked concepts in statistical analysis: test power. We'll explain what it is, why it's critical for study design, and how it can be estimated in practice using computer modeling. The goal of this video is to move from theory to simulation. Rather than relying on complex formulas, we'll use a clear Monte Carlo method to estimate the power of two key tests—the Student t-test and the Mann-Whitney U-test—under various conditions. We'll also solve the inverse problem: determining the minimum sample size required for a t-test to detect the difference of interest with acceptable reliability. In this video, we'll cover key concepts and approach in detail. The power of a statistical test is the probability that the test will correctly reject a false null hypothesis. In other words, it's the ability of the test to detect a true effect. Low power (e.g., less than 0.8) means a high risk of a false negative result, where you might not detect a real difference. We'll estimate power experimentally using Monte Carlo simulation. This involves repeatedly simulating a scenario in which we know in advance that a difference between groups exists. We'll repeatedly generate pairs of samples with a given effect, apply a statistical test to them, and record the proportion of simulations in which the test was able to detect this difference (i.e., produced a p-value less than 0.05). This proportion will be our empirical estimate of power. In the practical section, we'll create such an experiment in Python. First, we'll simulate a t-test. We'll set up various conditions: different effect sizes (e.g., difference in means), different data spreads (variances), and different sample sizes. You'll see how power consistently increases with increasing effect size and sample size and decreases with increasing data spread. We'll then conduct a similar evaluation for the Mann-Whitney U test under the same conditions. This will allow us to clearly compare which test is more powerful with a normally distributed data set, and which with outliers or skewness. We'll plot power versus sample size for both tests. Finally, we'll address a classic experimental design problem: how do we determine the minimum required sample size? We'll formulate the requirements: the desired effect size we want to detect, the expected variability in the data, and the required power level (usually 0.8 or 0.9). Using a loop or optimization, we'll select a sample size such that our simulation yields a power estimate no lower than the target. This will give us a practical figure that can be incorporated into the study design even before real data are collected. To complete this tutorial, you will need the following libraries: numpy for data simulation and loops, scipy.stats for performing t-tests and U-tests, matplotlib and seaborn for plotting visual graphs illustrating the dependence of power on various factors.