Hypothesis Testing
Refresher reading access
Introduction
Hypothesis testing is covered extensively in the pre-read. This learning module builds on that coverage and assumes a functional understanding of the topic gained there or elsewhere.
Lesson 1 summaries the hypothesis testing process by exemplifying its use in f inance and investment management. Lesson 2 brings forward the impact of errors in the hypothesis testing process. Lesson 3 introduces nonparametric tests and their applications in investment management.
- The steps in testing a hypothesis are as follows:
- State the hypotheses.
- Identify the appropriate test statistic and its probability distribution.
- Specify the significance level.
- State the decision rule.
- Collect the data and calculate the test statistic.
- Make a decision.
- A test statistic is a quantity, calculated using a sample, whose value is the basis for deciding whether to reject or not reject the null hypothesis. We compare the computed value of the test statistic to a critical value for the same test statistic to determine whether to reject or not reject the null hypothesis.
- In reaching a statistical decision, two possible errors can be made: reject a true null hypothesis (a Type I error, or false positive), or fail to reject a false null hypothesis (a Type II error, or false negative).
- The level of significance of a test is the probability of a Type I error when conducting a hypothesis test. The standard approach to hypothesis testing involves specifying a level of significance (i.e., the probability of a Type I error). The complement of the level of significance is the confidence level.
- For hypothesis tests concerning the population mean of a normally distributed population with an unknown variance, the theoretically correct test statistic is the t-statistic.
- To test whether the observed difference between two means is statistically significant, the analyst must first decide whether the samples are independent or dependent (related). If the samples are independent, a test concerning differences between means is employed. If the samples are dependent, a test of mean differences (paired comparisons test) is employed.
- To determine whether the difference between two population means from normally distributed populations with unknown but equal variances, the appropriate test is a t-test based on pooling the observations of the two samples to estimate the common but unknown variance. This test is based on an assumption of independent samples.
- In tests concerning two means based on two samples that are not independent, the data are often arranged in paired observations and a test of mean differences (a paired comparisons test) is conducted. When the samples are from normally distributed populations with unknown variances, the appropriate test statistic is t-distributed.
- In tests concerning the variance of a single normally distributed population, the test statistic is chi-square with n − 1 degrees of freedom, where n is sample size.
- For tests concerning differences between the variances of two normally distributed populations based on two random, independent samples, the appropriate test statistic is based on an F-test (the ratio of the sample variances). The degrees of freedom for this F-test are n1 − 1 and n2 − 1, where n1 corresponds to the number of observations in the calculation of the numerator, and n2 is the number of observations in the calculation of the denominator of the F-statistic.
- A parametric test is a hypothesis test concerning a population parameter, or a hypothesis test based on specific distributional assumptions. In contrast, a nonparametric test either is not concerned with a parameter or makes minimal assumptions about the population from which the sample was taken.
- A nonparametric test is primarily used when data do not meet distributional assumptions, when there are outliers, when data are given in ranks, or when the hypothesis we are addressing does not concern a parameter.
Learning Outcomes
The candidate should be able to:
- explain hypothesis testing and its components, including statistical significance, Type I and Type II errors, and the power of a test;
- construct hypothesis tests and determine their statistical significance, the associated Type I and Type II errors, and power of the test given a significance level;
- compare and contrast parametric and nonparametric tests, and describe situations where each is the more appropriate type of test.
1 PL Credit
If you are a CFA Institute member don’t forget to record Professional Learning (PL) credit from reading this article.