Refresher Reading

Portfolio Performance Evaluation

2024 Curriculum CFA Program Level III Portfolio Management and Wealth Planning

Introduction

Performance evaluation is one of the most critical areas of investment analysis. Performance results can be used to assess the quality of the investment approach and suggest changes that might improve it. They are also used to communicate the results of the investment process to other stakeholders and may even be used to compensate the investment managers. Therefore, it is of vital importance that practitioners who use these analyses understand how the results are generated. By gaining an understanding of the details of how these analyses work, practitioners will develop a greater understanding of the insights that might be gathered from the analysis and will also be cognizant of the limitations of those approaches, careful not to infer more than what is explicit or logically implicit in the results.

We will first consider the broad categories of performance measurement, attribution, and appraisal, differentiating between the three and explaining their interrelationships. Next, we will provide practitioners with tools to evaluate the effectiveness of those analyses as we summarize various approaches to performance evaluation. We will cover returns-based, holdings-based, and transactions-based attribution, addressing the merits and shortcomings of each approach and providing guidance on how to properly interpret attribution results. Again, by reviewing how each approach generates its results, we reveal strengths and weaknesses of the individual attribution approaches.

Next, we will turn to the subject of benchmarks and performance appraisal ratios. We will review the long-standing tests of benchmark quality and differentiate market indexes from benchmarks. We will also review different ratios used in performance appraisal, considering the benefits and limitations of each approach.

Lastly, we will provide advice on using these tools to collectively evaluate the skill of investment managers. This advice relies heavily on understanding the analysis tools, the limitations of the approaches, the importance of data to the quality of the analysis, and the pitfalls to avoid when making recommendations.

Learning Outcomes

The member should be able to:

explain the following components of portfolio evaluation and their interrelationships: performance measurement, performance attribution, and performance appraisal;
describe attributes of an effective attribution process;
distinguish between return attribution and risk attribution and between macro and micro return attribution;
describe returns-based, holdings-based, and transactions-based performance attribution, including advantages and disadvantages of each;
interpret the sources of portfolio returns using a specified attribution approach;
interpret the output from fixed-income attribution analyses;
discuss considerations in selecting a risk attribution approach;
distinguish between investment results attributable to the asset owner versus those attributable to the investment manager;
discuss uses of liability-based benchmarks;
describe types of asset-based benchmarks;
discuss tests of benchmark quality;
describe problems that arise in benchmarking alternative investments;
describe the impact of benchmark misspecification on attribution and appraisal analysis;
calculate and interpret the Sortino ratio, the appraisal ratio, upside/downside capture ratios, maximum drawdown, and drawdown duration;
describe limitations of appraisal measures and related metrics;
evaluate the skill of an investment manager.

Summary

Performance evaluation is an essential tool for understanding the quality of the investment process. Practitioners must take care, however, to understand how performance results are generated. They need a good understanding of the performance methods used, the data inputs, and the limitations of those methods. They particularly need to be careful not to infer results beyond the capabilities of the methods or the accuracy of the data. In this reading, we have discussed the following:

Performance measurement provides an overall indication of the portfolio’s performance.
Performance attribution builds on performance measurement to explain how the performance was achieved.
Performance appraisal leverages both returns and attribution to infer the quality of the investment process.
An effective attribution process must reconcile to the total portfolio return/risk, reflect the investment decision-making process, quantify the active portfolio management decisions, and provide a complete understanding of the excess return/risk of the portfolio.
Return attribution analyzes the impact of investment decisions on the returns, whereas risk attribution analyzes the risk consequences of the investment decisions.
Macro attribution considers the decisions of the fund sponsor, whereas micro attribution considers the decisions of the individual portfolio manager.
Returns-based attribution uses returns to identify the factors that have generated those returns.
Holdings-based attribution uses the holdings over time to evaluate the decisions that contributed to the returns.
Transactions-based attribution uses both holdings and transactions to fully explain the performance over the evaluation period.
There are various techniques for interpreting the sources of portfolio returns using a specified attribution approach.
Fixed-income attribution considers the unique factors that drive bond returns, including interest rate risk and default risk.
When selecting a risk attribution approach, practitioners should consider the investment decision-making process and the type of attribution analysis.
Attribution is used to calculate and interpret the contribution to portfolio return and volatility from the asset allocation and within-asset-class active/passive decisions.
Liability-based benchmarks focus on the cash flows that the assets are required to generate.
Asset-based benchmarks contain a collection of assets to compare against the portfolio’s assets.
Valid benchmarks should be unambiguous, investable, measurable, appropriate, reflective of current investment opinions, specified in advance, and accountable.
Benchmark misspecification creates subsequent incorrect performance measurement and invalidates the attribution and appraisal analyses.
Alternative investments are difficult to benchmark because they are typically less liquid, have fewer available market benchmarks, and often lack transparency.
Investment performance appraisal ratios—including the Sortino ratio, upside/downside capture ratios, maximum drawdown, and drawdown duration—measure investment skill.
Appraisal ratios must be used with care, noting the assumptions of each ratio and affording the appropriateness to the measured investment process, risk tolerance, and investor time horizon.
Although appraisal ratios help identify manager skill (as opposed to luck), they often are based on investment return data, which are often limited and subject to error.
Evaluation of investment manager skill requires the use of a broad range of analysis tools, with fundamental understanding of how the tools work, how they complement each other, and their specific limitations.