How Machine Learning is Transforming the Investment Process

Attempts to incorporate artificial intelligence (AI) in the investment industry have come in fits and spurts since the late 1980s, when fuzzy logic and early versions of neural nets were making headlines. The enthusiasm around those technologies, however, gradually faded and the industry reverted to conventional statistical models. This time, however, with more advanced technology now affordable and accessible, there is good reason to believe that AI will have lasting implications for the investment process.

For one thing, the use of AI tools is already going mainstream. Most fundamental investors have now incorporated quantitative screens and models into their research. And according to the latest Invesco Global Systematic Investing Study, nearly half of quantitative investors have integrated AI into their investment processes in some form, with 10% using it extensively. Moreover, in a separate study by Deloitte, an overwhelming majority of investment managers using AI-based solutions in the pre-investment phase claimed it helps them generate alpha.

Although only 29% of systematic investors currently use AI to develop and test investment strategies, more than three-quarters anticipate doing so in the future. AI’s more prevalent current and expected investment-focused use cases include identifying patterns and trends in market behavior, and optimizing portfolio allocation and risk management (see Figure 1).

In line with expectations of greater AI adoption across the industry, CFA Institute’s research on Future of Work found that 64% of investment professionals are currently pursuing, or plan to pursue, skills development in AI and machine learning (ML), rising to 71% among the younger cohort.

It is no wonder, then, that AI topped the list of potential disruptors in a 2022 survey of CFA charterholders (See Figure 2).

Clients, too, are taking notice of AI’s potential. The 2022 CFA Institute Investor Trust Study revealed that 81% of institutional investors are more interested in investing in a fund that relies primarily on AI and big data tools than a fund that relies primarily on human judgment to make investment decisions. The same survey found 87% of respondents said they trust their asset manager more because of the increased use of technology.

The advent of ML-based return prediction algorithms has kickstarted a paradigm shift in the construction of quantitative investment models across the globe, in developed and emerging markets, for large-cap and small-cap investment universes, and with single-country or multi-country strategies. In general, practitioners have found that ML-derived alpha models outperform those generated from more traditional linear models in predicting cross-sectional equity returns.

Generating alpha has always involved rigorous analysis of fundamental and technical indicators to identify patterns which can be used to predict opportunities and risks. The difference with ML algorithms is that investment managers can now reach much deeper into granular data to identify complex patterns that traditional methods might overlook.

As OpenAI Chief Scientist Ilya Sutskever quipped on social media, “Machine learning is just statistics. On steroids. Lots and lots of steroids.”

But there could be more to it. Traditional quantitative analysis essentially involves simplifying the world through such methods as targeting specific factors that can drive investment returns. ML, on the other hand, allows the development of models based on a less simplified, more realistic world.

AI pioneer and Nobel laureate Herbert Simon theorized that decision makers can either find optimum solutions for a simplified world, or satisfactory solutions for a more realistic world

“A well-specified artificial intelligence approach will allow us to deal with this by identifying very different companies using the same system” said Dan Philps, an AI researcher and Head of Rothko Investment Strategies. “This is a step change from common approaches to quantitative investing today, which risk introducing overfit by using many ‘contextual models’ for different company types, while introducing biases by oversimplifying the investment environment into factors.”

“Say you have a changing earnings number and increasing debt at the same time that cash flows are flatlining and earnings quality has fallen and you should be trying to understand what this pattern actually means from a fundamental perspective. That’s what fundamental analysts essentially try to do, and that’s what AI now allows us to automate, avoiding behavioral biases and getting a more objective stock selection outcome,”said Philps.

Compared to investment management, ML has made much bigger strides in several other domains, ranging from streaming video and online shopping recommendations to autonomous driving and drug discovery.

There are three main factors that potentially dilute ML’s effectiveness in the investment industry, holding back its wider adoption.

First, financial data have a low signal-to-noise ratio, meaning any given metric generally does not have a huge impact on a security’s performance. For instance, a company’s stock price could fall even after it announced strong earnings if those earnings were below expectations or there was an interest rate hike. Meanwhile, the signal-to-noise ratio is much higher for streaming video recommendation systems –if a person enjoys a movie in a given genre, there is a good chance they will like other movies in that genre.

Second, applying ML in financial markets is hindered by a lack of available financial data. The number of monthly financial data points for a given security is at most 1,200, compared to billions and trillions in other domains such as social media. Considering that in real-time testing, a simple ML algorithm trained on a large data set outperforms a sophisticated ML algorithm trained on a relatively smaller data set, this could be a significant impediment.

And third, financial markets pose a challenge for ML because they are non-stationary and evolve over time, while domains which have been successfully modelled by ML– such as biological systems – are generally constant. As a consequence, not only do financial markets not have a long enough history to train ML models, but the rules and dynamics that govern the outcomes the models try to predict are not constant. This is further complicated by the irrationality of human behavior, which collectively determine the results of financial markets.

Despite these caveats, practitioners believe that AI and big data can deliver benefits, including more effective risk management and better insights (see Figure 3). To some extent, the increasing dominance of data and technology is reflected in the growing share of assets that are passively managed, with passive fund ownership of US stocks overtaking active for the first time last year. And the industry continues laying the groundwork for AI’s wider adoption.

AI’s use cases in the investment industry can be categorized under three main pillars: operations, business analytics and investment analytics, noted Richard Fernand, Head, Certificate Management at CFA Institute.

“Automation can take away some of the boring grunt work. And you can use machine learning—as in other businesses—to predict whether a client is likely to make a purchase or which product is suitable for them,” said Fernand.

And while most are not yet ready to use AI to pick stocks, applications like ChatGPT can serve as a powerful assistant for investment managers, according to Isaac Wong, an Assistant Fund Manager at eFusion Capital. It can, for example, streamline idea generation and pre-screening tasks.

“Perhaps the easiest implementation of ChatGPT is to produce summaries from text-based documents, images and slides, from which you can extract important insights,” said Wong.

“You can also use it to do some simple data analysis,” he added. After using AI tools to identify a selection of securities that meet certain criteria, investment professionals can conduct further manual review, validation and analysis before making recommendations, melding traditional fundamental and quantitative approaches.

AI is also making the execution of trades more efficient. And crucially, it is paving the way for investment professionals to tap rich sources of unstructured data, deriving much more nuanced perspectives than those provided solely by traditional data sources.

Given that an ML algorithm’s power is determined largely by how much data is available to it, this, in turn, can potentially enhance ML algorithms’ capacity to create robust and profitable alpha strategies. It could thereby bring about a tipping point for accelerated AI adoption in the industry.

But despite the expected rise of machines in the investment industry, human intelligence is expected to continue playing a key role for the foreseeable future. Because even though AI now more reliably delivers alpha, a combination of the two — “AI + HI” – still offers the most compelling way to augment the investment process.

Note: With thanks to Ingrid Tierens, Head of Data Strategy for Global Investment Research at Goldman Sachs, for sharing insights conveyed in this article.