Modernizing and future-proofing investment management software

All investment strategies generally follow a similar lifecycle. Strategies are thought up (generation), they are tested against historical data (validation), if testing proves well they are put into practice (implementation), and once implemented they are subject to continual monitoring and analysis (confirmation). Last, the cycle repeats itself as strategy improvements follow the generation, validation, implementation, and confirmation loop.

In 2015 we were contacted by a client that had already made a full trip around the strategy lifecycle with an investment process that was originally conceived of in the early 1980s. At a time when quantitative and computer-driven strategies were exceedingly rare, this firm leaped ahead of their competition by implementing a strategy in the Fortran programming language that scanned thousands of individual data points to generate lists of portfolio candidates. In the more than three decades since then, the strategy had proven itself a winner and the firm continued to utilize the program’s outputs as a part of their larger investment process.

With a winning strategy and a nearly unprecedented amount of historical performance in hand, this firm had little to complain about, except for the fact that the program was still written in Fortran.

The program was written well and had operated nearly flawlessly since it was created more than thirty years ago, but it was becoming increasingly difficult to adapt to newer operating systems, required a significant amount of training to operate, and could only ingest data that was hand-input. The system also represented a single point of failure since its database was hosted on whatever machine it ran on. Attempts to clone the database and share it amongst multiple instances of the program had failed, and even if they had worked would represent only a temporary fix to the larger problem.

It was clear that the program needed to be brought into the 21st century. The new system needed to recreate the original program’s analysis with a new user interface in a cloud-based environment that supported multiple users, automatically ingested market data, and that was redundant in case of system failure or corruption. Additionally, the client expressed an interest in adding backtesting capabilities that would allow flexible testing of a variety of the existing strategy’s criteria.

To meet this demand we decided to use a combination of the Laravel PHP framework and Python, using MySQL as our database and Redis as a cache. Laravel enabled us to efficiently create the user-facing portion of the web application as well as handle server-side tasks such as scheduling ETL processes, managing worker pools, and efficient object relation. For some portions of the backtesting engine, we employed Python as it allowed us to perform some calculations and data handling processes much faster than PHP was able to. Specifically, matrix operations in PHP can be a pain.

The entire infrastructure is hosted on Amazon Web Services. The application resides on a rather large EC2 instance, and we host the MySQL database on a separate RDS instance.

While we can’t discuss the specifics of the application infrastructure, our intentionally flexible design, AWS hosting, and unique combination of Laravel and Python have enabled us to continue to work with the client to optimize the strategy via machine learning.