A leading financial services institution (henceforth referred to as simply The Firm) places strong emphasis on securing and frequently monitoring the sensitive data, confidential records, and frequent trade transactions of its more than eight million clients. The Firm is also committed to strong AI Governance. In addition to adhering to data governance standards and promoting AI explainability, The Firm conducts daily compliance checks on all data assets to prevent market manipulation, assess risk, and prevent clients from potential financial exploitation.
With millions of daily financial transactions, The Firm’s compliance teams rely on the deployment of numerous machine learning (ML) models in production to identify and predict indicators of potential account-level risk. These models support more than 125 field supervision agents with their hourly, daily, or monthly compliance checks.
Roadblocks to Efficiently Deploying Compliance Models
Prior to implementing Dataiku in 2022, The Firm leveraged several desktop-based, open-source solutions. In addition to a general lack of governance plus difficulty with centralized deployment and performance monitoring, the team faced a slew of challenges that made the maintenance of these models cumbersome and inefficient:
- Exorbitant time-to-production (eight months, on average): The time to optimize the model code and deploy complex, open-source models leveraging data from various source systems required nearly 1,440 standard deployment hours from tech support engineers.
- Low return on investment (ROI): Given the amount of time it took to deploy ML models, generating continuous value from compliance ML models was an uphill battle.
- Lack of rigor: Compliance models residing within desktop open-source solutions meant MLOps teams could not fully embrace agile and DevOps methodologies for version control and for building end-to-end continuous integration and continuous deployment (CI/CD) automated ML pipelines.
- Lack of collaboration: Models built on desktop solutions made it difficult to encourage cross-functional project collaboration between data scientists and ML engineers.
In addition, there were certain types of models The Firm simply could not implement working only with models residing within desktop solutions. For example, conditional pipelines designed to perform specific actions based on the threshold levels and percent change results detected within the model output.
Agile Model Deployment & Elevated AI Governance With Dataiku
In the spring of 2022, The Firm developed an MLOps team to collaborate with data scientists and work with various cross-functional departments to support the end-to-end deployment processes of operationalizing ML models.