Novartis: Streamlining Analytics & AI Across the Organization

In the fast-paced world of pharmaceuticals, where every decision can impact real lives, Novartis faced critical business challenges that demanded a revolutionary solution. 

Enter the dynamic duo – Novartis’s data engineer and data science teams. Faced with these challenges, they embarked on a mission to streamline analytics, machine learning (ML), and AI (including GenAI) across the organization. 

Revolutionizing Healthcare Market Research With the Dataiku LLM Mesh

Novartis’s dedication to understanding the healthcare landscape and conducting market research led them to hold extensive interviews with healthcare professionals, generating vast amounts of valuable data. However, the process of analyzing hundreds of 20-page transcripts was tedious and time-consuming, hampering productivity and innovation. This inefficiency not only slowed down their ability to extract crucial insights but also limited their flexibility in responding to rapidly changing market conditions.

The solution to this challenge is a GenAI chatbot, whose development typically follows an extensive product development lifecycle, potentially extending project timelines by several months. Dataiku offered a complete solution, including the LLM Mesh and GenAI builder capabilities, that markedly compressed development time, thereby expediting the overall time to market:

• Provider-agnostic architecture via the LLM Mesh for easy deployment and management of LLMs: For Novartis, the LLM Mesh abstracted intricate model configurations — like model parameters and prompt formats — into holistic, user-friendly UI options, thereby streamlining the experimentation process with various models across multiple service providers.
• Prompt Studios gave Novartis a highly optimized workspace for iterative prompt refinement, allowing for the development of high-efficiency prompts. Dataiku simplified the evaluation of disparate model behaviors through an intuitive selection interface, with most model-specific parameters pre-configured and readily accessible via the UI.
• Dataiku Answers enabled Novartis to use a highly adaptive, enterprise-grade solution to deliver an interactive chatbot UI to end users. With Retrieval Augmented Generation (RAG), the chatbot uses embeddings produced by the Novartis AI pipeline, integrating them with an LLM to address user inquiries based on retrieved data and conversational context. Plus, given that the chatbot can be shared across teams via a public URL, leadership was able to easily interact with the final product.
• LLM-powered recipes provided Novartis with turnkey solutions for establishing robust AI pipelines crucial for classification and summarization tasks. These pipelines enable comprehensive metadata extraction, content extraction, and embedding creation, seamlessly integrating into the RAG framework to power the end application.
• LLM monitoring via Dataiku’s scenario and bundle capabilities streamlined the oversight of LLM usage across various applications for Novartis — all without incurring additional operational overhead.

The platform’s transparency improved risk management and governance, while its automation capabilities freed up developer resources for more strategic initiatives. By significantly reducing the complexity of AI application development, Dataiku empowered data practitioners and citizen data scientists alike. The result was a 90% reduction in time to insights on the GenAI use case (from 21 days to two), as well as improved insights extraction, democratization of data, and a newfound agility in responding to market dynamics.

The Dataiku LLM Mesh offers substantial strategic benefits, notably accelerating time to market for GenAI chatbot development. Deepthi Sanam Group Lead, Data Engineering, Novartis

So Long, Tedious Tasks

Tasked with weekly updates of data in Excel to generate crucial metrics, the team found themselves entangled in a web of manual calculations and decisions with far-reaching consequences.

To attain its goals, Novartis harnessed the power of Dataiku, a robust platform offering modular components for project design, batch process automation, real-time API scoring, and AI Governance. The team created a customized simulation dashboard, empowering decision-makers with automated analyses of budget allocation and field-force allotments, unlocking the potential for better decisions and seizing demand-growth opportunities.

Dataiku not only facilitates the development of tailored, ML-based forecast models, but also presents the results of these models in a user-friendly environment. The platform’s informative wikis for data sources and processes, coupled with support for global variables and parameterization, ensured not just efficiency but also an organized approach to problem-solving.

From Ideation to Impact

At Novartis, the day-to-day change was palpable. Adopting Dataiku was a natural evolution for the data teams, exposing them to new functionalities that quickly found their way into other teams’ workflows. The self-explanatory workflows and parallel execution of scenarios reduced implementation time, transforming the once endless rabbit hole of user queries into a streamlined, one-stop shop for all questions.

In the areas of marketing, sales, and customer relationship management, in particular, Novartis witnessed the tangible impact of Dataiku’s value generation. The platform’s pushdown design ensured optimum performance and efficiency, even when handling massive workloads. Metrics, checks, and testing capabilities provided by Dataiku brought a new level of quality assurance to the models, elevating the team’s confidence in decision making.

The true value of Dataiku became evident in the efficiency gains. The platform’s automation features, including real-time data refresh and analysis, automatic periodic reports, and pre-built visualizations, became the cornerstone of Novartis’s success. The ability to track actual versus forecast variance through intuitive dashboards and sliders brought clarity and precision to decision-making. With Dataiku, in the spreadsheet use case Novartis experienced a significant acceleration in data ingestion time by 600% from a week to a matter of six hours.

Novartis Pharmaceuticals found in Dataiku not just a solution implementation tool but a game-changer that aligned seamlessly with its vision to reimagine medicine. The platform’s features empowered the team to automate ML model solutions, build LLM-powered applications with speed and agility, create customized forecasts, and ultimately reach their business goals. In its quest to enter the AI- and data-driven future, Novartis Pharmaceuticals has not only reduced costs and saved time, but has moreover increased its teams’ trust in data-based decisions, fostering a culture of innovation and excellence.