Data Governance and Scalability With Hybrid Cloud

The use of more than one cloud for data science, machine learning, and AI is inevitable. Data, analytics, and IT leaders must prepare for a multi-cloud and hybrid cloud world, where data governance, security, compliance, and integration become more complex than ever before.

A recent Gartner research survey on cloud adoption revealed that more than 80% of respondents using the public cloud were using more than one cloud service provider (CSP).

Gartner, December 2019

What Is Hybrid Cloud?

Hybrid cloud is a solution that combines on-premise data centers and private cloud with one or more public cloud services, with proprietary software enabling communication between each distinct service. A hybrid cloud strategy provides businesses with greater flexibility by moving workloads between cloud solutions as needs and costs fluctuate. 

It’s also a good solution if an organization needs to offer services both in private data centers and via an on-cloud subscription. They would then be able to build web applications and services or machine learning models and use them both on-premise and on-cloud, as well as take advantage of their hybrid architecture to maintain communication between applications or data flow between the cloud and the on-premise infrastructures. 

Watch Video

Challenges to Hybrid Cloud Adoption

According to Gartner, the main challenges of hybrid cloud adoption are the following:

  • Hybrid data architectures that span cloud and on-premise environments are becoming more and more common and practically inevitable for AI and machine learning at scale. However, the architectural considerations for dealing with a hybrid database management system (DBMS) cloud environment are neither inherently obvious nor consistent. This has important financial and performance implications for data and IT teams, as well as lines of business.
  • It is critical yet increasingly complex, especially for non-technical or non-IT stakeholders, to understand how data flows, both in volume and direction, and be able to measure how data location impacts performance, application latency, high availability and disaster recovery strategies, and financial models in hybrid DBMS cloud scenarios.

After observing a few projects, you can confirm these two problems but also realize that they are two faces of the same coin. The ability to create a sustainable Hybrid Cloud data platform lies in the location of the data but also the circuit through which the data circulates during processing.

If this is obvious on the macro scale (that is to say on the inter-application scale), this is not always obvious on a more reduced scale of data processing. It’s mainly due to the fact that in data science projects,  you traditionally add more and more data sources over time. Thus for a given use case it’s rare to have a preconception of what the final data flow will be. 

It is therefore a question of choosing the right location but also of the technologies and storage formats compatible with the operation carried out in order to maximize productivity during the design period and optimize and control costs during the production phase.

Go Further: The IT Architect's Guide

Get the Guide

Dataiku DSS for Seamless and Secure Hybrid Cloud Deployment

Dataiku DSS is now considered a cloud-native platform because we have integrated compute execution into native hosted services from all major public cloud providers. All Dataiku DSS processing, notebooks and webapps can be integrated with a scalable hosted service in the public cloud. Artificial intelligence and any data driven project rely on a resilient data ingestion at some point. Bringing elasticity to every atomic process of your workflow will drastically reduce if not erase refactoring:

  • Between the design or lab environment and production
  • Between execution on-premise and execution on-cloud
Watch Video

Governance, Sicherheit und Monitoring

Dataiku macht es Administratoren leicht, Datensätze zu durchsuchen und zu organisieren sowie den Zugriff auf und Nutzeraktivitäten in diesen Datensätzen zu verfolgen.

Learn More

Skalierbare Kollaborations- und Governance-Funktionen

Die Kollaborations-Funktionen von Dataiku erleichtern den Wissensaustausch und die Einarbeitung neuer Mitarbeiter.

Learn More

Effectively Managing Enterprise-Wide Risk

The age of AI presents additional risks across the enterprise that require a tighter - yet more flexible - governance structure.

Learn More

Managing Data Regulation and Privacy Compliance

How can IT organizations scale to meet the demands of the modern AI-powered company?

Learn More