Data Governance and Scalability With Hybrid Cloud

A recent Gartner research survey on cloud adoption revealed that more than 80% of respondents using the public cloud were using more than one cloud service provider (CSP).

Gartner, December 2019

What Is Hybrid Cloud?

Hybrid cloud is a solution that combines on-premise data centers and private cloud with one or more public cloud services, with proprietary software enabling communication between each distinct service. A hybrid cloud strategy provides businesses with greater flexibility by moving workloads between cloud solutions as needs and costs fluctuate. 

It’s also a good solution if an organization needs to offer services both in private data centers and via an on-cloud subscription. They would then be able to build web applications and services or machine learning models and use them both on-premise and on-cloud, as well as take advantage of their hybrid architecture to maintain communication between applications or data flow between the cloud and the on-premise infrastructures. 

Watch Video

Challenges to Hybrid Cloud Adoption

According to Gartner, the main challenges of hybrid cloud adoption are the following:

• Hybrid data architectures that span cloud and on-premise environments are becoming more and more common and practically inevitable for AI and machine learning at scale. However, the architectural considerations for dealing with a hybrid database management system (DBMS) cloud environment are neither inherently obvious nor consistent. This has important financial and performance implications for data and IT teams, as well as lines of business.
• It is critical yet increasingly complex, especially for non-technical or non-IT stakeholders, to understand how data flows, both in volume and direction, and be able to measure how data location impacts performance, application latency, high availability and disaster recovery strategies, and financial models in hybrid DBMS cloud scenarios.

After observing a few projects, you can confirm these two problems but also realize that they are two faces of the same coin. The ability to create a sustainable Hybrid Cloud data platform lies in the location of the data but also the circuit through which the data circulates during processing.

If this is obvious on the macro scale (that is to say on the inter-application scale), this is not always obvious on a more reduced scale of data processing. It’s mainly due to the fact that in data science projects,  you traditionally add more and more data sources over time. Thus for a given use case it’s rare to have a preconception of what the final data flow will be. 

It is therefore a question of choosing the right location but also of the technologies and storage formats compatible with the operation carried out in order to maximize productivity during the design period and optimize and control costs during the production phase.

Dataiku DSS for Seamless and Secure Hybrid Cloud Deployment

Dataiku DSS is now considered a cloud-native platform because we have integrated compute execution into native hosted services from all major public cloud providers. All Dataiku DSS processing, notebooks and webapps can be integrated with a scalable hosted service in the public cloud. Artificial intelligence and any data driven project rely on a resilient data ingestion at some point. Bringing elasticity to every atomic process of your workflow will drastically reduce if not erase refactoring:

• Between the design or lab environment and production
• Between execution on-premise and execution on-cloud
  

Watch Video