In order to succeed in creating a data driven enterprise it is clear that choosing the right data architecture is now critical. This session explores the evolution of data and analytics architecture and looks at what is needed to shorten time to value and create a data driven enterprise. It looks at the pros and cons of data lake, lakehouse and data mesh architectures and asks: Is there a best approach? Is a lot more than this needed to succeed?

Target Audience: Data architects, CDOs, CAOs, enterprise architects, data scientists, business analysts
Prerequisites: Basic understanding of data architectures used in supporting analytical workloads
Level: Advanced

Extended Abstract:
In many companies today the desire to become data driven goes all the way to the boardroom. The expectation is that as more and more data enters the enterprise, it should be possible to understand and use it to quickly and easily drive business value. In order to succeed in creating a data driven enterprise it is clear that choosing the right data architecture is now critical. However, data and analytics architecture has been evolving over recent years to a point where now there are multiple options. Is it a data lake that is needed? Is it a lakehouse? Or is it a data mesh? Should this be the focus or is it just vendor hype to fuel their own interests?  What are the pros and cons of these options? Is there a best approach? Is a lot more than this needed to succeed? This session explores the evolution of data and analytics architecture and looks at what is needed to shorten time to value and create a data driven enterprise.

• Data and analytics - where are we?
• Data and analytics architecture evolution
• Architecture options and their pros and cons - data lake Vs lakehouse Vs data mesh
• The shift to data fabric, DataOps, and MLOps to industrialise pipeline development and model deployment
• Using a data and analytics marketplace to putting analytics to work across the enterprise

The data lakehouse is the new popular data architecture. In a nutshell, the data lakehouse is a combination of a data warehouse and a data lake. It makes a lot of sense to combine them, because they are sharing the same data and similar logic. This session discusses all aspects of data warehouses and data lakes, including data quality, data governance, auditability, performance, historic data, and data integration, to determine if the data lakehouse is a marketing hype or whether this is really a valuable and realistic new data architecture.

Target Audience: Data architects, enterprise architects, solutions architects, IT architects, data warehouse designers, analysts, chief data officers, technology planners, IT consultants, IT strategists
Prerequisites: General knowledge of databases, data warehousing and BI
Level: Basic

Extended Abstract:
The data lakehouse is the new kid on the block in the world of data architectures. In a nutshell, the data lakehouse is a combination of a data warehouse and a data lake. In other words, this architecture is developed to support a typical data warehouse workload plus a data lake workload. It holds structured, semi-structured and unstructured data. Technically, in a data lake house the data is stored in files that can be accessed by any type of tool and database server. The data is not kept hostage by a specific database server. SQL engines are also able to access that data efficiently for more traditional business intelligence workloads. And data scientists can create their descriptive and prescriptive models directly on the data.  

It makes a lot of sense to combine these two worlds, because they are sharing the same data and they are sharing logic. But is this really possible? Or is this all too good to be true? This session discusses all aspects of data warehouses and data lakes, including data quality, data governance, auditability, performance, immutability, historic data, and data integration, to determine if the data lakehouse is a marketing hype or whether this is really a valuable and realistic new data architecture.

A popular new architecture for offering frictionless access to data is the data fabric. With a data fabric, existing transactional and data delivery systems are wrapped (encapsulated) to make all of them look like one integrated system. A data fabric enables all data consumers to access and manipulate data. Technically, data is accessed and used through services. But data fabrics cannot be bought, they need to be designed and developed. This session discusses key guidelines for designing data fabrics.

Target Audience: Data architects, enterprise architects, solutions architects, IT architects, data warehouse designers, analysts, chief data officers, technology planners, IT consultants, IT strategists
Prerequisites: General knowledge of databases, data warehousing and BI
Level: Advanced

Extended Abstract:
Companies are becoming increasingly dependent on data. Having access to the right data at the right time is essential. This implies that users need frictionless access to all the data, wherever it is stored, in a transactional database, a data warehouse, or a data lake. It does not matter to users where data comes from as long as it meets all their requirements. Users do not want to be hindered by all the data delivery silos. They want one system that gives them access to all the data they need.

The solution to provide frictionless access cannot be data warehouse-like, wherein all the data is copied (again) to one big central database. In this second era of data integration, integration must be achieved without copying. A new solution must be based on a single universal entry point to access all data. Where and how the data is stored, whether it is stored in various databases, must be completely hidden from data users.

A popular new architecture that supports this approach is data fabric. With a data fabric, existing transactional and data delivery systems are wrapped (encapsulated) to make all the independent systems look like one integrated system.  

A data fabric is formed by a software layer that resides on top of all the existing transactional silos and data delivery silos, enabling all data consumers to access and manipulate data. Technically, data is accessed and used through services.  

A real data fabric supports any type of service, whether this is a more transactional or analytical service. And especially the second group of services is complex to develop. Maybe analytical services based on predefined queries are not that complex to develop, but how are such services developed that need to deal with ad-hoc queries?

This session explains the need for data fabrics that support all types of services and discusses key guidelines for designing data fabrics. Technologies are discussed that help with developing such services.

•  What a data fabric is, and why you need one
• How you can architect a service-centric fabric to gain flexibility and agility
• The data management and integration capabilities that are most relevant
•  Where to start your journey to data fabric success
•  What is logical data fabric?

SWICA historically runs a data warehouse built by a centralized team and in parallel, multiple isolated solutions for domain specific analyses, which afford high maintenance and an extensive effort to stay compliant.

Modernizing our analytical environment, we are building a collaborative platform on MS Azure, utilizing the Data Mesh paradigms of data domain and data product.

We aim to deliver a managed data marketplace for all data domains to provide their data products on a modern platform with low maintenance and built-in security & compliance.

Target Audience: Data Analysts, Data Engineers, Project Leaders, Decision Makers
Prerequisites: Basic understanding of the data mesh concept, data warehouse architectures and the challenges of diverse analytical use cases from multiple lines of business
Level: Advanced