TDWI Konferenz 2020

München, 26. – 28. Oktober 2020

Sessionsdetails

Vortrag: Di 5.3
Datum: Di, 23.06.2020
Uhrzeit: 14:45 - 16:15
cart

The Changing Face of ETL: Event-Driven Architectures for Data Engineers

Uhrzeit: 14:45 - 16:15
Vortrag: Di 5.3

 

In this talk, we'll discuss the concepts of events, their relevance to data engineers and their ability to unify architectures in a powerful way. We'll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world. The talk will conclude with a hands-on demonstration of these concepts in practice using Apache Kafka and commentary on the design choices made.

Target Audience: Data Engineers, Architects
Prerequisites: None
Level: Basic

Extended Abstract
Data integration in architectures built on static, update-in-place datastores inevitably end up with pathologically high degrees of coupling and poor scalability. This has been the standard practice for decades, as we attempt to build data pipelines on top of databases that do a poor job modelling the fundamental objects that drive our businesses and systems: events.

Events carry both notification and state, and form a powerful primitive on which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. Everyone gains from using the standards-based, scalable and resilient streaming platform.

In this talk, we'll discuss the concepts of events, their relevance to both software engineers and data engineers and their ability to unify architectures in a powerful way. We'll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world. The talk will conclude with a hands-on demonstration of these concepts in practice using Apache Kafka and commentary on the design choices made.

Join this talk to learn:

  • What is Apache Kafka
  • The power of events and unbounded data
  • Streaming is not just for real-time applications – it's for everyone
  • Where a streaming platform fits in an analytic architecture
  • How event-driven architectures can enable greater scalability and flexibility of systems both now and in the future

 

Sponsoren TDWI München 2020

  • Ab Initio Software Germany GmbH Platinsponsor
  • Adastra GmbH Platinsponsor
  • Data Virtuality GmbH Platinsponsor
  • Denodo Technologies Platinsponsor
  • Exasol Platinsponsor
  • INFOMOTION GmbH Platinsponsor
  • itelligence AG Platinsponsor
  • NTT DATA Deutschland Platinsponsor
  • pmOne Group Platinsponsor
  • Qlik Platinsponsor
  • SAP Deutschland SE & Co. KG Platinsponsor
  • Sopra Steria Platinsponsor
  • Talend Germany GmbH Platinsponsor
  • Teradata GmbH Platinsponsor
  • Trivadis GmbH Platinsponsor
  • 2150 GmbH | Distilling business insight. Goldsponsor
  • adesso AG Goldsponsor
  • Alation Goldsponsor
  • BE-terna GmbH Deutschland Goldsponsor
  • blueforte GmbH Goldsponsor
  • CIMACON GmbH Goldsponsor
  • CP Corporate Planning AG Goldsponsor
  • Dataiku Goldsponsor
  • erwin, Inc. Goldsponsor
  • EVACO GmbH Goldsponsor
  • heureka e-Business GmbH Goldsponsor
  • infologistix GmbH Goldsponsor
  • INFORM DataLab Goldsponsor
  • INFORMATION WORKS GmbH Goldsponsor
  • integration-factory GmbH & Co. KG Goldsponsor
  • Marmeladenbaum GmbH Goldsponsor
  • Neo4j Goldsponsor
  • OPITZ CONSULTING Goldsponsor
  • PROCON IT AG Goldsponsor
  • SYCOR GmbH Goldsponsor
  • Syncwork AG Goldsponsor
  • TIBCO Goldsponsor
  • Vertica Goldsponsor
  • Alligator Company Silbersponsor
  • areto consulting GmbH Silbersponsor
  • Axians IT Solutions GmbH Silbersponsor
  • biGENiUS Silbersponsor
  • bluetelligence GmbH Silbersponsor
  • BSgroup Data Analytics AG Silbersponsor
  • cimt ag Silbersponsor
  • CINTELLIC Consulting Group Silbersponsor
  • Data Insights GmbH Silbersponsor
  • DataMerlin Silbersponsor
  • dataspot GmbH Silbersponsor
  • datasqill - The Post-ETL Silbersponsor
  • Disy Informationssysteme GmbH Silbersponsor
  • DWH42 Silbersponsor
  • Genesee Academy Silbersponsor
  • graphomate GmbH Silbersponsor
  • m2data Silbersponsor
  • PRODATO Integration Technology GmbH Silbersponsor
  • QuinScape GmbH Silbersponsor
  • saracus consulting GmbH Silbersponsor
  • Solecon Silbersponsor
  • sonra Silbersponsor