TDWI Konferenz mit BARC@TDWI Track 2019

München, 24. - 26. Juni 2019

Partner

Sessionsdetails

Vortrag: SDdi4.4
Datum: Di, 25.06.2019
Uhrzeit: 11:25 - 12:00
cart

Adastra: Easy and Scalable Stream Processing with Spark Structured Streaming and Kafka

Uhrzeit: 11:25 - 12:00
Vortrag: SDdi4.4

 

The number of sources generating continuous, streaming data has exploded in recent years and Enterprises need to process and analyze ever-growing volumes of streaming data to deliver on business critical use cases. However, building high-volume streaming data pipelines can be a challenge given the complexities of handling streaming data frequency and its processing in scalable distributed fashion. Two of the most widely used technologies together for tackling this challenge today are "Kafka" and "Spark Streaming" for building high throughput, low-latency, and fault-tolerant stream processing pipelines. Since Spark 2.0, Spark Streaming has a new high level API, Structured Streaming, a new stream processing engine built on Spark SQL, which is revolutionizing how developers could write scalable and fault-tolerant stream processing applications. In this session, we will see how to build streaming data pipelines using Spark Structured Streaming to process data from Kafka and/or send it back to Kafka, along with advancedcapabilities offered in Structured streaming for streaming complexities to make streaming applications ready for Production. We will also cover a code based demo and talk about real challenges, lessons learned in building streaming data pipelines.

Jetzt registrieren / Register now

 

Sponsoren TDWI München 2019

  • Ab Initio Software Germany GmbH Platinsponsor
  • Adastra GmbH Platinsponsor
  • Databricks Platinsponsor
  • Denodo Technologies Platinsponsor
  • Exasol Platinsponsor
  • Fujitsu Platinsponsor
  • INFOMOTION GmbH Platinsponsor
  • itelligence Platinsponsor
  • ITGAIN Consulting Gesellschaft für IT-Beratung mbH Platinsponsor
  • Microsoft Platinsponsor
  • MicroStrategy Platinsponsor
  • NTT DATA Deutschland Platinsponsor
  • SAP Deutschland SE & Co. KG Platinsponsor
  • Sopra Steria Consulting Platinsponsor
  • Talend Germany GmbH Platinsponsor
  • Tech Data GmbH & Co. OHG Platinsponsor
  • Trivadis GmbH Platinsponsor
  • WhereScape Europe Platinsponsor
  • 2150 GmbH | Distilling business insight. Goldsponsor
  • Actian Goldsponsor
  • adesso AG Goldsponsor
  • Alation Goldsponsor
  • ASG Technologies Goldsponsor
  • b.telligent GmbH & Co. KG Goldsponsor
  • blueforte GmbH Goldsponsor
  • CIMACON GmbH Goldsponsor
  • Collibra UK Limited Goldsponsor
  • Cubeware GmbH Goldsponsor
  • Data Virtuality GmbH Goldsponsor
  • Dataiku Goldsponsor
  • Dell Boomi Goldsponsor
  • Deloitte Goldsponsor
  • erwin, Inc. Goldsponsor
  • EVACO GmbH Goldsponsor
  • Genesee Academy Goldsponsor
  • heureka e-Business GmbH Goldsponsor
  • IBM Goldsponsor
  • Informatica GmbH Goldsponsor
  • INFORMATION WORKS GmbH Goldsponsor
  • integration-factory GmbH & Co. KG Goldsponsor
  • Marmeladenbaum GmbH Goldsponsor
  • mayato GmbH Goldsponsor
  • MID GmbH Goldsponsor
  • Neo4j Goldsponsor
  • OPITZ CONSULTING Goldsponsor
  • pmOne Group Goldsponsor
  • Poslovna inteligencija d.o.o. Goldsponsor
  • PPI AG Goldsponsor
  • PROCON IT AG Goldsponsor
  • SEVEN PRINCIPLES AG Goldsponsor
  • SISENSE Goldsponsor
  • Snowflake Goldsponsor
  • Syncwork AG Goldsponsor
  • T-Systems Multimedia Solutions Goldsponsor
  • TIBCO Goldsponsor
  • TIMETOACT Software & Consulting GmbH Goldsponsor
  • Vertica Goldsponsor
  • accantec consulting AG Silbersponsor
  • AnalyticsCreator Solutions GmbH Silbersponsor
  • areto consulting GmbH Silbersponsor
  • Attunity Ltd. Silbersponsor
  • Axians IT Solutions GmbH Silbersponsor
  • BARC GmbH Silbersponsor
  • BBF GmbH Silbersponsor
  • biGenius Silbersponsor
  • BIVAL GmbH Silbersponsor
  • brainLight Silbersponsor
  • cimt ag Silbersponsor
  • CINTELLIC Consulting Group Silbersponsor
  • Cognizant Digital Business Silbersponsor
  • Confexx Consulting GmbH Silbersponsor
  • Data Insights GmbH Silbersponsor
  • Data Reply Silbersponsor
  • data42morrow Silbersponsor
  • datasqill - The Post-ETL Silbersponsor
  • Digital Ratio GmbH Silbersponsor
  • Disy Informationssysteme GmbH Silbersponsor
  • Empalis Consulting GmbH Silbersponsor
  • gmc² GmbH Silbersponsor
  • i-refact Silbersponsor
  • initions AG Silbersponsor
  • Lawrence Harvey Silbersponsor
  • linkFISH Consulting GmbH Silbersponsor
  • Lufthansa Industry Solutions Silbersponsor
  • m2data Silbersponsor
  • ORAYLIS GmbH Silbersponsor
  • Passio Consulting Silbersponsor
  • PRODATO Integration Technology GmbH Silbersponsor
  • QuinScape GmbH Silbersponsor
  • Solecon Silbersponsor
  • TCI Consult GmbH Silbersponsor
  • TDWI Young Guns Silbersponsor
  • TEDAMOH Silbersponsor
  • WE ARE DATA Silbersponsor
  • Woodmark Consulting AG Silbersponsor
  • Contiamo Start-Up
  • DataLion GmbH Start-Up
  • iNNOVO Cloud Start-Up
  • SYNABI Start-Up