Equalum, providers of the first Data Beaming platform, today announced the addition of full batch ETL support to its end-to-end solution – enabling organizations that rely on batch processes to take advantage of the same speed and enterprise-grade reliability as those using Equalum’s real-time data streaming technology.
Equalum's Data Beaming platform seamlessly teleports data across the enterprise, in real-time or in batch, connecting any source to any target application or system, and allowing organizations to unlock valuable insights. Data Beaming replaces unscalable legacy batch tools or unwieldy open-source implementations, providing a robust end-to-end solution with full Change Data Capture (CDC) support, easy setup and a zero-coding approach.
While many applications call for instant extraction, transformation, and loading of large amounts of data for real-time insights, others require handling data in periodic batches. Data Beaming’s full batch support provides substantially improved performance over legacy batch ETL tools - in a recent implementation at a large Fortune 100 manufacturer, Equalum’s solution consistently outperformed previously installed ETL systems by a factor of 15x.
Two key technological innovations underpin Equalum’s breakthroughs in batch performance: in-memory lookup and proprietary modifications of the Apache Spark Big Data framework. Advances in in-memory lookup – which enables Equalum to efficiently perform multi-source transformations and enrichment – are powered by new applications of Equalum’s state-of-the-art CDC technology, which keeps meta-data caches accurate and up-to-date by picking up changes in source data. Additionally, Equalum has invested in a comprehensive wave of tunings, bug fixes, and enhancements to Apache Spark to make it substantially more performant in ETL deployments.
Recent results confirmed the findings of previous testing in the industrial and financial services sectors, where Data Beaming’s real-time performance has been found to provide latencies of seconds instead of minutes and increased throughput relative to other solutions.
“Organizations that rely on batch processes have struggled for too long to manage complex, unscalable ETL solutions,” said Nir Livneh, CEO, Equalum. “I’m pleased that our breakthroughs in performance and scalability are helping enterprises take control of their data flow – outperforming legacy software by orders of magnitude on speed, computing resources, and time to value.”