The Critical Edge

When and Why Real-Time Data Matters

by Cesar Rojas

Hero Image
June 25, 2019 9:00am

Blog

Attendees at any big data or data science conference might very well leave believing that the future will be dominated entirely by the use of real-time data. Batch is dead. Long live real-time!

There’s certainly no shortage of heady optimism about the future of real-time data. But proponents haven’t always been rigorous in defining why and in what situations real-time data matters. As a result, some technology leaders have rightfully pushed back, questioning whether batch data and legacy ETL processes are “good enough.”

The reality is that the use of batch data for analysis and decision-making isn’t going away any time soon – because there is still a place for it. Architecting a streaming data solution in order to report on last month’s financial numbers would be unnecessary.

That being said, real-time data is a must for any application where the cost of data latency is high. Businesses think of the cost in many ways, but generally, it falls into the bucket of lost revenue (e.g., from customer churn or inventory shortages) or actual financial outlays (e.g., for equipment repair or security remediation).

Here are a few situations where real-time data is critical – and what industry leaders are doing to take advantage of the opportunity afforded by real-time data technologies:

  • Customer workflows: Customer expectations for immediacy and personalization are rapidly changing, and businesses from retail to financial services are struggling to keep pace. The cost of data latency in customer-facing experiences can have serious consequences: irrelevance (leading to erosion in brand perception/loyalty) or friction in the customer journey (leading to drop-off and lower conversion). For example, a retailer serving a display advertisement for a product that a customer just purchased creates an alienating user experience. Similarly, a customer seeking an auto loan is likely to favor the bank with an instant loan review and approval process over the one that takes minutes or even hours.
    • Example: a leading media company correlates viewership and social media data to inform ad buying decisions in real-time – investing in the content and ad platforms that are most relevant to their viewing audiences.
  • Cost containment: Real-time data can afford critical insights for preventing cost escalation. For example, real-time supply chain optimization can help companies predict and remediate critical inventory issues before shortages result in missed sales or require costly interventions. Similarly, industrial manufacturers may rely on real-time analysis of machine data to optimize preventative maintenance, preventing equipment damage that can be devastating to manufacturing output. And health providers interpreting results from network-connected devices can detect anomalies in real-time, preventing patient health emergencies like strokes or heart attacks.
    • Example: a Fortune 100 industrial manufacturing company makes use of a digital twin to identify anomalies in the and optimize preventative maintenance on equipment.
  • Threat detection: The growth of cybersecurity threats has placed an increased premium on threat detection (including network, application, endpoint, cloud, and wireless security). While security breaches represent can result in staggering direct and indirect costs to businesses, response speed has a significant impact on the ultimate cost of the breach.
    • Example: leading financial institutions monitor network traffic in real-time in order to detect anomalies that could signal intrusion attempts.

Ultimately, real-time data can provide a critical edge that helps enterprises navigate today’s fast-paced business landscape.

Ready to Get Started?

Experience Enterprise-Grade Data Ingestion at Infinite Speed.