We’re at a pivotal moment in the evolution of big data.
That was the feeling shared by the thousands of industry professionals who descended upon the glimmering Javits Center in New York City for the Strata Data Conference in late September. Engineers, business leaders, and technology professionals mingled over coffee, discussing everything from data pipelines to open-source frameworks. The conference pulsed with optimism and whirred with the hum of vendor-issued electronics (including Equalum’s next-gen yo-yos, a perennial crowd favorite.)
Amidst a conference jam-packed with keynotes, breakout sessions, and tutorials, the Strata organizers put together a board to take the pulse of attendees. On one side, conference participants affixed Post-It notes with their hopes. On the other side, they posted their fears.
Taken together, the comments from these boards provide a vivid picture of the industry. Touching on topics as diverse as politics, gender representation in tech, and the industry’s universal love of coffee, these comments sum up technology professionals’ excitement and anxiety about the new age in which we find ourselves.
What were the most important takeaways from Strata 2017? The board provides a heat map of what’s top of mind for industry leaders.
“I hope that I get more data.”
The takeaway: Industry players are desperate to harness the power of increasingly massive datasets – in real-time. (Or, as one conference participant tersely summarized her hopes for the conference: “REAL TIME.”) Industry leaders are looking for ways to fix leaky data pipes and legacy ETL processes that are holding them back from realizing the full value of an increasing volume and variety of data.
“I fear that AI will take my job.”
The takeaway: Tech leaders are architecting increasingly sophisticated real-time analytics environments. The board is awash with participants eager to learn about how other companies are leveraging machine learning, deep learning, and artificial intelligence. Numerous Strata talks focused on applications of real-time analytics (for example, real-time anomaly detection). As one participant put it, somewhat less optimistically: “I fear we are already inside the matrix.”
“I fear those insights won’t translate to business value.”
The takeaway: Leaders haven’t yet bridged the worlds of IT and domain-specific business applications. Industry professionals are hungry to use their data to drive growth. But despite the focus on connecting more data sources in real-time, and the growing ubiquity of real-time analytics, many companies still find their data hopelessly siloed and inaccessible to business users to make real-time decisions. In the words of one participant: “I hope to find a way to monetize my data.”
Ultimately, it was the tension between industry leaders’ optimism about the value of big data – and their frustration at the challenges in capturing that value – that best captured the mood of Strata 2017. Or, as one participant succinctly stated: “I hope that I get ideas for real solutions.”