Megabuyte, September 2012. Original article here (£).
The Early View
Big Data, Big Bang: Change is the new constant
The technological Big Bang, courtesy of the coming-of-age of the internet, shows no sign of abating. For businesses, it’s proving useless to wonder where the merry-go-round of change will stop next because in all likelihood it will just keep gathering pace, and nowhere is this pace of change more prevalent than in the phenomenon that is Big Data.
“Digital innovations are transforming the economic landscape, far more profoundly than other big shifts in our economic history such as deregulation, oil shocks or mining booms,” said Deloitte’s access economics director Dr Ric Simes in a new report on digital disruption. The consultancy wrote this for the Australian market, but the lessons are global: supply chains are being scrambled and categories are being blurred, and while the timescales will vary, the disruptions apply to every single industry. Technology has leapt forward before, but how it’s different this time is that this is not a blip, it’s a new reality where ongoing change is the new permanent state.
Life in Big Data
At the core of this change is a growing realisation that the internet isn’t a virtual experience anymore, but it’s becoming an aspect of real life. The generational element to the blurring boundaries between virtual and physical experiences was eloquently summed up by Peter Czerski’s ‘We, the Web Kids’, which has become something of an internet meme: “We do not use the internet, we live on the internet and along it. […] The web is a process, happening continuously and continuously transforming before our eyes, with us and through us.”
Everything the “web kids” do comes draped in a digital layer which is starting to feel as real as anything in the physical world, explains Czerski, but what enables the internet to exist in our lives like this is the proliferation of Big Data. This includes the sheer amount of data now available for us to pick out of the ether, but equally interesting is the amount of information we ourselves put out there, via location-tagging apps, online shopping histories, Google searches, links to newspaper articles on Twitter, and updates to Facebook. In the past two years, the capacity of the internet has doubled, to 77 terabits per second, according to analysis from Telegeography. Big Data is getting obese, meaning there is a significant opportunity for start-ups that can sift through the noise and present information in an intelligent and ultimately useful manner. Here are some companies that seem to be getting the message at the core of Big Data: the internet is no longer separate from us, but it has become real life.
Visual effects
Space-Time Insight enables users to spot connections they would otherwise have missed by visualising geospatial data in map-form. The West Coast group has just raised $14m in a Series B round to further develop its technology and take it to the mainstream. The company’s funding, which totals an estimated $16m, this time came from Novus Energy Partners, EnerTech Capital and ClearSky Power & Technology Fund. The energy- and utility markets are Space-Time’s main customers at present, but the group is starting to see traction in areas such as oil and gas exploration and transportation. Space-Time enables companies with geospatially diverse assets to visualise and manage factors beyond user data to also include factors ranging from weather to social media.
Another geospatial startup freshly injected with cash is SpaceCurve in Seattle, whose second round of funding brought in $3.5m from Triage Ventures, Reed Elsevier and Divergent Ventures. Total funding is now $5.2 million. SpaceCurve doesn’t actually launch its product until next year and the descriptions of what they do are a bit vague: “The technology will tie people or entities to a precise point in time and space, immediately discover the social and semantic relationships between them, and deliver this real-time intelligence instantly to identify new opportunities, and support better decisions and more profitable actions.” A lot remains to be seen regarding SpaceCurve, but there is no shortage of potential and the backers seem to be pretty excited.
Security and mobility
Sqrrl recently raised $2m in seed money, from Atlas Ventures and Matrix Partners, to aid in their mission to make Big Data applications more secure. This will hopefully make Big Data utilisation more palpable for customers in sensitive industries such as financial services and healthcare. Massachusetts-based Sqrrl’s mission is aided by co-founder and CEO Oren Falkowitz’s roots of working with Big Data questions at the US National Security Agency. The CEO explained Sqrrls methods to GigaOm: “An electronic health record might have a hundred or a thousand components and if any one of those requires an extra control, like the patient’s social security number, the current method is to control that entire record. We have fine-grained access model so you can assign different security to the patient name, to allergies, to medications, to doctors names – they can all be individually controlled.”
Palo Alto’s Factual has spent the summer launching a couple of interesting applications for mobile. As providers scramble to make better location-specific offerings, Factual’s Geopulse API provides everything the Big Data group has on file about a specific location, mined from its constantly growing databanks. In Texas, Datafiniti is only one year old but comes with a compelling mission: to become a search engine for data. The hope of founder-CEO Shion Deysarkar, formerly of data crawler 80legs, is that Datafiniti will make finding relevant data as easy as finding relevant web sites through Google.
World Wide Kaggle
On the subject of mind-boggling Big Data ventures, it’s hard not to be excited by Kaggle. The San Francisco-based start-up has a unique solution for how Big Data can be used to predict future behaviour, by linking scientists to companies sitting on mountains of scrambled data. Kaggle’s method is to offer cash prizes to the global PhD community, asking them to find the best solution to the problem on behalf of clients such as NASA and Ford. At the moment, insurer AllState is offering money via Kaggle to scientists who can come up with the algorithm that best predicts which customers are going to renew their car-insurance policies when there is six months left. Boffins worldwide can take a crack at this using historic data from 2008 to 2011 work out a predictive model, and similar methods are being used to solve problems for music companies wanting to know which songs will be most popular, and banks wondering which customers will be most likely to default on a loan.
Kaggle’s founder is Australian native Anthony Goldbloom, whose previous employment was macroeconomic modeling for the Reserve Bank of Australia. Speaking to Silicon Valley news outlet PandoDaily, Goldbloom told of how he taught himself to code after learning about Big Data and becoming fascinated with what it would mean if the right information was linked to the right people. Goldbloom launched the first version of Kaggle in 2010, and last November the group raised $11m in a Series A round led by Index Ventures, Khosla Ventures and PayPal co-founder Max Levchin. Kaggle claims its unorthodox approach means its competitions have always yielded a solution that outperforms in-house efforts, and often the best models will come from unpredictable candidates which may not have been hired for the job in a traditional setting. There is something wonderfully egalitarian about a system that promotes solutions purely based on talent instead of people’s backgrounds, but Kaggle proves this counter-intuitive system also makes the best business sense. Dreamt up by a “web kid” born in 1983, Kaggle’s audacious concept shows just how intriguing the results can be when Big Data is put to work on big problems in a world where the distinctions between “digital” and “real” life are becoming increasingly irrelevant.
Jessica Furseth 