Big Brother and the big data dividend – diginomica

Big Brother and the big data dividend – diginomica

Satya Nadella, Microsoft

Microsoft’s new CEO Satya Nadella conjured up some evocative language to stir enthusiasm for big data and the Internet of Things at the launch of SQL Server 2014 in San Francisco this week:

“The core evolution of silicon, software and hardware is putting computing everywhere humans are present. And it’s generating a massive data exhaust of server logs, sensor data and unstructured social stream information. We can use that exhaust to create ambient intelligence for our users.”

Carefully chosen language plays an important role in steering cultural attitudes. Data is often seen as difficult and nerdy; big data gleaned from sensors has Orwellian undertones of Big Brother. This week Microsoft sought to recast such attitudes, mainly because it sees a massive market opportunity in harvesting all that ‘data exhaust’ and turning it into ‘ambient intelligence.’

According to research by IDC (and commissioned by Microsoft to coincide with the event), businesses that act to harness all this data can share in a global ‘data dividend’ worth $1.6 trillion in extra economic activity. That return justifies a lot more spending even than the $5 billion-a-year run-rate that Nadella revealed SQL Server reached last quarter.

Thus in addition to the new, in-memory release of its flagship database product, Microsoft also revealed details of a new Azure Intelligent Systems Service to capture and analyze machine data, delivered its Analytics Platform System (APS) that bridges SQL Server with Hadoop, and promoted Office (and Excel in particular, heaven help us) as the front-end for making sense of it all.

Customer stories

At a parallel event in London, Microsoft shared the stage with three customers who already use its technology to carve out their share of the data dividend:

Media agency network MEC Global uses Microsoft’s Parallel Data Warehouse product (the predecessor of APS) to perform ‘in-flight’ analysis of digital campaigns. Having cut query times from four hours down to minutes, it is able to make and test changes in keyword spending to enhance results. “We got true ROI,” said director of technology Trevor Attridge. “It really allowed us to move our client’s investment around to maximize performance.”
Games provider Mediatonic uses Microsoft’s Hadoop-based HDInsight service running on Azure for real-time analysis of player behavior. With revenue dependent on in-app purchasing, the analysis is vital to understanding how to tune gameplay to keep players engaged.
Social enterprise HACT is using HDInsight on Azure to aggregate data from not-for-profit social housing providers. By analyzing a dataset of almost half a million homes from 20 participating housing associations, HACT expects to deliver operational savings by helping them improve targeting of services and repairs, better prediction and management of arrears and voids, and identifying tenancy fraud.

Spooked by data

But this being Europe, some of the attendees were spooked by the type of data HACT is planning to assemble in a proposed follow-up project. Chief executive Matt Leach described the potential for putting sensors in homes that could yield even more useful data. He gave the example of sensors detecting that an elderly tenant’s gait had changed. That could alert the housing provider to put in a grabrail at a cost of £25 ($42), pre-empting the need to spend £10-20k ($17-34k) relocating them to a different property.

It seems reasonable enough, but there’s something uncomfortable about the notion of collecting metrics on social housing tenants as they move around their own homes. Will this be opt-in or will it simply be imposed, in the same way that media and gaming companies collect metrics about our movements around the digital world so they can get us to buy more stuff? Strangely enough, most of us have no qualms about the latter, so maybe we should blithely acquiesce to the former too.

Leach pointed out that assisted housing often already uses monitors and sensors, but at much greater cost than is needed for connected digital sensors, where the computing can be done in the cloud:

“In the past a lot of this was based on putting an expensive box in the home. If you can pull everything back into the cloud then actually you’re getting a long way for very little money …

“Looking forward to instrumented homes, this is the mainstreaming of technology that is already embedded in [some] people’s homes.”

Things and humans

Cultural attitudes to data collection are changing, and the language we use is crucial to how ready we are to accept these changes. The Internet of Things is a phrase that depersonalizes sensor data collection, but the data is only useful when it relates to human outcomes. As Tim O’Reilly pointed out this week, what we are actually instrumenting is #IoTH: The Internet of Things and Humans:

… when you think about the Internet of Things, you should be thinking about the complex system of interaction between humans and things, and asking yourself how sensors, cloud intelligence, and actuators (which may be other humans for now) make it possible to do things differently.

O’Reilly makes a different point than mine — he’s reminding us that we humans are ourselves often acting as sensors, processing information or performing actions within these networked applications (for example the Uber driver and his or her passenger). I’m wondering whether we’re walking into these networked relationships with our eyes fully open.

Europe stirs the crucible of data privacyLong before the Snowden revelations made privacy laws a battlefield of international diplomacy, the …Oct 21

We enjoy the benefits of services that work better because they know more about us. This is a trade-off we’re happy to make because it makes our lives better. But we should also understand how those trade-offs often cede control to external agencies in ways that may not be in our best interests.

No doubt we’ll have a more prosperous society as a result of that projected $1.6 trillion data dividend. But will we have paid a price in terms of individual autonomy? I don’t know the answer, I simply want to make sure the question gets asked.

Some links to coverage of the Microsoft announcements that I found useful while researching this post:

Microsoft’s Big-Data Angle: Office as a Friendly Front-End
Microsoft’s Data Culture: It Just Might Work
Inside the Hekaton: SQL Server 2014′s database engine deconstructed
Microsoft: Let us be your one stop for big data analytics 
Microsoft delivers preview of new Azure cloud service for Internet of Things
Microsoft Azure Intelligent Systems: 4 Facts

Image credits: Digital home © WavebreakmediaMicro –; Satya Nadella presenting – Microsoft; Data dividend infographic – IDC


Latest Posts

Phil Wainewright has been a thought leader in cloud computing as a blogger, analyst and consultant since 1998. As well as documenting the transformation of 21st century enterprises by digital technology, he is co-founder of industry advocacy group EuroCloud.

The Met Office and how to make accurate data ‘worse than useless’ – July 23, 2014
IFS CEO: mobile is a gamechanger, no demand for big data or SaaS – July 18, 2014
From crowdsourcing to DRaaS: cloud has diverse business impact – July 16, 2014


Big Brother and the big data dividend – diginomica

Share this post