UX and Big Data
I’ve been lurking, and recently posting, on Edd Dumbill‘s Google+ “community” discussion about “big data” since he set it up a few weeks ago (dunno if it’s a public group – G+ is opaque about these things – and I’m too lazy to find out). Dumbill works for O’Reilly Media, and helped popularise the term “big data” to describe a rather nebulous phenomenon of corporations and other entities using (some would say abusing) very large amounts data so as to spot interesting patterns. Naturally, this piqued my interest in terms of the ramifications for UX, but first I needed to get a handle on the definition of what “big data” might actually be. Perhaps also not without coincidence, O’Reilly have been involved in popularising new concepts with buzz words in the past – “web 3.0” being one of the most obvious – so I was a bit wary of possible hype. Stephen Few has also recently come out against the term (PDF) on the grounds that it over-states the capabilities of technology in order to sell software solutions to the gullible.
However, for the impatient (or simply lazy) UX-ers out there, I can report back on my investigations on what is the Interweb’s latest buzz phrase – and what it might mean for UX.
Right now, the vision behind “big data” is the construction of a sort of nervous system for the digital world. The first area of interest is the technology of that system. Here we see a wonderful geek’s playground of Hadoop, MapReduce and things like NoSQL and R. I neither know nor care much about these, but a heck of a lot of people on Edd’s discussion board do. How do you go about summarising 200 million data points before breakfast, or keep hold of a petabyte a day of data without simply throwing it all away almost as soon as you have it due to the cost of storage? The wider concepts of big data depend on these issues being solved – but the geeks will solve them I’m sure.
More interesting for me is what gives rise to all this techno-lust. We have an ever-increasing variety of telemetry available about people and things on the net. Request logs on web sites is just the start. We can swiftly move on to considering things like the prevailing weather at the time of a given event (most commonly a purchase), correlations with demographics such as age or income, deltas with other data, power consumption, lat/long – the list is probably endless. There is no reason to believe that at some point it might be possible for me to be generating data on where in my house I take my phone out of my pocket, or who is in the room with me when I use my microwave oven, when I ate last or whether a trip to the loo precedes my leaving the house. You get the idea.
Privacy issues aside (and there are big, elephant ones of gargantuan proportions here), what I’m most interested in is what’s the deal for UX? Much like some other “game changing” things such as responsive design or (once upon a time) eye-tracking, the answer is “more of the same.” Big data is what it says: more of what we have right now. There is nothing about having access to vast amounts of facts that really changes anything about how to design. A human still has to interpret the data as we do today (even if that data is pitiful compared to the coming tsunami). You have to be deeply familiar with the domain and what you understand about the people in it before you can find a new way of brokering insurance claims, let alone anything like having the next Facebook on your hands. Personally, I find that in the course of design, the more quantifiable data you have about something, the harder it is to make decisions and do the work of design at all.
So, I think until further notice, you can safely ignore “big data” as being anything approaching a revolution in terms of experience design. Mind you, I do fantasize that one day big data and its analysis will give us ways of predicting how, to give just one example, people interact with pricing or perceptions of value. Having verifiable principles that tell you why exposure to certain types of information under certain circumstances is likely to influence behaviour is surely the holy grail of experience design. So far though, in the discussions on Edd’s board, I see almost nobody thinking in those terms. This makes me think I’m either wrong or ahead of my time. We shall see.