Careful with that data, Eugene

Update:

Heather Hopkins has an update to her post in which she goes into a bit more detail about where the data for the chart comes from. And just to be clear, I wasn’t suggesting that the data is wrong — just that it’s probably not wise to be drawing sweeping conclusions from it.

Original post:

There’s a Hitwise traffic report up that looks at (or purports to look at) the demographic differences between those who spend most of their time at Yahoo vs. those who spend most of their time at Google — a report that led Duncan Riley at TechCrunch to boldly proclaim that “poor people use Yahoo, and those better off use Google.” While I am not a sociologist or market researcher, I have to say that multiple warning bells went off when I read this. I think there are a whole series of leaps of logic going on, most of which are based on not a lot of evidence.

Like most such research, the chart included with the Hitwise report looks great — colourful blobs represent different demographic groups with catchy names like “Blue-collar Backbone” and “Small-town Contentment,” which are clustered at various points on a graph with Yahoo on one axis and Google on the other. “Struggling societies” is at one extreme and “Affluent Suburbia” is at the other. So that’s it, right? End of story.

hitwise1.jpg

Except that there are some pretty large holes in all of this — or at least information that we don’t have. According to the Hitwise report, the demographic groups with the cute names are based on “offline data” from Experian, which appears to be a credit reporting agency with a marketing consulting arm (Note: Experian apparently acquired Hitwise last year). Where does that information come from and how reliable is it? That’s one question. Then there are the blobs: their size apparently refers to how many in that group have spent $500 or more online.

But what does that tell us? It tells us whether people who spend money online (or at least people who say they spend money online) are more likely to use Google or Yahoo, not whether those people are rich or poor. Maybe it’s just geeks vs. non-geeks, as more than one commenter on TechCrunch pointed out. In any case, we have one questionable set of data about spending habits balanced on top of another set of questionable data about demographic groups. Not much to draw conclusions from, I wouldn’t say. Sociologist danah boyd has some thoughts that tie into her research as well.

(*The title is based on a Pink Floyd song, for no other reason than it popped into my head. No, I don’t know why).

Leave a Reply

Your email address will not be published. Required fields are marked *