Category: Gigaom

The Google book settlement — which the search giant signed with the Authors Guild and the Association of American Publishers in 2008, after a dispute over the company’s scanning of books — was recently struck down by a judge as too far-reaching, which is arguably true (although Google would undoubtedly disagree). But the fact that the arrangement has been rejected might not be such a bad thing, because it puts the spotlight back where it should be: on the fact that Google is doing nothing wrong, legally or morally, in scanning books without the permission of the authors or the publishers of those books.

Just to recap, Google started scanning books sometime in 2002, as part of its expressed desire to “index all of the world’s information.” In addition to deals with certain publishers and various university libraries — deals that are not affected by the book settlement or the legal ruling — Google also began sourcing and scanning books that were either in the public domain or were “orphaned” (a term used to refer to books that are still under copyright, but whose author or publisher can’t be found).

So far, so good. But Google also started scanning and indexing books that were under copyright, and then offered authors and publishers the ability to “opt out” of the program and have their books removed. Some felt that this was a good bargain — especially since Google was going to help promote their books (by revealing them in search and at the Google Books site) and give readers an easy way to buy them. Others, however, said that scanning and indexing their books without explicit permission was wrong, and filed the lawsuits in 2005 that led to the agreement.

The crux of this argument is that scanning a book makes a copy of that book, and that copying is not permitted unless a copyright holder specifically agrees. The authors and publishers made this argument despite the fact that Google only ever shows a small fraction of a text when they display a book online. It’s not as though the company planned to make copies of all books freely available to anyone through some kind of Google Books version of Napster. But the plaintiffs argued that simply scanning them was bad enough.

This is a ridiculous position, and always has been. Scanning something makes a copy of it in the same way that my viewing a web page makes a copy of it in the RAM of my computer — I’m surprised that authors and publishers haven’t tried to argue that this is secondary copyright infringement as well.

The reality is that Google’s use of selected extracts from books or any other work is protected by the principle of fair use (PDF link), which allows anyone to make use of published content of all kinds (text, images, etc.) without asking for permission from the creator or the rights holder. It’s the same principle that allows Google to index and show search results for images, web pages and other content without having to ask every single site publisher or photographer.

Why is this important? Because without that ability, search engines as we know them couldn’t exist, and they are a positive force for society as a whole — just as having a single way to search (and buy) every published book in the world would be a positive thing. Imagine if we were setting up public libraries now: would any author or publisher agree to have copies of their books just sitting there on shelves, for free, with anyone allowed to borrow them for as long as they wanted to? Unlikely (and e-book publishers like Amazon are trying to roll back borrowing abilities for digital works as well). If I want to buy a book and rip it apart and then scan it and save a copy on my hard drive, I should be free to do so, and so should Google.

The big problem with the Google book settlement, as noted by the judge who struck it down (PDF link), is that the settlement gave the web giant the exclusive right to do whatever it wished with all scanned works, including selling orphan books, which is arguably over-reaching. But that doesn’t change the fact that Google’s initial impulse was the right one: it does have the right to scan and display extracts from books, regardless of what the Authors Guild and the AAP say, and it should be allowed to continue doing so.

I didn’t get a chance to write about the launch of the New York Times subscription plan last week for a number of reasons (okay, I was on a beach) but I’ve since read most of what others have written about it, and the general consensus seems to be that it is a) confusing, and b) a sign of desperation. But while both of these things are arguably true, my big problem with the newspaper’s money grab is that it is fundamentally backward-looking. More than anything else, it feels like a defensive move to buy some time while the paper figures out what it wants to be when it grows up.

There’s no question that the details of the plan are somewhat bewildering, as Felix Salmon of Reuters has noted: occasional readers get to see 20 “items” for free in a month — with the definition of an “item” subject to the restrictions described in the newspaper’s FAQ on the topic — and then they have to sign up for a plan. Paying $15 a month gets you access to the website and the iPhone app, but (somewhat surprisingly) not the iPad app. For $20 a month you get access to the website and the iPad app, but that doesn’t get you a subscription to the iPhone app. If you want access across all platforms, you have to pay $35 a month.

Once you get past these nuances, however, it becomes fairly obvious that the pay plan has little or nothing to do with promoting the iPad app or the iPhone app, or even the newspaper’s website. Instead, it seems pretty clearly designed to protect the subscription numbers for the printed version of the Times: if you subscribe to virtually any version of the paper, including the Sunday-only option, everything digital comes along with it for nothing. In other words, you can pay as little as $30 a month and get the entire contents of the newspaper in whatever form you want.

In that sense, the Times pay plan seems to be motivated by the same impulse as other paywalls at newspapers such as the Times and the Sunday Times in Britain — where News Corp. erected subscription plans last year — and at New York Newsday, which launched one in 2009. As I pointed out in a post at the time Rupert Murdoch was planning to launch paywalls at his British papers, the main point of these walls was to keep people in, not keep people out. Since print continues to deliver the majority of revenue for newspapers such as the Times and the NYT, it’s crucial to keep readers from cancelling their print subscriptions and simply moving to read everything for free online.

Is this a compelling financial rationale for a pay wall or subscription plan? Perhaps. There’s no question that, as the New York Times admitted when it announced the new plan, newspapers need to find new sources of revenue to replace declining advertising income. But I’m skeptical that the pay plan is going to produce the $100 million or so in new revenue that some seem to think it will.

Even more than that, however, the Times’ subscription model seems fundamentally reactionary, and displays a disappointing lack of imagination. The newspaper seems to be saying: “What we do is valuable, and you have been getting it for free, so it’s time to pay up.” Some readers may accept that rationale, and sign up — but others are going to go elsewhere, or reduce their NYT consumption to links that arrive from blogs, Twitter and Facebook, which the newspaper has (wisely) decided will be exempt from the subscription wall. And so the paper will continue to use its new digital assets primarily to subsidize its declining print business.

The success of aggregators like The Huffington Post — which NYT executive editor Bill Keller recently ranted about — is only the most recent sign that the way many people consume their news has changed forever. It is no longer about picking a specific outlet like the Times, and then relying solely on that for news and opinion about the world. Traditional media like the NYT have lost the control they historically had over distribution and consumption, and attempts to reimpose the scarcity that such things once had is ultimately futile.

The exemption for Twitter and other social media is a sign that the Times understands this on some level at least, but why not go further? Why not offer people a subscription to special Twitter direct messages or Facebook Q&A sessions with writer Nick Kristof as he reports on the uprisings in the Middle East? What about offering real-world events that involve some of its most prominent voices, where people can meet them and network with each other at the same time? The music industry seems to be waking up to the fact that individual songs are simply a loss leader that drives demand for other services, but the New York Times is still trying to charge people monthly fees for undifferentiated news content.

Will the subscription plan bring in some money? No doubt. But putting meters on its existing content is not going to save the Times. In order to really take advantage of the revolution that is underway in the content business, it needs to start thinking about what it does in a whole new way, and there are few signs of that happening any time soon.

Post and thumbnail photos courtesy of Flickr user Mark Strozier

Birthdays are a natural time for reflection, even if you’re only five years old, which is the age that Twitter officially turned today. It may not seem like much, but that’s about 35 in Internet years, which means the company is close to being middle-aged — and Twitter has definitely been struggling with some mid-life challenges lately. Another much-hyped web startup also just had a birthday recently: Digg, which turned six in December, has been struggling as well, after a failed redesign and the departure of founder Kevin Rose. And Digg’s decline from pioneering service to also-ran contains some lessons for its fellow social-media service.

In some ways, it’s hard to believe that Twitter has been around for five years. The service that Jack Dorsey and Biz Stone originally launched as a side project within Evan Williams’ company Odeo — which was later shut down, with Williams taking over from Dorsey as CEO of Twitter, in a move that reportedly caused some bad blood — didn’t seem like much when it first launched, even to Om. Like most users, I thought the service was fairly useless when I joined in early 2007, and I spent months wondering what I was supposed to do with it before a critical mass of friends and other interesting people joined, and it began to come to life.

Twitter’s status as a powerful real-time news platform didn’t really become clear until it was used to transmit updates about the forest fires in California in 2007 and during an earthquake in China in 2008. Gradually, people started to see it as something other than just a way of talking about what you were having for lunch — and when Janis Krums used Twitter to post a picture of a plane crash-landing in the Hudson River in 2009, the reality of Twitter as a news-publishing system started to go mainstream. Every subsequent event, from terrorist attacks in Jakarta to the earthquake in Haiti, has reinforced the idea that the service lowers the barriers to entry for publishing, as Evan Williams put it last year.

The most recent example was the use of Twitter and Facebook by dissidents in Tunisia and Egypt to co-ordinate demonstrations and uprisings against their governments, and the compelling stream of news from participants that was carried out of Tahrir Square by Twitter to the world, thanks in part to real-time news aggregators like Andy Carvin of National Public Radio, who created what was effectively a one-man wire service.

More than anything else, however, Twitter has become a platform for community — whether it’s a community of people interested in revolutions in the Middle East, or a community that is obsessed with the latest product release from Apple, or a community that wants to know what John Cusack or Steve Martin think about current events. And one of the hallmarks of a social service like Twitter and Facebook is that the more people use it to connect with each other around things they are passionate about, the more they feel like they own it to some extent — and that feeling is what Twitter is currently fighting as it tries to mature as a company.

You can see that in the outraged responses to the recent Quick Bar fiasco, and to the shutting down of third-party clients like Bill Gross’s UberMedia — which has been trying to develop its own competing monetization strategy for the social network — and to the rollout of services such as Promoted Tweets and Promoted Trends. As I’ve argued before, users have grown so used to seeing Twitter as a utility that every move the company makes to add money-making layers or to control its ecosystem is seen as an affront in some sense, like someone invited you to a party at their house and now is asking you for money or putting up turnstiles and imposing all kinds of rules on your behavior.

Although the two services are different in many ways, Digg has also been struggling with the same kinds of issues — and some of those struggles are directly related to Twitter, since Digg’s link-sharing features, which were once a pioneering example of what some called Web 2.0, have arguably been overshadowed by the growth of Twitter. But Digg has also rolled out its own poorly-received design features: the service launched Digg v4 last August and the new design was roundly criticized as unstable and (more importantly) a breach of faith with the traditional Digg community. The site’s traffic plummeted, the new CEO rolled back most of the new features and laid off almost 40 percent of the staff, and founder Kevin Rose is moving on to start a new venture.

So what are the lessons that Digg has to teach Twitter? One is that even pioneering services, whose founders appear on the covers of leading business magazines, can be overtaken by events, and by other services that don’t even exist yet. Yes, it’s true that Twitter is supposedly worth $10 billion, and is much larger than Digg ever was — but that lesson still applies (as MySpace is well aware). And the other lesson is that the core of a social network is the community of users, and arguably in Twitter’s case the community of developers as well, or the “ecosystem.”

Alienating either or both of those groups is a very risky strategy, as Digg has discovered. It could pay off in Twitter’s case, but it could also ruin one of the key features that make the network so powerful and compelling, and that is something that would be very difficult — if not impossible — to recapture.

Post and thumbnail photos courtesy of Flickr user Will Clayton

In another shot fired across the bow of the Twitter ecosystem — or another volley in the ongoing Twitter wars of 2011 — the company has come out with new terms on which all developers must operate, which makes it clear that Twitter plans to own the majority of the value in the system, and relegate third-party apps to the periphery. As with the company’s other recent moves, including shutting down misbehaving apps, the response has not been friendly from many parts of the network. And while Twitter can probably get away with this kind of behavior, it is taking a real risk of losing much of the goodwill it has built up over the years.

Critics have accused the company of “nuking” the developers and services that helped it achieve its early growth in its drive to monetize its network, in much the same way that Hunch founder and angel investor Chris Dixon criticized the company last year for “acting like a drunk guy with an Uzi” after it acquired Tweetie. Some have given the company credit for at least laying out the rules in a clear manner with its latest API update, since much of the developer community has been unclear on what was permitted and what wasn’t, but those responses seem to be in the minority.

The point has become clear by now: anyone who is still under the impression that Twitter is the friendly, touchy-feely company that co-founder Evan Williams used to run — the one that admitted it “screwed up” relations with developers by moving too quickly — is living in a dream world. Twitter CEO Dick Costolo may have been a standup comedian at one point, but he is a businessman now, and Twitter is determined to do whatever it takes to come up with a business model to justify the huge valuations it is getting.

As MG Siegler has pointed out, what Twitter is doing is just business and not personal — but there is a reason that most businesses don’t operate the way the Mob does (other than the fact that killing people is illegal, of course). Acting that way, by routinely kneecapping people or setting their businesses on fire, is a risky proposition. Even if you *can* do it, it’s not clear that you *should* do it, especially if some of your business depends on goodwill (as opposed to fear), as Twitter’s clearly does, and especially if a large part of your success is due to that larger ecosystem.

Without the help of third-party apps like Tweetie and Tweetdeck, the company likely would not have been nearly as successful at building the network (and a ready-made client like Tweetie certainly wouldn’t have been sitting there waiting to be acquired). But the ecosystem didn’t just build demand for the network — it also helped build and distribute the behavior that now makes Twitter so valuable: the @ mentions, the direct messages, re-Tweets and so on, none of which were Twitter’s idea originally. That created a huge amount of goodwill, and led to the (apparently mistaken) idea of an ecosystem.

It’s all very well for Twitter to claim ownership of all those things now, since it is their platform. And obviously there are businesses that can get away with being arbitrary or dictatorial — Apple is well known for such behavior, after all, and it is one of the most valuable companies on the planet. But this only works over the longer term if your product is so unique and compelling that people will put up with it. Is Twitter in that category? Perhaps. The company managed to grow at an astronomical rate even when it was suffering repeated outages, because users (including me) were so addicted to it. That may have made Twitter a little cocky about how necessary it is.

It’s also true that there isn’t really much competition when it comes to micro-blogging, or whatever we choose to call Twitter. Open-source options such as Status.net have tried to get traction, and programmer Dave Winer has been lobbying for and trying to jump-start an open Twitter alternative for some time — even before the company made it obvious that it was planning to “prune” the ecosystem. So far nothing has come along that can compete, but Twitter’s behavior could serve to boost those efforts substantially. And there would be definite benefits to an open system — not just in terms of features, but for when governments decide to order companies like Twitter to release user information to the State Department about their espionage investigations.

In the short term, Twitter seems likely to get away with throwing its weight around and dictating the terms on which developers — and users, to a large extent — can access or make use of the network. And maybe the network has grown to the point where none of that matters any more. But sometimes when you bulldoze an ecosystem, what you wind up with is a lot of weeds and a corporate mono-culture in which growth no longer flourishes, and in some cases that growth subsequently moves elsewhere. That’s a risk Twitter seems willing to take — whether it is the right one remains to be seen.

If you’re a traditional journalist, or someone who works for a traditional media outlet, the easiest way to cast aspersions at a web-based or digital media company is to use the A word: that is, “aggregation.” New York Times executive editor Bill Keller stayed true to form in a piece he wrote for his newspaper Thursday, in which he categorized The Huffington Post and other unnamed online media outlets as pirates, who are in the business of “counterfeiting” content rather than engaging in “real” journalism. In only a few paragraphs, the NYT editor managed to say volumes about how little he understands where media is now, or where it is going.

Keller’s piece starts out as a humble discussion of his status as the 50th most powerful person in the world (according to Forbes magazine) and how he thinks this is absurd, since he just runs a newspaper. But it quickly becomes a complaint about how members of the media — and assorted “flocks of media oxpeckers who ride the backs of pachyderms, feeding on ticks,” as well as professional pundits such as Clay Shirky and Jay Rosen — spend too much time talking about media in the abstract instead of doing it.

Then he launches into an attack on aggregators, saying the media industry has “bestowed our highest honor — market valuation — not on those who labor over the making of original journalism but on aggregation,” an obvious reference to the $315-million acquisition of the Huffington Post by AOL.

And what does the term aggregation mean? That seems to depend on who does it. The NYT editor says that aggregation can mean “smart people sharing their reading lists, plugging one another into the bounty of the information universe,” and then he admits that this “kind of describes what I do as an editor.” So aggregation is journalism then? But wait — Keller goes on to say that aggregation often amounts to:

[T]aking words written by other people, packaging them on your own Web site and harvesting revenue that might otherwise be directed to the originators of the material. In Somalia this would be called piracy. In the mediasphere, it is a respected business model.

This is where he calls out the Huffington Post, whose founder is “the queen of aggregation,” having discovered that “if you take celebrity gossip, adorable kitten videos, posts from unpaid bloggers and news reports from other publications, array them on your Web site and add a left-wing soundtrack, millions of people will come.” The NYT editor goes on to say that while AOL called the acquisition of Huffington Post a key part of its content strategy, buying an aggregator and calling it a content play is “like a company announcing plans to improve its cash position by hiring a counterfeiter.” (Update: Arianna Huffington has posted a response to Keller’s piece at her site).

Keller seems to be missing the point that all media — both online and offline — is to some extent about aggregation. Even newspapers like the New York Times aggregate content from newswires and occasionally rewrite that content to make it their own. Yes, they pay those newswires for the privilege, and so does the Huffington Post: the difference is that it pays in attention, which it directs back to the original source, just as Google pays with links when it aggregates content at Google News. According to a Huffington Post staffer, news websites actually beg the site to aggregate their content, since it gets more traffic.

Aggregation is a term that covers a wide variety of behavior, some of it nefarious and much of it not. To take just one example, look at what Andy Carvin of National Public Radio has been doing by pulling in and republishing Twitter posts from hundreds of different people — both individuals and journalists, including New York Times writer Nick Kristof — as a way of covering the revolutions in Tunisia and Egypt.

Is that aggregation? Sure it is (or “curation,” as some prefer to call it). Is Carvin not taking reports from unpaid bloggers and news reports from other publications and republishing them? Of course he is. But he’s also engaged in a very real form of 21st-century journalism. And maybe if Bill Keller spent a little more time trying to understand how aggregation works instead of railing against it, the New York Times would be a little further ahead in this new media game, instead of playing catch-up with Arianna Huffington.

Ever since the web first started to become mainstream, there have been attempts to build the “Daily Me,” a personalized newspaper that learns what you like or are interested in (does anyone remember PointCast?). But as I noted in a recent post on the topic, many of these efforts are lackluster at best, and irritating at worst. They either require too much fiddling to tune them, or they don’t show any intelligence at all (or both). But that doesn’t stop companies from trying — and the most promising entrants in this race so far are those that try to build their recommendations on top of the social signals coming from Twitter and other networks.

The latest to join the field is a personalized magazine app for the iPad called Zite, whose name is a play on the German word “zeitgeist,” meaning “the spirit of the times.” The company behind the app is based in British Columbia, and has been funded by angel investors and research grants from the Canadian government, and CEO Ali Davar says Zite has been working on its recommendation engine for several years. An earlier version of the project, which is based on technology developed at the University of British Columbia’s Laboratory for Computational Intelligence, involved a browser extension called Worio that suggested related results when users did a Google search.

The Zite app pulls in your Twitter account and your Google Reader feeds (if you have them), and then suggests topics based on your interests. This was the first place where it fell down for me — it said that it didn’t have enough information about me, which I thought was odd, since I have been on Twitter for about four years, have posted more than 35,000 tweets and follow over 2,000 people. I’ve used Google Reader for years as well, and am subscribed to about 600 feeds. Although Zite got some of its suggestions right, it recommended Barcelona as a topic, which was totally out of left field — in fact, I can’t recall ever mentioning the Spanish town before.

Although Robert Scoble says that Zite doesn’t feel as slick as Flipboard, I thought the app worked quite well in terms of usability — you can swipe to move through articles, click to read them in a built-in browser, and share them easily (although you can’t save them to Instapaper, which is a shame). And you get asked with each one whether you like the content or want to see more of it, which is something that other apps and services such as Flipboard are missing. It requires some effort on a reader’s part to do this training, and many will probably not do it, but it is crucial for learning likes and dislikes.

One glaring omission from Zite is the lack of Facebook integration. Davar says that Facebook tends to provide sources that are too heterogeneous (that is, too diverse) to be a source of good recommendation data, and that might be true, but it’s still a giant social network and a huge part of many people’s online news consumption, so it seems odd to leave it out — especially when the data coming from the billions of “like” buttons scattered around the web could be a source of so much data on what people want to read (Yahoo Labs has just released an interesting survey of what that shows for some of the major news sites).

There’s one nagging question that keeps jumping out at me as I look at all of these apps and services, however, and that is: where is Google? The combination of smart aggregation and algorithm-driven personalization seems like something the search engine should be all over. Google News has added some personalization aspects, but they are anemic at best, and one of the original customized news-readers — Google Reader — hasn’t really capitalized on that opportunity much at all, although it does provide some recommendations.

The reality is that the RSS reader has been eclipsed (for the small proportion of the population who even used one) by Twitter and Facebook and other social news sources, or smart aggregators such as Techmeme and Mediagazer. Google has more or less failed to take advantage of that transition at all when it comes to news reading, although it is trying to add social signals to search. Why not take FastFlip and try to make it a Flipboard or Zite or News360 competitor?

And apart from the Washington Post’s new Trove project and the News.me spinoff from the New York Times that Betaworks is close to launching, newspapers — who should know a thing or two about filtering and recommending the news to people — are virtually nowhere in this game.

If there’s one thing that web users need more than ever, it’s smart filters to help them navigate the vast tsunami of information that comes at them every day (it’s not information overload, says Clay Shirky, it’s “filter failure”). Someone is going to solve that problem, and if they do it properly they could capture a significant share of the online news-reading market.

As the financial screws continue to tighten on traditional media companies, more and more are choosing to throw their eggs into the basket labelled “paywall,” despite a conspicuous lack of evidence that erecting barriers to non-paying readers — or turnstiles that charge them after they have read a certain number of articles — has any beneficial effects. The latest to go this route is the Dallas Morning News, which put up its wall this morning, and the New York Times (s nyt) is also said to be close to launching its metered-access plan. But in the long run, these walls are really just sandbags against a rising tide.

The Dallas Morning News paywall, which the paper has been working on since the middle of last year, does have some holes in it that are designed to mitigate the extent to which it shuts out readers: non-subscribers to the paper will be able to read headlines, blogs, obituaries, classifieds and any syndicated content for free, but local news will be blocked. And the news doesn’t come cheap: a subscription to the print newspaper and all of the Dallas publisher’s digital content (which includes an iPad app) is $33.95 a month, and an online-only subscription is $16.95 a month. By comparison, Rupert Murdoch’s new iPad app The Daily costs $4 a month or $39.99 for a year.

Last month, Dallas Morning News publisher Jim Moroney admitted that he was unsure whether the paywall would work or not, telling the Nieman Journalism Lab that “This is a big risk — I’m not confident we’re going to succeed. But we’ve got to try something. We’ve got to try different things.” Moroney was similarly blunt in a memo to his newsroom staff about the launch of the wall:

So why, beginning tomorrow, are we going to require a subscription to access much of the content we originate and distribute digitally? The reason is straightforward: Online advertising rates are insufficient at the scale of traffic generated by metro newspaper websites to support the businesses they operate. We need to find additional and meaningful sources of revenue to sustain our profitability.

The bet being made by papers like the Morning News — and Gannett, which is experimenting with paywalls at a number of its papers, and says it plans to roll the strategy out to other publications — is that a paywall can do two things: one is to keep existing print readers from cancelling their subscriptions so they can read for free online, and the second is to generate more revenue, not just from subscriptions but by convincing advertisers that readers who pay for their content are more desirable as targets of advertising.

This is the argument being made by News Corp., (s nws) which launched paywalls at two of its British newspapers late last year, and saw its online readership plummet by more than 90 percent. The company has said that it isn’t concerned about the decline, and that advertisers are proving to be receptive to its claims that the remaining readers are more engaged and therefore worth more. What impact that will have on the company’s actual finances remains to be seen, however.

The New York Times, meanwhile, is expected to launch its “metered access” plan soon, which is based on a similar model used by the Financial Times that provides a certain number of free articles per month before readers hit a wall. The NYT has experimented with a paywall before — in 2005 it launched TimesSelect, which put the paper’s columnists behind a wall, but the service (which former Guardian digital head Emily Bell credits with helping to jump-start The Huffington Post) was shut down in 2007. And some financial analysts are skeptical that the new wall will be any better in terms of helping the paper’s business: William Bird of Lazard Capital recently rated the stock a “sell,” saying it was like buying “a declining annuity,” and that the paywall was unlikely to help.

The reality is that the biggest problem for traditional newspaper companies — a combination of high costs and falling ad revenues — isn’t something a paywall is going to help solve. At best, it is a stop-gap measure that might slow their decline, and an ultimately futile attempt to reimpose scarcity on their content in an age when the supply of free content is virtually unlimited.

There’s been a lot of discussion recently about Facebook-powered comments, which have been implemented at a number of major blogs and publishers (including here at GigaOM) over the past couple of weeks. Supporters argue that using Facebook comments cuts down on “trolling” and other forms of bad behavior, because it forces people to use their real names instead of hiding behind a pseudonym, while critics say it gives the social network too much power. But the reality is that when it comes to improving blog comments, anonymity really isn’t the issue — the biggest single factor that determines whether they are any good is whether the authors of a blog take part in them.

According to TechCrunch’s MG Siegler, the addition of Facebook comments seems to have improved the quality of the comments that the blog receives, but has reduced the overall number of them, which he says may or may not be a good thing — since some people may be declining to comment via Facebook as a result of concerns about their privacy, etc. A bigger issue, says entrepreneur Steve Cheney, is that using Facebook as an identity system for things like blog comments forces users to homogenize their identity to some extent, and thus removes some of the authenticity of online communication.

Although Cheney’s argument caused Robert Scoble to go ballistic about the virtues of real names online, Harry McCracken at Technologizer had similar concerns about the impact that Facebook comments might have, saying it could result in comments that are “more hospitable, but also less interesting.” And social-business consultant Stowe Boyd is also worried that implementing Facebook’s comments is a continuation of the “strip-malling of the web,” and that

Facebook personalizes in the most trivial of ways, like the Starbucks barristas writing your name on the cup, but they totally miss the deeper stata of our sociality. But they don’t care: they are selling us, not helping us.

There’s no question that for some people, having to put their real name on everything they do online simply isn’t going to work, because they feel uncomfortable blending their personal lives with their professional lives, or vice versa. Those people will likely never use Facebook comments, and that is a real deterrent to hitching your wagon to Facebook entirely.

But the biggest reason not to rest all of your hopes on Facebook comments is that Facebook logins are not a cure for bad comments, real names or no real names. The only cure is something that takes a lot more effort than implementing a plugin, and that is being active in those comments — in other words, actually becoming part of an ongoing conversation with your readers, even if what they say happens to be negative in some cases. This is a point that Matt Thompson of National Public Radio made in a blog post, in which he talked about the ways to improve the quality of comments:

Whether online or offline, people act out the most when they don’t see anyone in charge. Next time you see dreck being slung in the bowels of a news story comment thread, see if you can detect whether anyone from the news organization is jumping in and setting the tone.

As Thompson notes, the standard defense for not doing this is a lack of time, and responding to reader comments definitely takes time. But it’s something that we feel strongly about here at GigaOM, and it’s something that we are determined to do, to the best of our ability — regardless of whether it is through our regular comment feature, or through the Facebook plugin. In the end, it’s not the tool that matters, it’s the connection that it allows.

According to multiple news reports this morning, AOL has agreed to acquire hyper-local news aggregator Outside.in for a sum that is reported to be less than $10 million, substantially below the $14.4 million that the company has raised from venture funds and other sources. After four years of trying, the service has more or less failed to become much more than a local aggregator, pulling in automated feeds of news, blogs and keyword searches based on location.

There is a business in doing this, but not a very big one — and that’s because simply aggregating data isn’t going to produce enough traffic or engagement to get advertisers interested. As Marshall Kirkpatrick notes, the field is littered with hyper-local experiments that have not really succeeded. Why? I think it’s because many of these, including Outside.in, focus too much on the how of hyper-local — the automated feeds and the aggregation of news sources, which sites like Everyblock (which was bought by MSNBC in 2009) and Topix do with algorithms based on location — rather than the why. And the why is simple: to serve a community. Unless a site or service can do that, it will almost certainly fail.

So how do you do that? The most successful community news operations — like a startup called Sacramento Press, which continues to grow rapidly despite the presence of a traditional newspaper competitor in the McClatchy paper The Sacramento Bee, or a Danish newspaper project called JydskeVestkysten, which has thousands of community-based correspondents who submit content for a series of hyper-local sites — come from the communities that they serve. They aren’t data aggregators that are imposed on those towns and regions by some external source, but come from within them.

The easiest way to see whether a hyper-local site is working or not is to look at the comments. Are there heated discussions going on in the comments on stories? If not, then the site is likely to be a ghost town. History is filled with local news experiments like Backfence — which was founded by former Washington Post staffer Mark Potts and shut down in 2007 — and Dan Gillmor’s Bayosphere, which never really managed to connect with the communities they were supposed to be serving, despite all the best intentions. Among the startups trying to take a community-first approach is OpenFile, a kind of pro-am local journalism startup based in Toronto.

In the comments at Read/Write Web, the founder of Everyblock, programmer and entrepreneur Adrian Holovaty, said that his service is trying to add more community to its sites by focusing on comments and discussion around the issues — and that’s a good thing, because without it, there is nothing but a collection of automated data, and no one is going to form a strong relationship with that.

Topix, which says it is one of the largest local news services on the web, started out doing the news aggregation thing just like Outside.in and Everyblock, co-founder Chris Tolles said recently in an interview with me, and then almost accidentally started to become a community hub for lots of small towns and regions that didn’t have anywhere else to talk about the issues. Topix has focused on expanding those kinds of discussions, by targeting local hubs with features such as election-based polls during the recent mid-term elections, in order to spark more debate and engagement.

This is the central challenge for AOL and its Patch.com effort, which has already spent over $50 million launching hyper-local news operations in almost a thousand cities across the United States. The sites are designed to be one-man or one-woman units, with a local journalist (in many cases, one that came from a traditional media outlet) as the core of the operation, writing local news but also pulling in other local content from blogs, government sources and elsewhere. And most of the sites highlight the comments from readers prominently, which is smart.

But can this massive, manufacturing-style effort from a web behemoth manage to connect with enough towns on a grassroots level and really become a core part of those communities? Because without that, AOL is pouring money into a bottomless pit.

The secret to online success for newspapers doesn’t depend on the choice of technology, or decisions about content, or even specific kinds of knowledge about the web, says Emily Bell — the director of the Tow Center for Digital Journalism at Columbia University, and the former head of digital for The Guardian. All it requires, she says, is a firm commitment to be “of the web, not just on the web.” Speaking at a journalism event in Toronto last night, Bell said the biggest single factor in the success that The Guardian had online was the determination to be part of the web, and to embrace even the controversial aspects of the online content game — including user-generated content and the use of tools to track readers and traffic. “Its useful to have the digital skills,” she said, “but more important to have a digital mindset.”

One of the most controversial things The Guardian did early on, according to Bell, was to launch the Huffington Post-style Comment Is Free platform in 2006, which allowed anyone to submit opinion or commentary pieces and have their blog posts run alongside the traditional columnists employed by the paper.

It was this last part of the project that really caused a furor within The Guardian, said Bell, because the traditional columnists didn’t want their pearls of wisdom to be appearing alongside the rantings of non-journalists, and they expressed their displeasure in no uncertain terms to Guardian editor-in-chief Alan Rusbridger. To his credit, Bell says the editor stood firm.

Bell also noted that one of the big factors in the rise of The Huffington Post was the New York Times‘ (s nyt) decision to put all of its columnists behind a pay wall, which it did in 2005. The wall was dismantled in 2007, but while it was in effect it locked the NYT’s opinion leaders away from the web, and effectively removed them from the discussion stream — which created a perfect opportunity for Arianna Huffington, and helped her build a business that AOL just acquired for $315 million (s aol). It remains to be seen what kind of impact the NYT’s new “metered” pay wall will have once it launches, which is expected to happen soon.

Bell said one of the mistakes most newspapers made was to not pay close enough attention to the technology side of the online content business, and to ignore the obvious impact of social networks such as Twitter and Facebook. Bell said she met with Google (s goog) executives in 2004, and they warned that the traditional media industry was out of touch with what readers and advertisers wanted. But newspaper executives thought “that was just about search, and that wasn’t our business — but the more I thought about it, the more I thought it was our business.” The same thing happened with the rise of social media, she says: “People thought, oh that’s not our business — but it was.”

The former Guardian executive said that using tools to track what readers click on doesn’t mean that “we will all just write about Britney Spears without her clothes on,” but simply means that journalists can keep an eye on what people are interested in reading about. The idea that paying attention to such metrics is somehow undercutting journalism is “just plain wrong,” she said. Bell also noted that newspapers have seen the digital side of their business as the risky part, when the reality is that the legacy print operations are actually more risky. “Even if you don’t know what is going to happen in your legacy business, you know what is happening now — you are losing money,” she said.

When asked during the Q&A session about how newspapers should blend their traditional newsrooms with their new digital operations, Bell said that “the jury is still out” on whether merging newsrooms is a good idea. But she said one thing was clear: that having traditional print editors telling digital staff what to do was “a recipe for disaster.” A number of newspapers that have merged their newsrooms — including the Washington Post (s wpo), which used to have its print and online operations in two completely separate buildings, with separate management — have suffered after the merger because, as journalism professor Jay Rosen and others have pointed out, the “print guys won.”

Bell’s views on who should be driving the innovation at newspapers echo those of publisher John Paton, CEO of the Journal-Register Co., which owns a chain of regional daily and weekly papers in New Jersey and Connecticut. In a digital manifesto he wrote for the company last year, Paton said that newspapers need to “be digital first,” and that the best way to do that is to “put the digital guys in charge of everything.”

The writing has been on the wall for some time in the book publishing business: platforms like Amazon’s Kindle (s amzn) and the iPad (s aapl) have caused an explosion of e-book publishing that is continuing to disrupt the industry on a whole series of levels, as Om has written about in the past. And evidence continues to accumulate that e-books are not just something established authors with an existing brand can make use of, but are also becoming a real alternative to traditional book contracts for emerging authors as well — and that should serve as a massive wake-up call for publishers.

The latest piece of evidence is the story of independent author Amanda Hocking, a 26-year-old who lives in Minnesota and writes fantasy-themed fiction for younger readers. Unlike some established authors such as J.K. Konrath, who have done well with traditional publishing deals before moving into self-publishing their own e-books, Hocking has never had a traditional publishing deal — and yet, she has sold almost one million copies of the nine e-books she has written, and her latest book appears to be selling at the rate of 100,000 copies a month.

It’s true that the prices Hocking charges for these books are small — in some cases only 99 cents, depending on the book — but the key part of the deal is that she (and any other author or publisher who works with Amazon or Apple) gets to keep 70 percent of the revenue from those sales. That’s a dramatic contrast to traditional book-publishing deals, in which the publisher keeps the majority of the money and the author typically gets 20 percent or even less. If you sell a million copies of your books and you keep 70 percent of that revenue, that is still a significant chunk of change, even if each book sells for 99 cents.

(Update: As a number of commenters have noted, only books that are priced at $2.99 or higher are eligible for Amazon’s 70-percent royalty rate; books priced cheaper than that are eligible for a 35-percent royalty rate).

The overwhelming appeal of that kind of mathematics has other authors moving away from traditional publishing deals as well, including Terrill Lee Lankford, who wrote recently about how he turned down a deal with a major publisher in the middle of negotiations over a new book because the publisher wanted him to agree to a deal for a future e-book that would have given the publishing house 75 percent of the revenue — and tried to entice him with a hefty advance for the original book. But the author said no to both deals, saying:

I see it as a permanent 75% tax on a piece of work that generates income with almost no expense after the initial development and setup charges.

Just as the music industry did, many book publishers seem to be clinging to their traditional business models, despite mounting evidence that the entire structure of the industry is being dismantled, and the playing field is being leveled between authors and publishers. And it’s not just individual authors who are taking advantage of this growing trend — author and marketing consultant Seth Godin has created something called The Domino Project in partnership with Amazon, which he sees as a new kind of publishing middleman that can help authors take advantage of the e-book wave. More traditional publishers should be paying attention, or they will find their lunch is being eaten.

Making the transition from traditional print publishing to being digital-first media outlets hasn’t been easy for newspapers — in fact, many have stubbornly resisted this change, and tried to dip their toes into digital waters gradually without really investing any substantial effort or resources. As media analyst Frederic Filloux pointed out in a post yesterday at The Monday Note, this strategy (or lack of a strategy) is turning out to be a slow-motion train wreck. As author Clayton Christensen described in The Innovator’s Dilemma, it is almost impossible to cope with market disruption by making incremental changes, and newspapers are a perfect example of that principle at work.

Filloux uses financial results from the Washington Post to make the point. In some ways, the newspaper company is better off than a lot of other media entities, because it generates a lot of revenue from its educational arm — more than 60 percent of the company’s revenue comes from it, as well as 60 percent of its operating income — which creates a nice cushion for its newspaper business. And that business needs the help, Filloux notes, because advertising revenue for the paper side of the business has continued to decline at a rapid rate, and even though online revenue has grown, it hasn’t even come close to making up the gap. This is the “digital pennies in exchange for print dollars” problem:

As Filloux points out, the math in this graph is not pleasant: over the last seven years, the Washington Post has lost five dollars in print revenue for every dollar that it has added in the form of online ad revenue — losing almost $90 million in print revenue while its online business has grown by less than $20 million. Other newspapers may have somewhat different numbers, but the trend is likely to be very similar. And Filloux correctly diagnoses the main reasons for this online-revenue problem:

• Too much free content, which has diluted the value of editorial brands like the Washington Post

• The rise of competitors such as The Huffington Post, who have taken advantage of digital technology to build audiences at much lower cost

• The downward pressure on ad prices created by the explosion of content, as billions of pageviews depress the market for banner ads

The big problem for newspaper companies is that incremental change is not really helping them adapt, or as Filloux puts it: “mere adaptive tactics won’t save the traditional news industry in their multi-front war against disruptive technologies.” The Washington Post has done as good a job as any paper of trying to build a business online — online revenue accounts for 43 percent of overall revenue, up from just 10 percent in 2004, according to the figures that Filloux quotes — but overall its business continues to decline because online ads are worth so much less than their print counterparts.

There are no signs that this is going to change any time soon — if anything, online advertising just keeps getting cheaper (newspaper companies are forming private ad networks, but this seems both too little and too late). Newspapers are fighting the law of diminishing returns.

So what is to be done? Many companies, including Rupert Murdoch’s News Corp. and the New York Times, are trying to fight a rear-guard action by putting up paywalls to protect some of their print revenue, or pinning their hopes on iPad apps and subscription revenue, despite Apple’s 30-percent fee for doing business on its platform. Filloux argues that “radical re-engineering is needed,” and I think he is right — print may still be producing a large proportion of revenues, but it is also the source of a large proportion of a media company’s costs, and that spells doom if you are competing with digital-only outlets such as AOL and Yahoo that have a dramatically cheaper business model.

The radical restructuring that Filloux describes, which involves a much smaller newsroom, lower costs and a digital-first approach to publishing, sounds very much like what Journal-Register CEO John Paton is trying to do with the company he took over last year after it emerged from bankruptcy. Above all, Paton says, media outlets need to become digital-first, because the print side is only dragging down their businesses and preventing them from being as competitive as they should be. So far, that’s a message too few traditional newspaper publishers have heard.

News360 says it does recommendations using semantic filtering, etc. Washington Post launched a service called Trove that it says will do the same — and aspects of recommendations appear in other apps as well, and the New York Times recently launched its own recommendation service attached to its site.

These kinds of recommendations are largely algorithmic, although the NYT has some behavioral info that comes from anyone who has joined its internal Times social network, which was an interesting experiment — although it’s not clear how many people are actually using it.

The biggest source of recommendation-type data is Facebook, which can see when your friends like something, when they share it, etc. Huffington Post has driven a lot of traffic to the site by making smart use of Facebook integration to recommend stories that people you follow have liked or read, and as I keep pointing out to people — social networks like Twitter and Facebook are inherently farther ahead when it comes to recommendations because of all the social signals that are embedded in my social graph, the relationships with the people I follow and my friends and social network.

There are some services and apps that make use of the links that get passed around via Twitter and Facebook — there’s Twitter Tim.es and Paper.li for showing you links from your Twitter stream, and PostPost does something similar for Facebook links, producing a kind of personalized newspaper. News.me, the social news platform that Betaworks and the New York Times have partnered on, shows you content from other peoples’ streams, which is an interesting twist. And Flipboard pulls in links from your Twitter stream, your Facebook graph and RSS feeds and shows it to you.

But no one is really doing much when it comes to recommendations. I’ve tried playing with Trove, and it is not much better than a random sampling of news that is being shared on the web — and News360 seems equally haphazard. It’s possible that they could get better over time, of course, although there doesn’t seem to be any way to tell either service that it is wrong when it suggests a particular story to you. And News360’s choice of a visual interface with photos sliding by is interesting, but I’m not convinced it’s particularly useful.

Others are trying to solve the recommendation conundrum, including **, which former ** is developing as a kind of **. But so far, if you want recommendations about what to read, your Twitter stream and Facebook graph are probably the best solution — and anyone who wants to do better is going to have to leverage both of them to do it.

Blogging is on the decline, according to a New York Times story published this weekend — citing research from the Pew Center’s Internet and American Life Project — and it is declining particularly among young people, who are using social networks such as Twitter instead. Pretty straightforward, right? Except that the actual story said something quite different: even according to the figures used by the New York Times itself, blogging activity is actually increasing, not decreasing. And as the story points out, plenty of young people are still blogging via the Tumblr platform, even though they may not think of it as “blogging.”

The NYT story notes that blogging among those aged 12 to 17 fell by half between 2006 and 2009 according to the Pew report, but among 18 to 33-year-olds it only dropped by two percentage points in 2010 from two years earlier — which isn’t exactly a huge decline. And among 34 to 45-year-olds, blogging activity rose by six percentage points. The story also admits that the Blogger platform, which is owned by Google, had fewer unique visitors in the U.S. in December than it had a year earlier (a 2-percent decline), but globally its traffic climbed by 9 percent to 323 million.

In many ways, this “blogging is dying” theory is similar s to the “web is dying” argument that Wired magazine tried to float last year, which really was about the web evolving and expanding into different areas. It’s true that Facebook and Twitter have led many away from blogging because they are so fast and easy to use, but they have also both helped to reinforce blogging in many ways.

What’s really happening, as Toni Schneider of Automattic — the corporate parent of the WordPress publishing platform (see disclosure below) — noted in the NYT piece, is that what blogging represented even four or five years ago has evolved into much more of a continuum of publishing. People post content on their blogs, or their “Tumblrs,” and then share links to it via Twitter and Facebook; or they may post thoughts via social networks and then collect those thoughts into a longer post on a blog. Blog networks such as The Huffington Post get a lot of attention, but plenty of individuals are still making use of the long-form publishing that blogs allow, including programming guru Dave Winer, and Hunch founder and angel investor Chris Dixon.

One of the reasons why Tumblr seems to have taken off, particularly with younger users, is that it is extremely easy to set up and use — but it also offers many of the same real-time sharing options that have become popular with Twitter and Facebook. For example, Tumblr makes it easy for users to follow others, and then with a simple click they can “re-blog” another user’s post, which redistributes it to all their followers in much the same way that a “retweet” does on Twitter.

So what we really have now is a multitude of publishing tools: there are the “micro-blogging” ones like Twitter, then there are those that allow for more interaction and longer-form or multimedia content like Facebook, and both of those in turn can enhance existing blogging tools like WordPress and Blogger. And then there is Tumblr, which is like a combination of multiple tools. Not every platform is going to appeal to every user, but the fact that there are multiple methods available means there is even more opportunity for people to find a publishing method they like.

So while “blogging” may be on the decline, personal publishing has arguably never been healthier.

Disclosure: Automattic, the maker of WordPress.com, is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Did you hear that noise? It sounded like a cannon shot. And it was: a cannon shot fired from Twitter headquarters, directly across the bow of UberMedia — and, by extension, across the bow of every third-party developer whose app competes in some way with the micro-blogging service. With little or no warning, Twitter flipped the “kill switch” and shut down several of UberMedia’s apps on Friday afternoon, including UberTwitter and the popular Android app Twidroyd.

Twitter says the reasons were simple: trademark infringement and breaches of the terms of service. But there is more to this than just a squabble over usage, and Twitter’s heavy-handed behavior is drawing some fire even from the company’s supporters.

The first notice that anyone had of serious issues between Twitter and UberMedia came when users suddenly couldn’t access the network through UberTwitter and Twidroyd. Shortly afterward, a blog post appeared on the Twitter support blog saying that the apps had been shut down for “violating our policies” — but even that explanation only came after a description of the “official” clients for Twitter (with some helpful links to them) and a generic-sounding statement about how the company asks applications “to abide by a simple set of rules that we believe are in the interests of our users, and the health and vitality of the Twitter platform as a whole.”

The blog post didn’t even describe what the actual violations by UberTwitter and Twidroyd were — those details didn’t come out until someone posted a question on the Q&A site Quora about the shutdowns, which drew a comment from Twitter communications staffer Matt Graves that included the statement the company sent to the media. According to the statement:

The violations include, but aren’t limited to, a privacy issue with private Direct Messages longer than 140 characters, trademark infringement, and changing the content of users’ Tweets in order to make money.

Graves said the company had “had conversations” with UberMedia about some of the violations since April 2010, including the use of terms such as “tweet” and “twitter” in product names, and that the company hoped “that they will bring the suspended applications into compliance with our policies soon.” Meanwhile, UberMedia founder Bill Gross was busy doing damage control, posting on Twitter that the company was making changes to bring its applications into compliance (including changing the name of UberTwitter to UberSocial, something he said had been in the works for some time) and issuing a news release with the details.

I wrote recently about the potential for a serious collision between UberMedia and Twitter — based on Gross’s accumulation of Twitter clients, his attempts to launch a competing advertising product, and a recent financing that saw a series of venture funds put $17.5 million into the company — and this seems an obvious signal that Twitter is not going to take UberMedia’s potential competitive threat lying down.

If it had wanted to handle things quietly, Twitter could easily have negotiated something with UberMedia via back-room diplomacy. Instead, it clearly decided to send a message, both to UberMedia and to other third-party developers: Namely, don’t step out of line.

Obviously, Twitter has the right to manage its network and provide access to whoever it wishes. But the heavy-handed way in which it terminated UberMedia’s apps drew criticism even from some of the company’s supporters, including venture investor Mark Suster. A partner with GRP Partners, Suster — who doesn’t have a stake in either Twitter or UberMedia — wrote a sharply critical Quora note and a somewhat friendlier blog post about the incident, saying he didn’t appreciate being cannon fodder in the war between Twitter and one of its third-party app developers. Angel investor Dave McClure, meanwhile, yanked the company’s chain with a tweet about the company not having to worry about any Google-style “don’t be evil” mantra.

When Twitter started buying up applications and clamping down on third-party apps last year, it was obvious that the company was no longer the free-wheeling, “everyone join the party” kind of operation it seemed to be in the early days, when third-party apps were seen as partners who could help the network grow and no one worried about things like trademark infringement (TwitPic and Tweetmeme and other apps and services continue to function without any problems — so far). But the no-holds-barred attack on UberMedia suggests that Twitter is even more willing to throw its weight around now, especially since there is a potential $10-billion valuation on the line. No more Mr. Nice Guy.