Econometa

The mentions of “Vickrey auctions” I recently came across via various posts led me to this paper. It’s a really interesting analysis concerning how bidding works on search engines like Google and Yahoo, and how advertisers are not necessarily being as well-served as they could be. It also clarifies what “Vickrey” means, and how it was mis-used by both Google and the NYT article referenced in the posts above.

A much-touted feature of many search engine keyword auctions is that even if you bid a high amount on a given keyword, the amount you actually pay is reduced to be a penny above the next-lowest bid. This tends to make advertisers feel safer in placing bids that are high; in fact, at least in the past, it was argued that it made the best approach “truth-telling,” or bidding the actual value of what the ad is worth to you. Google even justified this by referring to Nobel prize winning research showing that such truth-telling had been proven to be the optimal strategy for bidders in “Vickrey auctions” like Google’s.

Truthful bidding is simple and easy for advertisers, and by discouraging low-bidding, tends to increase search engine revenues; so it would seem everyone wins. But as the paper shows, the system used by search engines isn’t a true Vickrey mechanism and does *not* lead to truth-telling being the optimal strategy. In fact, the paper shows that truth-telling under this system always leads to higher prices for advertisers than in a true Vickrey system.

Here’s a quick summary of the paper. If there were only one position in sponsored search listings, reducing the top bidder’s price to be essentially equal to the next highest bidder would be a “second price” auction. In reality there are multiple positions, with different values in terms of CTRs. In this case the obvious generalization, which Google and Yahoo use, is to reduce each winning bidder’s price to the next-highest bidder; this is called “generalized second price” or GSP. Vickrey-Clarke-Groves (VCG) is a more complicated system than this. VCG reduces each winning bidder’s price to be equal to the value lost to all lower winning bidders by being knocked down a position. As the paper shows, this price is always less than under GSP, and leads to truth-telling being the optimal strategy.

So a true VCG auction would better serve advertisers by really making truth-telling the best strategy and by lowering prices paid. But it would probably reduce search engine revenues as compared to the current GSP approach, and would eliminate the systems used by some sophisticated advertisers to constantly try to game GSP, which is inherently unstable and presents fleeting opportunities for gain via low-bidding. The paper points out that Ask and MSN, having less invested into GSP, have an opportunity to attain a comparative advantage over Google and Yahoo by using VCG to reduce these inefficiencies.

All this reminds me of another inefficiency in the online ad market: the high CPCs currently being paid for contextual placements due to mixing search keyword bids with contextual keyword bids. Google’s AdSense is certainly the most relevant example here. In the same way that advertisers are encouraged to bid their true value even though it might not be the most effective strategy, advertisers are also encouraged to extend their keyword bids to contextually targeted ads, even though such ads are known to have lower value than search ads. In fact, although it’s possible to enter different bids for contextually targeted ads, or to skip them altogether, the default behavior is for keyword bids to apply to all ads placed by Google anywhere; and defaults matter.

The result is that for publishers, AdSense has two enormous advantages over other ad networks:

(1) Every search advertiser on Google is by default an advertiser under AdSense. This automatically provides a large and varied pool of advertisers.

(2) The CPC paid by advertisers is by default set by the value of an audience who is actively searching for the keywords bid on. This leads to much higher prices than those actually based on the value of an audience who is just reading text containing the keywords.

In general, these advantages come at the expense of advertisers; it seems to me that this situation presents another great opportunity for competitors like Ask and MSN to differentiate themselves.

An interesting question is raised here: assuming conversion rates for contextual clicks are lower than those for search clicks, automatic inclusion in AdSense must lower the overall conversion rates for advertisers on Google. A recent report shows that indeed, Google comes in dead last among major search engines when it comes to conversion rates. The report attributes this to demographics, but maybe AdSense has just as much to do with it.

The real question, though, is this: have lower conversion rates led to lower keyword bids at Google as compared to other search engines? If so, this represents an effective transfer of money from search profits, where Google is already dominant, to buying market share in the placing of ads on other properties, where Google clearly plans to become dominant. As with the GSP/VCG issue, it’s impossible to say how much of this is accidental and how much is deliberate; but in the end, it amounts to both an impressively effective strategy for Google and a significant opportunity for competitors as these inefficiencies become understood and are wrung from the market.

Update: Ed Clarke, the Clarke in VCG, shows up in the comments! Blogs are amazing.

A (separate) emailed comment notes that Google’s “smart pricing” automatically lowers bids for contextual ad space using an algorithm that is supposed to take into account its lower value. I hadn’t been able to find any mention of this on the Google site, but searching more broadly, I found this and this: looks like this has been around since ‘04. Thanks for the correction, and sorry I missed this!

The thing is, it makes me wonder even more: if “smart pricing” is really taking ROI into account accurately, why are overall conversion rates so low on Google? Maybe it really is demographics…or maybe the algorithm still results in higher than justified CPCs paid. It’s hard to know for sure, since the alorithm is not public, but two facts seem to remain: compared to other search engines, Google conversion rates are low; and compared to other contextual ad networks, Google CPCs paid are high.

This entry was posted on Saturday, March 4th, 2006 at 3:32 pm and is filed under Economics, Marketing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.