Post-Gates: Microsoft, Competition and the Semantic Web

Microsoft has made many acquisitions for many reasons over its history - 122 to date, according to the list maintained at the Wikipedia.  Almost 100 of these have been consummated in the last decade, as the company that triumphed in operating system and office productivity software has sought (often unsuccessfully) to achieve similar success in other domains.  Other purchases have demonstrated pragmatic "build versus buy" decisions, serving to add functionalities to products that needed them more quickly and efficiently than in house efforts could achieve.

In its earlier days, Microsoft was much more likely to mimic the products of other companies rather than buy them, in part reflecting its engineering-driven culture, and in part its hardball approach to competition.  When it did add features this way, it invariably added them for free into its existing products to make them more desirable.  The result was often to drive the originators of those features out of the marketplace, since who would buy what they could get for free?  Sometimes, the motivation was more desperate, as with the crash development, and bundling, of Internet Explorer in Window, when Netscape threatened to open a critical breach in Microsoft's control personal computing.

If that sounds vaguely familiar, it should, since Google is following the same course, albeit in a kinder, gentler way, as it adds service upon service, all for free, and all in the service of racking up more and more ad revenues.  That's disturbing, because when your goal is ad revenues and not great technology, you may not necessarily produce great technology.  But as Google's dominance continues to grow, who will be able to credibly compete against it in those technologies, to ensure that innovation continues?

That’s a troubling question, and the example of Microsoft in the past does little to allay such fears, in part because there are good things as well as bad that cam come from such dominance.  After all, the overall result of Microsoft’s success was a far cheaper, feature-rich, and seamless desktop than would have been likely to have come into being so quickly by any other means.  And, of course, the extinction of competition in a host of areas that might have developed in far more interesting directions, had there been a remaining economic incentive for others to push their envelopes.  Antitrust regulators in the US (until George W. Bush took office) and in Europe (on an ongoing basis) have taken due note, and brought a series of actions and investigations that have not, to date, taken a meaningful toll on the software giant economically, or served to dramatically restrict its ability to act in the marketplace.

With the retirement of Bill Gates, this yin and yang has naturally been much discussed.  Essentially, the debate boils downs to the question of whether the strategy of Bill Gates and, to a lesser extent, Steve Ballmer, has been a Good Thing or a Bad Thing for the personal computer industry, and for those that use PCs.  Predictably, most commentators have straddled this fence.

Recently, of course, Microsoft is finding itself more challenged than it has ever been in the past – as demonstrated in part by that same accelerating list of acquisitions: it completed only 4 acquisitions in each of the years 2001 through 2004, and then 12 in 2005, 19 in 2006 and 15 in 2007.  Notwithstanding the distractions of its (thus far) failed attempt to combine with Yahoo, it has already completed 9 acquisitions this year.

Reviewing that list of acquisitions reveals much about how reality has been changing for Microsoft in the last few years.  And the most recent transaction – the acquisition of Powerset on the Monday following the Friday upon which Gates retired – provides what I think is a fitting indication of how much has changed for Microsoft, what the state of competition in the marketplace will be like going forward, and how Microsoft’s new less-powerful status in the marketplace may even serve sometimes to help restore, rather than extinguish, innovation and competition in the marketplace.

Why?  Because when it comes to the Web, the new Microsoft appears to be Google.  Not, I hasten to say, tactically – no one has accused Google of engaging in the same tactics to achieve its level of dominance in search that helped Microsoft to conquer the desktop.  But from a position of influence, Google’s 60% marketshare in search, and the strategy that it is employing, could have the same impact on innovation that Microsoft’s dominance in operating systems achieved in the past.

Witness, for example, the profusion of seemingly unrelated – and free – services that Google provides.  Google Alerts, those handy Web searches that you can set up in a few seconds have presumably wiped out the "clipping service" industry.  Why pay thousands of dollars a year for something that is free?  Or email – note how the number of gmail addresses in your inbox continues to swell.  Or RSS syndication.  The list goes on, of pretty good service after pretty good service that may not be totally to your liking – but hey – it’s free, easy to set up, and easy to learn. 

That’s a tough value proposition to deal with, and no one knows this better than Microsoft, which played the same game so effectively in the past.  And even though Google is pledged to "do no evil," that does not equate to a commitment to "do ultimate good."  Just as with features that Microsoft added to Windows and Office for free, what of these free services from Google?  Once enough people are using a service to achieve the goal of providing it, why continue to upgrade it?  Where’s the business case for that, when greater investment will reap no greater reward for shareholders?  Hence, even if sins of commission are inconsistent with the corporate ethos, sins of relative omission may be compelled by simple common sense.

In such a situation, Microsoft’s still powerful position can be an instrument to maintain competition in the marketplace, since few other players exist that are yet capable of challenging Google in its core areas of competence.  Sometimes, a Microsoft decision – such as the purchase of Powerset – can be doubly influential, by helping propel a technology over the finish line that might never make it otherwise, due to the very substantial investment in time, effort and technical commitment that would be required.

And that brings us to the final significant part of the title to this blog entry, which is the Semantic Web, a topic that I’ve been covering in detail ever since I conducted the longest, and still one of the most read, interviews of Semantic Web visionary Tim Berners-Lee, which was the Feature Article to my June 2005 issue (titled, "The Future of the Web") of what is now called Standards Today.

The Semantic Web has always been a high stakes gamble with a high risk of failure.  Converting the world to implementing the series of standards created by the World Wide Web Consortium to enable it is a great undertaking, requiring the buy in of web developers the world over.  Moreover, the vision for the Semantic Web is not itself a revealed truth – the next generation of the Web could be approached in many ways, of which this is only one.  And even within its general conceptual framework, there are many subdecisions that could have been otherwise.

In its first phase, the reputation and commitment of Berners-Lee has been instrumental.  It is hard to imagine the project having achieved the degree of penetration it has absent his vision and leadership.  But that penetration has still only been partial, arising here and there in areas such as health sciences, where the needs and rewards are more commensurate with the demands.  True, there are other ventures, some venture backed (like Powerset) and others from interested parties in a position to benefit from making a commitment (such as Reuters, which recently created a free on-line tool to "semanticize" pages, to mutual benefit).

But this sort of implementation creep is slow, and might be ultimately successful, perhaps overtaken by other more chaotic efforts, such as social tagging.  What’s needed is an 800 pound elephant to do for the Semantic Web what global retailing monster Wal-Mart did for RFID tags – to legitimate the opportunity such that others will take notice, and conclude that its worth taking the gamble to get on the implementational bus.  Someone, as they say, needs to buy the first telephone.  Better yet when that first adopter buys a million of them – the network effect then springs full blown into existence, and the race to link up is on.

Will this happen with Microsoft and the Semantic Web?  That’s hard to tell.  Powerset apparently has a novel approach to Semantic Web technology.  Perhaps Microsoft is dabbling, rather than signaling a dramatic and broad based.  Perhaps, as suggested in the announcement at the Live Search blog at the Microsoft Web site (reproduced in full below for archival purposes), the main goal was simply to acquire "talented engineers and computational linguists in downtown San Francisco" that could generically support the existing Microsoft natural language strategy.  That and a few patents could easily justify the $100 million price tag.  For now, commentators are largely left to guess what the real strategy may be.

For my part, I’ll be hoping that this is a signal that Microsoft is making a major commitment to the Semantic Web, and will provide leadership and credibility to bringing the world on board to that effort.  Taking search to the next level is an enormous undertaking, and one that is not likely to happen without such endorsement.  And to the extent that it requires global infrastructural commitment through the adoption of standards, it is unlikely to happen without competitive pressure.

Just as innovation in so many areas of office productivity software and browsers became mired down once Microsoft came to dominate those areas, the world can’t afford to see search languish for years at a time.  Innovation thrives on competition, and Microsoft is pumped to give it to Google.

That’s a good thing, in my book – or at least it could be.  Google is a great company, but like any great company, it needs great competitors to keep it at the peak of its game.  But it takes a huge company to give a dominant player serious competition.  With Yahoo seemingly going sideways, that means that for some time it will need to be Microsoft, or no one, that is capable of rising to the challenge.

The question is how?  Go to Microsoft’s site now and you’ll be greeted by a gray screen, with an "Install Silverlight" popup.  Will that be the post-Gates approach with search as well – an effort to create yet another new Microsoft ecosystem?  Or will it be a more enlightened competitive attitude, that seeks to level a playing field through adoption of open Semantic Web standards, and compete on the merits of its solutions based on those standards?

For now, we’ll just have to wait and see whether the post-Gates world is a new one that brings back innovation and competition, or a continuance of the old one, where the old rules rule.

 

For further blog entries on the Future of the Web, click here

sign up for a free subscription to Standards Today today!

* * * * * * * * * * * * * * * * * *

Powerset joins Live Search

We’re excited to announce that we’ve reached an agreement to acquire Powerset, a San Francisco-based search and natural language company.

Powerset will join our core Search Relevance team, remaining intact in San Francisco. Powerset brings with it natural language technology that nicely complements other natural language processing technologies we have in Microsoft Research.

More importantly, Powerset brings to Live Search a set of talented engineers and computational linguists in downtown San Francisco. This is a great team with a wide range of experience from other search engines and research organizations like PARC (formerly Xerox PARC).

We’re buying Powerset first and foremost because we’re impressed with the people there. Powerset CTO and cofounder Barney Pell is a visionary and incredible evangelist. When he introduced our senior engineers to some of the most senior people at Powerset — Search engineers and computational linguists like Tim Converse, Chad Walters, Scott Prevost, Lorenzo Thione, and Ron Kaplan — we came away impressed by their smarts, their experience, their passion for search, and a shared vision.

That shared vision is to take Search to the next level by adding understanding of the intent and meaning behind the words in searches and webpages.

We know today that roughly a third of searches don’t get answered on the first search and first click. Usually searchers find the information they want eventually, but that often requires multiple searches or clicks on multiple search results. Two specific problems are the most common reasons for this:

  • Differences in phrasing or context between a user’s search and the way the same information is expressed on webpages. Search engines don’t understand today that "shrub" and "tree" are similar concepts. We don’t understand that "cancer" sometimes refers to a disease and sometimes refers to a horoscope and when a query or a webpage refers to which.
  • Lack of clarity in the descriptions for each webpage in the search results. Sometimes a result looks relevant from its short description on the results page but turns out to be not so relevant when you visit the actual page. As a result, searchers frequently click results and then rapidly click back when they realize they aren’t what they’re looking for.

These problems exist because search engines today primarily match words in a search to words on a webpage. We can solve these problems by working to understand the intent behind each search and the concepts and meaning embedded in a webpage. Doing so, we can innovate in the quality of the search results, in the flexibility with which searchers can phrase their queries, and in the search user experience. We will use knowledge extracted from webpages to improve the result descriptions and provide new tools to help customers search better.

Working with our existing Search team and other Microsoft teams that focus on natural language, Powerset will help us address all of those problems and opportunities.

We’re looking to add even more talented engineers to the San Francisco team to accelerate our shared progress. If you’re interested in joining the team, drop us a line.

We’ll have more to say about the things we’re doing in understanding searches and webpages through natural language technology in the coming months. In the meantime, please join me in welcoming Powerset to Microsoft!

Satya Nadella, Senior Vice President, Search, Portal, and Advertising

Comments (2)

  1. I think that your points are good ones, but also that you might have missed one of mine.  I don’t in any way think that Google is "hijacking" anything, but I am concerned about the understandable impact of providing free services.  So even though Google has no motivation to lessen innovation, its business model could nevertheless have that impact.

    My concern in this regard isn’t limited to Google, by the way.  I’m concerned that more and more of the economics of technology development is being driven by advertising revenues – of which there is a finite supply.  If you can get something that’s pretty good for free, how much are you willing to pay for something that is better, how much better does it have to be, and how many people need to be willing to pay the extra amount?  I’m not an economist, but it seems to be that there is a real danger of a "mediocritazation" of technology, simply because of the business models that are being adopted.

    Open source software, in contrast, actually drives improvement in software, and also becomes more feasible due to the broad sharing of the expense of development.  This will certainly help, but I think it’s probably right to assume that this wouldn’t provide a complete fix, since not every problem will attract or support a community, and community-driven processes (at least thus far) haven’t  necessarily produced the same types of results as competition between proprietary vendors – such as more consumer-friendly products.

      –  Andy

    • Those are quite valid concerns – I must apologize for having missed them in your original article. However, as I said, Google is a small player in the technology domain. Then again, "freebies" have long been a strategy to drive adoption of a product in the market. Put these two together, and we see Google (and others like Google) are simply following an established practice to gain market-share.

      On the other hand, the Open Source eco-system generates most of it’s revenues (direct revenues impacting profitability, as well as indirect revenues absorbing costs of operation) from a service-based model. So I see the vast majority of software technology in the future being funded by such service models. This model works very well for infrastructure software – i.e. OS, databases, common productivity tools, application servers – but doesn’t work for specialized software. Companies like Microsoft will not be able to sustain their current model very far into that future. In the future, most specialized software will be built by in-house teams (or consultancies) on top of a standards-based infrastructure software. This is true even now for many cases – take Banks for example (all software comprises of custom applications built on top of the J2EE software stack running on GNU/Linux blade servers – front-ends may be C#, VB or web-based clients).

      So, although there are more "freebies" being generated by media companies (like Google), they do not form the bulk of software technology as it stands today. And I see service-based models continuing to be the major driving force behind software technology – aka service-based models will have the greater impact in defining most of the software technology landscape into the foreseeable future.

      However, your concerns are quite valid most especially when considering the effect a monopoly could have on pushing policies that could have a major impact on the future of the web. Google has been benevolent so far. But any monopoly – however benevolent it seems at the time – should always be cause for concern.

Comments are closed.