MASSACHUSETTS AND OPENDOCUMENT:
A BRAVE NEW WORLD?
It should be reasonably obvious for a layperson who reflects on the concept of public records that the government must keep them independent and free forever.
||Massachusetts Secretary of Finance and Administration Erich Kriss
|Abstract: For a period of 20 months, the Information Technology Division (ITD) of Massachusetts has been considering certain amendments to its internal information technology policies relating to the use of “open formats” when saving documents created by the Massachusetts Executive Agencies. The impetus for such a change is to prevent vendor “lock in”, and also to lessen the likelihood that public information will not become inaccessible in the future due to changes in proprietary software, or the discontinuance of support for such software. On September 21, 2005, the proposed amendments became final, and Massachusetts became the first jurisdiction in the world to mandate the saving of documents using only software that complies with the OpenDocument OASIS Standard or the Adobe PDF format. This article describes the history of both the process followed by the ITD as well as that of the OpenDocument OASIS Standard, summarizes and assesses the arguments for and against the amendments made by those that offered public comments, and finally seeks to evaluate the potential impact of the Massachusetts decision on further government information technology policy evolution around the world.
Introduction: On August 29th, 2005, a long-watched process in Massachusetts quietly slipped into its final phase. On that date, the Information Technology Division (ITD) of the Commonwealth posted a new version of an information technology policy amendment on its Website, together with a statement that it would become final following a brief, eleven-day comment period. No press release was issued, and the first widely noticed note of the event appeared as a two sentence posting by Richard Waters, a Financial Times correspondent living in San Francisco, to the Financial Times Website. The second sentence described Massachusetts, with its 6,349,097 inhabitants, as "one of the most populous states in the US."
In this humble event some see the first victory of a revolution in government information technology (IT) policy that will spread throughout the world, seeking to liberate end-users from dependence on limiting proprietary solutions and the risk of abandonment, and guaranteeing that long term access to public documents will be assured. Others fear that implementing the new policy will prove to be a disastrous adventure in IT utopianism, resulting from a process that perhaps advanced too quickly.
What happens next in Massachusetts will be watched closely not just in the United States, but around the world as well, where government support for open source software supported by open standards is already much stronger. Still, Massachusetts is the first government in the world to take the step that it did on September 21st, when it adopted the policy amendments referred to above: mandating compliance with a new file format rule that would preclude those subject to the policy from saving any document after January 1, 2007 in Microsoft Word, Excel, PowerPoint, or any format of Microsoft or any other vendor that does not conform to OpenDocument 1.0, a standard newly adopted by OASIS, or as an Adobe PDF file.
How this decision came about, and what the future holds, merits study from a variety of viewpoints: Was the process by which the decision was made appropriate? Were the criteria developed by Massachusetts optimal from a technical and practical view? Was the proper balance between benefits and negative impacts found? And most critically, will Massachusetts be successful in implementing the policy?
The process by which the Commonwealth formulated the latest amendments to its IT policies provides a case study of the manner in which decisions are being made today on the cusp between the old world of proprietary systems, and a potential new open order of information and communications technology procurement. The outcome of the decisions made by Massachusetts will therefore inform the actions of other governments considering similar departures from past policy, and may therefore accelerate, or impede, the making of such decisions.
This article is based on public information available at the links embedded in this article, supplemented by live and email interviews conducted between September 20 and 26 with the following individuals, each of whom was (and is) at the center of the events in question:
- Michael Brauer, Sun Microsystems and Chair of the OASIS OpenDocument Format for Office Applications Technical Committee
- Patrick Gannon, President and CEO of OASIS
- Mary McRae, OASIS Manager of Technical Committee Administration
- Peter Quinn, CIO of the Commonwealth of Massachusetts
- Morgan Reed, Vice President of Public Affairs, Association for Competitive Technology (ACT)
- Alan Yates, Microsoft General Manager of Information Worker Business Strategy
- Jonathan Zuck, President, Association for Competitive Technology
A near-final draft of this article has been reviewed for by representatives of Microsoft, OASIS and ACT as to the accuracy of the respective statements and the positions attributed to them. [Disclosure: OASIS (the Organization for the Advancement of Structured Information Standards) is a client of the author and his law firm.]
A perennial question: Few definitions defy consensus in our modern technology world as totally as the simple phrase “open standards.” Despite the insistence of some that one definition should suffice for all purposes, it would be unnecessarily rigid and counterproductive to impose a single definition upon all situations. For example, those that wish to create open source software for release under any of the licenses approved by the Open Source Initiative could not permit usage of many of the licensing terms that would be required (and considered to be unobjectionable) by the companies involved in creating many types of standards today, or by the standards organizations within which such standards are being developed.
But definitions of “openness” underlie increasingly important procurement requirements in the public as well as the private sector. And when substantial IT budgets are involved, the nuances of such definitions inevitably become the subject of protracted and emotional debate.
Historically, that debate has been energized in part by the tension between what a vendor may gain from adoption of a standard, and what it may be required to give up if it’s intellectual property would be infringed when that standard is implemented by others. If a steady royalty stream can be earned, such a result is highly desirable to many companies. In other cases, a patent owner is happy to provide its intellectual property without cost, due to the other benefits that it expects to receive, such as the rapid development of a new and profitable market, or the increased opportunity to sell other products protected by patents. But if a company enjoys a dominant market share in a given product space, then no amount of royalties would offset the loss occasioned by the broad adoption of a standard that could destroy such a desirable market position.
Recently the degree of energy invested in such debates has risen due to the increasing quality of open source software, the development of which is enabling the acquisition of an ever-wider range of products that are license fee-free (but hardly inexpensive, when all costs of ownership are factored in), providing serious challenges to entrenched vendors and new opportunities to other providers that have waited enviously on the sidelines for many years
Policy context: This year, all of these forces converged when Massachusetts announced the latest changes to the Enterprise Technical Reference Model (ETRM) that informs the decisions of the ITD, which in turn controls the procurement decisions for the Executive Agencies of the Commonwealth. The ETRM is a living document, now instantiated in Version 3.5, as most recently updated .
Without question, the most controversial section of the amended ETRM is titled "Information Domain", and contains new rules relating to the forms in which documents may be saved beginning January 1, 2007. Those rules will permit documents to be archived using software that utilizes one of (at this time) only two formats that are deemed to be sufficiently “open” to increase the likelihood that their contents will be accessible over the long-term: OpenDocument for Office Applications 1.0, a standard developed and maintained by OASIS, and the Portable Document Format (PDF), which is owned, but freely licensed, by Adobe Systems Incorporated.
The decision to credential Adobe PDF as well as OpenDocument is noteworthy, given that Microsoft Office is not on the approved list, notwithstanding recent changes in policy and promises regarding amendment of licensing terms that have been made by the Redmond software developer, and the fact that Microsoft had earlier announced in no uncertain terms that it had no intention of conforming its productivity suite to the requirements of the OpenDocument format, regardless of the eventual policy adopted by Massachusetts
The history of OpenDocument: The format that would eventually become OpenDocument is based upon an office suite that has experienced many years of evolution and actual use. The original code was developed by a German Company called StarDivision, which began development of the product in 1994. In August of 1999, Sun Microsystems purchased the code for US $73.5 million for purposes of offering it’s own office suite in competition with Microsoft. Initially, the suite (now called StarOffice 8.0) was sold as a commercial product, but in 2000 Sun contributed most of the code to the newly organized OpenOffice.org project, to serve as the basis for an open source office suite.
Part of the vision for OpenOffice.org from inception was to create an interoperable, vendor independent and standardized file format for office applications. One key to achieving this goal was the development of the OpenOffice.org XML file format in an application-independent form, meaning that the same format could be used by other vendors as the basis for their own product offerings.
While the process of achieving this goal was begun in OpenOffice.org, the primary focus of that organization had been to produce and maintain a high-quality office suite. As a result, a group of interested companies and others proposed the formation of a Technical Committee to standardize the OpenOffice Format within OASIS, a software standards consortium whose rules and process were deemed to be compatible with the creation of a standard based upon an open source application. While OASIS was a logical home for the project from a process point of view, standardizing a file format for complete office suites was an ambitious undertaking, in contrast to the computer-to-computer, transactional processes upon which OASIS has historically focused. As a result of this early transition, the majority of the format project was accomplished within OASIS rather than OpenOffice.org.
OASIS chartered what was originally called the OpenOffice XML Format Technical Committee (TC) in December of 2002. The original members of the TC were nothing if not diverse: Arbortext, Boeing, Corel, CSW Informatics, Drake Certivo, National Archive of Australia, New York State Office of the Attorney General, Society of Biblical Literature, Sony, Stellent and Sun Microsystems. Later, other organizations joined the committee, including KOffice and IBM, each of which has created its own office suite that supports OpenDocument. Some, but not all, of the suites that support OpenDocument are based upon open source software developed by OpenOffice.org, which continues to offer an open source office suite that is committed to compliance with the OpenDocument OASIS Standard.
With the transfer of rights in the file formats to OASIS, the relationship between the OpenOffice.org code and architecture and what is now called the OpenDocument format was reversed. Given the diverse composition of the OASIS TC, the needs of end users, archivists and XML experts as well as vendors interested in productizing the format were directly represented in the development process, resulting in a balancing of interests that does not naturally exist within the development process of a single vendor that is strongly motivated by cost and time concerns as well as market considerations.
The new TC did not simply begin where OpenOffice.org left off, but instead spent more than a year analyzing the existing format in detail in order to determine what to retain, what to change, and what to add, thereby ensuring that the finally adopted standard would be vendor neutral, application independent, and as interoperable as possible. More than a year of additional work was required to take the resulting format to a third and final vote by the TC in March of 2005. The draft standard was also posted for public review during the process, and the many comments received from non-members were reviewed for merit and inclusion by the TC. The resulting OpenDocument 1.0 was approved by the full membership of OASIS two months later, becoming an OASIS Standard in May of 2005.
Most recently, OpenDocument 1.0 was submitted by OASIS in September to the Joint Technical Committee of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), a committee formed by the two “Big Is” to create IT standards, and more simply known as ISO/IEC JTC 1. Since OASIS is an international standard setting organization in its own right, the submission was made using the ISO/IEC JTC 1 “Publicly Available Specification” (PAS) approval process, rather than through a national member body of ISO/IEC JTC 1 (such as ANSI, the American National Standards Institute). The PAS process was created specifically for such purposes, in order to provide a fast-track approval option for globally adopted specifications that have been created through an open, consensus-based process, but outside of an accredited software development organization (ebXML, another OASIS Standard, was earlier approved via a similar route and become ISO 15000).
The TC that created OpenDocument will continue to operate indefinitely, and remains open to new voting and observer members. Currently, it is at work on version 2.0, and its future work plans include both the addition of new features and capabilities, as well as meeting needs of office suite users as they continue to evolve. According to Patrick Gannon, OASIS expects that the OpenDocument Format, and products that conform to it, “will be of particular interest to state and federal governments, universities, research centers, and other large international enterprises that (like Massachusetts) may have a diversity of users working with different office client applications."
The history of the Massachusetts proposal: Massachusetts, like all other states, has an extremely large and complex IT infrastructure that has grown by accretion over many years. As noted by Massachusetts CIO Peter Quinn, “We’ve got one of everything,” leading to enormous challenges of maintenance and support. In an effort to better manage this historical legacy and ensure the greatest utility at the minimum cost to taxpayers, the ITD has developed a number of policies and guidelines that are regularly updated as needs and available solutions evolve over time.
Moving towards "Open Formats:" As a part of this ongoing process, the ITD announced a brief Enterprise Open Standards Policy on January 13, 2004, based upon an earlier draft that had been released for comment the preceding year. This final version of the policy sought to resolve what some commentators had declared was confusion on the Commonwealth’s part in distinguishing open source software from open standards. In part, the policy reads as follows:
The Commonwealth must ensure that its investments in information technology result in systems that are sufficiently interoperable to meet the business requirements of its agencies and to effectively serve its constituencies. This policy addresses the importance of open standards compliance for IT investments in the Commonwealth. For the purpose of this policy, open standards is defined as follows:
Open Standards: Specifications for systems that are publicly available and are developed by an open community and affirmed by a standards body. Hypertext Markup Language (HTML) is an example of an open standard. Open standards imply that multiple vendors can compete directly based on the features and performance of their products. It also implies that the existing information technology solution is portable and that it can be removed and replaced with that of another vendor with minimal effort and without major interruption (see current version of the Enterprise Technical Reference Model).
The ETRM, in turn, provides the detailed roadmap for carrying this policy into action, and forms the cornerstone of Massachusetts’ concerted effort to transform its IT infrastructure into a lower cost, more durable, vendor-independent, cohesive platform for its operations
One year later, Kriss made a public statement titled Informal Comments on Open Formats. In these comments, he announced a new commitment to vendor-independent file formats, stating in part as follows:
We are now ready to extend the concept of Open Standards to the next stage in an informal announcement today….We will extend the definition of Open Standards to include what we will be calling Open Formats. Open Standards, as you know, are specifications for systems developed by an open community and affirmed by a standards body. An example is XML, a method of exchanging data. Open Formats are specifications for data file formats based on an underlying open standard, developed by an open community, and affirmed by a standards body; or de facto format standards controlled by other entities that are fully documented and available for public use under perpetual, royalty-free, and nondiscriminatory terms. An example is TXT text and PDF document files. We plan to formally promulgate in February 2005 an additional list of approved Open Formats. …Why do we care about formats? Electronic file formats sit at the core of concern about future access to today's public records. Simply put, the question is whether, when we look back a hundred years from now, we will be able to read the records of what we did today. It should be reasonably obvious for a lay person who reflects on the concept of public records that the government must keep them independent and free forever. It is an overriding imperative of the American democratic system that we cannot have our public documents locked up in some kind of proprietary format, perhaps unreadable in the future, or subject to a proprietary system license that restricts access.
A question of great interest to multiple constituencies thus arose: what would qualify as an open format for purposes of future Massachusetts IT acquisitions?
In the same informal remarks, Secretary of Administration and Finance Kriss indicated that the OASIS OpenDocument format would be a likely choice. He also discussed ongoing discussions with Microsoft regarding certain licensing terms that Massachusetts found to be inconsistent with its definition of open standards, stating:
[Microsoft has] made representations to us recently they are planning to modify that license, and we believe, if they do so in the way that we understand that they have spoken about (we will leave it obviously to them to describe exactly what they are going to do), it is our expectation that the next iteration of the Open Format standard will include some Microsoft proprietary formats. These formats, like DOC files, will be deemed to be Open Formats because they will no longer have restrictions on their use.
According to Alan Yates, Microsoft believed that the ITD later confirmed that the Microsoft XML Reference Schemas were acceptable and would be included in the final list of permitted formats.
Public comments: During the nine months that followed, a public meeting and many private consultations were held between members of the ITD and representatives of technology companies, industry associations, and open source advocates, among others, culminating with the announcement on August 29th that a final decision would be made on the open format issue following a brief, eleven-day comment period. Notwithstanding the short notice and window of opportunity, some 157 comments were received in letter or email form. Some of the comments received were short (typically either very positive or very negative), while others were extremely detailed.
Five entities (Adobe Systems, Inc., Corel Corporation, IBM, Sun Microsystems, and Hiser + Adelstein ( a consulting firm)) provided comments in support of the proposed amendments that are individually posted at the ITD website. The only negative comments that are separately posted are those of Microsoft. A single datafile includes all 157 comment submissions, including those that are separately posted.
Of the total of 157 submissions, 97 are best classified as endorsements of the proposed amendments, and 46 submissions are best classified as critical. The remaining 13 submissions are difficult to classify, since they were offered solely to offer constructive comments, or included both praise and criticism. Comments came from a great diversity of individuals, including from the Communications Officer of the United Nations (positive), from many states, and from as far a field as Australia, England, Italy, Portugal, and Sweden (all positive).
Of the negative comments, seven are submissions by Massachusetts public officials, and six are brief, identical form letters. The great majority of the remaining negative submissions were sent by individuals that are blind, or by representatives of organizations concerned with those with disabilities. All of the submissions by major corporations are positive, with the exception of the comment letter received from Microsoft.
13 of the submissions were sent by various types of organizations: of these, three were sent by organizations concerned with the rights of the blind (all negative), one by the Massachusetts District Attorneys Association (negative), and two by public policy associations (negative). Of the seven letters submitted by technology trade associations, however, five were laudatory (including all of the associations based in Massachusetts) and only two were negative.
A final opportunity to comment and ask questions was provided at a meeting held on September 16 under the auspices of the Massachusetts Technology Leadership Council.
On the evening of September 21, the ITD quietly posted the as-amended ETRM at its website, together with a detailed Frequently Asked Question (FAQ) page that addressed many of the questions that had been pressed most vigorously by opponents of the draft policy.
Analyzing the results: Several factors contributed to the energy level, confusion and differences of opinion that have been expressed regarding the wisdom and propriety of the decision by the ITD. First, the Commonwealth is entering into new territory, with a new standard to which additional functionalities will continue to be added. Second, the high stakes for vendors, on the one hand, and the emotion of true believers in open standards and open source, on the other hand, have not been conducive to the most objective and precise expressions of belief. And finally, the rapid end game events bracketed the Labor Day holiday weekend, therefore requiring rapid responses from commentators and the media alike.
As the dust begins to settle, it is important to make an objective assessment of the process followed by Massachusetts, and the prospects for success as it begins to implement its newly adopted plan of action. The following is an attempt to begin that process, first, by addressing one company-unique issue that has been much bandied about in the press, and then by seeking to objectively present the views of both the supporters as well as the critics of the new policy.
The pursuit of antitrust prosecution by other means? There has been much speculation that the commitment of Massachusetts to open source and open standards arises in part from the very public legal differences it has had with Microsoft in the past. In 1998, Massachusetts was one of twenty states to join the U.S. Department of Justice in a massive and prolonged antitrust suit against Microsoft. In 2001 the DOJ reached a settlement with Microsoft, in which most of the states joined, leaving Massachusetts as the only state objecting to the settlement and seeking stronger restrictions on Microsoft. Ultimately, a federal appeals court rejected Massachusetts' challenge, and in June of 2004 approved the settlement earlier negotiated on its behalf by the DOJ.
While such speculation has made for interesting news coverage and even more blog fodder, there appears to be no basis for any such link between the opinions of the Attorney General's office and those of Eric Kriss or Peter Quinn. Each person interviewed in connection with this article (including Alan Yates of Microsoft and Peter Quinn of Massachusetts), regardless of their position on the proposal, firmly refused to give any credence to this speculation.
At the same time, it would be true to state that those involved in making the decisions on behalf of the Commonwealth are strongly committed to achieving permanent freedom from "lock in" to proprietary formats. Or, in the novel words of Eric Kriss during the final public meeting held by the ITD on September 16, Massachusetts will not sacrifice its "sovereignty" over its IT infrastructure to any proprietary vendor. As a result, while the new policy is not directed at Microsoft, it will be disproportionately felt by that vendor as a result of its dominant market share in office productivity software.
The case for the amendment: Unlike corporate end users and consumers, governments have long-term document retention obligations. And while detailed rules provide that various types of information may be discarded after set periods of time in order to minimize the burden of retention, much information still remains that must be archived on a permanent basis. Over the short course of the IT age, there have already been numerous market transitions in hardware and software, resulting each time in the need for difficult and expensive conversions of data, a trend that many government planners expect to extend into the indefinite future.
The ability to adopt standardized formats that would minimize the burden of indefinite retention is therefore extremely attractive to governments of all types, and would also facilitate the assembling of software and systems that could more easily share documents and data on a current basis. If these formats are well maintained and are mandated by increasing numbers of customers, incentives will exist for multiple vendors to adopt them, resulting in broad product offerings that will drive down costs. Best of all, if the formats are instantiated in open source software, acquisition costs (if not the significant support, maintenance and other costs of ownership that users of open source software still incur) will largely be eliminated, and the cost of competing proprietary versions of the same products should fall as well.
While initial conversion to such a system would be painful, the hope would be that the pain would be a one-time event, in contrast to the prospect of ongoing conversions of equal difficulty as proprietary vendors make substantial changes to their own products or the collapse or acquisition of a vendor leads a customer to switch software entirely.
The result, post-conversion, would be the ability to maintain a lower cost, more interoperable IT infrastructure that is less susceptible to disruption and is immune over the long term from lock in by a single vendor and monopolistic price increases, and which is less at risk of abandonment as well.
The case against the amendment: Not surprisingly, Microsoft believes that the future would not be nearly so rosy. But the President and Vice President of Public Affairs of ACT also argue persuasively that there can be significant risks to adoption of the new policy.
From Microsoft's perspective, a primary question is why the concessions it offered to Massachusetts were deemed to be inadequate. As pointed out in the comment letter submitted by Alan Yates, Microsoft announced on September 19, 2004 that it would include the source code of Office 2003 in its Government Security Program. Under this program, qualifying government customers are able to gain access to both the source code for Office as well as the related XML Reference Schemas under terms of the Microsoft Government Shared License (Windows 2000, Windows XP, Windows Server (TM) 2003, and Windows CE were already included in the program). Other benefits of the program include opportunities to communicate directly with Microsoft staff regarding current and proposed software, and access to additional documentation and training opportunities.
Microsoft also included a variety of more subjective comments in its letter, questioning the financial benefits and practical problems of the new proposal, the robustness of the OpenDocument standard, the validity of permitting the use of Adobe PDF but not Microsoft Office, the potential for the policy to prevent Massachusetts from adopting new innovations, and whether undue burdens would be placed on Massachusetts residents and businesses. Finally, Microsoft questioned whether the ITD had conformed with Massachusetts regulatory requirements involving notice, public comment, due process and internal procedures.
ACT raised similar concerns in two comment letters, one dated September 8, 2005, and a second undated letter that it delivered to Massachusetts, as well as additional criticisms relating to adverse impacts that it feared the policy could have on its small and mid-size business members.
Which is right? Given the variety of points raised by various constituencies, there is no single answer to this question. Preliminary conclusions on most issues, however, can be reached on a point-by point-basis. Of particular benefit in this regard is the Frequently Asked Question (FAQ) document that was placed at the ITD site simultaneous with the posting of the final version of the amended policy. Many of the answers in that document provide additional details that may not have been unknown to various commentators, and other responses seem to indicate a retreat by the ITD from certain positions that it had earlier taken.
How burdensome will the new policy be to state government? While conversion to the new policy will still represent a significant challenge, the FAQ includes one very important statement that appears to be new: no conversion of existing documents is now contemplated – only documents created after the effective date of the policy (January 1, 2007) will be required to be saved in an approved format. For the same reason, contentions (as in the Microsoft comment letter) that conversion will "significantly change countless legacy documents that are not fully supported by the newly designated format" will also not be a factor. As clarified by the FAQ, it appears that the evolving plan is closer to a commitment that the Executive Agencies (only) will make archival copies of new documents using approved formats rather than a campaign to convert state government, and those doing business with the state, entirely to an OpenDocument environment. This will still create burdens for the Executive Agencies, however, which will need to create documents in two formats (one for internal and one for external use), or convert at least some OpenDocument Format documents into other forms for transmission beyond the Executive Agencies.
How burdensome will the new policy be to other Massachusetts governmental entities? Microsoft contended in its letter that:
[T]here would be significant, and entirely unnecessary, costs incurred by all state agencies, departments, cities, counties, and school districts to procure new software applications that support the OpenDocument format for their individual users."
This statement is not accurate. Only documents saved by the Executive Agencies of the Commonwealth must be in OpenDocument format. According to Peter Quinn the day before the announcement of formal adoption of the policy, its implementation will be "invisible" to the citizens and other public officials of Massachusetts. Presumably this means that although documents must be archived in the OpenDocument Format, they will be made available from Massachusetts websites in PDF or other formats that are already in common usage. Similarly, the FAQ document makes it clear that no documents will need to be submitted to the Executive Agencies in OpenDocument Format.
Will existing proprietary format software need to be replaced? No. Any Executive Agency employee may continue to use existing software (including Microsoft Office), provided that those document types that are currently supported by OpenDocument must be saved in that format, whether directly (if supported by the software in use) or through independent conversion, as would be necessary if the document was created using Microsoft Office.
Will limiting acquisitions to a single format stifle innovation? There are many ways to answer this question. One is to reflect that Microsoft has never adopted a number of wonderful features that WordPerfect included in its products almost twenty years ago, such as the "show codes" feature and WordPerfect's remarkably easy macro creation keystroke sequence. It is true that if increasing numbers of governments require OpenDocument Format compliance in their procurement of office productivity software, that this would tend to increase the focus of developers on developing compliant products. To the extent that the OpenDocument Format imposes limitations on their design of such products (as all standards necessarily must do), this could be considered a limitation on innovation. But it also may be agreed that competition is a powerful driver of innovation, and that limiting purchasing to the products of a single vendor that decides which features to offer can hardly be better than having a degree of enforced limitation on formats and allowing vendors to compete based on all other aspects of their compliant products. And until such time (if ever) as OpenDocument Format supported office suites overwhelm the market, there will be more than ample incentive for those that serve on the OASIS OpenDocument TC to make that format as competitive as possible in the marketplace.
Does the policy limit Massachusetts to a single proprietary format? This is the clearest example of efforts to spread FUD (fear, uncertainty and doubt) in the marketplace by opponents of the Massachusetts proposal. As earlier noted, the OpenDocument Format is not only based upon a product that has been in use in the marketplace for many years, but the OpenDocument Format is already the foundation for several other existing products, both proprietary as well as open source. Version 8.0 of StarOffice, the original package from which the OpenDocument format evolved, was released by Sun Microsystems on September 27, 2005 for use on the Windows, Solaris and Linux platform. It is also fully compliant with OpenDocument 1.0. In its press release, Sun aggressively (and certainly not coincidentally) promotes the appropriateness of its product for those needing an OpenDocument alternative to Office:
StarOffice software provides excellent compatibility with Microsoft Office. StarOffice 8 software further improves import and export of Word, Excel and PowerPoint documents, even password-protected MS Word and MS Excel files and presentations with complex animations, autoshapes and slide transitions. StarOffice 8 software also provides features that look more familiar to Microsoft Office users. The Format Paintbrush allows simple transfer of styles from one section of a document to another, and the Impress multi-pane user interface simplifies creation of high-impact presentations.
Although StarOffice includes the oldest code and is the first to claim full OpenDocument 1.0 compliance, it is hardly the only alternative that will be available to the Executive Agencies well before the conversion deadline. Other alternatives include OpenOffice, IBM Workplace, and KOffice. Although as of this writing Corel has not made an announcement that it will implement OpenDocument, it is a founding member of the OASIS TC and retains a meaningful number of government users for its office suite. It also submitted a letter of strong support to the ITD, and is assumed by some to have a compliant product under development. [Editor's Note: Corel later announced that it will support OpenDocument, but did not announce a release date for a compliant version of WordPerfect Office.]
Did the ITD violate Massachusetts regulations? The comment letters of both Microsoft and ACT review the provisions of a number of Massachusetts statues, regulations and administrative procedures in detail, questioning whether the ITD failed to follow required procedures in advancing the proposal. Peter Quinn unequivocally denies that this was the case, stating that the Massachusetts process was extensively reviewed by legal counsel. Among other reasons for concluding that compliance was not required, he stated that the changes to the ETRM do not constitute rule making subject to regulation. However, he also stated that the ITD believed that public comment was useful and appropriate independent of statutory requirements, and that a decision had therefore been made to welcome public involvement on a less formal and regimented basis.
The comments of both Microsoft and ACT are tentative on this point, either through lack of conviction or perhaps due to the lack of public information available regarding what the ITD may or may not have done by way of internal vetting of the policy. Although more than twenty months of open, public review of the policy amendments in their various drafts were provided, as well as several face to face meetings, the specific formal public notices mandated for some types of government action were not provided. Compliance with other formalities called for under other possibly applicable laws, such as consultation with the Massachusetts Information Technology Advisory Board, would have been less visible, and is therefore more difficult to evaluate.
Constraints of time and space (not to mention the still limited amount of factual data) prevent a detailed examination of these issues in this article. Nor is it likely, absent a legal challenge, that these contentions will ever be conclusively addressed. However, further detail may be found in the FAQ on the internal vetting process followed by the ITD, as well as answers to specific contentions relating to procedural requirements. It remains to be seen whether some of these responses (e.g., “There is no legal requirement that the ITD seek advice from the IT Advisory board on any IT initiatives commenced in the Executive Department; rather, a legal requirement is imposed on the Board to provide such advice to ITD”) will satisfy others in government that may be concerned with whether the ITD complied with the spirit as well as the letter of the law.
Is the exclusion of Microsoft Office and the inclusion of Adobe PDF arbitrary? This is perhaps one of the most difficult decisions by the ITD to reconcile. It is certain that the Adobe format does not meet the ITD's own original definition of "open standards" as announced in its January 13, 2004 policy statement: "Specifications for systems that are publicly available and are developed by an open community and affirmed by a standards body." In fact, Secretary Kriss had already backed away from this definition at an "Open Formats Summit" meeting with stakeholders held on June 9, 2005. The brief public notes of that meeting reflect apparent agreement that, "There is no one definition of the term "open"; rather, there is a continuum of openness." While it is not possible in many cases to tell conclusively which statements in the notes of the meeting represent agreement and which are the record of opinions expressed by private sector attendees, the notes do conclude with three "Next Steps," one of which is to "Identify a continuum of acceptable open document standards for the Commonwealth."
Once having agreed to back off the original “bright line” definition that required adoption by a standards body, it becomes more difficult to define and justify the dividing line between what is, and what is not, acceptably “open.” The FAQ answers the question as follows
While the MS XML schema is licensed under a somewhat open patent license, its license is not as open as Adobe’s copyright license for PDF. Adobe’s copyright license for the data structures, operators and written specification constituting the interchange format called the Portable Document Format or “PDF” imposes minimal legal restrictions on developers.
Suffice it to say that it remains the prerogative of a customer to define the requirements for its purchasing, and the ITD has decided that the dividing line lies somewhere between Adobe PDF and Microsoft’s XML Reference Schema for Office and the licensing conditions related to their use. But one cannot help feeling that the ITD would have benefited all concerned had it been more explicit in the FAQ response quoted above (the only guidance included in the FAQ document on this topic). Such a vague statement provides no assistance to Microsoft in resolving the issue, or for other governments that may consider adopting their own open format policies.
At the end of the day, it may be that the approval of the PDF format may have been based more on pragmatism than on consistency in applying qualifying criteria: there are available implementations (both open source and proprietary) that offer an alternative to most features of Microsoft Office, but there is no “more open” alternative to the PDF format.
Will the benefits outweigh the challenges? The comments of ACT (both public and as conveyed during this author's telephone interview with ACT's President, Jonathan Zuck, and Vice President of Public Affairs, Morgan Reed) are worthy of careful consideration. According to Jonathan Zuck, about 2/3's of its membership had an opinion on the Massachusetts proposal, and those with an opinion were mostly critical of the proposed amendments. In the case of ACT members that are systems integrators and consultants, there was concern that their services would become less valuable if large segments of the marketplace were to narrow their purchasing options to a small number of products. But there was also unhappiness over what members saw as a poorly vetted, aggressively pushed process that imposed broad-brush solutions that they believe would require "pounding square pegs into round holes."
More specifically, ACT expressed concerns that traditional standards processes may not only be slower than commercial development, but that the need to complete format upgrades before development could begin would automatically lengthen the process of innovation and implementation. Rather than providing a platform for community innovation, they also stated that they expect individual vendors will compete to add features to their product that will ultimately defeat the initial goals of the policy. And they also pointed out a mirror image concern: if Massachusetts needs a new feature or change that is inconsistent with the existing OpenDocument format and is unable to persuade OASIS to include that feature, what then?
Zuck and Reed also believe that the last-minute addition of language to the policy recognizing the needs of disabled persons in response to the dozens of protests received on that topic gives further evidence of undue speed and inadequate public review. Overall, they believe that Massachusetts will have a very difficult time making the new policy work. At the same time, they also responded, when asked, that they expect that Microsoft would in fact have little difficulty adapting Office to support the OpenDocument format, notwithstanding its protests that doing so would cause irresolvable backward compatibility problems.
One question that arises from the FAQ is whether further analysis by the ITD may be leading it to trim back the implementation of the new policy to the point where the benefits will shrink even as the burdens merely shift, or even grow. As presented, only newly created Executive Agency documents will be covered, and the FAQ recognizes that, “Implementation plans will take into account the need to maintain interoperability through the use of a variety of acceptable formats.” In other words, OpenDocument will be used to achieve the core purpose of creating a hopefully safer long-term archive, but it appears that the Executive Agencies may be operating on the basis of redundant systems (and the continuing need to pay software license fees on any remaining software that does not support the OpenDocument Format) for some time to come
Conclusions: Not surprisingly, whether Massachusetts has made an expensive mistake, or a brave, bold move that will later be recognized as a turning point in public procurement, can only be determined over time. The outcome of that determination will depend less on the degree of difficulty that the ITD encounters in making what it freely admits is a difficult transition than on the future of the OpenDocument Format. If those already offering or developing software based on that format participate fully and forthrightly in ongoing OASIS development efforts, and if systems integrators, independent software vendors and open source projects embrace the opportunity to work with those that buy and sell such products, then Massachusetts’ bet on breaking the lock-in cycle will be well rewarded.
The biggest variable in whether this will occur is whether or not other governments will follow the lead of Massachusetts and adopt the OpenDocument OASIS Standard. If they do, then it will be a customers’ rather than a vendors’ game, and many of the concerns voiced by ACT will be mooted by the self interest of the vendors themselves, leading each to compete on services, features and quality for the largest share of the government market possible. Similarly, if other governments join in- the work of the OASIS OpenDocument Technical Committee, the future course of the OpenDocument Format will stay closely attuned (as it is now) to government needs.
Either way, Massachusetts will need to take a long view, and can expect that it will have a bumpy road ahead in the short term. But someone must lead the way, and every technology requires a first adopter. The commitment of the ITD and its leadership must be applauded, given how easy it would be to maintain the status quo as one more sheep in the herd, and how difficult it will be to blaze a trail into what may prove to be a better future for all – but only if someone takes that first, most difficult, step.
Copyright 2005 Andrew Updegrove