Home > Standards Blog

Advanced Search 

Welcome to ConsortiumInfo.org
Saturday, October 25 2014 @ 07:01 AM CDT

Email Article To a Friend View Printable Version

Bill Introduced in Minnesota to Require Use of "Open Data Formats"

OpenDocument and OOXML

I received an email yesterday pointing me to a bill, introduced on March 27, that would require all Executive branch agencies in the state of Minnesota to "use open standards in situations where the other requirements of a project do not make it technically impossible to do this."  The text of the bill is focused specifically on "open data formats," and would amend the existing statute that establishes the authority of the Office of Enterprise Technology (OET), and the duties of the states Chief Information Officer.  While the amendment does not refer to open source software, the definition of "open standards" that it contains would be conducive to open source implementations of open standards. The text of the affected sections of Minnesota Statutes Chapter 16E, showing the amendments proposed, can be found here.

The fact that such a bill has been introduced is significant in a number of respects.  First, the debate over open formats will now be ongoing in two U.S. states rather than one.  Second, if the bill is successful, the Minnesota CIO will be required to enforce a law requiring the use of open formats, rather than be forced to justify his or her authority to do so.  Third, the size of the market share that can be won (or lost) depending upon a vendor's compliance with open standards will increase.  And finally, if two states successfully adopt and implement open data format policies, other states will be more inclined to follow.

The amendment, and the statutory framework in which it exists, provides an interesting contrast to the open standards policy adopted by the Massachusetts Information Technology Division (ITD) in many respects.  For example, it appears from the existing text of the statute that the Minnesota OET already has clear authority to mandate the use of open data formats, reading in part as follows:

When state agencies have need for the same or similar public data, the chief information officer, in coordination with the affected agencies, shall manage the most efficient and cost-effective method of producing and storing data for or sharing data between those  agencies. The development of this information architecture must include the establishment of standards and guidelines to be followed by state agencies. The office shall ensure compliance with the architecture.

Second, the proposed amendment contains an extremely detailed (and rather eclectic) definition of "open standards," including not only the traditional concepts of availability to all on reasonable and nondiscriminatory ("RAND") terms,  but a good deal more besides.  For example, it requires that all permitted standards must permit royalty free implementation, and also includes a number of requirements that are far more detailed than would normally be found in the description of an open standard, but which might be agreed upon as necessary by the members of a working group developing a standard.  The following is one example taken from the proposed definition of an "open standard" found in the amendment:  "[An open standard] is documented, so that anyone can write software that can read and interpret the complete semantics of any data file stored in the data format."  (I've included the full text of the proposed definition at the end of this blog entry.)

In addition to the very restrictive definition of open standards and open data formats, the amendment is also intolerant of making exceptions, providing:

For a particular project involving the access, storage, or transfer of data, a restricted data format may be chosen when satisfaction of essential project requirements precludes the use of an open data format. Neither the current storage format of previously collected data, nor current utilization of specific software products, is a sufficient reason, in absence of other specific overriding functional requirements, to use a restricted format;

Moreover, the amendment would require periodic review of all "existing data stored in a restricted format, to which the state of Minnesota does not own the rights, every four years to determine if the format has become open and, if not, whether an appropriate open standard exists;"  The amendment, if enacted, would therefore impose a very tight collar on what types of software could be purchased and used. 

On its face, the amendment is vendor neutral.  It does, however, include one provision that may have been directed at Microsoft, which has at times been criticized for adding proprietary extensions to otherwise standards-compliant product features.  That provision is found in the definition of an "open standard," and requires that if a standard, "allows extensions, ensures that all extensions of the data format are themselves documented and have the other characteristics of an open data format;"

There are a number of other interesting points that I note in reading the amendment at the Minnesota site, one being that the state legislation portal is set up to provide data on a bill's sponsors and current status - something that is sadly lacking in Massachusetts.  There also appear to be a number of differences between the responsibilities and authority of the CIO in Minnesota, as compared to Massachusetts, that Peter Quinn might have appreciated.  I may look more deeply into those differences and return to them in a future entry.

Curiously, I have been able to find out almost nothing on the Web about the amendment, indicating that thus far it has received little public notice.  The news item submission (written by the submitter) that pointed me to the bill, however, reads in part as follows:

A consortium of Minnesota businesses and citizens has moved to put forward legislation that promises to assist the State in overcoming the negative effects of time, innovation and the market. Referred to as the Open Data Formats Bill, House File 3971 defines the means by which Minnesota could take control over how it stores information so as to not be bound, technically or legally, to anything other than its own technical objectives.

The Bill is an attempt to counter the fact that searches on the State website display information that is primarily stored in formats owned by vendors... that could go out of business, get acquired, or turn into the next Enron. It is also a response to what some refer to as the 'eight track tape effect' where information is stored in a way that is both popular and looks permanent, but then is quickly replaced by newer technologies. In some instances the information has been lost forever.

The Bill is not biased towards any one technology and advances a policy where at all times and in all instances the State has the ability and legal right to review, fix or improve the information it uses to conduct business. The Bill is not expected to increase State spending on technology. It is, nonetheless, expected to receive stiff opposition despite improving competition for State contracts, enhancing the ability of Minnesotans to access State services and data, and improving communication between State systems.

I will keep an eye on this new bill and report further as additional information becomes available. For now, however, it is significant to note that the debate over open data formats has now begun in a second state.  It will be interesting to watch how the forces align, and the discussion becomes focused as the process moves forward.

The full text of the two new statute sections appears below.

For further blog entries on ODF, click here

subscribe to the free Consortium Standards Bulletin
(and remember to Buy Your Books at Biff's)

Language to be added to Minnesota Statutes 2005 Supplement, section 16E.03, subdivision 1:

(f) "Open standards" means specifications for the encoding and transfer of computer data that:  

(1) is free for all to implement and use in perpetuity, with no royalty or fee;

(2) has no restrictions on the use of data stored in the format;

(3) has no restrictions on the creation of software that stores, transmits, receives, or accesses data codified in such way;

(4) has a specification available for all to read, in a human-readable format, written in commonly accepted technical language;

(5) is documented, so that anyone can write software that can read and interpret the complete semantics of any data file stored in the data format;

(6) if it allows extensions, ensures that all extensions of the data format are themselves documented and have the other characteristics of an open data format;

(7) allows any file written in that format to be identified as adhering or not adhering to the format;

(8) if it includes any use of encryption, provides that the encryption algorithm is usable on a royalty-free, nondiscriminatory manner in perpetuity, and is documented so that anyone in possession of the appropriate encryption key or keys is able to write 2.20 software to unencrypt the data.

(g) "Restricted format" means any data format that is accessed, stored, or transferred 2.22 and is not open standards compliant.

 Language to be added to Minnesota Statutes 2005 Supplement, section 16E.04, subdivision 2: 

(g) The office shall assist state agencies to avoid the purchase or creation of data processing devices or systems that do not comply with open standards for the accessing, storing, or transferring of data. The office shall:

 (1) ensure any new data standards which the state of Minnesota defines and to which it owns all rights are open standards compliant;

(2) use open standards in situations where the other requirements of a project do not make it technically impossible to do this. For a particular project involving the access, storage, or transfer of data, a restricted data format may be chosen when satisfaction of essential project requirements precludes the use of an open data format. Neither the current storage format of previously collected data, nor current utilization of specific software products, is a sufficient reason, in absence of other specific overriding functional requirements, to use a restricted format;

(3) reexamine existing data stored in a restricted format, to which the state of Minnesota does not own the rights, every four years to determine if the format has become open and, if not, whether an appropriate open standard exists;

(4) make readily accessible, from a central location on the Internet, documentation on open data formats used by the state of Minnesota. When data in open format is made available through the state's Web site, a link shall be provided to the corresponding data format documentation.

 

 

Bill Introduced in Minnesota to Require Use of "Open Data Formats" | 24 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Wednesday, April 05 2006 @ 08:45 AM CDT
Hi,

definitely a good move. Open playground enforces more competetion and that's (after cooperation) the second best way of working. I am affraid M$ is in the "ignoration" working category ;(.

Take care,
Jan

[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Wednesday, April 05 2006 @ 01:40 PM CDT
As I read this, this goes WAY beyond what Massachusetts has done.  This affects backups, archives and potentially even file systems.

Is the ZIP format an open standard?  Would that qualify for us under these proposed changes?
[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Wednesday, April 05 2006 @ 02:21 PM CDT
I already feel sorry for whatever representative introduced or sponsored this bill. It is almost guaranteed that he or she will be personally attacked in the media, etc. before this process is over, just as Peter Quinn in MA was.


[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Wednesday, April 05 2006 @ 02:21 PM CDT
Don't publicize this!

Great.  Now MS is mobilizing their army of lobbyists to kill this thing before it can pass.

[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Wednesday, April 05 2006 @ 08:08 PM CDT

 

My reading is that clause G. (2) implies that, if passed, nobody in Minnesota government will be able to produce and store any new documents in .doc (Ms-Word) or .xls (Excel) format.  If that is truly the case, then the CIO will be very busy doing a lot of behavior / change management  to "ensure compliance"  

[ # ]
*Shhhh!*
Authored by: Anonymous on Wednesday, April 05 2006 @ 08:41 PM CDT
If this hadn't been reported, maybe it could have been snuck through without notice from Microsoft... :(

Might be better to hear only of its passing (unless Microsoft was already obstructing it somehow, in which case, the more noise the better!) ...

[ # ]
  • *Shhhh!* - Authored by: Anonymous on Thursday, April 06 2006 @ 03:44 PM CDT
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Thursday, April 06 2006 @ 01:09 PM CDT

the bill is not trying to create a million ISO-like organizations or processes every time someone releases a protocol, application, etc.. This is not its objective explicitly or implied.  Given this, you are right: your company would be conforming to the letter of the law if it acted as you  described. But that fact is what your company has released IS a standard/specification. That they didn't go and get some sort of certification is not as important as ensuring they don't encumber the standard with patents, copyrights, encryption, etc. At first this requirement may seem almost silly. Certainly not worth of a bill. But if the application becomes popular -popular in the same way PDF, HTML or Word has- this lack of encumbrances will be critical. And this is what the bill is getting at. Not the creation of committees.


[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Friday, April 07 2006 @ 08:57 AM CDT
"The Bill is an attempt to counter the fact that searches on the State website display information that is primarily stored in formats owned by vendors... that could go out of business, get acquired, or turn into the next Enron. It is also a response to what some refer to as the 'eight track tape effect' where information is stored in a way that is both popular and looks permanent, but then is quickly replaced by newer technologies. In some instances the information has been lost forever."

my favorite part - finally someone gets it - maybe other states will get smart and follow suit like any competent cio should.
like the part especially about the next enron.
[ # ]
Bill Introduced in Minnesota to Require Use of "Open Data Formats"
Authored by: Anonymous on Friday, April 07 2006 @ 11:34 AM CDT
sorry to get the reply thing messed up. and not that it's an excuse, but typing anything at a waterpark in wisconsin is tough.

nonetheless, i see your point. and frankly the bill doesn't really addess the time/how/why/who at which a specification becomes a standard.

don't get me wrong: standards bodies and what they try to do is a really good thing. that said, this bill should not get dumped becuase it doesn't go one more step past removing encumbrances and dictate the the mechanisms involved in elevating specifications to the status of 'Standard'.

it probably shouldn't. and why should it? if it did i have no doubt it would die a quick death.

But a better way to respond  this critiscm, though, is the quesiton: what good is a review/maint. committee if the specifications are patented and at any point can be used to sue anyone who attempts to implment the standard? this is kinda the nature of the agreement between sun and microsoft now relative to open office.

there is probably not enough room in the word 'standard' for all of us. i might be ok with 'open specifications' but i'm sure this would be a problem for others.








[ # ]