“Our state government possesses vast treasure troves of valuable information and reports: from health, business and public safety data to information on parks, recreation, labor, and transportation … The Open New York web portal will allow researchers, citizens, business and the media direct access to high-value data, which will be continually added to and expanded, so these groups can use the data to innovate for the benefit of all New Yorkers.” – Governor Cuomo, State of the State Address, January 9, 2013
Government is the public’s business and the public should have access to the records of government. New technologies have dramatically changed both the way government conducts business and the public’s expectations about access to government information. As part of this transformation, New York State launched Open.ny.gov, on March 11, 2013, dedicated to increasing public access to one of the State’s most valuable assets – data. Its goals are to spark innovation, promote research and economic opportunities, engage public participation in government, increase transparency, and inform decision-making.
Concurrent with the launch of Open NY, Governor Andrew Cuomo issued Executive Order No. 95, “Using Technology to Promote Transparency, Improve Government Performance and Enhance Citizen Engagement.” This unprecedented Executive Order directed covered state entities, for the first time, to identify and catalogue their data, and make publishable state data available on the new transparency website.
The concept of “Open Data” describes data that is freely available, machine readable, and formatted according to uniform technical standards to facilitate visibility and re-use of published data. New York’s open data platform is a quality by design web-based public data portal that catalogues data and enables data to be discoverable. The portal offers access to standardized data that can be easily retrieved, downloaded, sorted, searched, analyzed, and re-used by citizens, business, researchers, journalists, developers, and government to process, trend, and innovate utilizing a singular dataset or combinations of datasets.
OpenNY puts tools for transparency, accountability, and innovation directly into the hands of New Yorkers and people all around the world through a centralized, user-friendly interface. This increased visibility provides derivative value as the public is able to analyze and utilize government data, and better understand what is happening in government on all levels – federal, state, and local.
This Open Data Handbook is intended as a general guide for government entities participating in data.ny.gov , as well as the general public. The Handbook provides guidelines for identifying, reviewing, and prioritizing publishable state data for publication – with a foundational emphasis on high quality, and metadata and documentation requirements. These guidelines are intended for use by both covered state entities and other government entities not covered by Executive Order 95 (including localities). These guidelines are also intended for use by the public in order to understand how New York State makes its publishable data sets available.
The breadth of data and agency participation are continually being enhanced and expanded on data.ny.gov, making it a dynamic, living platform. This Handbook, providing guidelines for publication of publishable State data onto data.ny.gov, is the first step in a major shift in the way New York State government agencies share information to promote efficiency, accessibility and transparency; a major shift in the way New York State government engages citizens and fosters innovation and discovery in the scientific and business communities. It begins the process of standardizing the state’s data, which will make it easier for both government workers and the public to discover and use the data. This, in turn, advances “interoperability” so the data can be more easily shared and analyzed.
Working in collaboration with state agencies, this Handbook will be supplemented, as needed, with technical and working documents addressing specific formatting, data preparation, and data refresh and data submission requirements. We welcome input from academics, researchers, developers, businesses, entrepreneurs, and the general public to help identify data that would be useful to them and the best ways of providing it.
In advancing a transparency agenda, New York State is committed to continuous publication of publishable State data. Data discovery is an iterative and on-going process. The following is a high-level timeline for the implementation of Executive Order 95.
Throughout this handbook, some terms are used as defined in Executive Order 95. These definitions are included below.
Covered State Entity:
Chief Data Officer (CDO): The New York State Chief Data Officer in the Office of Information Technology Services or a designee thereof.
Data: Final versions of statistical or factual information that:
The term “data” shall not include image files, such as designs, drawings, photos or scanned copies of original documents; provided, however, that the term “data” shall include statistical or factual information about image files and geographic information system data.
Data set: A named collection of related records maintained on a storage device, with the collection containing data organized or formatted in a specific or prescribed way, often in tabular form.
ITS: The New York State Office of Information Technology Services.
Publishable State data: Data that is collected by a covered State entity where the entity is permitted, required or able to make the data available to the public, consistent with any and all applicable laws, rules, regulations, ordinances, resolutions, policies or other restrictions, requirements or rights associated with the State data, including but not limited to contractual or other legal orders, restrictions or requirements. Data shall not be Publishable State data if making such data available on the Open Data Website would violate statute or regulation (e.g., disclosure that would constitute an unwarranted invasion of personal privacy), endanger the public health, safety or welfare, hinder the operation of government, including criminal and civil investigations, or impose an undue financial, operational or administrative burden on the covered State entity or State.