Maureen C. McCormick. Shedding Light on Transparency: An Analysis of the Breadth and Depth of Federal Agency Implementation of the Open Government Initiative in Online Environments. A Master’s Paper for the M.S. in L.S degree. March, 2012. 44 pages. Advisor: Christopher A. Lee.
The Open Government Initiative (OGI) was signed by President Obama on January 21, 2009. The stated aims of the OGI are to promote transparency, participation, and collaboration in executive agencies. Implementation of the OGI focused initially on transparency and data integrity, particularly in online environments.
Open government compliance is of interest to library and information science (LIS) professionals because it utilizes several core competencies taught during LIS graduate programs including data and knowledge management, outreach, collaboration, and data visualization. Managing OGI participation could be a growth area for LIS professionals because of its use of many components of library and information science.
More than three years after the OGI was signed, no broad investigation of the level of agency participation with the OGI has been done. This paper reports on a study to determine how and why agencies are meeting or exceeding compliance minimums concerning online data transparency in online environments.
Headings:
Shedding Light on Transparency: An Analysis of the Breadth and Depth of Federal Agency Implementation of the Open Government Initiative in Online Environments
by
Maureen C. McCormick
A Master’s paper submitted to the faculty of the School of Information and Library Science of the University of North Carolina at Chapel Hill
in partial fulfillment of the requirements for the degree of Master of Science in
Library Science.
Chapel Hill, North Carolina March 2012
Approved by
Table of Contents
Introduction ... 2
Background ... 2
Problem Statement ... 2
Importance, Implications, and Scholarly Contribution ... 4
Literature Review ... 5
I. Introduction ... 6
II. Open Government ... 7
Importance ... 7
Innovation ... 8
Why Open Government? ... 8
Transparency vs. Accountability ... 10
Core Concepts ... 10
Implementation ... 11
Open Government Critiques and Public Reception ... 13
III. Conclusion ... 15
Methodology ... 16
Methodological Background ... 16
Table 1: Definition of Key Terms ... 17
Table 2: Open Government Directive and Memorandum Analysis ... 19
Quantitative Metric Developed for this Study and Variable Relationship Analysis ... 20
Table 3: Quantitative Metric ... 21
Coding Elements ... 23
Content Analysis as a Methodological Choice ... 24
Limitations and Threats to Validity and Reliability ... 25
Findings ... 27
Table 4: Overall Compliance by Participating Agencies with Compliance Minimums by Entity Type ... 27
Table 5: Statistical Test of Budgetary Significance ... 30
Table 6: Case Summary Table for FTE over/under Median Comparisons ... 31
Table 7: Crosstabulation Results for FTE over/under Median Test ... 32
Table 8: Chi-Square Results for FTE Significance ... 32
Table 9: Representation on Open Government Working Group ... 34
Table 10: Compliance Minimums Met or Exceeded Based on Budgetary Quartile Comparison (Measured in Percentages) ... 34
Table 11: Compliance Minimums Met or Exceeded Based on FTE Quartile Comparison (Measured in Percentages) ... 35
Conclusion and Recommendations ... 36
Introduction
Background
On January 21, 2009, President Barack Obama’s first full day in office, he signed the Open Government Initiative (OGI). It is a call for executive agencies to incorporate a culture of transparency, participation, and collaboration into every aspect of operation. Open government enforcement, including its advancement and compliance, are growing areas in the LIS field. It includes several traditional LIS fundamentals, such as data, information and knowledge management, outreach, and records management. In essence, open government sits directly at the intersection of government and library and
information science.
OGI implementation efforts have thus far focused primarily on transparency. As a result of the initial focus on transparency and the push for greater online access to government information, this paper examines only efforts made towards greater transparency in online environments. Compliance and enhanced participation with the transparency portion of the OGI were measured among participating agencies in online environments, as were factors influencing whether or not an agency chose to participate.
Problem Statement
Open government has been advanced as an Obama Administration policy in the hopes of encouraging innovation, giving citizens access to information without them specifically requesting it, and encouraging participation and collaboration. Agencies are using a variety of methods to achieve these lofty goals, but some of the common methods include the use of social and new media, more solicitation of public comment, and the publication of datasets both on their own websites and on data.gov, the government data repository launched in conjunction with the OGI (Staff Writers, 2009).
To date, there has been no analysis of the OGI’s implementation in the executive branch overall, and, as a result, there is a fundamental lack of understanding as to how the OGI has been received or implemented across the federal government. For LIS professionals to gain a foothold in open government administration, they must understand what has been done so far and in which agencies and areas there is room for significant improvement.
Currently, there is a lack of knowledge about how or why agencies may choose to meet or go beyond the compliance minimums set by the OGI. This study attempts to address that shortcoming by providing information about relationships between enhanced participation and several independent variables (further explained in the “Methodology” section). To date, no study of this kind has been conducted.
I identified three primary research questions to guide and inform my study: 1. Are agencies that are required to comply with the OGI meeting the compliance
2. Are agencies not required to comply with the OGI choosing to do so? Is there a difference between the larger, better-funded agencies and those that are smaller with less funding?
3. How does representation on the Open Government Working Group affect compliance?
Importance, Implications, and Scholarly Contribution
As noted previously, the first phase of the open government movement, as implemented in the United States federal government, dealt with data transparency and increased data access. This phase sought to increase innovation by citizens and enhance that engagement with government data. Although larger agencies likely have more data to release, it is nonetheless important for smaller agencies to release data as well. In addition, many believe that there is a societal good to having a transparent government that engages its citizens (Sifry, 2010 and Tapscott, 2010). This requires participation across the federal government for maximum impact.
Literature Review
This section is divided into three subsections. The first gives a general overview of the history of the Open Government Initiative and Directive, the policy statements that form the cornerstone for this study. The second section dissects the Open Government Movement, its implementation in the U.S., adoption, and several key components of both the Movement and the OGI. Finally, the third section acts as a conclusion to the
literature review, reiterating several key points from the literature.
This paper examines the extent of voluntary participation and mandatory
compliance with the OGI. As detailed earlier, the research questions address the extent of compliance or participation and how different variables -- agency size and funding, representation on the Open Government Working Group, and presence on the White House Open Government Scorecard -- affect compliance and participation.
I. Introduction
In general, the literature has spoken to several key areas of open government: the importance of the OGI and the Open Government Movement; core concepts and points of the OGI; agency implementation; and critiques of the OGI and its public reception. This paper focuses only on steps taken to open government up on online platforms, because, in the words of Beth Noveck, “We are drawn to collaboration enabled by the Internet” (Noveck, 2009, p. 189) and as Carl Malamud states, “Today public means online” (Malamud, 2009, p. 46).
The third wave has coincided largely with calls to open up government and make it more transparent. Here, too, is where LIS professionals have an opportunity to make their mark on the open government movement. With technical expertise and prior
experience dealing with records management and information requests, LIS professionals can step in during this third wave and improve open government implementation from an LIS perspective. Each of these waves has built upon the previous and been accompanied by major reforms. In the past, these reforms have included publication of rules and regulations and better public record laws. Now, reform is taking place in the form of the open government movement.
II. Open Government
Importance
The OGI officially became part of the Obama agenda when it was signed on his first day in office. Unofficially, open government was an espoused part of the
Innovation
Currently, we stand at a crossroads. The Internet, social media, and web 2.0 technologies are making it possible for millions of people dispersed across vast geographic areas to simultaneously participate in one, singular conversation (Reich, 2009, p. 132) for the first time in history. In addition, increased government transparency is a major priority at the highest levels of government.
Furthermore, while earlier attempts at e-government focused primarily on making existing services available online, or “paving the cow paths” (Tapscott, 2010, p. xvi), current open government efforts are actually improving upon government services, or even creating new ones (ibid). This is resulting in not only new services, but better services and service delivery. By opening up datasets and providing the citizenry with more complete access to information, citizens are in turn able to tailor their experience with government. This can now be done by developing applications to make data useful (O’Reilly, 2010, p. 17) or by communicating with government representatives through new media (Drapeau, 2009, p. 127 and Reich, 2009, p. 131).
Why Open Government?
go forward with the “proactive release” of high-value datasets (Horvit, 2011). Open Government architects in the White House also challenged agencies to incorporate the spirit of the OGI and the subsequent Directive into their agency culture. As a result, agencies have begun to do so in their trainings (Jones, N., French, C., & Willard, E., 2010, p. 9) and in some cases, even reorganizing to reflect a more open culture.
Second, the ongoing fiscal reality is that “most governments will have to do more with less, both today and in the future” (Tapscott, 2010, p. xvii). The transparency measures implemented as part of the OGI allow citizens to do some of the “heavy lifting” by developing applications (apps) and transforming data into useful products (ibid) that the government may not be able to do itself (Burton, 2010, p. 6) or anticipate the need for.
Third and finally, the Internet age and open government actions on online platforms have made it possible for everyday citizens to influence the way that
2009, p. 33). These legs (participation and collaboration) of the OGI are not addressed in this paper, but are nonetheless important parts of both the OGI and the Movement.
Transparency vs. Accountability
Previously, many efforts at increased government transparency were aimed at making government more accountable. This did not necessarily make government leaders receptive to further transparency efforts since these accountability efforts frequently led to “a ‘gotcha’ game in which the information provided by open
government measures is used by journalists, advocacy groups of the right and left, and political opponents to catch official wrongdoing” (Fung & Weil, 2010, p. 106). Now, however, transparency efforts aim not necessarily to increase reactive accountability, but the proactive release of information (ibid).
Core Concepts
There are three main parts of the OGI: transparency, participation, and
collaboration (Orszag, 2009). The first stage of implementation has focused primarily on data transparency in government agencies. Data.gov, a central repository for
government-published datasets, was established more than two years ago (Staff Writers, 2009). The idea behind data.gov was not just to consolidate datasets, but also to make data more accessible and, according to Peter Orszag, encourage “citizen feedback and new ideas” (Orszag, 2009).
Since the launch of data.gov, citizens have been able to access raw data from government agencies, and “ordinary citizens have the ability to seek out, use, and transform data like never before” (Malamud, 2009). Because data.gov allows for open
are able to create applications that allow people to use the data as they choose (O’Reilly, 2010, p. 17). Use of this technology “helps people to route around the logjam” (Noveck, 2009, p. 29) and solve problems for themselves using government information. As Beth Noveck notes, this is both practical and helpful because “innovation is not emanating from Washington; instead, the practices of government are increasingly disconnected from technological innovation” (Noveck, 2009, p. 34).
Carol Lukensmeyer and others have identified five distinct types of participatory activities: informing, or giving the public balanced and objective information; consulting, or obtaining feedback from the public; engaging, or working with the public to ensure the concerns are considered; collaborating, or working with the public in every aspect of the decision; and empowering, or giving final decision-making authority to the citizenry (Lukensmeyer et al, 2011, p. 14). These types of activities are all encouraged under the OGI.
Implementation
agencies party to the audit did not respond to its FOIA request within the appointed time frame (ibid, p. 11).
On a procedural level, some of the most common strategies that agencies employed when first implementing their open government plans included social media, crowdsourcing, and collaborative online ideation platforms such as IdeaScale
(Lukensmeyer et al, 2011, p. 20). Additionally, some agencies are in the process of making the switch from using primarily proprietary software to open source, notably the Department of Defense (O’Reilly, 2010, p. 17). This change from proprietary, closed software platforms to openly available ones whose code is public appears to embrace the spirit of the Open Government movement, which calls for a more complete public understanding of how the government operates. More directly, the OGI requires that agencies publish in open formats when possible.
One of the most popular methods used by agencies has been to create blogs written by agency employees. This can have the effect of personalizing the agency and getting away from the official line of the press office, making the agency more
accessible. For instance “Blogger Bob,” who blogs for the Transportation Safety Administration has “been empowered by his organization to write with a personal
viewpoint that showcases the personality of a human rather than the coarseness of official jargon” (Drapeau, 2009, p. 127).
technical minimums of the OGI, there has been little measurement of whether or not it is being implemented in the spirit intended by the original architects.
Open Government Critiques and Public Reception
Despite its uncontroversial nature on the surface, the open government movement was not welcomed with open arms by all. Even some who have an important stake in the success of the movement, such as citizens and investigative reporters, have been critical. The editor of a journal for investigative reporters remarked in an opinion piece that, even if open government was working incredibly well, it would still be far too dependent on the “whims of those in power” (Horvit, 2011). He further remarked that “any policy that relies too much on government to open itself is flawed, because it puts the balance of power in the wrong half of the equation” (ibid).
Related to the limitations of the open government movement that Horvit finds, others have implied that perhaps the OGI does not do enough to truly make government more open. Carl Malamud writes: “The principle that primary legal materials should be available to all is a principle that needs to be driven by the leadership of the executive branch and applied to all levels of government” (Malamud, 2009, p. 46). However, as a legal matter, the OGI cannot apply to other branches of government, and therefore applies only to executive branch agencies.
(particularly given the current financial climate), there are also compelling reasons not to participate. In the past, sunshine groups looking to make government more transparent have sought “information that most organizations would not voluntarily disclose” (Fung & Weil, 2010, 111). Examples of this include the National Security Archive and the sunshine groups that sought to overturn Executive Order 13233, which limited access to the records of former presidents. From the agencies’ perspective, it is frequently
“cumbersome to comply with these requirements” (ibid.). One of the architects of the OGI, Beth Noveck, even argued that “more active involvement in government by self-selecting citizens would only increase the risk of corruption” (Noveck, 2009, p. 41) and that open government is potentially harmful as a result. This implies that she was
concerned from the outset that the “self-selecting citizens” who choose to participate in a more open government would be so concerned with niche causes that they would actually harm open government efforts and make it more corrupt.
The public has not been entirely pleased with the way that the open government movement has been implemented, either. An analysis of more than 5,100 citizen survey responses to agencies’ open government websites reveals that citizen trust in government is even lower now than it was a year ago (Steirnstein, 2011). Citizens do not believe the public officials are being frank with them, despite the strides made to open the
government (Steirnstein, 2011).
by citizens via Facebook, Twitter, and YouTube (Sifry, 2010, p. 118). However, on the day of the event, “there was no mechanism established to determine which ones were indeed popular” (ibid.). As a result, staffers simply chose which questions he would answer, producing, according to Sifry, “an event that was less spontaneous and less town-hall-like than if all the questions had come from citizens live at the event using no
technology at all” (ibid).
III. Conclusion
While a significant amount of information and literature exists about the open government movement in America, there is also a great deal of debate about best practices, implementation, and in some cases, even the utility and helpfulness of it generally. Additionally, the LIS field has been noticeably absent from the discussion, despite what should be a vested interest in the success and continuation of open government.
Open government has been advanced as a policy of the Obama Administration in the hopes of encouraging innovation and encouraging participation, giving citizens access to information without them having to specifically request it. Agencies are using a
Methodology
This section is divided into three subsections. The first gives an overview of the methodological background and initial analysis. The second outlines the information that I analyzed and the metric that I developed to measure quantifiable points of compliance and participation. The final section addresses content analysis as a methodological choice. A short review of relevant literature regarding content analysis is included at the end.
Methodological Background
For this study, I identified compliance minimums in the OGI (Table 2) and conducted a content analysis of agency websites. Only agencies with a /open domain (e.g. www.justice.gov/open) were considered to be “participating in the OGI” (see Table 1 for definitions of key terms referenced in this paper) because this is the public-facing open government web address for all federal agencies. Without the public-facing open government web address, it would be impossible for a member of the general public to know that the agency was participating in the OGI in a meaningful or official way. Mindful of the importance of publicly available and public-facing information for open government efforts, I only attempted to access publicly available data and information. After I determined participation, I then conducted content analysis of the Open
Government webpages of agencies that participate in the OGI.
analysis in the aggregate. The PVAT is intended for use by the agency officials who develop open government plans, in order to maximize the plans’ public utility. In other words, the PVAT is meant to be a qualitative internal evaluation tool, not a tool for outsiders to use in order to check compliance or make comparisons between agencies. Since there is no applicable tool for evaluating Open Government participation, I have developed a metric to quantify and analyze data collected related to OGI participation and compliance.
Table 1: Definition of Key Terms
Term Definition and Source
Boards, Commissions, and
Committees
"These organizations were established by congressional or Presidential action, and their functions are not limited to supporting a parent
agency." From: usa.gov
CFO Act The CFO Act is a law signed in 1990 by President George H.W. Bush.
It was an attempt to create greater financial accountability within 24 high-profile executive agencies. It broadened the responsibilities of OMB and established financial accountability officers (CFO's) within the agencies affected by the Act. Information and full text located here: http://www.gao.gov/special.pubs/af12194.pdf
Compliance Agencies were determined to be in compliance with various elements
of the OGI if they met the minimum requirements set by OMB in the Directive and Memorandum.
Data Integrity The appointment of an upper-level manager to be "accountable for the quality and objectivity of, and internal controls over, the Federal spending information publicly disseminated through such public venues as USAspending.gov or other similar websites." From: USAID /open page, but common definition across many agencies Executive
Department
Those entities part of the Executive Branch of the government whose heads serve as part of the President’s Cabinet. From: usa.gov
Corporation agencies are responsible for keeping the government and economy running smoothly." From: usa.gov
Open
Government Directive and Memorandum
The memorandum issued by then-OMB Director Peter Orszag on December 8, 2009 to heads of executive agencies that laid out the groundwork for the open government movement in America. Located at: http://www.whitehouse.gov/open/documents/open-government-directive
Open
Government Initiative
Term referring specifically to the open government movement in America. Began with President Obama's issuance of the
Memorandum on Transparency and Openness issued January 21, 2009. Located at:
http://www.whitehouse.gov/the_press_office/TransparencyandOpenG overnment/
Social/New Media
"Federal agencies are increasingly using recently developed
technologies (commonly referred to as "Web 2.0" technologies) that offer flexible, sophisticated capabilities for interaction with
individuals, allowing agencies and the public to publish comments, photos, and videos directly on agency-sponsored Web pages." From: http://www.gao.gov/assets/130/125110.pdf. Includes social
networking platforms, wiki spaces, and blogs. Specific platforms include, but are not limited to: Facebook, Twitter, FourSquare, LinkedIn, Flickr, MySpace, YouTube, Vimeo, and iTunes. White House
Scorecard
Also known as: "White House Open Government Dashboard." A matrix on the White House Open Government page that tracks the progress of various executive agencies in achieving open government. Located at: http://www.whitehouse.gov/open/around
The first step in my data collection and analysis was to analyze the Open
Table 2: Open Government Directive and Memorandum Analysis (Requirements in BOLD analyzed in this study)
Measurement
Number Requirement Goal Location
1 Create /open website
that includes
mechanisms for public to:
1) Give feedback on information published; 2) Provide input on what info to publish (prioritization); 3) Provide input on agency's Plan.
Increase public awareness of open government activities
Memorandum, pg. 3
2 Publish information
online in addition to other formats
Transparency Memorandum,
pg. 2
3 Publish 3 new,
high-value datasets on data.gov Increase transparency and access Memorandum, pg. 2 "Proactively disseminate data" (do not wait for FOIA requests) Increase transparency and access Memorandum, pg. 2
Publish FOIA report on /open page Increase transparency and access Memorandum, pg. 3
Reduce FOIA backlog by 10% each year
Increase transparency and access Memorandum, pg. 3 Implement Presidential open government activities: 1) data.gov 2) eRulemaking 3) IT Dashboard 4) recovery.gov 5) USAspending.gov
Increase
transparency and access
Create agency buy-in
Memorandum, pg. 3
4 Develop and publish an
open government plan
Update the plan every two years Improve transparency, incorporate participation and collaboration into the culture
Memorandum, pg. 5
5 (This point was considered and monitored, but not tested for statistical significance because of the makeup of the group (detailed later).)
OMB will convene a working group Improve transparency, incorporate participation and collaboration into the culture; create a forum to discuss best practices
Memorandum, pg. 5
6 Publish in open format Be
machine-readable/searchable and promote re-use
Memorandum, pg. 2
7 Respond to input
received from public Engage citizens Memorandum, pg. 3
8 Designate a senior
official to be in charge of info quality
Improve
accountability Memorandum, pg. 3
Detail internal controls over info quality
Improve accountability
Memorandum, pg. 4
Comply with OMB guidelines, report spending quarterly Increase spending transparency Memorandum, pg. 4
OIRA, the CIO, and the CTO will review policies to identify impediments to the use of new technologies
Allow the use of emerging
technologies for open government; find and eliminate roadblocks
Memorandum, pg. 5
Quantitative Metric Developed for this Study and Variable Relationship Analysis
Working Group). This analysis was later used in order to compare levels of participation with agency characteristics, such as size and capacity. The metric accounts for both non-compliance and exceptional non-compliance for the quantitative areas. Points for non-compliance, non-compliance, and enhanced compliance (that which goes beyond the requirements laid out in the Directive and Memorandum) were assigned in the following manner:
Table 3: Quantitative Metric
Level Points Earned
Non-compliance -1
Complete (basic) compliance 0
101-150% compliance 1
151-200% compliance 2
Greater than 200% compliance 3
Thus, hypothetically, if an agency published 23 datasets on data.gov, 8 of which it had designated as high-value, I would award 3 quality points for data publication.
For the remaining Boolean value transparency requirements (see Table 2 for my analysis of the Open Government Directive and Memorandum) in the OGI, either 0 or 1 point was assigned. For instance, if there was an agency did not publish a plan on its /open website (a compliance requirement), this yielded 0 points. A link to the published plan yielded 1 point.
statistical software to run t-tests and chi-square tests as appropriate to determine relationships between the independent and dependent variables.
Although a federal budget has not been passed since 2010, I chose to use the 2012 fiscal year proposed budgets from the agencies. I decided to use the proposed budgets rather than the continuing resolutions because I believed they would more accurately represent the agencies’ perceived needs. The 2012 budgets were used for both FTE and budget figures.
After measuring participation and capacity, I used statistical software to run t- and chi square tests, determining relationships between the independent and dependent
variables. Generally, the dependent variable measured was agencies’ OGI transparency as reflected by data and navigation on data.gov and /open domains.
A list of coding elements is listed later in the methodology section, but in general, the independent variables that I analyzed were:
• Position in agency hierarchy (explained below);
• Representation on the Open Government Working Group; • Presence on the White House Scorecard; and
• Capacity (measured by two different variables: number of FTE positions and funding quartiles).
In addition to this analysis, I completed an analysis of participating agencies’ social media presence to determine the rate at which agencies participating in the OGI maintain a social media presence. I did this analysis because social media use is one of ways that the Obama administration has effectively reached new groups of constituents, and
arguably one of the most important advancements of the administration (Harfoush, p. 46). Additionally, social media tools commonly used by government entities, such as
were used by agencies with lower capacity since they require few additional resources. Finally, the terms of service agreements that allow social media use by government entities were negotiated by the General Services Administration during the Obama administration in response to the Open Government Directive (U.S. General Services Administration, 2011). To determine social media usage, I looked first on agency /open pages for widgets directing visitors to social media platforms (e.g. “Like us on
Facebook!” or “Follow us on Twitter!”). If nothing was found on the /open page, then I searched for the agency on Facebook because it is the most commonly used social media platform (Leggatt, 2011).
Coding Elements
Before coding elements, agencies were sorted according to their position on the hierarchy of agencies. Each agency designation was populated according to the list on usa.gov. The hierarchy is:
1. Executive department;
2. Independent agency or corporation; 3. Board, commission, or committee; and
4. Other participating offices (Note: Entities that fell under the “Other Participating Offices” designation were not included in the statistical analysis, because no comprehensive list of quasi-independent offices exists. The “other participating offices” found were found solely because of their representation on the White House Scorecard or the Working Group).
Then, agencies were coded for the following elements:
• Agency Name
• Main Website (web address) • /open Page? (yes or no)
• Number of Data Sets Released on Data.gov
• Data Set Quality Points (according to quantitative metric)
• Social Media Binary Measure (1 or 0) • Published Plan (yes or no)
• Binary Plan Measure (1 or 0)
• PLAIN Language Published? (yes or no) • Represented on Working Group? (yes or no) • Binary Working Group Measure (1 or 0)
• Represented on Scorecard? (yes or no, also called “White House Open Government Dashboard”)
• Binary Scorecard Measure (1 or 0) • FTE Employees (2012 Requested) • FTE Quartile (within Agency level) • Total Budget (2012 requested)
• Budgetary Quartile (within Agency level) • Budget over $1 billion (1 or 0)
• Budget $500 million - $1 billion (1 or 0) • Budget $100 million - $500 million (1 or 0) • Budget $50 million - $100 million (1 or 0) • Budget $25 million - 50 million (1 or 0) • Budget less than $25 million (1 or 0) • Date Data Accessed
Content Analysis as a Methodological Choice
As discussed earlier in the methodology section, I used quantitative content analysis techniques to determine levels of agency compliance and voluntary participation with the OGI. Content analysis, an “often descriptive” method (Riffe, D., Lacy, S., & Fico, F., 2005, p. 33) has a long history among researchers, dating back to the 18th
century in Scandinavia (Rosengren, 1981, as quoted in Hsieh, H.-F. & Shannon, S., 2005, p. 1278).
analysis permits “statistical comparisons between” the variables (Mehmetoglu, M., 2004, p. 178).
When conducting a content analysis study it is important for researchers to remember that, even though the text is being picked apart and categorized based on its meaning, there remains an “underlying, often abstract theoretical concept” (Riffe, D., Lacy, S., & Fico, F., 2005, p. 24). Herein lies one of the great criticisms of content analysis from a methodological standpoint: internal validity. For this study, I was the only coder, and due to constraints and the limitations presented by overwriting
information on websites, it was impossible to conduct an inter-coder reliability test. Interestingly, “quantitative descriptive content analysis often represents the earliest study in an area” (Riffe, D., Lacy, S., & Fico, F., 2005, p. 34). For this reason, it seems particularly fitting that this paper relies on the method, as this is the first known study of its kind.
Limitations and Threats to Validity and Reliability
A more technical limitation to the study relates to recent government budgetary problems. To date, a federal budget has not been passed for the 2012 fiscal year. No budget was passed for the 2011 fiscal year. Agencies have been operating on a continuing resolution, but this resolution does not account for changes in agency size, capacity, or priorities. In an attempt to account for this, the proposed 2012 budget was used for data regarding agency capacity, despite the fact that it was not passed or enacted. This was a calculated choice made in the hopes of accounting for what agencies and the executive branch in general perceive to be changes in agency capacity and need.
Findings
As previously established, participating executive departments, independent agencies and corporations, and boards, commissions, and committees (full list located in Appendix F) were evaluated on several criteria. Data were collected over a one-week period in July 2011, and analyzed between August 2011 and January 2012. Table 4 shows the overall breakdown of how many agencies per entity type were compliant with each requirement of the OGI.
Table 4: Overall Compliance by Participating Agencies with Compliance Minimums by Entity Type
Entity Type Compliance Point Number in
Compliance/Number Participating
Executive Department Create /open website 15/15
Publish information online 15/15 Publish 3 new, high-value
datasets on data.gov
15/15 Develop and publish an
open government plan
15/15 OMB will convene a
working group
15/15 Publish in open format 15/15 Solicit and respond to input
received from public 15/15 Designate a senior official
to be in charge of info quality
15/15
Independent Agency Create /open website 49/49
Publish information online 48/49 Publish 3 new, high-value
datasets on data.gov 24/49 Develop and publish an
open government plan
33/49 OMB will convene a
working group
to be in charge of info quality
Boards, Commissions, and Committees
Create /open website 4/4 Publish information online 4/4 Publish 3 new, high-value
datasets on data.gov
0/4 Develop and publish an
open government plan
3/4 OMB will convene a
working group
0/4 Publish in open format 2/4 Solicit and respond to input received from public
4/4 Designate a senior official
to be in charge of info quality
2/4
Overall, analysis showed that the highest percentage of participating agencies by category was among Executive Departments, followed by Independent Agencies, and finally Boards, Commissions, and Committees (Figure 1). This mirrors the general hierarchy of federal agencies (explained in the “Methodology: Methodological Background” section). Moving down the hierarchy, the entities become less likely to participate in the OGI. Figure 1 shows this phenomenon.
Likewise, moving down the hierarchy, participating agencies are less likely to meet or exceed the compliance minimums set by the OGI. Figure 2 shows the percentage of participating agencies that were meeting or exceeding compliance minimums at the time of data
0 10 20 30 40 50 60 70 80 90 100
P
er
ce
n
t P
ar ti ci p at in
g
Agency Category
Figure 1: Participation Percentage by Category of Entity
Board, Commission, or
Commi-ee Independent
Agency Execu9ve
collection, broken down by category. The figure illustrates that Executive Departments and participating
Independent Agencies are meeting or exceeding minimums more often than Boards, Commissions, and Committees.
Statistical analysis of the data
indicates that both budget and FTE capacity are statistically significant to participation, at the 95% confidence interval. This is represented in Tables 5 and 8 by a chi-square or t score of more than 2. With regard to the FTE capacity indicator, this means that agencies with an FTE level above the overall median (which was 1365 FTE employees) were more likely to participate in the OGI, and agencies with an FTE level at or below the median were less likely to participate (Table 8). Tables 6 and 7 show the general breakdown of cases over the median number of FTEs and likelihood of participation of cases over median, respectively.
Budget was also found to be statistically significant (Table 5). Those agencies in the highest budgetary band (over $1 billion) were far more likely to participate, and those in the lowest budgetary band (under $25 million) were
Figure 3: Social Media Use Among All Participating
Agencies
Use Social Media (n = 32) Do not use Social Media
0 20 40 60 80 100
1 2 3
P
er
ce
n
t M
ee
ti
n
g
Co m p li an ce M in im u m
s
Figure 2: Percentage of Participating Agencies Meeting
Compliance Minimums by Category
Agency Type
Board Commission
Commi-ee Independent
Agency Execu9ve
bands in the middle did not indicate any statistical significance.
Figure 3 shows the analysis of my binary measure of participating agency use of social media. As established earlier, the binary social media measure was not meant to advance a quality of participation argument, but to determine if agencies were using social media. It has only been since the beginning of the open government movement that agencies have been allowed to use social media tools and terms of service negotiated with providers (U.S. General Services Administration, 2011), and it was purely a means to determine whether or not agencies were using the tools. As seen in Figure 3, less than half of all participating agencies had a discernable presence on social media platforms. This was measured by advertised participation on agencies’ open government webpages as well as searches run on Facebook (further described in the “Methodology” section).
Table 5: Statistical Test of Budgetary Significance
t
95% Confidence Interval of the Difference Lower Upper
budgetband1 Equal variances
assumed
-4.217 -.573 -.206
Equal variances not assumed
-5.559 -.529 -.250
budgetband2 Equal variances
assumed -.856 -.152 .061
Equal variances
not assumed -.982 -.139 .047
budgetband3 Equal variances
assumed
-1.428 -.257 .042
Equal variances
budgetband4 Equal variances assumed
.304 -.103 .141
Equal variances
not assumed .293 -.109 .146
budgetband5
Equal variances assumed
.037 -.105 .109
Equal variances
not assumed .037 -.107 .111
budgetband6 Equal variances
assumed 5.729 .341 .703
Equal variances
not assumed 5.419 .329 .716
The t-tests in Table 5 on the budgetary bands shows that the lowest band (budgetband1, under $25 million) and the highest band (budgetband6, over $1billion) were both
statistically significant on the dependent variable of participation. This is represented by the t-score above 2 or less than -2.
A chi-square test was performed to determine the significance of FTE capacity on participation. The tables below indicate that agencies with more than the median number of employees (1365) are statistically more likely to participate in the OGI. Likewise, those agencies with fewer than the median are less likely to participate (Table 6).
Table 6: Case Summary Table for FTE over/under
Median Comparisons
Cases
Valid Missing Total
N Percent N Percent N Percent Participation *
FTEovermedian 63 .4 87 .6
150
I also ran a cross-tabulation analysis to determine the number of agencies above and below the median number of FTEs participating and not participating with the OGI. As seen in Table 7, there are eight agencies with fewer than 1365 FTE employees that are not participating in the OGI at all. Likewise, there are 24 agencies with fewer than 1365 that are participating.
Table 7: Crosstabulation Results for FTE over/under Median Test
Participation * FTEovermedian Crosstabulation FTEovermedian
Total
.00 1.00
Participation 0 8 2 10
1 24 29 53
Total 32 31 63
Finally, the chi-square test represented in Table 8 tests the impact of the number of FTE employees that an agency has on participation. As seen by the chi-square value, the number of FTEs is statistically significant on participation.
Table 8: Chi-Square Results for FTE Significance
Value df
Asymp. Sig.
(2-sided)
Exact Sig.
(2-sided)
Exact Sig.
(1-sided)
Pearson Chi-Square 4.057 1 .044
Likelihood Ratio 4.311 1 .038
Presence on the White House Scorecard and representation in the Working Group were not tested for statistical significance because the vast majority of the agencies on the Scorecard and in the Working Group were Executive Departments, and all of them were required to comply with the OGI. Table 9 shows the breakdown of agencies on the White House Open Government Working Group (“Working Group”).
A full list of key definitions of terms in this paper, including the Working Group can be found in Table 1 in the Methodology: Methodological Background section. For the purposes of this study, the Working Group is a group of senior-level officials from various executive agencies whose mission is to promote the tenets of the OGI within their agencies. Convened by the White House, agencies represented on the Working Group are compelled to comply with the OGI. The Working Group is comprised of CFO-Act agencies, as well as representatives of agencies whose missions are significantly tied to the principles advanced by the OGI, such as the National Archives and Records
Administration.
Table 9: Representation on Open Government Working Group
Category Number
Represented on Working Group
Number Participating
Percent of Participants Represented Executive
Department
15 15 100
Independent Agency 13 49 26.53
Board, Commission, or Committee
0 4 0
Overall, I found that three-quarters of the participating Boards, Commissions, or Committees are using social media, whereas fewer than half of the agencies overall are using social media (Figure 3).
Table 10: Compliance Minimums Met or Exceeded Based on Budgetary Quartile Comparison (Measured in Percentages)
Quartile 1: Quartile 2 Quartile 3: Quartile 4: Meet Exceed Meet Exceed Meet Exceed Meet Exceed Executive
Department
25 75 0 100 0 100 25 75
Independent Agency
20 10 10 20 40 30 22 44
Concerning levels of participation within agency designations, the data show that FTE capacity and budgetary capacity largely mirror one another. For the most part, as the agency quartile for both FTE and budget increased within the agency designation, the percentage of agencies exceeding OGI compliance minimums also increased. In the budgetary quartile comparison, there was one exception to this, as one agency in the top quartile for budget only met the minimums and did not exceed them (Table 10).
second quartile met and exceeded compliance minimums as a higher rate than in the third quartile. The data indicate that, as capacity increases (as measured by either FTE
capacity or budget, relative to other agencies in the same designation), so does the level of participation. Table 10 shows this finding for the budgetary quartile comparison, and Table 11 illustrates the finding when comparing FTE quartiles. Boards, Commissions, and Committees were not included in Tables 10 and 11 because so few were participating (four), making it impossible to draw conclusions based on quartiles.
Table 11: Compliance Minimums Met or Exceeded Based on FTE Quartile Comparison (Measured in Percentages)
Quartile 1 Quartile 2 Quartile 3 Quartile 4
Meet Exceed Meet Exceed Meet Exceed Meet Exceed Executive
Department
25 75 25 75 0 100 0 100
Independent Agency
44 0 25 25 22 11 12.5 75
Conclusion and Recommendations
The Open Government Initiative, a major Obama Administration policy, has been adopted by many federal agencies in varying degrees. This paper examined agency participation with transparency portions of the OGI in online environments and found that both agency budget and FTE employee capacity are statistically significant to agency participation. Likewise, the data show that, for the most part, as agencies increase in capacity (as measured by budget and/or FTE employees), the likelihood of compliance or participation increases. Budgetary extremes at both the top and the bottom of the range of federal budgets impacted an agency’s decision to participate in the OGI. Additionally, there was a steady decline in participation moving down the agency hierarchy. This decline also holds true for representation on the White House Scorecard and in the Working Group.
the larger agencies. For instance, many small agencies have developed a social media presence in recent years. The terms of service agreements for many social media platforms were negotiated only after the OGI was signed (U.S. General Services Administration, 2011). This implies that smaller agencies may be using only the tools that work best for their missions and capacities, rather than straining already limited resources to fully comply with the OGI.
LIS professionals interested in this area must be mindful of suggested courses of action and be sure that each suggested step makes sense for the mission and character of the agency. If an LIS professional working for a small agency knows that the agency has a number of datasets, then adding them to the centralized repository at data.gov may be a good step to increase transparency. However, if a small agency has collected few
datasets but has a mission tied to outreach or education, then perhaps an enhanced social media presence would be a better choice.
I recommend that further studies be conducted to make concrete
recommendations to agencies and government entities with limited resources and capacities about how to employ open government techniques and principles. In particular, two further studies would be useful:
1. A study that determines if participation really is only a matter of funding and resources, or if there are other motivating factors driving participation, such as agency mission and culture, including characteristics of agency leaders and organizational structure.
some benefits themselves, but I believe that more research would be useful to establish best practices for open government implementation at various agency levels.
These studies would help open government activists understand why some agencies choose to participate and others do not, even if the agencies appear to be peers. They would also help sunshine groups make a practical case for participation to agencies that remain undecided.
The Open Government Initiative establishes compliance minimums and
References
Burton, M. (2010). A Peace Corps for programmers. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 1-9). Sebastopol: O’Reilly Media.
Chopra, A. (2010, April 8). Making government accountable for openness (Web log post). Retrieved from: http://www.whitehouse.gov/blog/2010/04/08/making-government-accountable-openness.
Drake, M. (2010). Life, liberty, and the pursuit of transparency. Information Today, 27(5), 1; 42-43.
Drapeau, M. (2009). Two way street: Government with the people. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 123-129). Sebastopol: O’Reilly Media.
Eaves, D. (2009). After the collapse: Open government and the future of civil service. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 139-151). Sebastopol: O’Reilly Media.
Fung, A. & Weil, D. (2010). Open government and open society. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 105-113). Sebastopol: O’Reilly Media.
Ginsberg, W. R. (2011, January 28). The Obama Administration’s open government initiative: Issues for Congress. Retrieved from:
http://www.fas.org/sgp/crs/secrecy/R41361.pdf.
Horvit, M. (2011). Opening up the government. The IRE Journal, 34(2), 4.
Hsieh, H.-F., & Shannon, S.E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.
Kundra, V. & Fitzpatrick, M. (2009, June 16). Enhancing online citizen participation through policy (Web log post). Retrieved from:
http://www.whitehouse.gov/blog/2009/06/16/.
Leggatt, H. (2011, Feb. 2). Facebook most popular, not most successful, social media tool (Web log post). Retrieved from: http://www.bizreport.com/2011/02/facebook-most-popular-not-most-successful-social-media-tool.html
Lukensmeyer, C., et al (2011). Assessing public participation in an open government era: A review of federal agency plans. The IBM Center for The Business of Government.
Mehmetoglu, M. (2004). Quantitative or qualitative? A content analysis of Nordic research in tourism and hospitality. Scandinavian Journal of Hospitality & Tourism, 4(3), 176-190. doi: 10.1080/15022250410003889.
Malamud, C. (2009). By the people. In D. Lathrop & L. Ruma (Eds.) Open
government: Collaboration, transparency, and participation in practice, (pp. 41-47). Sebastopol: O’Reilly Media.
Noveck, B. (2009). Wiki government: How technology can make government better, democracy stronger, and citizens more powerful. Washington: Brookings Institution Press.
Noveck, B. (2010, April 8). Open government plans: A tour of the horizon (Web log post). Retrieved from: http://www.whitehouse.gov/blog/2010/04/08/open-government-plans-a-tour-horizon.
Obama, B. (2009, January 21). Transparency and open government. Retrieved from: http://www.whitehouse.gov/the_press_office/Transparency_and_Open_Governm ent/.
OMB Watch Staff. (2010, February). Federal agency open government review criteria points. Retrieved from: http://www.ombwatch.org/files/info/criteria.pdf. OMB Watch Staff. (2010, February 23). Leaders and laggards in agency open
government webpages. Retrieved from: http://www.ombwatch.org/node/10785/. OMB Wiki. (n. d.). Retrieved October 30, 2011 from the White House Open
Government Wiki: http://expertnet.wikispaces.com/Getting+Started.
O’Reilly, T. (2010). Government as a platform. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 11- 39). Sebastopol: O’Reilly Media.
http://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-06.pdf.
Reich, B. (2009). Citizens’ view of open government. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 131-138). Sebastopol: O’Reilly Media.
Riffe, D., Lacy, S., & Fico, F. (2005). Analyzing media messages: Using quantitative content analysis in research (2nd ed.). Mahwah: Lawrence Erlbaum Associates. Rosengren, K. E. (1981). Advances in Scandinavia content analysis: An introduction. In
K. E. Rosengren (Ed.), Advances in content analysis (pp. 9-19). Beverly Hills, CA: Sage.
Schuler, D. (2010). Online deliberation and civic intelligence. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 91-104). Sebastopol: O’Reilly Media.
Sifry, M. (2010). “You can be the eyes and ears”: Barack Obama and the wisdom of the crowds. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. 115-122). Sebastopol: O’Reilly Media.
Staff Writers. (2009). Government records: A one-stop shop for government data. Information Management, Sept/Oct 2009, 8.
Steirnstein, A. (2011). Tracing transparency. Government Executive, 43(1), 24-29. Sunstein, C. R. (2010, April 7). Social media, web-based interactive technologies, and
the Paperwork Reduction Act. Retrieved from:
http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGui dance_04072010.pdf.
Tapscott, D. (2010). Foreword. In D. Lathrop & L. Ruma (Eds.) Open government: Collaboration, transparency, and participation in practice, (pp. xv-xvii). Sebastopol: O’Reilly Media.
U.S. General Services Administration Staff (website). (2011). 8.1 Terms of service agreements: GSA paves the way for government use of new media. Retrieved f rom: http://www.gsa.gov/portal/content/104320.
open government plans. Retrieved from:
http://www.whitehouse.gov/open/documents/leading-practices-open-govt-plans. White House Staff. (2010). Open government plan evaluation criteria. Retrieved from:
http://www.whitehouse.gov/open/documents/evaluation.
White House Staff. (n. d.). Around the government (open government scorecard). Retrieved from: http://www.whitehouse.gov/open/around.
White House Staff. (Unknown). Open government working group. Retrieved from:
http://www.whitehouse.gov/open/documents/open-government-directive/working-group.
Wilshusen, G.C. (2010). Challenges in federal agencies’ use of Web 2.0 technologies. Government Accountability Office. Retrieved from:
http://www.gao.gov/assets/130/125110.pdf.
