On Wednesday, White House press secretary Josh Earnest sent The New York Times a letter asking journalists to give the Obama administration credit for government transparency, arguing that media columnist Jim Rutenberg did “did not acknowledge the important and unprecedented steps that the Obama administration has taken to fulfill the president’s promise to lead the most transparent White House in history” in his latest column. Since January 2009, the federal government has become more transparent in some ways and far less so in others, which naturally leads the press and public that relies on those assessments to view the success of President Barack Obama’s pledge to have the “most transparent administration in history.” In June, Brian Stelter, the host of CNN’s Reliable Sources, suggested that this is the least transparent administration in history, pointing to limited access to the White House, official secrecy and over-classification, and record numbers of denials to Freedom of Information Act (FOIA) requests for agencies, much as Rutenberg did in his column. The Washington Post’s Margaret Sullivan hit the same points in a recent column on the administration’s record. On Wednesday, Andrew Beaujon critiqued Earnest’s letter, calling attention to the full record. Erik Wemple took an inventory of journalists who disagreed with the White House press secretary’s assessment, drawing from the Committee to Project Journalists’ scathing 2013 report on the Obama administration and the press. In his letter, Earnest asked: “If journalists don’t acknowledge steps that the Obama administration has taken to strengthen transparency, then who will?” We’d ask him to look beyond the press corps in the briefing room and cable news shows. The White House press secretary ignored the role that government watchdogs have played in evaluating the strengths and weaknesses of the Obama administration’s open government initiatives, from the Project on Government Oversight to OpenTheGov.org to Public Citizen to the Government Accountability Project to Sunlight. Over the past seven years, we have both acknowledged this administration’s progress on open government, decried its failures and advocated for reform. The challenge that we and the rest of the country face in judging the veracity of Earnest’s argument is that there is no consensus for what the “most transparent administration” in U.S. history is, nor how we should assess it.

In that context, what would be the right gauge for judging adherence to transparency and open government? Here are some approaches to evaluating an administration’s openness that go beyond the headlines, and some thoughts about how to sort through them.

Compliance with FOIA

Media access to the president

Number of press conferences

Public access to government scientists and research

Support of open government reforms moving through Congress

Support for whistleblowers

Protection of domestic press freedom

Open data and proactive disclosure

Civic engagement and open innovation

Sunshine laws

National security and classification

Corruption, influence and scandals in the White House

Promotion of open governance globally

A mixed record. Taken together, one might achieve some kind of useful determination about how the Obama administration compares, even if it’s hard to speak to FOIA since it’s only existed for 50 years. On each count, answers exist that contradict the Obama administration’s “most transparent ever” claims — but context matters. On many ways, this administration not only compares well with history but has made history. On others, not so much.

This administration has made unprecedented releases of data with a measurable impact on many sectors of society at the same time that the White House and agencies have stonewalled the press asking tough questions. When it comes to the use of the internet to disclose data and documents, there are only two prior administrations that can be relevant. President Obama’s choice to make open government part of his first day in office reflects an optimistic worldview that connected democracy and legitimacy to transparency and accountability in the wake of a decade of secrecy around an expanding national security state. While the actions or inactions of his administration contradicted this in deed, it would be an error to dismiss the ideas themselves or the investments in time, money and technology as meaningless or divorced from an important aspect of American democracy that this presidency has deepened. The most definitive statement we can make today is that President Obama will leave office with a mixed record on open government, although it’s not too late to embrace reforms that will cement the aspects of this legacy that have merit or to connect these initiatives to the ideals and principles that the American experiment was founded upon. We hope that he and his staff do so, in public, ensuring that whoever occupies the Oval Office can not only build upon the successes but learn from the failures. Public trust in our government depends on it.

More From Sunlight Foundation: