Too good to count?

COEUR d’ALENE — Earlier this year, a team of high achievers from Coeur d’Alene was pumped up to win a gold, silver, or bronze in their field of expertise. Their team had a proud history of winning medals in competitions with teams nationwide.

This year’s group had worked hard together, from the grizzled veterans to the young rookies, to make sure they would excel when tested. In fact, they did. In one event, every single member of the 56-person team did as well as or better than their peers. In another event, only two fell below the average.

When it came time to give out the awards, though, this Coeur d’Alene team was nowhere near the medal platform. In fact, they weren’t even mentioned.

Why?

It wasn’t the first year that it had happened. Since 2015, this team had consistently overachieved — and seemingly been punished for it. Those who gave out the awards were not the ones who deemed them unworthy. Instead, it was an Idaho organization that refused to share their results with the medals committee.

As recently as 2014, this team had won gold as the best team in all the Gem State, and was recognized nationwide as one of America’s best teams. But after 2014, the middleman decided that there weren’t enough underachievers on Coeur d’Alene’s team, and that their performance results shouldn’t be shared. Since then the local heroes have gone unrecognized for their achievements, and disrespected by the other teams they bested across the state and nation.

After several years of this problem, one of the team’s biggest fans got together with the bureaucrats and the medals committee to figure out a way around the problem. They worked out an arrangement that seemed to satisfy all parties. The local team eagerly awaited the day when they would once again stand on top of the medals podium.

However, the middleman didn’t sign and return the agreement on time. This year’s medals ceremony came and went, and once again the team of high achievers from Coeur d’Alene — one of the best in the entire United States — went unwept, unhonored, and unsung.

That team is Coeur d’Alene Charter Academy. The medals committee is U.S. News and World Report, which annually gives out gold, silver, and bronze medals to the nation’s top high schools in its “Best High Schools” publication. The middleman is the Idaho State Board of Education, which has interpreted a law designed to protect students’ privacy in a way that has prevented Coeur d’Alene Charter Academy, its students, teachers, and community, from being ranked or even recognized.

GOING BACK

In 2013, Coeur d’Alene Charter Academy was ranked the 59th-best high school in the nation by the Washington Post’s “High School Challenge Index,” and the 89th-best in the nation by Newsweek’s “America’s Best High Schools.” In 2014, Charter was ranked the 47th-best high school in America by U.S. News and World Report’s “Best High Schools,” and 66th-best by the Washington Post. It was hailed as not only a success story for charter schools, but a gem in the Gem State’s educational crown. The local institution, paid for by tax dollars and open to students regardless of background, was the only high school in Idaho that scored so highly in the national rankings by these publications.

But beginning in 2015, it dropped off the map.

The students at Coeur d’Alene Charter Academy hadn’t suddenly developed amnesia. They hadn’t started flunking the state’s assessment test — the Idaho Standards Achievement Test — en masse. The only thing that changed was the Idaho State Board of Education’s application of state law.

In 2014, legislators concerned about the creeping collection of data on their students passed a law now known as Idaho Code 33-133. This regulates what data the state can collect on students, and with whom the state can share that data.

Idahoans concerned about snooping federal agencies rested assured that their students’ blood pressure, heart rates, and other personal data wouldn’t be shared willy-nilly. In accordance with existing laws designed to protect ethnic minorities, the state also required that no data for small groups be divulged. As state board data chief Carson Howell explained it, you wouldn’t want to share test scores for Native Hawaiians in a school where there might be only one or two. Anyone could recognize which students’ scores they were looking at if data were shared in that way, he explained.

The practice of data suppression exists in education and in many other fields for this reason. The state calls it “redaction,” but the result is the same. It means that when the public sees information on one of the public schools they pay for, such as Coeur d’Alene Charter Academy, they see far less data than actually exists. Howell explained that after the 2014 law, the state’s Data Management Council established 10 as the smallest number of students whose scores they’d report on. More recently, they dropped that minimum to five.

Howell interpreted 33-133 to not just refer to visually identifiable groups, such as males and females, or people of various races. Instead, he applied it to any group smaller than the minimum number, whether or not the students’ identities could be ascertained by a reasonable person. Because of this, when it came time for the state board to report Coeur d’Alene Charter Academy’s ISAT scores to publications such as U.S. News and World Report, the state didn’t share the data.

Why?

Howell said there were so many proficient students at Coeur d’Alene Charter that releasing the school’s ISAT data would show a tiny number who scored below proficient.

“They were excluded unfortunately because they had so few kids scoring low on the test,” he said in a phone interview Wednesday.

To Coeur d’Alene Charter Academy Principal Dan Nicklay, that interpretation is inexcusable.

“The law says you can’t publish a data set with fewer than 10 students in an ‘identifiable subgroup’ (race, ethnicity, gender, etc.),” Nicklay said. “If the state releases our data on 300 kids and only five are non-proficient, there is no distinguishing trait (ethnicity, etc.) that makes it possible to identify those five students.

“Isn’t it patently ridiculous that we can’t be recognized because our scores are too good?”

GOING FORWARD

This shouldn’t be so complicated, Nicklay said. All the state has to do is provide the overall test results for the school and the requirements of the law will be satisfied.

Idaho Code 33-133 says a student’s personally identifiable information includes, but is not limited to, information such as the student’s name, address, and Social Security number. In addition, it includes “other indirect identifiers, such as the student’s date of birth, place of birth and mother’s maiden name; and other information that, alone or in combination, is linked or linkable to a specific student that would allow a reasonable person in the school community, who does not have personal knowledge of the relevant circumstances, to identify the student with reasonable certainty or information requested by a person who the educational agency or institution reasonably believes knows the identity of the student to whom the education record relates.”

According to Howell and ISBE spokesman Mike Keckler, this includes Coeur d’Alene Charter Academy’s overall scores on the ISAT math and ISAT English language arts tests.

“This is a small school and it probably isn’t hard for students to have some idea who is struggling in math,” Keckler said. “By including that small percentage number, anyone familiar with the school can do the math and probably figure out who the student is that tested at the lowest possible level. Federal and state law require that performance data not be released of any group of five or fewer in order to protect student privacy. That is why these data masking procedures are in place.”

Keckler requested that The Press withhold information on Coeur d’Alene Charter Academy’s ISAT results for this reason. The Press declined and Nicklay provided the data.

During the 2016-17 school year, 95 percent of Coeur d’Alene Charter Academy’s 74 sophomores earned proficiency in English language arts. Seventy-three percent of them achieved proficiency in math, according to the same state data provided by Nicklay.

Comparing the school with the rest of the state is even more illuminating. During the 2017-18 school year, 100 percent of Coeur d’Alene Charter Academy’s 56 sophomores tested proficient in English language arts, with 66 percent testing at the highest level possible. That stands in sharp contrast to the 40 percent of sophomores statewide who failed to achieve proficiency, with 17 percent scoring the lowest level possible.

In math, 89 percent of Coeur d’Alene Charter Academy’s sophomores achieved proficiency this year, with 56 percent testing at the highest level possible. Only 2 percent of its 55 sophomores tested at the lowest possible level in math, whereas 39 percent of the entire state’s sophomores tested at that level and 67 percent failed to achieve proficiency.

“Simply saying two students in seventh grade failed a test does not offer any information that would make those two students identifiable,” Nicklay said. “The fact that the person in charge of data for the state can’t understand this is simply unacceptable.”

The firm used by U.S. News and World Report for its “Best High Schools” publication is RTI International. In its four-step methodology used to rank high schools, Step 1 requires state assessment data from each high school’s state. Robert Morse, chief data strategist at U.S. News and World Report, told the Press on Thursday that “Coeur d’Alene Charter Academy was not eligible for Step 1 since the Idaho state assessment data was suppressed by the state.”

Howell, however, insisted that “any change would have to be legislative.”

CALL FOR CHANGE

Terry Ryan has worked with charter schools in Ohio and Idaho for 20 years. He said Friday that he had never run into this issue before the state board began balking at releasing Coeur d’Alene Charter Academy’s data a few years ago.

“All the schools that deserve recognition should get the recognition,” said Ryan, CEO of nonprofit educational organization Bluum. “We as taxpayers should know what’s going on with our schools. When they’re recognized nationally that’s great for everybody — the state, the kids, and families in that school. We shouldn’t make it hard for that to happen.”

Nor should it be this hard for the state to come up with a way to work with a nationally respected research firm, said Ryan, but three years later he was once again shocked to find that Coeur d’Alene Charter Academy was unranked in the “Best High Schools” publication.

This time the shock was not just over questionable data suppression, but over basic competence.

On Jan. 5 of this year, Ryan held a conference call with Howell, RTI, and chief technology officer Chris Campbell from the Idaho State Department of Education. Following that call, the state board and RTI created a 16-page memorandum of understanding that would allow data for Coeur d’Alene Charter, and other Idaho schools with similarly small numbers of below proficient students, to be shared and give them national recognition in the “Best High Schools” publication. According to Keckler, state board staff received the MOU on April 2 with a deadline for the signed MOU and the relevant ISAT data to be returned to RTI by April 6. If the state didn’t return the MOU and the data to RTI by that date, Coeur d’Alene Charter would again go unranked.

And that is exactly what happened.

“The MOU was signed both by our Executive Director Matt Freeman and by Carson [Howell] on April 3. The mistake occurred when we failed to provide the data in time to meet their April 6 deadline,” Keckler said.

He added, “It was our mistake and we will do better next year.”

Sen. Mary Souza said she is “very frustrated at the lack of follow through” shown by the state. She had been aware of problems with Coeur d’Alene Charter Academy’s data for the past two years but was surprised to learn that now state officials are citing 33-133 as a reason for suppressing it.

“If legislation is needed next session to clarify the data law, I’m ready to help,” she said.

Rep. Ron Mendive said he, too, is willing to step in and help.

“The intent of the law was to prevent data from being collected on every student from being widely accessible,” he said. “I think the state has taken code that was designed to protect individual privacy and broadened it to protect individual schools’ outcomes from being evaluated against other school outcomes. Parents, teachers, and legislators need to know which schools are succeeding and which schools are failing. I am personally disappointed that the state did not do their part to correct this and return the MOU on time. Cd’A Charter deserves to be recognized for their accomplishments.”

He added, “In our next legislative session we need to meet with appropriate state agencies to review and address any need for clarification of this policy.”

ANOTHER VIEW — AND CONCLUSION

Rep. Paul Amador took a different stance, saying he agrees with the state’s actions to suppress Coeur d’Alene Charter Academy’s data.

“The generally accepted standard in education-related research to protect individual privacy is that a category within a data set should never have fewer than 20-25 individuals,” Amador said. “The reason for this standard is that it is quite easy to cross reference between categories to narrow down to a particular individual, therefore compromising their privacy. By excluding or eliminating categories because there are fewer than 20-25 individuals, one would risk providing an incomplete and inaccurate analysis of the overall data. Generally speaking, this is not advised in conducting educational research.

“In this instance, I believe it is appropriate to err on the side of caution in protecting the privacy rights of Idaho families. I am also concerned that by pursuing the release of limited and specific categories within the student performance data at one particular school, we risk providing an incomplete and inaccurate analysis as compared to other schools within the report.”

Whether the law is clarified in the next legislative session remains to be seen. But for now, Howell will no longer oversee the state’s educational data.

He has been promoted.

“Carson has been promoted to Chief Financial Officer for the State Board and assumes his new duties next week,” Keckler said Thursday. State board principal research analyst Cathleen McHugh will take Howell’s place as state board chief research officer and chair of the Data Management Council, Keckler said.

•••

To read the “Best High Schools” rankings, go to: bit.ly/CDACharter