SALT LAKE CITY — The group running TestUtah.com’s statewide testing system has declined to participate in a proficiency test that other major Utah labs conducting COVID-19 testing have agreed to join in order to verify each others’ accuracy.
© Ivy Ceballo, Deseret News
Health care workers operate a COVID-19 drive-thru testing site at Timpanogos Regional Hospital in Orem on Monday, April 6, 2020.
Utah’s state epidemiologist Dr. Angela Dunn said Thursday TestUtah.com’s lab was the only lab not to participate in the proficiency test designed specifically to verify sensitivity of different tests.
Rather, the group has agreed to a more scaled down test with the state’s lab — an experiment that may not produce as comprehensive results related to test sensitivity, according to the ARUP medical director who is over the larger proficiency test.
Proficiency testing is a standard process accredited labs use to verify each others’ testing accuracy
“I have no idea,” Dunn said when asked why the TestUtah.com groups have declined to participate in the larger proficiency test being conducted by ARUP.
But officials with MountainStar, the health care system that owns Timpanogos Regional Hospital in Orem where TestUtah.com tests are processed, and the firm that produces the testing kits say TestUtah.com’s tests have already been validated, and they’ve been willing to participate in proficiency testing — but they’ve taken issue with this proficiency test’s process, indicating issues with transparency and independent review.
“We have been more than happy to participate in proficiency testing and will be happy to participate in further proficiency testing with the state as long as all other testing locations are included and the process for such a validation is agreed upon by all parties as well as an independent reviewer,” said MountainStar spokesman Mike Graul in a prepared statement. “That hasn’t been the case to this point.”
Co-Diagnostics, the firm that has produced TestUtah.com testing kits, said its test has earned excellent performance ratings in third-party assessments and would be happy to participate in any independent, and transparent, analysis for the state.
“When the state of Utah reached out, Co-Diagnostics indicated a willingness and continues to be willing to participate in a transparent evaluation of test performance,” a company statement read. “The state indicated that if samples were available, they would come from ARUP. As of today, those samples have not been received but we hope they are forthcoming.
“The company has participated in several independent evaluations of test performance which have consistently demonstrated a high degree of sensitivity and specificity.”
Dr. Mike Baumann, MountainStar’s chief medical officer, contested any claim that MountainStar’s Orem lab has been unwilling to participate in proficiency testing.
“The notion that the Timpanogos lab is unwilling to undergo proficiency testing is incorrect,” Baumann said in a prepared statement. “Labs nationwide are working under an emergency use authorization (EUA) because COVID-19 is caused by a novel virus. We have even submitted lab data to the FDA.”
Graul said MountainStar has communicated the Timpanogos lab’s willingness to participate in the proficiency test to the Utah Department of Health “and have yet to hear back or be invited to participate in further discussion about proficiency testing criteria.”
Dunn, when asked about Graul’s statement, said officials are only now finishing the first proficiency test’s first round, and there hasn’t been a need yet to complete another round of testing. For that next round, Dunn said the officials will work to determine if TestUtah.com will be a part of that experiment.
Questions of TestUtah.com’s testing accuracy surfaced after state data last month showed TestUtah.com’s rate of positive results was much lower than other Utah labs have reported.
TestUtah.com’s positivity rate was 1.3% in April compared to 7% from the Utah Public Health Laboratory, 5.9% from HCA Healthcare, 5.3% from ARUP, and 4.6% from Intermountain Healthcare, according to an April 28 email from Nathan Checketts, director of medicaid and health financing in the state’s health department, obtained by the Deseret News.
“Overall, TestUtah has identified a lower percentage of positive cases than what other labs have reported, even after controlling for the fact that TestUtah has tested far more asymptomatic individuals than other labs,” Checketts wrote. “We have been watching TestUtah’s positivity rate closely and have had multiple discussions with them about different practices that may be impacting their testing results. For example, at our recommendation, they changed their sample collection procedures from an oral swab to a nasal swab in order to improve the specimens they collected.”
Health department spokesman Tom Hudachko told the Deseret News last month there are “a few potential reasons” for lower TestUtah positivity rates, including that they are testing asymptomatic people, and they were originally collecting samples using oral swabs before switching to nasal swabs.
“The test is the same type of test as what is being used at other labs, although I can’t confirm that other labs are using the same test kit manufacturer,” Hudachko said.
Geographic location of TestUtah’s sites could also influence positivity rate, with sites in areas including Ogden, Heber, Orem, Provo, Vernal and St. George.
The aim of the proficiency test is to gain an understanding of Utah’s testing accuracy, Dunn said.
“It’s good practice to do a proficiency test to determine how much virus is needed in a sample to trigger a positive so we can better understand the accuracy of these tests,” Dunn said.
In a compromise with TestUtah, Dunn said the health department proposed an alternative to the proficiency test. TestUtah agreed to send 50 test samples to the state’s lab, and the state lab sent 40 specimens to TestUtah. The state lab will compare and analyze the results, expected to be finished this week.
“Because TestUtah is a new venture and a new partner, not a traditional lab or health care system we’ve worked with in the past and the state of Utah has a contract with, we’re doing our due diligence to make sure the results are as accurate as they can be,” Dunn said.
But it’s not clear whether those test results will be public, Dunn said.
“We don’t know yet,” she said, adding that state officials are working with attorneys to determine whether the results can be made publicly available.
Earlier this month, all of Utah’s major labs conducting COVID-19 testing — including ARUP Laboratories, Intermountain Healthcare and the state’s public health lab — agreed to participate in the proficiency testing, sending samples with varying concentrations of the virus to test the sensitivity of their tests, according to David Hillyard, an ARUP medical director.
“Labs around the world are grappling with how to best test for COVID-19,” Hillyard said. “We want to do our very best to be able to pick up positives.”
Hillyard said rates of detection of COVID-19 cases depend on not just the population being tested and their geographic location, but also on how the test samples are collected and how sensitive the test is, or its ability to detect varying concentrations of the virus.
“So naturally we, like many of our colleagues, want to better understand the comparative sensitivity of different tests,” Hillyard said. “A test that lacks sensitivity won’t pick up as many positives. It’s a fundamental and very important question.”
Hillyard said the idea of a “blind” proficiency test of samples with varying concentrations of the virus was “enthusiastically embraced” by a working group representing state testing labs.
“The more we know about the performance of tests and collection devices, the better off we are in understanding viral transmission and how to protect populations (amid the pandemic),” Hillyard said.
Asked about TestUtah’s alternative test being conducted with the state’s lab, Hillyard said an exchange of a smaller number of random samples “is not a good way to understand” sensitivity of the tests.
“It’s not very informative about sensitivity just to exchange a few samples,” he said.