VA touts patient satisfaction, but its findings are questionable
PHOENIX — Leaders at the Department of Veterans Affairs have asserted for more than two years that medical care and service for America’s ex-warriors should be measured not by how long patients wait for appointments, but how they respond to surveys.
Indeed, Secretary Bob McDonald and his team tout industry-recognized satisfaction surveys as evidence that VA hospitals are more customer friendly than other medical centers.
Despite months of requests, however, department administrators have failed to produce clear proof of those claims.
Instead, what emerges from VA interviews and literature is a dizzying maze of metrics used to evaluate success or failure of the nation’s largest integrated health-care system.
While there are all kinds of spreadsheets and reports, this is the bottom line: The VA has no tool that fairly compares satisfaction scores in veterans’ medical centers with those at non-VA facilities.
And, while the department has internal systems to evaluate care at its roughly 150 hospitals, administrators refuse to publicly disclose the rankings.
Dr. Peter Almenoff, special adviser to the secretary, was asked what surveys his bosses are referencing about patient satisfaction.
He did not identify one.
When asked again how one might honestly compare veterans’ hospitals with non-VA facilities, he told The Arizona Republic, “I can’t answer that.”
And the survey says?
The VA argument that measuring wait times is unfair — and patient satisfaction is a more appropriate gauge of health services — began to swell in April 2014.
Just days after The Arizona Republic first reported that veterans were dying while awaiting treatment in Phoenix's Carl T. Hayden VA Medical Center, a department news release boasted about a 2013 survey known as the American Customer Satisfaction Index.
The ACSI is used to measure customer response in 43 industries and at more than 200 private-sector companies.
According to VA officials, survey results show their hospitals are equal to or better than non-VA medical centers.
In an October 2014 speech for the Institute of Medicine, McDonald declared: “Since 2004, the ACSI has consistently shown that veterans receiving both VA inpatient and outpatient care give VA higher satisfaction ratings than patients at private hospitals.”
Just one problem: ACSI spokesman Chaat Butsunturn said the company does not do VA health-care surveys.
Instead, Butsunturn said, the Veterans Health Administration contracts with a separate spin-off firm, CFI Group, to do polls using ACSI’s model. “Though the Department of Veterans Affairs has called it an ACSI report, that is an error,” he added.
A CFI spokeswoman did not respond to inquiries.
During a phone interview with The Republic this summer, Deputy Secretary Sloan Gibson expressed shock when he was told ACSI LLC does not do patient polling for the VA.
“I will tell you that’s completely misleading from my perspective, and unacceptable,” he said.
Gibson also was advised that ACSI LLC does conduct a survey on overall public satisfaction with key federal agencies, including the Department of Veterans Affairs. In 2015, ACSI placed VA among the U.S. government’s “least satisfying departments” with a score of 60 on a 100 scale — 14 points down from 2012.
“Myriad problems experienced by Veterans Affairs in delivering health services to a swelling number of veterans are likely contributing to its low score,” the report notes.
When apprised of that survey, Gibson said, “You are catching us completely and totally blind.”
CFI’s 2015 report on veteran surveys contains a single comparison showing inpatient satisfaction results for the VA are 12% higher than the overall industry scores, and outpatient scores are 6% higher. However, appendices suggest the numbers are misleading — based on apples-to-oranges comparisons.
Methodology in the survey also raises questions. The VA gave CFI names and contact information for the patients to be surveyed. And the report says VA officials selected survey objectives different from the goals used to gauge satisfaction at private-sector hospitals.
‘Never, Never Land’
The VA refrain about satisfaction hit a crescendo in May during a McDonald sit-down with the Christian Science Monitor.
“When you go to Disney, do they measure the number of hours you wait in line?” McDonald asked. “What’s important is, ‘What’s your satisfaction with the experience?’ ”
Political backlash came in a swarm of soundbites. For example, House Speaker Paul Ryan, R-Wis., rebuked McDonald, saying, “This is not make-believe, Mr. Secretary. Veterans have died waiting in those lines.”
Oops, VA data missing
Directors at VA medical centers nationwide also harp on the patient experience, rather than appointment openings.
At the Edward Hines Jr. VA Hospital in suburban Chicago, Director Mark Magill put it this way: “The real thing that’s important about access isn’t days (waiting). It’s whether the patient is getting the right care in the right place at the right time. I think (we) have really excelled on access when you look at it in those terms.”
Magill was asked for survey results and rankings at Hines, and promised to follow up. His spokeswoman sent a link to Medicare's website containing evaluations for U.S. hospitals — including satisfaction measurements. The database contained this notation for Hines and all other VA medical centers: “Not available.”
Last month, Paste BN reported the backstory: “The Department of Veterans Affairs over the summer quietly stopped sharing data on the quality of care at its facilities with a national database for consumers, despite a 2014 law requiring the agency to report more comprehensive statistics to the site so veterans can make informed decisions about where to seek care."
“For years, the VA provided data on a number of criteria to the Hospital Compare web site run by the Centers for Medicare & Medicaid Services in the Department of Health and Human Services. ... But the VA confirmed to Paste BN ... that it stopped reporting its information July 1."
Seeing stars?
During his recent interview with The Republic, Gibson mentioned other ways of gauging VA success.
For example, he said, about 800,000 additional veterans have signed up for VA care during the last two years despite outside alternatives and a deluge of negative media coverage.
That kind of popularity is "a true litmus test,” he said. “Veterans are voting with their feet.”
Gibson and Almenoff also touted a VA report card for hospitals known as SAIL, for Strategic Analytics for Improvement and Learning. The SAIL report measures all kinds of service and quality issues, ranking VA medical centers from one to five stars.
Gibson said it is now used to grade the performance of hospital administrators — an improvement incentive that has driven up scores by 60%. “You hear about accountability all of the time,” he added. “That’s what you call sustained accountability. It’s not about taking somebody (who has underperformed) and firing them.”
Almenoff declined to make public the VA’s top-to-bottom scoring for medical centers nationwide. “That’s an internal ranking,” Almenoff said. “We don’t put those out externally. ... We try not to focus on the stars.”
The VA uses another tool to evaluate hospitals — the Consumer Assessment of Health Providers and Systems Survey — which allows patients to score hospitals on a 1-10 scale.
But the results published online contain no comparison with non-VA medical centers, and the database does not rank veterans’ hospitals from top to bottom.
Earlier this year, two-thirds of that survey's respondents gave VA health care an overall score of 9 or 10.
Gibson, meanwhile, seemed to revert to wait times as a success measure.
“If you look at the data across the enterprise, we have come light years — light years! — in how quickly we are able to close consults,” he said. “We’ve improved access to care, period. Period! And we’ve got the data to prove it ...”
Follow Dennis Wagner on Twitter: @azrover