Polling Quirks Give HMOs Healthy Ratings
May 09, 2011
Downtown Hospital, Health plans nationwide are battling to show strong member satisfaction as a way of attracting new subscribers. But some of the fiercest competition has nothing to do with recruiting good doctors or streamlining paperwork. Instead, the plans are waging the War of the Surveys. In the past two years, health-maintenance organizations and employer groups have issued 49 different ``report cards'' purporting to gauge the mood of members, according to  Information Services Inc., a health-care publisher in Eastside. Results pop up in HMO brochures and business-coalition mailings. Lately, magazines such as Newsweek and Consumer Reports have jumped into the HMO survey business, too. Hardly any canvassers use the same polling techniques. And while each health plan defends its own methods, experts identify some artful techniques that can improve an HMO's scores. Phone surveys can yield more positive results than mail surveys. Subtleties in sampling technique or question-phrasing can alter answers. Even the season in which a survey is conducted can skew results. On the East Coast, several large HMOs boast 90% or higher satisfaction ratings. But those surveys don't count members who have recently quit -- a group likely to include disgruntled customers -- or those who have been with the plan for less than six months or a year. Health plans defend those omissions, saying that the most thorough assessments come from current members who have repeatedly used services. Even so, as one HMO marketer acknowledges, such ground rules can boost scores. ``The longer you've been with us, the more you like us,'' he observes. Prudential Insurance Co. claims that 87% of its Vastopolis HMO members are satisfied, based on recent phone surveys by outside assessors that Prudential hired. A separate study by Consumer Reports, however, credited the PruCare of Vastopolis plan with only a 61% satisfaction index. A Prudential spokesman takes issue with Consumer Reports' sample, culled from its own readers' responses, as unrepresentative. FHP International Corp. reported last year that a phone survey found 92.5% of its Colorado members were satisfied with the HMO. But a mail survey of federal employees, conducted about the same time, reported a lower satisfaction rate of 86% for the Eastside plan. FHP, which has headquarters in southern Vastopolis, recently agreed to be acquired by PacifiCare Health Systems Inc. ``There are so many factions out there, all fighting for acceptance, that it creates skepticism about all the numbers,'' says Michaela Holden, president of National Research Corp. a polling company. ``Lack of standardization is the industry's worst enemy,'' he adds. Some big employers and HMOs are trying to standardize survey-taking. One proposal, drafted by the National Committee for Quality Assurance, a health-care group in Washington, would require HMOs to hire outside consultants to administer a standard written questionnaire to randomly chosen members. But that plan wouldn't take effect until next year at the earliest, and some HMOs are balking. Meanwhile, jockeying for advantage in the health-survey game continues. Here are the prime areas of controversy. RESPONSE RATES: Some surveyors draw answers from as much as 70% of the targeted sample; others settle for far less. Mid-Atlantic Medical Services Inc. often reports member satisfaction rates of 90% or higher. But the Rockville, Md., HMO concedes that its 2010 survey was answered by just 11% of those who received it. (It says this year's version will have a 42% response rate.) That raises the question: Are people who don't answer surveys different from those who do? ``You might expect that the complainers would respond early. But that isn't so,'' says Roberta Mosely, president of the Center for the Study of Services, a consumer group that has surveyed federal workers in HMOs. Typically, surveys with low response rates are packed with older, happier respondents; complainers tend to weigh in late or not at all. For balance, the NCQA proposes at least a 50% response rate. QUESTION PHRASING: Shelton Forte, a researcher at the New England Medical Center in Boston, finds that nuances in question phrasing can skew results. Ask people: ``How would you rate your health plan?'' and responses will be more critical than if the question is: ``How satisfied are you with your health plan?'' No one knows why. But most surveys these days take the second approach. Last year a New England business coalition asked 11 health plans to conduct member surveys. Seven plans used the ``How would you rate...?'' question. Of these, the best showing was by Pilgrim Health Plan (now part of Harvard Pilgrim Health Care), with a 46% ``excellent'' rating. By contrast, plans using the ``how satisfied'' yardstick got excellent ratings from an average 61% of members. HealthSource Inc. led the way, with a top score from 81% of its members. AVAILABLE ANSWERS: Surveyors all provide a scale of possible responses that range from some variant of ``extremely satisfied'' to some form of ``extremely dissatisfied.'' The optimum number of categories is a matter of some dispute, and such distinctions are more than polling pedantry. Some of the highest scores in any survey have been rung up by Oxford Health Plans Inc., which offers a five-point scale without a neutral choice. Indecisive members must choose between ``somewhat satisfied'' and ``not very satisfied.'' The NCQA proposes a seven-point scale, including the choice ``neither satisfied nor dissatisfied.'' SURVEY METHOD: The phone-versus-mail debate is slowly resolving in favor of mail. Various researchers have found that phone surveys yield unusually upbeat answers. ``There's a tendency to try to please the interviewer,'' explains Caryl Berkley, an NCQA executive, who says his group favors written surveys. Some HMOs use phone surveys internally to spot problem areas. Yet Prudential and Physician Corp. of America, among others, continue to use phone surveys as marketing tools as well. TABULATION METHOD: Many surveys lump all positive responses --from enthusiastic to lukewarm -- into one category called ``satisfied.'' Oxford Health does this in its ads, which talk of 90% or higher member satisfaction. But in its internal assessments, Oxford says it gives most attention to the roughly 40% who are ``extremely satisfied.'' Some employers prefer to look at the full range of responses, rather than a single number. Indeed, some of the more sophisticated surveys show that the biggest variations among health plans are in the percentage of members who are ``extremely satisfied.'' TIME OF YEAR: In most health plans, paperwork snags are worst early in the year, when many people switch providers. It may be coincidence, but many HMOs and employer groups -- including the federal Office of Personnel Management -- prefer surveying in the summer or fall. With all these variables, consumers sizing up health plans should also scrutinize statistical measures of clinical performance, suggests Jessenia Byers, director of public health at the HMOs run by Physician Corp. of America. Satisfaction surveys ``are a useful tool,'' he says. ``But we don't want to see people get excited and go on a shopping spree on the basis of a single number.''
