Assessment of Medicare Part D Communications to Beneficiaries

Research on health communications has gained prominence over the past few years.¹ In the United States, healthcare consumers are expected to read and act on communications from various sources, including federal and state governments, the Social Security Administration, private insurance plans, managed care organizations, and voluntary health agencies. Written materials are not the only means of acquiring health information, but they are the most widely used tool for disseminating crucial information. Therefore, it is necessary to evaluate the quality of such communications for accuracy and to reduce redundancy and errors.

According to a 1992 national survey, most American elderly beneficiaries read at the 5th-grade level.² There is an appropriate concern that healthcare materials are written at higher grade levels and may not adequately educate or benefit the intended population.^3-6

Most studies in health communications measure patient knowledge of a specific disease state.^3-6 It is, however, important to measure patient comprehension of healthcare materials in addition to their knowledge. Such comprehension may be affected by a variety of factors, including, but not limited to, inadequate health literacy, readability of materials, and complexity of addressed topic(s), as well as readers’ interest levels and cognitive abilities.

The Medicare program is divided into 4 parts: A, B, C, and D. Part D is the recent outpatient prescription drug benefit introduced through the Medicare Modernization Act (MMA) of 2003, implemented on January 1, 2006. Most people pay a monthly premium for this prescription coverage. Enrollment in Part D is voluntary, with penalties for late sign-ups.

Medicare Part D
Under Medicare Part D, distinct types of plans are offered to beneficiaries—stand-alone prescription drug plans (PDPs) or Medicare Advantage Prescription Drug (MA-PD) plans incorporating prescription drug coverage with other managed care benefits.⁷ In principle, based on the MMA standard benefit design, a Medicare beneficiary is involved in cost-sharing. Part D’s standard benefit design has a unique feature known as the coverage gap or doughnut hole.

Each year this coverage gap is expected to widen, and in 2010 the standard benefit package involves an annual deductible ($310), 25% of drug costs up to the initial limit for the doughnut hole ($2830 in total costs for covered drugs), and little cost-sharing beyond the upper limit for the doughnut hole ($6440). This means that beneficiaries who have a standard benefit plan and do not receive low-income subsidies would be responsible for $4550 in out-of-pocket costs before reaching the catastrophic coverage limit of $6440.⁸ Although the standard benefit must be offered by PDPs and MA-PDs, most plans also offer actuarially equivalent plans. Some of these plans may involve partial or complete coverage during the doughnut hole.⁸

Several new concepts were introduced within Part D, including the doughnut hole, formulary management, step therapy, and quantity limits.

It is important to realize that many elderly beneficiaries may not have been familiar with some or all of these concepts in the previous Medicare program. Beneficiaries receive information from the Centers for Medicare & Medicaid Services (CMS) through its Medicare & You handbook, the Medicare website (www.medicare.gov), and a toll-free telephone number.

In Part D, formularies vary considerably within different plans and among different PDPs or MA-PDs.⁹ Quantity limits are put in place by a plan sponsor to restrict the amount of drug prescribed (eg, 3 months). Step therapy requires that a certain drug be tried out before the prescribing of a newer and/or more expensive therapy to prevent improper utilization. Prior authorization requires healthcare providers to seek approval before providing certain drugs to beneficiaries.

These concepts are managed care tools used to control prescription drug utilization. Although many people with previous insurance may be familiar with these terms, many elderly beneficiaries may not be familiar with these terms, either because of a lack of previous insurance, dependence on caregivers/spouses/children for insurance matters, and/or their own cognitive or educational limitations.

Medicare Communications with Beneficiaries
The Balanced Budget Act of 1997 mandated that general and managed care plan comparison information be mailed to all current beneficiaries by October 15 of each year, beginning in 1999. CMS (formerly known as the Healthcare Financing Administration) initiated a National Medicare Education Program (NMEP) to inform and educate beneficiaries about Medicare+ Choice plans and provide them with general and comparative information about their health insurance options.¹⁰ “The specific objectives of the campaign are to ensure that beneficiaries have access to accurate and reliable information, are aware of the different health plan choices available to them, understand the consequences of choosing different plans, and are able to use the information provided to them when making decisions.”¹⁰

The Medicare & You Handbook CMS would like Medicare beneficiaries to view the Medicare program and its private sector partners as trusted and reliable sources of information.^10,11 The agency developed the consumer handbook Medicare & You to explain health plan options to beneficiaries. This handbook (formerly known as the Medicare Handbook) was pilot tested in 5 states and the Kansas City metropolitan statistical area in the fall of 1998, when CMS mailed the handbook to 5.1 million beneficiaries.^12-14

Table 1

The Medicare & You 2000 handbook was mailed to all 39 million elderly and disabled beneficiaries in the fall of 1999. Later versions were mailed each fall to beneficiaries. The education campaign also provided a toll-free telephone helpline; an Internet information database; support and training; counseling services; and state- and community-based outreach efforts (Table 1).

The Medicare & You handbook is one of the key sources of information for Medicare beneficiaries— information that is extremely detailed and legitimate, because it comes from CMS and from several other sources, such as plan materials, state and local organizational information, and the Medicare website. In 2008, there were 59 geographic-specific versions of the handbook with drug and health plan comparison charts for particular states. The 2008 handbook had approximately 120 pages (depending on the version used) and was issued in English, Spanish, Braille (English only), audio, and large print. Each fall, CMS mails a geographicspecific version of this handbook to all households of persons with Medicare coverage.

Assessment of Medicare Communications
Relatively few studies have examined the appropriateness of the Medicare & You handbook for informing and educating beneficiaries. The handbooks and worksheets used to compare plan information would be useful to new beneficiaries if they were mailed up to 1 year in advance, and would increase the likelihood of new beneficiaries actually using the worksheets to compare plans and make better-informed choices.¹²

Our study focused exclusively on this handbook, because it is the most comprehensive document regarding Medicare. Beneficiaries have access to other sources of Medicare-related information, but analyzing these pieces of information is beyond the scope of this study. In 2006, the US Government Accountability Office (GAO) analyzed 6 of 70 CMS documents on Medicare Part D and indicated that reading levels for analyzed passages ranged from 7th grade to post-college. The majority of American seniors read at or below the 5th-grade level, suggesting a significant scope for improvement.15 Findings from a more recent GAO study indicate that CMS’s model annual notice of change did not communicate drug plan changes effectively to beneficiaries.15 This study showed that “the language contained in the mailings was at a reading level too high for beneficiaries, and it contained irrelevant information.”¹⁵

Other studies conclude that based on reading materials, individuals aged ≥65 years are less proficient than younger adults in locating information in documents to make health-related decisions.¹⁶ A 2003 national survey on adult literacy demonstrated that 27% of Medicare beneficiaries were unable to understand information in short, simple texts.¹⁷

In light of evidence that some older individuals face challenges in reading and retaining written information,¹⁵ the design and evaluation of appropriate writing materials for the elderly population are particularly important. Readability testing of written communications may be a first step toward compiling healthcare materials that are comprehensible and beneficial for the intended readers.

Evaluating Readability 
Readability of healthcare materials is an emerging yet underrated area of academic research. Readability is the ease or comfort of reading text and includes legibility (ie, words can be read) and comprehension (ie, understanding the text). In the 1930s, psychologists studying the processing of written information concluded that longer sentences (>20 words) are difficult to grasp, and readers find it easier to understand simple words.¹⁶ Indices used to measure readability depend on sentence length and the number of “hard words” that appear in each sentence.¹⁸

Readability analyses—primarily used in schools to ascertain that students can read and comprehend materials at particular grade levels—are a necessary and useful tool when preparing important and timely information, such as Part D prescription drug coverage. Previous studies have shown that readability of written healthcare communication is consistently beyond average patients’ reading grade levels.^3-6

As mentioned earlier, approximately 20% of the US adult population cannot read beyond a 5th-grade level.2 It is therefore important that all adult healthcare materials be written at a 5th-grade level or lower to adequately educate the target population.

Methods
Our study measured the readability of the CMSproduced 2008 Medicare & You handbook, a comprehensive, lengthy document detailing plan options and benefits for Medicare beneficiaries.

Data
We downloaded the 2008 English version of the Medicare & You handbook from the CMS website. Based on the online Lexile Framework for Reading (www.lexile.com) measurement instructions, all passages with >250 words were saved individually in Windows XP Notepad as text files. Passages with <250 words were combined with the next continuing passage.

A total of 64 passages were analyzed using the Lexile Framework for Reading (permission to use and include results in the study was obtained before beginning the study). Tables, illustrations, and figures were not included in the analysis.

Instrument
Lexile measures have been used in the assessment of adult communications.^19-21 Assessing the readability of healthcare materials is a fairly new endeavor. The Lexile Framework for Reading was developed with federal funding in the 1980s. Older methods were more routine for school level assessments (Flesch-Kincaid, Dale-Chall, Simple Measure of Gobbledygook) and have limited use in measuring the readability of adult communications.

The Lexile formula is based on sentence length and word frequency counts. Based on the Lexile theory of comprehension, passages with higher scores (between 1800L and 1900L) are more difficult to read than passages with lower scores (between 1300L and 1400L).

Grade levels were calculated by averaging the corresponding grade regions of the Lexile scores. For example, a Lexile score of 670 would fall into 2 regions—3rd and 4th grade. This score was assigned the average of these 2 grade levels—3.5.

Table 2

Results
Nearly 30% of the Medicare & You handbook (19 of 64 passages) scored above 12th-grade readability levels, and 70% of the handbook (45 of 64 passages) scored from 5th- to 12th-grade readability levels (a href="/images/images/2010-Sept-Oct/Table2-pg313.png" target="_blank">Table 2). An average grade level of 10.23 for the handbook suggests that there were more passages at higher reading levels.

Beyond the 12th grade, the number of years of education to achieve advanced degrees may vary; therefore, averaging the number of years of schooling beyond the 12th grade may lead to errors in estimation. Average grade levels were therefore not computed for the 19 passages (ie,˜30% of the material) that were beyond the 12th-grade readability range (a href="/images/images/2010-Sept-Oct/Table2-pg313.png" target="_blank">Table 2).

Table 3

Table 3 presents the grade-level readability of the 45 passages that were within the 5th- to 12th-grade readability range. Few passages scored (Lexile scores) at lower grade levels. Only 2 passages were found to correspond to an approximately 7th-grade level (actual level, 6.5), and only 1 passage scored a Lexile value that corresponded to an approximately 5th-grade reading level.

Subanalysis: Lexile Scores for the Part D Sections 
Because Medicare Part D introduced new concepts that may have been unfamiliar to beneficiaries accustomed to the Medicare program for Part A and Part B, a subanalysis of Part D sections was necessary.

Part D sections were found on pages 37 to 74 in Section 2 of the handbook, and on pages 76 to 88 in Section 3. Some information in Section 3—such as information about grievances—is common to all the different Medicare programs (Parts A, B, C, D). However, because information to file for appeals and exceptions may be more important in the context of Part D, this information was included in the subanalysis.

Lexile measures were computed for Part D passages. There were a total of 38 passages with ≥250 words. The mean Lexile score for these passages was 1243.68L, with a minimum score of 970L and a maximum score of 1690L.

Of the 38 passages on Medicare Part D, 10 passages (26%) showed a difficulty level beyond 12th grade. Approximately 10 passages (26% of the Part D sections) had a 12th-grade reading level and 10 passages (26%) had a 10th-grade reading level. Only 1 of 38 (2.6%) passages had an approximate 6th-grade reading difficulty level, and the remaining 7 passages (18%) had gradelevel difficulties ranging from 7th to 9th grade.

Discussion
The Medicare & You handbook could be a much more useful tool for informing and educating beneficiaries. It contains pertinent information about plan choices, appeals, grievances, and exceptions, but omits a basic understanding of the societal context of the program, fails to list current challenges facing the program, and does not do an adequate job of explaining the meaning of Medicare reform for beneficiaries and their families.22 Changes in the recent congressional legislation will warrant explanation of governmental influence on the Medicare program.

The problem with written educational materials is that there is often a gap between the reader’s Lexile measure (ability to read and understand text) and text Lexile measures. The Medicare handbook’s Lexile scores indicate an average grade level of approximately 10th grade. This may be higher than the average beneficiary’s reading level and their grade level of education. Nineteen passages of 64 (30% of the handbook) were beyond the 12th-grade education level, indicating that these passages may not be readable or comprehensible by the average beneficiary.

The Medicare handbook is a standardized document that provides information on how to enroll, types of plans, procedures for grievances, and a definition of terms, but it does not conform to an individual’s specific situation. For example, if a beneficiary who may be eligible for a low-income subsidy is enrolled in a plan and would like to change it, there is very little information in the handbook on how to do so.

Healthcare communicators face numerous problems when dealing with a large subset of the population. With the increased complexity of Part D and the need to secure proper and safe use of pharmacotherapy, beneficiaries and their families have a greater need to understand instructions, follow procedures, interpret coverage information and forms, and then act on all these steps to make the best decision for their health.

This is vital to properly selecting a drug plan for their prescription benefit. This process—daunting enough for people with adequate literacy skills—can compromise the health and safety of persons with low literacy skills, as well as of US residents with limited English proficiency.²³

At the beginning of the implementation of Part D in 2006, there were many plans (approximately 30-60, depending on the state) available to Medicare beneficiaries.²⁴ During the first 2 years of implementation, 22.5 million (53%) beneficiaries signed up for Part D.²⁵ A significant majority of Part D beneficiaries reported that the benefit was too complicated, and observers suggested that such complexity might have thwarted some beneficiaries in finding a suitable plan.²⁶ Assessing the impact of Part D on healthcare utilization by the elderly is extremely important to evaluating the program’s viability.

The introduction of private plans into Medicare has created a market scenario in which beneficiaries are free to choose from a number of different plan offerings. Whether offering multiple plans affects healthcare utilization, adherence to medication, or improved quality of life for the elderly is not yet known. This may be partly because Part D is a newer program, and obtaining a longitudinal database of Part D beneficiaries to demonstrate such outcomes will take time. CMS is just now assembling such data for researchers.

To evaluate Part D’s sustainability, it is important to understand whether elderly beneficiaries are able to make the type of informed choices that are expected of them. Medicare is a social policy program introduced as a benefit by right of citizenship. In this social policy, the introduction of private parties has also brought about managed care techniques, such as formulary restrictions, copayments, coinsurances, step therapy, quantity limits, the doughnut hole, penalties for late sign-ups, and annual enrollment periods. It is important to recognize that these concepts may be unknown to many elderly beneficiaries, who may not be cognitively intact to process them along with the additional burden of weighing and sifting multiple plan options, some of which may not be significantly different from one another.

For beneficiaries who are disabled or have cognitive impairments, the burden of selecting, enrolling in, and utilizing plan benefits may rest on their caregivers, families, or even the long-term care facilities where they reside. Because of penalties for late sign-ups that may last for the beneficiary’s entire plan period, it is important to ensure that beneficiaries sign up in a timely manner. Decision-making for elderly beneficiaries to enroll in a Part D plan is certainly not easy, and caregivers who have assisted seniors will agree that this decision-making requires a certain level of knowledge, literacy, and cognition to navigate the plethora of choices available. Existing beneficiaries also need to reevaluate their plan choices to ensure their optimal outcomes.

Because CMS is the administering agency for Medicare and Medicaid, it can be assumed that any communication coming from the agency would be complete and be accurately designed to benefit seniors. As previously noted, CMS ideologically claims that it would like beneficiaries to make informed choices and perceive the Medicare program and its private partners as trusted and reliable sources of information.¹⁰ However, Medicare communications have not always been written at beneficiaries’ educational and/or health literacy levels.

The readability problem with Medicare documents is neither new nor unique to Medicare publications. In the case of David v Heckler, the husband of a female patient in New York received a letter from the Department of Health and Human Services (DHHS) in 1984 when his wife died of cancer.²⁷ Like many Medicare beneficiaries, the patient’s husband received far less remuneration than what he had expected. The letter explained why he received such little remuneration. His inability to understand the letter brought Legal Services Corporation to file a class action suit on behalf of all Medicare beneficiaries in the state of New York. Legal Services pointed out that 48% of the Medicare population had less than a 9th-grade education. Dr Edward Fry (originator of the Fry Readability Graph) testified that the letter was written at grade-16 level, or at a level suitable for persons with a college-level education. As a result, the judge ordered DHHS Secretary Margaret Heckler to take “prompt action” to improve the readability of Medicare communications.²⁷

There is still significant scope for improvement. According to Dr Fry’s testimony, inclusion of tables and pictorial depictions may improve the readability of a document.27 The 2008 Medicare & You handbook is comprised mostly of text, with few tables and charts. Almost all the tables and figures are placed toward the end of the book, and for the Part D sections, 26% of the 38 passages had a readability level beyond 12th grade.

The Part D private market is not inherently stable. Since 2006, several plans opted out of the program, thereby necessitating that beneficiaries reevaluate their choices for the next year.

Policymakers may benefit by continuing to monitor beneficiary satisfaction with plans and learn important information through research on how well beneficiaries comprehend plan benefits and key factors behind such decision-making. Few studies have assessed elderly beneficiaries’ knowledge about Medicare, and even fewer have evaluated comprehension of the newer Part D concepts.

As the federal government anticipates the rollout of various programs to reform the healthcare system, comprehension of all new directions is vital for the public’s understanding, support, and benefit. In 2010, President Obama signed the Health Care Reform Bill, which includes future recommendations and implications for Medicare, particularly Part D.

Effects of the 2010 Healthcare Reform on Part D
Under the new Patient Protection and Affordable Care Act (PPACA) of 2010, there is a provision for a “voluntary agreement with the Pharmaceutical Research and Manufacturers of America (PhRMA) to provide discounts of 50% for brand-name drugs used by Part D enrollees in the Part D coverage gap. Manufacturers of prescription drugs will be required to enter into agreements with Medicare Part D drug plan sponsors to provide discounts on drugs provided to plan enrollees in the coverage gap period beginning January 1, 2011.”⁹

The discount amount, along with the actual amount paid by the enrollee, will be counted toward costs incurred by the enrollee. Beneficiaries receiving any low-income subsidies or manufacturer discounts are not eligible for this discount. The PPACA mandates participation in this program by manufacturers, further stating that, “Drugs sold and marketed in the US by a manufacturer will not be covered under Part D unless the manufacturer agrees to participate in the discount program.”⁹

Section 1101 of the Health Care and Education Reconciliation Act of 2010 added provisions to close the coverage gap (ie, doughnut hole) over the course of 10 years, by 2020.²⁸ Medicare Part D beneficiaries entering the doughnut hole in 2010 would also receive a $250 rebate. Cost-sharing in the doughnut hole for brandname drugs (minus the $250 rebate) has dropped from 100% to 25%.⁹ Subsequent reductions in cost-sharing on the enrollee’s part will occur over the span of 10 years, thereby closing the coverage gap completely by 2020. Generic drugs are not a part of the 50% discount program, and beneficiary cost-sharing in the doughnut hole will be reduced to 25% by 2020 (Part D will pay for 75% of the generic drug’s costs).⁹

Limitations 
Readability formulas do not measure persistence, an important aspect of comprehension. Engaging reading material written at the appropriate comprehension level will likely induce persistence to read, which is important in today’s healthcare system. Elderly beneficiaries need to not only read and comprehend but also to continue reading information provided to derive maximum utility. Constant involvement to protect one’s health is a salient necessity in the US healthcare system of late, given the high prevalence of underinsurance amidst health equity issues. Several older studies in the field of readability research have shown significant relationships between persistence and readability.^29,30

Our study did not measure persistence, nor did it involve readers to assess their readability levels. Further analysis is needed to ascertain whether certain portions of the Medicare handbook text should be rewritten to improve comprehensibility for beneficiaries and their families.

Conclusions
Medicare communications to beneficiaries are vast and extensive. Few studies have examined the extent and usefulness of such materials in educating beneficiaries properly to make appropriate choices. In a consumeroriented society, it is crucial that beneficiaries understand their choices before they make them. The breadth and depth of information may serve to further confuse and overwhelm elderly beneficiaries.

Although the NMEP—of which the Medicare & You handbook is a component—gets evaluated on a periodic basis, the question remains whether this communication is comprehensible to, or even readable by, beneficiaries. It must be noted that the 2010 English version of the handbook has incorporated only few changes. For example, the glossary of terms is at the beginning of the book, unlike in previous versions. A summary of the 4 parts of Medicare is provided at the beginning for quick reference, including contact information for various services.

Nevertheless, the new Part D prescription drug benefit is philosophically and programmatically different from the previous original parts. As such, the following recommendations are offered for improving the communications, based on our analysis of the 2008 handbook.

1. Consider writing Part D as a separate supplement. Medicare Part D is a new benefit, which is very different from Part A and Part B. To better understand Part D concepts, it would be useful to have a separate supplement or addendum on Part D to differentiate pharmacy benefits from hospital and medical benefits.
2. Include new information about dual eligibility and low-income subsidies. For beneficiaries who fall into either category, a separate section on hardship and affording Medicare Part D may be appropriate, as opposed to inclusion in the overall text. The legislative objective of Part D, after all, was to alleviate financial barriers to drug access.
3. Include charts, figures, tables, or graphs to help some beneficiaries better understand their choices. Most beneficiaries may find reading continuous text difficult because of visual problems, declining cognition, or other problems. Therefore, inclusion of visual aids, such as charts or figures, may become more appealing to these beneficiaries and thereby aid in the learning process.
4. Add several case vignettes. In the entire section on Part D, only 1 case example was used (for step therapy). Using more case studies or examples to demonstrate or simulate real-life situations may help beneficiaries understand how these concepts work.
5. Arrange Part D sections in a single supplement to improve the continuity of content. Part D sections are spread out on pages 52-66, 84, 87-88, 99-101, and 107-111 of the handbook.
6. Provide for separate sections on enrollment and appeals/grievances. Writing the Part D supplement as 2 separate sections—plan enrollment and plan appeals/grievances—may help to improve the clarity of these concepts.

Periodically rechecking the document by CMS after incorporating changes would be a useful means of ascertaining whether readability of the material has improved. All these are practical targets for improving the reading material. Our analysis is especially relevant to the CMS staff and their private partners who need to consider our findings in reworking the Medicare & You handbook and all other communications to Part D beneficiaries. We recommend further periodic retesting of the handbook with readers for a better assessment of its comprehensibility.

Disclosure Statement 
Dr Aruru and Dr Salmon have nothing to disclose. 

References

1. Hornik RC, ed. Public Health Communication: Evidence for Behavior Change. Mahwah, NJ: Lawrence Erlbaum Associates; 2002.
2. National Center for Education Statistics. National Assessment of Adult Literacy (NAAL). National Adult Literacy Survey (NALS). 1992. http://nces.ed.gov/pubs 2001/2001457tech.pdf. Accessed March 9, 2010.
3. Davis TC, Crouch MA, Wills G, et al. The gap between patient reading comprehension and the readability of patient education materials. J Fam Pract. 1990; 31:533-538.
4. Berland GK, Elliott MN, Morales LS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001;285: 2612-2621.
5. Estrada CA, Hryniewicz MM, Higgs VB, et al. Anticoagulant patient information material is written at high readability levels. Stroke. 2000;31:2966-2970.
6. Smith H, Gooding S, Brown R, Frew A. Evaluation of readability and accuracy of information leaflets in general practice for patients with asthma. BMJ. 1998;317:264.
7. Gold M. The Landscape of Private Firms Offering Medicare Prescription Drug Coverage in 2006. Kaiser Family Foundation. www.kff.org/medicare/upload/7474.pdf. Accessed February 5, 2010.
8. US Department of Health & Human Services, Centers for Medicare & Medicaid Services. Medicare basics. www.medicare.gov/Publications/Pubs/pdf/10050.pdf. Accessed January 10, 2010.
9. Davis PA, Hahn J, Morgan PC, et al; for Congressional Research Service. Medicare provisions in PPACA (P.L. 111-148). CRS Report for Congress. April 21, 2010. www.aamc.org/reform/summary/PPACMedicareProvision042810.pdf. Accessed January 10, 2010.
10. Berkman ND, Bann C, Kuo M, et al. Analysis of Medicare beneficiary knowledge data using the Medicare Current Beneficiary Survey (MCBS). Phase 3.1. Final Report. RTI International. 2002. www.cms.gov/Reports/downloads/berkman_ 2002_7.pdf. Accessed January 10, 2010.
11. Goldstein E. Assessment of the National Medicare Education Program: supply and demand for information. Health Care Financ Rev. 1999;21:129-131.
12. Harris-Kojetin LD, McCormack LA, Jael EMF, Lissy KL. Beneficiaries’ perceptions of new Medicare health plan choice print materials. Health Care Financ Rev. 2001;23:21-35.
13. Goldstein E, Teichman L, Crawley B, et al. Lessons learned from the National Medicare & You Education Program. Health Care Financ Rev. 2001;23:5-20.
14. Carman KL, Short PF, Farley DO, et al. Epilogue: early lessons from CAHPS demonstrations and evaluations. Consumer Assessment of Health Plans Study. Med Care. 1999;37(suppl 3):MS97-MS105.
15. US Government Accountability Office. Medicare Part D. Opportunities exist for improving information sent to enrollees and scheduling the annual election period. 2008. www.gao.gov/new.items/d094.pdf. Accessed January 2010.
16. Hibbard J, Greene J, Tusler M. An assessment of beneficiary knowledge of Medicare coverage options and the prescription drug benefit. May 2006. http://assets.aarp.org/rgcenter/health/2006_12_medicare.pdf. Accessed September 9, 2009.
17. Kutner M, Greenberg E, Jin Y, Paulsen C; for the National Center for Education Statistics, Institute of Education Sciences. The health literacy of America’s adults: results from the 2003 National Assessment of Adult Literacy. 2006. US Department of Education (NCES 2006–483). http://nces.ed.gov/pubs2006/2006483_1.pdf. Accessed January 10, 2010.
18. Hall JC. The readability of original articles in surgical journals. ANZ J Surg. 2006; 76:68-70.
19. Wolf MS, Davis TC, Shrank WH, et al. A critical review of FDA approved medication guides. Patient Educ and Counsel. 2006;62:316-322.
20. Wolf MS, Davis TC, Tilson HH, et al. Misunderstanding of prescription drug warning labels among patients with low literacy. Am J Health Syst Pharm. 2006;63:1048-1055.
21. Wolf MS, Chang CH, Davis T, Makoul G. Development and validation of the Communication and Attitudinal Self-Efficacy scale for cancer (CASE-cancer). Patient Educ Couns. 2005;57:333-341.
22. Bernstein J, Stevens RA. Public opinion, knowledge, and Medicare reform. Health Aff (Millwood). 1999;18:180-193.
23. Villarruel AM, Portillo CJ, Kane P. Communicating with limited English proficiency persons: implications for nursing practice. Nurs Outlook. 1999;47:262-270.
24. Kaiser Family Foundation. Medicare Part D Plan Enrollment and Plan Charac - teristics. 2006. www.kff.org/medicare/upload/7426-02.pdf. Accessed February 6, 2010.
25. Polinski JM, Bhandari A, Saya UY, et al. Medicare beneficiaries knowledge of and choices regarding Part D, 2005 to the present. Ethics, Public Policy Med Econ. 2010;58:950-966.
26. Hoadley J. Medicare Part D: simplifying the program and improving the value of information for beneficiaries. Issue brief (Commonw Fund). 2008;39:1-15.
27. David v Heckler, 591 F.Supp. 1033 (E.D.N.Y., 1984). www.fastcase.com/Google/Start.aspx?C=cc0de5afbc82ca7abc8929ad8494150ba890ba54220920cc&D=a281b2e1feb0792396a4e9215f1b84ba2c9da8d14688d101. Accessed February 1, 2010.
28. Health Care and Education Reconciliation Act of 2010, H.R. 4872, §1101.
29. Schramm W. Measuring another dimension of newspaper readership. Journalism Quarterly. 1947;24:293-306.
30. Swanson CE. Readability and readership: a controlled experiment. Journalism Quarterly. 1948;25:339-343.