Published on in Vol 2, No 1 (2019): Jan-Jun

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/12134, first published .
Reading Level and Suitability of Congestive Heart Failure (CHF) Education in a Mobile App (CHF Info App): Descriptive Design Study

Reading Level and Suitability of Congestive Heart Failure (CHF) Education in a Mobile App (CHF Info App): Descriptive Design Study

Reading Level and Suitability of Congestive Heart Failure (CHF) Education in a Mobile App (CHF Info App): Descriptive Design Study

Original Paper

College of Nursing, University of South Florida, Tampa, FL, United States

*all authors contributed equally

Corresponding Author:

Ponrathi Athilingam, RN, ACNP, PhD

College of Nursing

University of South Florida

12901 Bruce B Downs Blvd

MDC 22

Tampa, FL, 33612

United States

Phone: 1 8139747526

Fax:1 8139749324

Email: pathilin@health.usf.edu


Background: Education at the time of diagnosis or at discharge after an index illness is a vital component of improving outcomes in congestive heart failure (CHF). About 90 million Americans have limited health literacy and have a readability level at or below a 5th-grade level, which could affect their understanding of education provided at the time of diagnosis or discharge from hospital.

Objective: The aim of this paper was to assess the suitability and readability level of a mobile phone app, the CHF Info App.

Methods: A descriptive design was used to assess the reading level and suitability of patient educational materials included in the CHF Info App. The suitability assessment of patient educational materials included in the CHF Info App was independently assessed by two of the authors using the 26-item Suitability Assessment of Materials (SAM) tool. The reading grade level for each of the 10 CHF educational modules included in the CHF Info App was assessed using the comprehensive online Text Readability Consensus Calculator based on the seven most-common readability formulas: the Flesch Reading Ease Formula, the Gunning Fog Index, the Flesch-Kincaid Grade Level Formula, the Coleman-Liau Index, the Simplified Measure of Gobbledygook Index, the Automated Readability Index, and the Linsear Write Formula. The reading level included the text-scale score, the ease-of-reading score, and the corresponding grade level.

Results: The educational materials included in the CHF Info App ranged from a 5th-grade to an 8th-grade reading level, with a mean of a 6th-grade level, which is recommended by the American Medical Association. The SAM tool result demonstrated adequate-to-superior levels in all four components assessed, including content, appearance, visuals, and layout and design, with a total score of 77%, indicating superior suitability.

Conclusions: The authors conclude that the CHF Info App will be suitable and meet the recommended health literacy level for American adult learners. Further testing of the CHF Info App in a longitudinal study is warranted to determine improvement in CHF knowledge.

JMIR Aging 2019;2(1):e12134

doi:10.2196/12134

Keywords



Heart Failure Education

Education is key to improve knowledge and persuade patients with congestive heart failure (CHF) to practice recommended daily self-management tasks of weight and symptom assessment, diet, exercise, and pharmacological therapy [1]. Evidence indicates that traditional patient education using printed materials does not support self-management skill development [1]. New patient-teaching strategies are needed for prolonged engagement of patients to support the development of tactical and situational skills [2]. The incidence of CHF is strongly dependent on age, with an estimated incidence of 1% at age 65 that approximately doubles with each decade thereafter, affecting 6.5 million Americans [3]. According to the Centers for Medicare and Medicaid Services, nearly 20% of all Medicare patients are readmitted to the hospital within 30 days and 34% are readmitted within 90 days of discharge [4]. As many as 79% of readmissions are considered preventable. The prospective EuroHeart Failure Survey (N=2331) from 24 countries followed patients with CHF for 12 weeks and reported that about half of the patients (49%) recalled receiving advice to weigh themselves at discharge [5]. American health care puts a great deal of emphasis on patient autonomy and patients' “right to know.” The American College of Cardiology/American Heart Association clinical guidelines include written educational instruction at the time of discharge as one of the core performance measures to improve CHF outcomes [6]. However, adequate communication skills and health literacy level is important for patients to understand the discharge instructions given, so they can follow them appropriately at home [7]. Inadequate communication skills may not mean resistance to the treatment plan or poor intellect, but rather a low literacy level. Thus, assessing and addressing health literacy levels of patients and reading levels of patient educational materials are warranted for patient safety and to improve outcomes among the elderly.

In recent years, patient-centered mobile health technologies have emerged as a way to actively engage patients in their health care decision-making process. Patients who are engaged as decision makers in their care tend to be healthier and have better outcomes. This is supported by the national survey of 1604 mobile phone users, where 76% of mobile phone users were constantly connected by technology [8]. African Americans and Latinos are reported to be 50% more dependent on mobile phones compared to the white population; about 80% of adults over the age of 65 use mobile phones and 42% are advanced-feature mobile phones [9]. Older adults aged 65 and over with no experience in technology are reportedly using mobile phones to manage daily self-management of chronic diseases [10,11]. Thus, embedding health education in a mobile platform is proposed to improve patient engagement, facilitate communication, help overcome health challenges, and improve disease management. However, health literacy could affect people's ability to navigate the health care system, including filling out complex health information forms and engaging in both self-care and chronic disease management [12]. This paper addresses the suitability and the reading-level assessment of patient educational materials included in the mobile app CHF Info App.

Health Literacy

Health literacy is the ability to understand health information and to use that information to make good decisions about one’s health and health care [13]. According to the US Department of Health and Human Services, over a third of US adults are reported to have difficulty with common health tasks, such as following directions on a prescription drug label, adhering to a childhood immunization schedule, or understanding the direction of a medication schedule from the instructions found in the medication container [13]. Health literacy also encompasses the educational, social, and cultural factors that influence the expectations and preferences of the individual [14]. According to the Centers for Medicare and Medicaid Services, about 70% of adults older than age 60 had difficulty using print materials and 80% had difficulty using documents such as forms or charts [15]. Limited health literacy affects adults in all racial and ethnic groups. The proportion of adults with basic or below basic health literacy ranges from 28% of white adults to 65% of Hispanic adults [13]. Approximately 90 million Americans have limited health literacy and have a readability level at or below the 5th-grade level [16]. Although half of adults without a high school education may have below basic health literacy skills, even high school and college graduates can also have limited health literacy [14]. The Joint Commission recommended that in order to self-manage their own health care, individuals must be able to locate health information, evaluate that information for relevance and credibility, and analyze risks and benefits. For those with limited literacy skills, self-management may be too much of a challenge to be overcome, especially if such challenges are undiscovered or ignored [7]. Suitability of educational materials (ie, ease of understanding and acceptance) and readability (ie, reading difficulty) at the level of patient education or understanding have been suggested as strategies to improve the knowledge of the CHF patient. Despite the American Medical Association (AMA) recommendation to provide all patient educational materials at a 6th-grade reading level, many educational materials do not abide by the rule [17].

With advances in medical science, health educational information can overwhelm people, even those with advanced literacy skills. Patient educational materials are potentially effective at improving patient comprehension and influencing health behaviors, especially if they are written at appropriate reading levels for patients. Currently, most institutions include links for patient education within electronic health records (EHRs). An assessment of reading levels of three commonly used patient education links in an EHR study reported varied reading levels: 11th-13th grade for EBSCO, 14th-17th grade for MedlinePlus, and 11th grade for Micromedex—grade levels above the 12th grade refer to the college or university level and beyond [18]. Similarly, an assessment of 339 online ophthalmic patient educational materials demonstrated a varied reading level ranging from the 10th to the 17th grade [19], which is higher than the AMA-recommended 8th-grade reading level. However, no reading levels are assessed on patient educational materials included in mobile apps. Recent reviews evaluated the quality of mobile apps regarding the ease of use, reliability, quality, scope of information, and aesthetics using the Mobile App Rating Scale, but not regarding the reading level of patient educational materials [20,21].

Therefore, the original theory-based development of the CHF Info App with 10 education modules was assessed for reading levels to meet those recommended by the AMA. The modules included in the app were based on educational materials from the Heart Failure Society of America (HFSA) with their approval; a link to detailed HFSA material is provided in the app for patients to navigate [22]. The CHF Info App was developed by educational technology and computer engineering students from the University of South Florida. The app was beta-tested on a small sample of patients and health care providers for usability [22] by the educational technology students. The content included was approved by cardiologists during beta-testing [22]. The educational material included in the CHF Info App evolved and additions were made based on input from patients and providers after beta-testing [22]. Given the influence of health literacy on understanding educational material, assessing for suitability and reading level of patient educational material was deemed a necessary next step before making the CHF Info App available for patient use. Therefore, the authors sought to determine the suitability and readability level of the materials included in the CHF Info App to meet the requirements of the AMA recommendations.


A descriptive design was used to assess the suitability and readability of the patient educational materials included in the CHF Info App. The Suitability Assessment of Materials (SAM) tool is composed of a 26-item Likert scale with four categories including message content, text appearance, visuals, and layout and design. The SAM instrument is a systematic tool that assesses printed health-related educational resources in a short amount of time. The validated SAM tool was used to assess the CHF Info App [23]. Two of the authors (PA and BR) independently assessed the suitability of the CHF Info App using the SAM instrument. These authors, both with doctoral-level education, objectively followed the SAM tool and coded each of the 10 modules according to content, appearance, visuals, and layout and design. Each item was scored as superior (2 points), adequate (1 point), or not suitable (0 points). The given score was divided by the total possible score to obtain a percentage; a score of 0%-39% is considered not suitable, 40%-69% is considered adequate, and 70%-100% is considered superior. Conflict between the coders was resolved by consensus and a workflow for improvement of the app was developed.

There are several predominant tools available online to measure the reading level of patient educational materials. The authors used the Text Readability Consensus Calculator, an online readability consensus assessment formula, which calculates reading level based on the seven most-common reading formulas to calculate the average reading grade level, reading age, and text difficulty of the text [24]. The prominent measures of readability documented in the literature are the Flesch Reading Ease Formula, which indicates that the best text should contain shorter sentences and words. A score between 60 and 70 is largely considered acceptable [25]. The Flesch-Kincaid Grade Level Formula presents a score as a US grade level, making it easier for teachers, parents, librarians, and others to judge the readability level of various books and texts [26]. The Simplified Measure of Gobbledygook Readability Formula estimates the years of education needed to understand a piece of writing, particularly for checking health messages [27]. The Gunning Fog Index estimates the years of formal education a person needs to understand a piece of text on the first reading. For instance, a Gunning Fog Index of 12 requires the reading level of a US high school senior [28]. The Fry Formula is calculated as the average number of sentences (y-axis) and syllables (x-axis) per 100 words. These averages are plotted onto a specific graph; the intersection of the average number of sentences and the average number of syllables determines the reading level of the content [29]. The Dale-Chall Formula is unique because, unlike other formulas that use word length to assess word difficulty, the Dale-Chall Formula uses sentence length and counts of “hard” words to calculate the US grade level [30]. In addition, the Linsear Write Formula is included to calculate the readability of technical manuals; also, the Automated Readability Index was designed to gauge the understandability of a text that produces an approximate representation of the US grade level needed to comprehend the text [24]. Therefore, the 10 education modules of the CHF Info App were assessed using the comprehensive Text Readability Consensus Calculator Formula to make sure the CHF Info App meets the AMA recommendations.


A suitability assessment of patient educational materials from the CHF Info App was performed by two of the authors using the SAM tool. Both of the authors (PA and BR) with doctoral-level education were qualified to assess the quality of the material objectively using the SAM tool. Each item was scored as superior (2 points), adequate (1 point), or not suitable (0 points), as mentioned above. The result of the assessment demonstrated adequate-to-superior levels of agreement in all four components: content, appearance, visuals, and layout and design. The only item found to be inadequate or that was not specifically addressed in the CHF Info App was cultural difference. The authors agreed that the educational material provided in the CHF Info App was general and culturally neutral and did not address any one cultural preference. The CHF Info App was assessed as superior with a score of 77%. See Table 1 for detailed results.

The reading grade level for each of the 10 CHF educational modules included in the CHF Info App was assessed using the comprehensive online Text Readability Consensus Calculator formulas [24]. The reading level included the text-scale score, the ease-of-reading score, and the corresponding grade level. The materials included in the CHF Info App ranged from 5th-grade to 8th-grade reading levels with an average of a 6th-grade level, which meets the AMA recommendation (see Table 2).

Table 1. Suitability Assessment of Materials for patient education of the CHFa Info App.
Appraisal of components and related questionsAssessment based on score

InadequateAdequateSuperior
Appraisal of message content



Does the material explain the purpose and benefits from the patient’s view?
X

Is the content limited to a few essential main points that the majority of the target population will benefit from?

X

Are behaviors and skills emphasized rather than just facts?

X

Are readers provided with opportunities for small successes?
X

Are key points reviewed at the end of each section or page?
X

Is the material sensitive to cultural differences?N/AbN/AN/A

Is the new information placed in the context of the patients’ lives?
X

Are readers told what they should get from the material and what they can do to improve their health?

X

Is the organization of the paragraphs and sentences conducive to easy reading?

X

Are instructions broken down into easy-to-read parts?

X

Is the material interactive? Does it encourage the patient to write, answer questions, ask questions, cut out forms, etc?

X
Appraisal of text appearance criteria



Is the font size no smaller than 12pt-14pt? Is zoom function available?

X

Is easy-to-read font used? Are there no fancy scripts or lettering?

X

Are bold and underline used instead of ALL CAPS and italics?

X

Are colors used to promote easy reading (ie, dark fonts on light backgrounds are best)?

X

Are overall sharp contrast and large font used?

X
Appraisal of visuals



Do the visuals all help communicate your messages in a literal manner (ie, no abstract symbols)?
X

Are the visuals culturally relevant and sensitive?
X

Are the visuals easy for your readers to follow and understand? For example, if showing a sequence, are the steps numbered and labeled?
X

Are internal body parts or small objects shown in context and in a realistic manner?
X

Are the visuals professional and appropriate for an adult audience?

X

Are the visuals free of distracting details that take away from the main idea?

X

Do all of the graphics contribute to your message?
X

Are examples given for any lists, charts, or diaries that readers are supposed to complete?
X
Appraisal of layout and design



Is the cover effectively designed?
X

Are messages organized so they are easy to act on and recall (headings, subheadings, chunking, etc)?

X

Is there a lot of white space? Is there no dense text?
X

Is the text easy for the eye to follow? For example, bullets, paragraph shape (40-50 characters wide is best), and text boxes.

X

aCHF: congestive heart failure.

bN/A: not applicable.

Table 2. Reading-level assessment of the CHFa Info App.
Scoring formulaText-scale scoreEase-of-reading scoreGrade level
Flesch Reading Ease Formula84.8EasyN/Ab
Gunning Fog Index6.8Fairly easyN/A
Flesch-Kincaid Grade Level Formula4.7Easy5th grade
Coleman-Liau Index8.0Fairly easy8th grade
Simplified Measure of Gobbledygook Index4.4Easy4th grade
Automated Readability Index5.9Easy5th and 6th grade
Linsear Write Formula6.1Easy6th grade

aCHF: congestive heart failure.

bN/A: not applicable.

Table 3. Comparison of words and text of standard US high school and adult readers and the CHFa Info App.
MeasureAverage for US high school and adult readersAverage in the CHF Info App
Sentence length (number of words), mean (SD)13-16 (1.3)13 (0.7)
Reading level7th and 8th grade6th grade
Three-syllable text, %12-144

aCHF: congestive heart failure.

In addition to the reading level, the online Text Readability Consensus Calculator provided a comparison of words and syllables, sentence length, and texts included in the CHF Info App corresponding with the standard of US high school and adult readers. The results showed appropriate sentence length, reading level at the 6th grade, and an average three-syllable text of 4%; the results also meet the AMA recommendation (see Table 3).


Principal Findings

The findings of the assessment of the patient educational material included in the CHF Info App indicated a reading level of 5th-8th grade, with a mean of a 6th-grade level. This result is consistent with the recommendation from the AMA [17]. The ability to obtain and understand basic information about health in order to make informed decisions is vital and contributes to the complex area of health literacy. Although the use of medical terminology in patient educational material is often unavoidable, it has a profound impact on readability because of the use of polysyllabic medical terms, and the CHF Info App included fewer polysyllabic terms. The CHF Info App was made up of 4% of three-syllable text, which is much lower than the recommended average three-syllable text of 12%-14%; in addition, the average sentence length of 13 words met the standard for US high school and adult learners. A study by Chen et al showed that health literacy is associated with CHF knowledge, longitudinally (P<.001), among 51 patients with CHF with a mean age of 65 years [31]. Therefore, the authors conclude that the CHF Info App will be suitable and meet the health literacy level recommended for adult learners. Low health literacy was consistently associated with more hospitalizations; decreased ability to demonstrate taking medications appropriately; decreased ability to interpret labels and health messages; and, among elderly persons, poorer overall health status and higher mortality rates. Health literacy was independently associated with knowledge (P<.001), however, it was not related to self-care [32]. Therefore, our next step is to test the CHF Info App in a longitudinal study to measure improvement in CHF knowledge and self-care.

The reading-level assessment was complemented by the use of the SAM instrument, which assessed the content, appearance, visuals, and layout and design of the materials. The components were assessed as adequate to superior for all 26 items on the SAM instrument, with a total score of 77% indicating superior. Except for cultural difference, the other items were found to be adequate or superior. The authors concluded that the contents included in the CHF Info App are general, using common words, and are culturally neutral. However, further testing is needed among a diverse population to determine its usability and potential efficacy, since poor health literacy partially explains racial disparities in some outcomes [33]. Evidence supports the use of mobile phone-based telemonitoring and educational support for patients with CHF [11].

Limitations

One limitation is that this was a descriptive study that assessed the readability and suitability of a mobile phone-based CHF education app—the CHF Info App. The other major limitation is that the CHF Info App is available only in the English language. Further testing on a larger sample including elderly persons and a longitudinal follow-up are warranted to determine if the patient educational material included in the CHF Info App will improve CHF knowledge and self-care. Although the readability measures used in this study do address reading grade level and ease of reading, through assessment of word difficulty and sentence length, they do not consider other factors that may affect comprehension of health educational materials, such as cultural appropriateness, learning stimulation, and motivation. Although the suitability of educational material includes many factors, reading grade level is foundational to any patient educational material.

Clinical Implications

All patient educational material, whether in paper form or an app version, should identify reading level and how it was measured to provide further guidance for patients and health care providers to make sure that they meet the AMA recommendations. Future research should focus on continued assessment of health educational materials used for diverse populations and settings and investigation of readability measures among CHF patients.

Conflicts of Interest

None declared.

  1. Alberti T, Nannini A. Patient comprehension of discharge instructions from the emergency department: A literature review. J Am Assoc Nurse Pract 2013 Apr;25(4):186-194. [CrossRef] [Medline]
  2. Dickson V, Riegel B. Are we teaching what patients need to know? Building skills in heart failure self-care. Heart Lung 2009;38(3):253-261. [CrossRef] [Medline]
  3. Benjamin E, Virani S, Callaway C, Chamberlain A, Chang A, Cheng S, American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics-2018 update: A report from the American Heart Association. Circulation 2018 Mar 20;137(12):e67-e492. [CrossRef] [Medline]
  4. CMS.gov. Baltomore, MD: US Centers for Medicare & Medicaid Services; 2017 Jun 13. Hospital Readmissions Reduction Program (HRRP)   URL: https:/​/www.​cms.gov/​Medicare/​Medicare-Fee-for-Service-Payment/​AcuteInpatientPPS/​Readmissions-Reduction-Program.​html [accessed 2019-03-29] [WebCite Cache]
  5. Cleland JG, Swedberg K, Cohen-Solal A, Cosin-Aguilar J, Dietz R, Follath F, et al. The Euro Heart Failure Survey of the EUROHEART survey programme. A survey on the quality of care among patients with heart failure in Europe. The Study Group on Diagnosis of the Working Group on Heart Failure of the European Society of Cardiology. The Medicines Evaluation Group Centre for Health Economics University of York. Eur J Heart Fail 2000 Jun;2(2):123-132 [FREE Full text] [Medline]
  6. Yancy C, Jessup M, Bozkurt B, Butler J, Casey DJ, Colvin M, et al. 2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA guideline for the management of heart failure: A report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America. Circulation 2017 Dec 08;136(6):e137-e161. [CrossRef] [Medline]
  7. “What Did the Doctor Say?": Improving Health Literacy to Protect Patient Safety. Oakbrook Terrace, IL: The Joint Commission; 2007.   URL: https://www.jointcommission.org/assets/1/18/improving_health_literacy.pdf [accessed 2019-04-10] [WebCite Cache]
  8. Krebs P, Duncan DT. Health app use among US mobile phone owners: A national survey. JMIR Mhealth Uhealth 2015 Nov 04;3(4):e101 [FREE Full text] [CrossRef] [Medline]
  9. Pew Research Center. Mobile fact sheet. Washington, DC: Pew Internet & American Life Project; 2018 Feb 05.   URL: http://www.pewinternet.org/fact-sheet/mobile/ [accessed 2019-04-10] [WebCite Cache]
  10. PMLive. 2013 Jun 28. mHealth: Patients open to app prescriptions   URL: http://www.pmlive.com/pharma_intelligence/mhealth_patients_open_to_app_prescriptions_486119 [accessed 2019-04-10] [WebCite Cache]
  11. Seto E, Leonard K, Cafazzo J, Barnsley J, Masino C, Ross H. Mobile phone-based telemonitoring for heart failure management: A randomized controlled trial. J Med Internet Res 2012 Feb 16;14(1):e31 [FREE Full text] [CrossRef] [Medline]
  12. Kutner M, Greenberg E, Jin Y, Paulsen C. The Health Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006–483). Washington, DC: National Center for Education Statistics, US Department of Education; 2006 Sep.   URL: https://nces.ed.gov/pubs2006/2006483.pdf [accessed 2019-04-10] [WebCite Cache]
  13. US Department of Health and Human Services. 2008. America's health literacy: Why we need accessible health information   URL: https://health.gov/communication/literacy/issuebrief/ [accessed 2019-04-10] [WebCite Cache]
  14. Nielsen-Bohlman L, Panzer AM, Kindig DA(. Health Literacy: A Prescription to End Confusion. Washington, DC: The National Academies Press; 2004.   URL: https://www.ncbi.nlm.nih.gov/books/NBK216032/pdf/Bookshelf_NBK216032.pdf [accessed 2019-04-12] [WebCite Cache]
  15. Centers for Disease Control and Prevention. Improving Health Literacy for Older Adults: Expert Panel Report 2009. Atlanta, GA: US Department of Health and Human Services; 2009.   URL: https://www.cdc.gov/healthliteracy/pdf/olderadults-508.pdf [accessed 2019-04-11] [WebCite Cache]
  16. Wilson M. Readability and patient education materials used for low-income populations. Clin Nurse Spec 2009;23(1):33-40; quiz 41. [CrossRef] [Medline]
  17. American Medical Association. Health Literacy and Patient Safety: Help Patients Understand. Chicago, IL: American Medical Association; 2007.
  18. Stossel L, Segar N, Gliatto P, Fallar R, Karani R. Readability of patient education materials available at the point of care. J Gen Intern Med 2012 Sep;27(9):1165-1170 [FREE Full text] [CrossRef] [Medline]
  19. Huang G, Fang C, Agarwal N, Bhagat N, Eloy J, Langer P. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol 2015 Apr;133(4):449-454. [CrossRef] [Medline]
  20. Masterson Creber RM, Maurer M, Reading M, Hiraldo G, Hickey K, Iribarren S. Review and analysis of existing mobile phone apps to support heart failure symptom monitoring and self-care management using the Mobile Application Rating Scale (MARS). JMIR Mhealth Uhealth 2016 Jun 14;4(2):e74 [FREE Full text] [CrossRef] [Medline]
  21. Athilingam P, Jenkins B. Mobile Phone Apps to Support Heart Failure Self-Care Management: Integrative Review. JMIR Cardio 2018 May 02;2(1):e10057. [CrossRef]
  22. Athilingam P, Osorio R, Kaplan H, Oliver D, O'neachtain T, Rogal P. Embedding patient education in a mobile platform for patients with heart failure: Theory-based development and beta testing. Comput Inform Nurs 2016 Feb;34(2):92-98. [CrossRef] [Medline]
  23. Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills. 2nd edition. Philadelphia, PA: JB Lippincott Company; 1996.
  24. Readibility Formulas. Myrtle Beach, SC: My Byline Media; 2003.   URL: http://www.readabilityformulas.com/ [accessed 2019-04-10] [WebCite Cache]
  25. Flesch R. Let's start with the formula. In: How to Write Plain English. New York, NY: Barnes & Noble; 1981.
  26. Kincaid JP, Braby R, Mears JE. Electronic authoring and delivery of technical information. J Instr Development 1988 Jun;11(2):8-13. [CrossRef]
  27. McLaughlin GH. SMOG grading: A new readability formula. J Read 1969;12(8):639-646 [FREE Full text]
  28. Bogert J. In defense of the Fog Index. Bus Prof Commun Q 1985 Jun 01;48(2):9-12. [CrossRef]
  29. Fry E. A readability formula that saves time. J Read 1968 Apr;11(7):513-516, 575-578 [FREE Full text]
  30. Dale E, Chall JS. A formula for predicting readability. Educ Res Bull 1948 Jan 21;27(1):11-20, 28 [FREE Full text]
  31. Chen AMH, Yehle KS, Albert NM, Ferraro KF, Mason HL, Murawski MM, et al. Nursing Research and Practice. London, UK: Hindawi Publishing Corporation; 2013. Health literacy influences heart failure knowledge attainment but not self-efficacy for self-care or adherence to self-care over time   URL: https://www.hindawi.com/journals/nrp/2013/353290/ [accessed 2019-04-10] [WebCite Cache]
  32. Chen A, Yehle K, Albert N, Ferraro K, Mason H, Murawski M, et al. Relationships between health literacy and heart failure knowledge, self-efficacy, and self-care adherence. Res Social Adm Pharm 2014;10(2):378-386 [FREE Full text] [CrossRef] [Medline]
  33. Berkman N, Sheridan S, Donahue K, Halpern D, Crotty K. Low health literacy and health outcomes: An updated systematic review. Ann Intern Med 2011 Jul 19;155(2):97-107. [CrossRef] [Medline]


AMA: American Medical Association
CHF: congestive heart failure
EHR: electronic health record
HFSA: Heart Failure Society of America
SAM: Suitability Assessment of Materials


Edited by J Wang; submitted 07.09.18; peer-reviewed by H Du; comments to author 08.11.18; revised version received 15.02.19; accepted 25.03.19; published 25.04.19

Copyright

©Ponrathi Athilingam, Bradlee Jenkins, Barbara A Redding. Originally published in JMIR Aging (http://aging.jmir.org), 25.04.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Aging, is properly cited. The complete bibliographic information, a link to the original publication on http://aging.jmir.org, as well as this copyright and license information must be included.