Predictive and Prognostic Models: Implications for Healthcare Decision-Making in a Modern Recession

The decision-making process in many aspects of the healthcare delivery system is often complicated and detailed, requiring many considerations before arriving at a course of action in patient care. Increasingly today, the perspectives of each healthcare stakeholder should be considered, which often leads to confusion and controversy in decisionmaking. For example, what is of value to a patient may not be of value to a provider, and what medical evidence determines to be of value may be very limited or subject to interpretation because of the paucity of definitive evidence. These differences in the interpretation of a treatment value contribute to decision-making controversy and confusion among participating stakeholders, which is further inflamed by third-party coverage guidance. Inefficient decision-making is further compounded by a lack of standardized processes that fully address the perspectives of multiple stakeholders.

Various modeling tools have been devised to assist with the decision-making process. Some are aimed at predicting a clinical outcome, whereas others focus on identifying patients who may be at risk for the development of a particular condition.

There are usually no "general" models used; each desired result is obtained by the use of a modeling tool that is specifically tailored to the circumstance at hand.¹ These prognostic and predictive mathematical camps, believers, or classes are created using formulas that are theoretically capable of assisting in the decision-making to achieve specific outcomes that could help guide allocation of resources in healthcare. Two such examples are prognostic and predictive models.^1,2

Prognostic Modeling and Clinical Outcomes
It is important to be able to predict future outcomes of diseases and/or treatments. In the past, such estimates were typically at the discretion of the clinician and were based on the provider's individual experience and professional opinion. The prognostic model has been devised to assist clinicians and other providers in making more accurate predictions based on information garnered in the present. Prognostic models are statistical tools that predict a clinical outcome based on at least 2 points of patient data.²

Prognostic models are based on prognostic information that generally addresses the patient rather than the disease or treatment. Examples include statements that predict chance or duration of survival, progression of disease (how the patient is expected to progress in the scope of disease—from low- to high-risk group, not from precancer to cancer), and prediction of certain clinical events related to therapy or treatment response (eg, patient not likely to respond to a chosen treatment).³

Prognosis is heavily reliant on other inherent aspects of clinical management, including the initial diagnosis, as well as the prescribed therapy and skills of the clinician. Prognosis is driven by several characteristics: for example, a patient's prognosis is strongly tied to the specific diagnosis (ie, benign prostatic hyperplasia vs prostate cancer), as well as to the therapeutic treatment decisions (watchful waiting vs surgery).

Prognostic models have many uses, including "guiding healthcare policy by generating global predictive scenarios; determining study eligibility of patients for new treatments; selecting appropriate tests and therapies in individual patient management including supporting decisions on withholding or withdrawing therapy."⁴

The 2 main types of prognostic models are (1) those at the patient population level, and (2) those at the individual patient level. Patient population models focus on recognizing trends or discrepancies in groups of patients for a specific criterion, whereas individual patient models are used to govern treatment advice and provide patient-centered consultation.⁴

Prognostic models are normally used to make the best possible choices for a patient or group of patients with regard to a specific clinical scenario. Before a model is used in clinical practice, there must be sufficient evidence to ascertain that it is an appropriate tool applicable to the current situation. Therefore, evaluation of the model is important. The 2 main types include (1) laboratory evaluation, which is concerned with the statistical performance of the model, and (2) clinical evaluation, which aims to determine if the model can effectively address a clinical scenario (ie, if it is clinically effective, not just statistically effective— its "real-world" effectiveness).3 In addition, "For a prognostic model to be clinically useful, it should fulfill 2 requirements: it must be clinically valid and methodologically valid."²

Many factors must be considered in the development of prognostic models to ensure their validity. The model should be user-friendly, so that a clinician can easily draw conclusions applicable to his/her patient's situation. It should also include large sample sizes, to ensure precise measurements; and clinicians should be involved in a discussion of clinically relevant predictors that may affect prognosis.²

Prognostic models have evolved from the use of simple decision rules to guide therapy into complex mathematical formulas developed based on large population databases. For example, the pneumonia severity index (PSI) is a statistically derived prognostic model developed with patient database information.5,6 The PSI can be used by physicians to stratify patients with community-acquired pneumonia (CAP) as having a high risk or a low risk of death.5,6 The initial PSI model predicted 2.4 times more deaths from CAP than actually occurred in practice.7 The model was recalibrated and has since been validated with the caveat that the PSI may not apply to all patients with CAP.⁷ Early experience with the PSI model alerted the medical community that statistically derived models are only as accurate as the data they are based on and may not apply to all institutions.

Concerns with published prognostic models include clinical credibility and evidence of accuracy, effectiveness, and generality.⁸ To ensure clinical credibility, certain criteria must be met⁸:

• Clinicians should be able to easily obtain the data required in a matter timely enough to make their prediction
• Calculations should be simple to allow the clinician to generate the prediction (algorithms are more useful than data entry)
• Relevant patient data should have been tested for model inclusion.

The accuracy of the prognostic model should also be apparent, otherwise clinicians will have no confidence in using this tool; therefore, the model should have a low incidence of false-negative and false-positive results.⁸ The generality of the model should be readily discernible, to assure clinicians that it can be applied from one population to another. This entails that the model has been tested separately, at another time and place, on a different test set. Finally, there should be evidence of clinical effectiveness, perhaps from well-documented clinical trials that exhibit the accuracy of the model.⁵

Predictive Modeling and Cost Management
Healthcare costs are increasing, a problem that is further compounded by our current economic crisis.^6,7 In response, employers and health plans alike are frantically searching for ways to reduce cost. Predictive modeling has become a popular way to assess and manage costs associated with healthcare. This modeling is used for the identification of patients at high risk for certain conditions and implementing interventions to prevent them from becoming high-cost.⁹

Predictive modeling is accomplished through risk assessment to determine susceptibility of a patient/ employee population to a specific condition.¹⁰ The value of such endeavors can be seen in successful identification of at-risk populations. This gives employers or health plans access to information that allows early identification of risk and appropriate and timely preventive actions.

Predictive modeling has been defined by Jonathan Weiner of Johns Hopkins Bloomberg School of Public Health as "a process that applies available data to identify persons who have high medical need and are 'at risk' for above-average future medical service utilization."¹¹ The availability of large amounts of data is critical for the development and effective implementation of a predictive model. These data are derived from pharmacy and medical claims and include diagnosis codes, demographic data, previous claim history, and laboratory results.

These information sets are analyzed by predictive models, which can use the data to identify cases that may move into a high-cost category in the future.11 Once the model does its job and identifies at-risk groups, action plans can be instituted by the employer or the health plan to manage their health.

The core "competency" of predictive modeling is its proactive nature; that is, not taking a reactive approach to disease management and cost containment but instead identifying trends and forecasting events that may have substantial implications for healthcare stakeholders. The 2 main categories of predictive modeling include (1) medical data–based models, and (2) prescription drug–based models. Medical data–based models contain clinical data from all sites of service; these are considered to have the highest predictive power of all models, meaning that they are most effective models for forecasting consumption of healthcare by a given population.¹²

The medical data that these models rely on, however, can take a substantial amount of time to become available electronically (12 months or more), whereas prescription drug–based data are available within 2 months.12 Generally speaking, the current disease burden of a population is most accurately represented by the most recent data. Prescription drug–based models are also advantageous in that prescription data are much easier to obtain than medical data from the exact same population, because of the variety of medical claims submitted for a patient each year, on both an inpatient and outpatient basis, whereas prescription drug data only require 1 database.^12,13

The key principles that should be considered when implementing a predictive model include¹⁴:

• A focus on total population and addressing of the entire spectrum of healthcare
• Emphasis on behavioral change on a long-term scale
• Creation of programs that are driven by data and aimed at addressing individual risk, learning, and health status
• Supportive health plan designs with incentives. When such criteria are met, predictive modeling is a valuable tool for employer plans in decreasing overall healthcare costs, while encouraging adherence to evidence-based medicine.

Predictive modeling has been successfully applied to identifying undiagnosed diabetes, predicting survival after in-hospital cardiopulmonary resuscitation, and determining which combat casualties are likely to require massive blood transfusions.^15-17

Implications to Various Stakeholders The Regulatory Process In the current financial liquidity restraint and investment uncertainty, principally because of the economic recession, a volatile environment across stakeholders around the complexities of decision-making in the US healthcare has ensued. Congress passed funding for comparative effectiveness research in February 2009, has entered the healthcare reform debate in March 2009, and has begun the process to seek change in the way healthcare is financed, reimbursed, and delivered to the American public. This has made the choice of modeling tools all the more important for clinical decision makers.

The Patient's Perspective
From a patient point of view, tools that place an emphasis on health and outcomes are desirable. When diagnosed with a particular disease or disorder, questions of utmost importance spring to mind: How much time is left for me? How do I manage this condition? What is my prognosis? Prognostic models can best address these concerns. This type of a model has become even more important in light of the economic recession, as patients are basing their treatment decisions more heavily on the cost that will be incurred.¹⁸

When used correctly, prognostic models can provide patients with more accurate and detailed information about their condition, allowing them to make more educated decisions about whether to undergo certain treatments or pursue other alternatives. For example, when a clinician uses a prognostic model and estimates a patient's survival time as only 2 months, the patient may choose to forgo expensive chemotherapies and instead enroll in an end-of-life care program. Utilizing such effective tools to aid in clinical decision prognosticating, in combination with financial information, can result in a better-informed decision today about a treatment tomorrow.

The Payer's Perspective
What may be in the best interest of the patient is not necessarily viewed as the best course of action for an employer or a health plan. Cost, and the ability to recognize the potential for patients to move into a higher expense category, is a driving factor of significant influence to healthcare administrators.^10,19 With such concepts heavily influencing the decisions of benefit providers, the ability to predict outcomes becomes all the more important, especially during the current economic downturn. Predictive models allow health plans and employers to identify at-risk patients and forecast their impact on health spending. This early recognition begets early action, so that data compiled and analyzed by the predictive model can serve as a catalyst for change and a basis for devising an action plan resulting in reduced expenditures.

The Provider's Perspective
Clinicians' perspectives must also be considered. The ability to properly diagnose and manage disease is an invaluable skill yet is not always held to a uniform standard.²⁰ Differences in professional opinion among healthcare providers, as well as a general lack of trust in the applicability of current modeling tools, have created a barrier to a widespread use of clinical decisionmaking aids. Prognostic models, although often maligned, can be a valuable asset to clinicians when properly designed and implemented.

In the current economic climate, there is increasing pressure on providers to prescribe not only the most effective therapies but also to control costs for all stakeholders involved. Ideally, prognostic models can be of asset to clinicians and other healthcare providers by allowing them to recognize a specific disease or chart its course more accurately, enabling them to save valuable time and money by selecting the appropriate treatments and action plans with more confidence, without wasting resources on testing procedures and therapies that have not been proved effective in evidence-based guidelines. If the design of prognostic models is streamlined so that their applicability and accuracy is more readily apparent to providers, their use may become more widespread, potentially decreasing total health expenditures through the use of more appropriate and cost-effective therapies.

From Research to Clinical Practice
Prognostic and predictive models are primarily kept in the domain of research and only a handful have found wider acceptance for adoption to clinical practice. The adoption of a prognostic or a predictive model is largely dependent on the ease of use and generalizability of the model in terms of its clinical applicability or effectiveness.

There is danger in moving too quickly to use these models without appropriate validation and understanding of their limitations. As the public and clinical decision makers seek better tools to determine the relative value of a clinical choice, it is equally important to seek the use of reasonable tools that do not misrepresent the information needed to make an informed decision about their spending for medical care.

Conclusions
Decision-making in healthcare is a confusing and involved process. This is compounded by the current condition of the US economy. Prognostic and predictive models have been devised to assist in the patient care decision-making. Each model has a particular purpose, strengths and weaknesses, and individual appeal to different healthcare stakeholders. A model is limited in its application in the way in which it can be used in clinical practice. In some cases, models may not be used for clinical practice directly; instead, they may be used indirectly to drive quality measures retrospectively. In light of the weakened economy, the implications of using either or both predictive and prognostic models have become all the more important. Their use, although still varied and controversial, may increase, as providers and patients alike seek to better understand clinical choices or options that may also reduce overall healthcare costs.

Acknowledgment
The author wishes to acknowledge the editorial and literature research assistance provided by Stephen Cutts, PharmD, University of Rhode Island, College of Pharmacy

References

1. den Boer S, de Keizer NF, de Jonge E. Performance of prognostic models in critically ill cancer patients—a review. Crit Care. 2005;9:R458-R463.
2. Perel P, Edwards P, Wentz R, Roberts I. Systematic review of prognostic models in traumatic brain injury. BMC Med Inform Decis Mak. 2006;6:38.
3. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54:17-23.
4. Abu-Hanna A, Lucas PJ. Prognostic models in medicine: AI and statistical approaches. Methods Inf Med. 2001;40:1-5.
5. Fine JM, Hanusa BH, Lave JR, et al. Comparison of a disease-specific and a generic severity of illness measure for patients with communityacquired pneumonia. J Gen Intern Med. 1995;10:359-368.
6. Fine JM, Singer DE, Hanusa BH, et al. Vali
dation of a pneumonia prognostic index using the MedisGroups Comparative Hospital Database. Am J Med. 1993;94:153-159.
7. Flanders WD, Tucker G, Krishnadasan A, et al. Validation of the pneumonia severity index. Importance of study-specific recalibration. J Gen Intern Med. 1999;14:333-340.
8. Wyatt JC, Altman DG. Prognostic models: clinically useful or quickly forgotten? BMJ. 1995;311:1539-1541 [Commentary].
9. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130:515-524.
10. National Cancer Institute. Costs of Cancer Care. Cancer Trends Progress Report–2007 Update. http://progressreport.cancer.gov/doc_detail.asp?pid=1&did=2007&chid=75&coid=726&mid. Accessed November 21, 2008.
11. Fuhrmans V. Consumers cut health spending, as economic down turn takes toll. The Wall Street Journal. September 22, 2008. http://online.wsj.com/article/SB122204987056661845.html. Accessed November 21, 2008.
12. HealthNEWS.Direct! Predictive modeling tools make further progress in healthcare. www.healthnewsdirect.com/?p=63. Accessed December 6, 2008.
13. Mahoney J. Value-based benefit design: using a predictive modeling approach to improve compliance. J Manag Care Pharm. 2008;14(6 suppl B):3-8.
14. Carlson B. Predictive modeling, sharp lens on near future. Manag Care. 2003;12:16-21.
15. Baan CA, Ruige JB, Stolk RP, et al. Performance of a predictive model to identify undiagnosed diabetes in a health care setting. Diabetes Care. 1999;22:213-219.
16. Danciu SC, Klein L, Hosseini M, et al. A predictive model for survival after in-hospital cardiopulmonary arrest. Resuscitation. 2004;62:35-42.
17. McLaughlin DF, Niles SE, Salinas J, et al. A predictive model for massive transfusion in combat casualty patients. J Trauma. 2008;64(2 suppl):S57-S63.
18. Noble L. Pros and cons of predictive modeling approaches. Healthc Financ Manage Assoc. November 3, 2004. www.hfma.org/publications/know_newsletter/archives/110304.htm. Accessed November 21, 2008.
19. Silva C. "Employers investigating predictive models to cut health care costs." Employee Benefit News. http://ebn.benefitnews.com/asset/article/536761/employers-investigating-predictive-models-cut-health.html. Accessed November 21, 2008.
20. McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-based medicine and the practicing clinician. J Gen Intern Med. 1999;14:236-242.