The value of drugs, whether individually or comparatively, and the definition of value itself, have emerged as acute concerns in oncology, where the cost of cancer care has evoked issues of financial toxicity.1-3 In the United States alone, the costs associated with cancer treatment have been forecast to increase 27% from their 2010 levels, to approximately $157.8 billion by 2020.2 Along with patients and payers, who bear the burden of these costs, physicians and policymakers have waded into the discussion of defining value in oncology care. Although providers and payers require new frameworks to assess the value of therapy options based on their individual perspectives and needs, payers require tools to better guide patient care.4 In addition, the pharmaceutical industry seeks to refine its scenario modeling of the value and financial pain points (ie, willingness of payers and ability of patients to cover the treatment costs) of increasingly complex drugs to achieve greater alignment among payers and their treatment-eligible populations.
Into this complex marketplace, 5 major value frameworks have emerged in the past 2 years, representing years of efforts by diverse organizations and institutions to quantify and evaluate the benefits, harms, and (in some cases) costs, and arrive at a composite value metric or metrics.5-10 But how robust and reliable are these very different frameworks? How can the healthcare industry best apply them to make better decisions, and what do drug manufacturers need to know about these frameworks and their potential for application in the care paradigm?
We compared the methodology of these frameworks using a side-by-side comparative approach in terms of the input, scoring methodology, and output of each framework. In addition, we gleaned stakeholder insights regarding these frameworks and their potential real-world application through dialogues with physicians and payers, as well as through secondary research and an aggregate analysis of previously published surveys. As detailed below, US physician and payer awareness and utilization of the frameworks indicate a rift between their current use and future application that should be noted by those stakeholders seeking to model the impact of these frameworks on their decision-making.
Frameworks Share Some Commonality, but Reflect Very Different Origins and Purposes
The American Society of Clinical Oncology (ASCO), a professional organization representing more than 40,000 physicians of all oncology subspecialties, launched its Value Framework in June 2015,11 targeting physicians and patients (Table 1).6-10 This framework is intended to be a physician-guided tool to facilitate shared decision-making between physicians and patients.6,11
Developed by the ASCO Value in Cancer Care Task Force, this approach provides a net health benefit score derived from efficacy, safety, and bonus points for secondary end points.6,11 It also includes a comparison to direct treatment costs and versions in the advanced (noncurative) disease and curative disease settings. In 2016, ASCO updated its Value Framework with changes to the scoring methodology, providing additional secondary end points, such as improvement in quality of life and significant survival improvement in the tail of the curve for which bonus points could be earned.6
The National Comprehensive Cancer Network (NCCN), a nonprofit alliance of 26 cancer centers throughout the United States, launched its evidence block framework in October 2015, to give “the healthcare provider and the patient [similar to ASCO’s] information to make informed choices when selecting systemic therapies based upon measures related to treatment, supporting data and cost.”7 Guided by staff from the NCCN, in consultation with the group’s members, this approach uses a standardized scale to provide consensus-based scoring of the efficacy, safety, and affordability of a drug or a regimen and the quality and consistency of the evidence associated with that drug or regimen. Each of the 5 measures in the NCCN’s approach is displayed as a solid block using a scale from 1 to 5, where 1 is considered least favorable and 5 is most favorable.7
The Memorial Sloan Kettering Cancer Center (MSKCC) DrugAbacus tool, conceived by Peter B. Bach, MD, MAPP, Director of the Center for Health Policy and Outcomes at MSKCC and launched in June 2015, targets physicians and policymakers (not patients as done by ASCO and NCCN) with an “interactive tool [that] takes more than 50 cancer drugs and lets you compare the company’s price to one based on value.”8 This system delivers a value-based price for a drug that graphically represents the user’s weighted preferences and estimated monthly costs relative to 52 cancer drugs.12
The Institute for Clinical and Economic Review (ICER), an independent nonprofit organization founded in 2005 by Harvard physician-researcher Steven D. Pearson, MD, MSc, launched its assessment program in July 2015, with guidance from an advisory committee of payers, patient organizations, physician organizations, and the biopharmaceutical industry. Targeting payers and policymakers (and not physicians as with the ASCO and NCCN frameworks and the DrugAbacus), ICER delivers a value-based price benchmark anchored in the real benefits that a specific drug brings to patients.9
The European Society for Medical Oncology (ESMO), a nonprofit professional medical oncology society that provides evidence-based recommendations for basic standards of cancer care, launched the Magnitude of Clinical Benefit Scale in May 2015 to assess new anticancer drugs approved by the European Medicines Agency. Similarly geared to payers and policymakers, this framework was designed to assist oncologists in evaluating the most effective anticancer medicines for their patients, providing a relative ranking for each drug on a magnitude of clinical benefit scale for curative and noncurative settings.10
Dissection and Cross-Comparison of the 5 Frameworks
Compiling a comparative cross-section of these frameworks from their source bodies,6-10 the methodology of each framework was further parsed into emphasis (Table 1), input (Table 2),5-10 scoring (Table 3),6-10 and output (Table 4).6-10
Together, this matrix has allowed us to draw comparisons between the frameworks’ scope, capabilities, and limitations. Given the diversity in composite output(s) of each framework, direct cross-framework comparisons should be discouraged. Certain frameworks (eg, ASCO’s) also discourage comparison within the framework, given the nature of the clinical trial input for each drug (ie, single-arm vs head-to-head clinical trial). Considering the systems collectively, several key recurring themes, which are outlined below, warrant further investigation, given their impact on the intended stakeholders’ (ie, patients, physicians, and/or payers) adoption of the frameworks.
Beginning with inputs (Table 2), randomized controlled trials, by the nature of their design to minimize selection bias, are often not representative of the demographic distribution of the actual patient population nor the practical choices that patients encounter.13 Thus, frameworks (eg, NCCN’s and ICER’s frameworks) that rely on consensus or combined analysis of multiple clinical trials and a variety of clinical end points may, in some circumstances, provide a better indication of the therapeutic value of a drug in the larger population.
A recent survey of 50 oncologists and 55 payers revealed a substantial lack of confidence in the ASCO Value Framework, which quantifies a single randomized controlled trial.14 Rather, survey responders’ preferences leaned toward comparison across clinical trials and a measure of cost within the core net health benefit score.14 Notably, these factors are provided through the NCCN Evidence Blocks,7 considering that value is derived from the consistency of quality evidence across multiple sources filtered through the prism of accumulated clinical experience.
Lack of Real-World Evidence
Across the oncology value frameworks, there remains a lack of real-world evidence and ready access to subpopulation analyses across patient types. Indeed, conclusions from a workshop of patients, patient advocates, and pharmaceutical and medical device manufacturers indicated a need to better account for the heterogeneity of the patient population that exists outside of a randomized clinical trial, specifically the gap between the randomized sample set utilized for evidence generation and the subsequent variability of response among patient subgroups.15 An anonymous attendee at that workshop observed an absence of clinical trial design models providing either inputs or outputs for the consideration of patient heterogeneity, or even subpopulations,15 which is echoed by the survey of 50 oncologists and 55 payers discussed earlier.14
Understanding the methodologies of the scoring algorithms (Table 3) is crucial to evaluating the value and limitations of the frameworks’ outputs. In ASCO’s framework, for example, the interchangeability of various primary end points, such as survival hazard ratio or overall survival with progression-free survival or recurrence rate, belies their subtle differences. Given the interplay between shared decision-making and the usability of information, Schwartzberg and colleagues have asserted that the information should be accessible to patients, as well as understandable and usable.16
Suboptimally, the framework and its outputs should at least be in a form that providers can easily relay to patients. However, from our interviews with payer and secondary research exploring usability, we note a dearth of investigation into patient and provider usability. Indeed, our informal discussions with physicians and payers regarding the analysis of value confirm a hesitancy in applying these frameworks in practice until they better understand how to apply and extract value from the frameworks’ inputs. This sentiment was particularly expressed in regard to the ASCO and the NCCN frameworks.
Evidence supporting the specific fixed weights of certain variables, as well as their binary inclusion, is not available. Similarly, the logic supporting the potential ranges of variable weights (of MSKCC DrugAbacus and a future version of ASCO’s framework) remain vague. For example, a side-by-side comparison of the ASCO and ESMO frameworks revealed disparity in criteria stringency.17 Similarly, the correlation between ASCO’s net health benefit score and ICER’s comparative clinical and incremental cost-effectiveness outcomes is low.18
Consequently, ESMO’s or ICER’s rating for a specific treatment is not predictive of the score it will receive under ASCO’s framework, or vice versa. It has been suggested that clinical benefits and toxicities be evaluated for context by clinicians who have extensive and active expertise in the disease area being examined.16 However, it is unclear how this methodology is standardized, if at all.
Schwartzberg and colleagues have suggested that organizations authoring value frameworks should be held to scientific standards via peer review.16 They posit that the publication of their methodologies in relevant peer-reviewed journals would invoke authorship standards for scientific documents, resulting in greater clarity about the characterization process, terminology utilized, and the quantitative relation of outcomes to calculated results.16
This would allow the assumptions and methodology to be more transparently documented, explained, referenced, and evaluated for potential bias. For example, Wilson and colleagues revealed significant scoring reliability issues when they asked 8 clinicians to complete the ASCO Value Framework for 11 anticancer medications.18
How these frameworks will be patient-tailored remains unclear, because individual patient disease characteristics are not considered by the frameworks developed by ASCO, the NCCN, or ESMO.6,7,10 Stakeholders at the workshop mentioned earlier noted that although value to the individual patient remains the most important consideration in any patient-centric treatment assessment tool, none of the current value frameworks considers short- or long-term value from the individual patient’s perspective.15
Total Cost of Care
Beyond limited individual drug cost reporting or comparative assessment of treatment options, the final framework outputs (Table 4) are often of little help in defining the total cost of care, given that pharmaceuticals represent a mere 5% to 20% of the total cost of cancer treatment.19 Neither ASCO nor MSKCC considers other medical costs, such as reducing the need for surgery or hospitalization.6,8 Conversely, the NCCN defines its affordability measure as the overall cost of an intervention, including the drug, infusions, supportive care, toxicity monitoring and management, and the probability of care being delivered in the hospital.7 To an ever-greater degree, ICER considers the total cost per patient along with the aggregate cost.9
However, in none of these frameworks, not even the patient-oriented frameworks, is affordability considered at the patient-tailored level.6-10 As such, how can providers assess what is affordable, especially because an individual patient’s budget impact and tradeoffs, length of treatment, and considerations related to insurance coverage maximums, copays, and dynamic formulary inclusion and exclusion criteria may vary between patients, as well as within the treatment period for that patient?15
Implications for Payers and Physicians
Healthcare stakeholders have adopted a wait-and-see attitude as the different frameworks iterate and develop followings. Our structured interviews with payers engaged in evaluating oncology drugs over the past 2 years and spanning more than 100 hours of discussion indicate a split in payer expectations of these frameworks to influence oncology drug assessments in the near future. This split was confirmed through an aggregate analysis of 3 separate studies between 2015 and 2016 comprising a total of 101 payers, where a mere 53% expect that these frameworks will influence their assessment of the value of oncology drugs in the near future.20-22 In addition, our dialogues with payers spanning more than 100 hours of discussion further confirm the findings of a recent study, demonstrating payer familiarity with at least ASCO’s and NCCN’s value frameworks.22
Our structured interviews with US physicians engaged in evaluating oncology drugs, over the past 2 years and spanning more than 100 hours of discussion, similarly confirm the results of previous research comprising 93 medical or hematologic oncologists, which showed that the greatest awareness among physicians remains with the ASCO and the NCCN frameworks.23
Although the use of value frameworks in oncology is increasing in clinical practice,5 especially regarding the ASCO framework, physicians have noted in discussion with us that the frameworks will be more useful in clinical practice in the future, as they become more established and their outputs are more widely accepted. Indeed, these physicians indicated to us that they were beginning to consider how to utilize value frameworks in the near future, emphasizing the importance of providing a comparative assessment of various treatment options available and their relative financial implications to patients.
This was confirmed through our combined analysis of 243 physicians from surveys collected in 2016, in which 51% of providers indicated they were considering utilizing value frameworks in their practice compared with 11% who were not considering utilizing them, and 38% who were unsure.20,23
Implications for Drug Manufacturers
Although a dominant value framework to guide the pharmaceutical industry has yet to emerge, the input of patients, providers, and payers, coupled with the current approaches for assessing value through the frameworks, can provide the drug manufacturers with additional considerations in modeling value in the analysis of their clinical and commercial strategy in oncology. However, the pharmaceutical industry would be wise to remain current with these value frameworks and their potential updates, as well as contribute to discussions with framework bodies (ie, ASCO, NCCN, ESMO) and seek resources to understand payer and prescriber perspectives about these models.
Integrating comparative analysis methods into strategy, risk management, and value modeling may allow pharmaceutical manufacturers to better position their oncology assets at all stages of development. Integrating value framework considerations into clinical trial design (ie, preliminary or interim clinical data that hint at value outputs) can provide an additional tool for modeling their oncology drugs’ value and the value of the associated clinical trials or subsequent forecast clinical trials, even before the trial is complete. Standardizing appropriate clinical trial comparators would also reduce the incomplete inputs that are crucial for internal value framework analysis and external analysis by various stakeholders, particularly payers.
Incorporating value framework considerations may further assist drug makers in their internal price modeling of an oncology drug when developing drug launch price strategies. Insight into the attitudes of payers toward individual value frameworks, as well as toward specific aspects of a particular framework, may be vital to developing optimal drug pricing and reimbursement strategies (eg, premium pricing).
By understanding the meaning of the output(s) generated by each framework, as well as the value of each framework to the various stakeholders, pharmaceutical companies have an opportunity to selectively utilize these frameworks to shape their value modeling in the analysis of their clinical and commercial strategies.
In addition, value framework assessments may prove valuable in identifying innovative access strategies to a specific therapy for access to favorable situations. Even with these potential benefits, stakeholder education through outreach strategies will still be crucial when communicating with various healthcare stakeholders, including payers, physicians, and patients; such context will be essential for understanding why one or several value frameworks are favored over others when reaching specific conclusions.
Our findings suggest that payers are aware of the frameworks and see the potential in them, but they are split on whether they will utilize the frameworks in the near future. Likewise, physicians are considering the use of these frameworks, yet they do not fully appreciate their applications and limitations in decision-making. Consequently, although incorporating elements of these algorithms may assist pharmaceutical companies in internal pricing and reimbursement strategies, our dialogues with pharmaceutical stakeholders confirm our primary and secondary research findings among physicians and payers, regarding the uncertainty as to which framework will gain the greatest traction among stakeholders, whether it is the patient, provider, or payer.
Although each of the 5 value frameworks has selectively targeted patients, physicians, and/or payers using a unique interpretation of value, the relevance of the 5 frameworks to direct (ie, providers, payers) and indirect (ie, drug manufacturers) stakeholders will depend on adoption of the framework. Factors that may drive value framework adoption include appreciation for real-world evidence, the relation of cost to affordability, transparency regarding the quantification process and outputs, and, where stakeholder-relevant, the framework’s applicability to individual decision-making. Which value framework becomes more established and widely accepted, and by what stakeholder, will influence how the pharmaceutical industry will shape the clinical and commercial development of its oncology drugs.
We wish to thank David Stanton, Vice President of Marketing Communications at GfK, for his input in the revisions and preparation of this manuscript.
Author Disclosure Statement
Dr Slomiany, Ms Madhavan, Mr Kuehn, and Ms Richardson reported no conflicts of interest.
1. Newcomer LN. Innovative payment models and measurement for cancer therapy. J Oncol Pract. 2014;10:187-189.
2. Mariotto AB, Yabroff KR, Shao Y, et al. Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst. 2011;103:117-128. Erratum in: J Natl Cancer Inst. 2011;103:699.
3. American Society of Clinical Oncology. ASCO in Action Brief: value in cancer care. January 21, 2014. www.asco.org/advocacy-policy/asco-in-action/asco-action-brief-value-cancer-care. Accessed October 10, 2016.
4. Stowell C, Akerman C. Better value in health care requires focusing on outcomes. Harvard Business Review. September 17, 2015. https://hbr.org/2015/09/better-value-in-health-care-requires-focusing-on-outcomes. Accessed March 10, 2017.
5. Chandra A, Shafrin J, Dhawan R. Utility of cancer value frameworks for patients, payers, and physicians. JAMA. 2016;315:2069-2070.
6. Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American Society of Clinical Oncology Value Framework: revisions and reflections in response to comments received. J Clin Oncol. 2016;34:2925-2934.
7. National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) with NCCN Evidence Blocks. www.nccn.org/evidenceblocks/default.aspx. Accessed October 10, 2016.
8. Memorial Sloan Kettering Cancer Center Drug Pricing Lab. The DrugAbacus Tool. http://drugpricinglab.org/tools/drug-abacus/. Accessed June 8, 2017.
9. Institute for Clinical and Economic Review. Evaluating the value of new drugs and devices. 2015. www.icer-review.org/wp-content/uploads/2014/01/Value-Assessment-Framework-9-71.pdf. Accessed October 10, 2016.
10. Cherny NI, Sullivan R, Dafni U, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol. 2015;26:1547-1573.
11. Schnipper LE, Davidson NE, Wollins DS, et al; for the American Society of Clinical Oncology. American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol. 2015;33:2563-2577.
12. Memorial Sloan Kettering Cancer Center Drug Pricing Lab. Methods. http://drugpricinglab.org/tools/drug-abacus/methods/#faqs. Accessed June 8, 2017.
13. Basch E. Toward a patient-centered value framework in oncology. JAMA. 2016;315:2073-2074.
14. Xcenda AmerisourceBergen. Assessing value in oncology: models & perspectives from payers and oncologists. www.xcenda.com/Documents/Assessing%20Value%20in%20Oncology%20-%20Models%20%20Perspectives%20from%20Payers%20and%20Oncologists%20FINAL%20for%20print%205-18-16.pdf. Accessed January 17, 2017.
15. Milken Institute; Avalere. Integrating the patient perspective into the development of value frameworks. March 2016. www.fastercures.org/assets/Uploads/value-coverage-framework-March-2016.pdf. Accessed January 17, 2017.
16. Schwartzberg L, Ettinger D, Jahanzeb M, et al. Op-Ed: our view on value frameworks in oncology: proposing principles for value framework development. OBR Green. 2016;11. http://obroncology.com/article/op-ed-our-view-on-value-frameworks-in-oncology-proposing-principles-for-value-framework-development/. Accessed January 17, 2017.
17. Subramanian R, Schorr K. How valuable is a cancer therapy? It depends on who you ask. Comparing and contrasting the ESMO Magnitude of Clinical Benefit Scale with the ASCO Value Framework in Cancer. Poster presented at the International Society for Pharmacoeconomics and Outcomes Research Annual International Meeting; Washington, DC; May 21-25, 2016.
18. Wilson L, Lin T, Wang L, et al. Evaluation of the ASCO Value Framework for anticancer drugs at an academic medical center. J Manag Care Spec Pharm. 2017;23:163-169.
19. Young RC. Value-based cancer care. N Engl J Med. 2015;373:2593-2595.
20. Green M, DiPaula K; for Kantar Health. The path toward drug value definition: implications of value initiatives in oncology. White paper. March 2016. www.kantarhealth.com/docs/white-papers/the-path-toward-drug-value-definition-implications-of-value-initiatives-in-oncology.pdf. Accessed May 8, 2017.
21. Seidman J, Choe SH. Most health plans do not use existing value frameworks to make coverage decisions. Avalere Health. July 26, 2016. http://avalere.com/expertise/life-sciences/insights/most-health-plans-do-not-use-value-frameworks-to-make-coverage-decisions. Accessed May 8, 2017.
22. Capuano C, Divi B, Singhania A, Suponcic S. How are US payers using emerging pharmaceutical value frameworks? Presented at the International Society for Pharmacoeconomics and Outcomes Research Annual International Meeting; Washington, DC; May 21-25, 2016.
23. Lederman L. Value tools at ASCO 2016: building a framework for prime time. ZS Associates. www.zs.com/publications/articles/value-tools-at-asco-2016-building-a-framework-for-prime-time.aspx. Accessed May 8, 2017.