The eighteenth-century essayist and satirist Jonathan Swift made the observation that “vision is the art of seeing things invisible.” So, too, is “the art of seeing things invisible” a key for the ongoing sustainability of health information exchange (HIE). HIEs have long been theorized to provide a number of tangible benefits. These benefits accrue through the provision of medical history at the point of care: decreases in redundant laboratory testing, improved provider efficiency, improved care coordination, increased quality of care, and the ultimate goal of an overall decreased cost of care.
More recently, we have witnessed a seismic movement from theory to practice with definitive dollar savings noted for HIE use in emergency departments in Indianapolis, IN,1 and in Milwaukee, WI2 ($26 and $29 savings per emergency department visit, respectively),1,2 as well as in Memphis, TN (approximately net $1.1 million in savings for the community at large).3 Moreover, in Memphis, the great majority of dollar savings (97.6%) resulted from the avoidance of inpatient admissions from the emergency department.3 Inpatient admissions account for the preponderance of dollars spent in healthcare. Costs for inpatient admissions in the United States are increasing; during calendar year 2004, the average inpatient admission cost was $10,0304; by 2008, this increased to $15,017.5
A community replicating the Memphis experience by mitigating inpatient admissions from the emergency department should experience financial savings as Memphis did. The Memphis experience showed that HIE availability within the emergency department decreases direct admissions from the emergency department.3 But can HIE availability in the emergency department indirectly impact admissions emanating from outside of the emergency department? Is the risk of any inpatient admission occurring altered by the presence of HIE in the emergency department? If so, the community benefits indirectly, as well as directly, from having said HIE occurs within the emergency department. However, achieving that benefit requires HIE sustainability, and HIE sustainability requires a stable source of funding. Enhancing the business case for HIE sustainability by uncovering such indirect or “hidden” value may help validate the need for external support and funding.
In our previous article promoting the “business case for payer support of a community-based HIE,” we described the relationship between Humana in Southeast Wisconsin and the local HIE.2 To briefly summarize, beginning in December 2008, Humana provided a financial incentive to the Wisconsin Health Information Exchange (WHIE) for promoting the querying of a database by emergency department clinicians (as a part of their workflow) for fully insured members presenting to the emergency department for care.6 WHIE links together disparate emergency departments across 5 competitive health systems in Milwaukee County.7
Our previous evaluation showed a positive direct financial outcome for our health plan, with an average savings of $29 per emergency department visit when clinicians queried the WHIE in the course of providing emergency department care as opposed to when the WHIE was not queried.2 We also realized a direct return on investment (ROI) of more than 2:1.2 Further analysis looking at other potential sources of value, some of which may result from indirect savings, may help further the business case for external HIE support and may also show that the value of HIE may even be higher than what we can quantitatively measure.
Study Design: Developing the Sample
Population for Evaluation
The Humana version of an Institutional Review Board, the clinical “Stage Gate Process,” provided approval for this pilot assessing the impact of HIE query in the emergency department. The planned evaluation included both observational and retrospective analyses. In developing the member pool from which to draw the evaluation population, Humana and the WHIE had agreed that the plan would provide to the WHIE a financial incentive to cover its costs for promoting emergency department clinicians’ querying of the WHIE database. Although queries apply to all patients, the incentive only covered eligible Humana members presenting to the emergency department for care.8
Eligible members were commercial, fully insured members only; self-funded group members, as well as members covered by governmental programs (eg, Medicare), were specifically excluded. Every quarter, WHIE provided Humana specific information about each individual health plan member who was fully insured by Humana and who sought emergency department care, as well as when the emergency department clinician accessed the WHIE database for that patient and at which facility. WHIE only provided clinical data as it would appear on a claim related to the encounter. All communications were HIPAA compliant and used encrypted files. The information that was provided allowed Humana to match emergency department claims data received from providers with the emergency department encounter by date of service and facility.
In working with claims data, we stipulated precise parameters for member inclusion in the evaluation sample. Inclusion criteria noted in the original analysis stipulated that2:
- All members included in the evaluation must have had at least 12 months of continuous coverage with our health plan
- Members would be excluded from the evaluation if they had either less than 6 months of coverage before the start of the program or less than 3 months of coverage after the start of the program
Because admissions from the emergency department or prolonged emergency department holds of “24-hour observations” would have impaired our ability to perform the original analysis on emergency department costs, we excluded those members (admitted from or held in the emergency department) from the analysis
These exclusions prevented potential skewing of the data for our analyses.
Study Design: Developing the Control Group
and the Test Group
In our previous article, we discussed in great detail how we determined who would make up the control and test groups.2 Humana identified members seen in the emergency department when the WHIE database was queried in both a first emergency department visit and a subsequent emergency department visit as eligible to be included in the test group; members seen in the emergency department where the WHIE database was not queried in neither a first emergency department visit nor in a subsequent emergency department visit (because the facility had not yet provided WHIE access at that time) were eligible to be included in the control group. A total of 428 plan members were deemed eligible for the test group, whereas 1054 plan members met control group eligibility.2
In addition, our evaluation deliberately assumed the need for propensity scoring, because that technique affords the best way to match members, while minimizing bias. Propensity scoring provides “the conditional probability of receiving the treatment given the observed covariates.”9 In their defining article, Rosenbaum and Rubin showed that “the adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates.”10 Furthermore, propensity scoring has been found to yield estimates that are not substantially different from typical multivariable methods.11,12
For the logistic regression yielding the propensity scores, we used all of the following combinations of cost-related and demographic variables to match the 2 groups: age, sex, medical net paid per participant per month (PPPM), prescription net paid PPPM, medical plus prescription net paid PPPM, medical inpatient net paid PPPM, medical outpatient net paid PPPM, and medical physician net paid PPPM. With the exception of age and sex, all of these variables represent dollar values, because dollar values are easy to calculate from claims and they were unrelated to the specific exposure (ie, WHIE database querying).
Propensity scores on which we matched the participants used the nearest neighbor algorithm. Matching allows for “sampling from a large reservoir of potential controls to produce a control group of modest size in which the distribution of covariates is similar to the distribution in the treated group.”9 For member matching, MATLAB version 18.104.22.168 was used.13
Once we completed matching 325 pairs of individuals for the test and control groups, we analyzed differences in the metrics of interest for the 2 groups. For descriptive chi-square and other statistics, SAS Enterprise Guide version 4.2 was used.14 We compared all claims for the 2 groups for a time period beginning 1 year before an individual’s first emergency department visit date to an end date of 1 year after that first emergency department visit date; therefore, each individual’s length of time in the pilot was 1 full year.
The pilot ran from December 2008 through March 2010. Within that 1-year time period, a group member would still need to have a second emergency department visit before the end date. The emergency department visit served to delineate a point in time where we evaluated member utilization; in other words, we looked at the member’s inpatient utilization at the time of a first emergency department visit and then again at the time of a second emergency department visit.
For this specific analysis, we looked at differences in inpatient admissions, inpatient days, and length of stay (LOS) to gauge a possible association for HIE impact outside of the emergency department. We used routine payer parameters to calculate differences in admissions per 1000 members, bed days per 1000 members, and in average LOS per admission at the times of a first and of a subsequent emergency department visit for the 2 populations of interest, with adjustment for trend between the 2 time periods.
Descriptive results before and after propensity score matching for all eligible control population and test population members are shown in Table 1. Table 2 outlines inpatient admissions per 1000 members for the propensity-matched cohort, as well as for the summed population results at the time of a given emergency department visit and by group designation. A chi-square test for independence of the groupings and the time period of emergency department visit show that the admissions per 1000 members of each group are not independent of the time period when a member of the group was seen in the emergency department. This finding implies that inpatient admissions, unrelated to an emergency department visit, may be impacted by the use of HIEs within the emergency department.
Table 3 describes the conditional probabilities within the 2 groups; we specifically examined the probability of an admission during a specific time frame given the possible use of HIEs in an emergency department. The conditional probability results show that first, the probability for having had an admission in either group is more likely at the time of a second emergency department visit than at the time of a first visit (eg, 67.4% of all admissions studied occurred by the time of a second emergency department visit).
Second, the probability of having had an admission at the time of a first emergency department visit is greater in the control group (37% vs 29%, in favor of admission with no HIE query in the emergency department), whereas the probability of having had an admission at the time of a second emergency department visit is greater in the test group (71% vs 63%, in favor of admission with HIE query in the emergency department).
Another way of viewing these results is to note that the absolute risk of any inpatient admission by the time of a first emergency department visit is 8% higher (relative risk [RR], 28% higher) in the control group, whereas the absolute risk of any inpatient admission by the time of a second emergency department visit is 8% lower (RR, 11% lower) in the control group.
Table 4 and Table 5 show the results of the other standard payer metrics, inpatient bed days per 1000 members, and average LOS for the propensity-matched cohort, by time period of emergency department visit and by group designation. As seen in Tables 4 and 5, when accounting for trend, the availability of HIE in the emergency department may be associated with shorter LOSs for admissions emanating outside the emergency department (4.27 days per admission) and a decreased number of inpatient days in total (771 bed days per 1000 members).
However, the noted results, especially for the average LOS calculations, may be skewed by an abnormally high number of catastrophic cases, as noted by the maximum LOSs in Table 6. Addressing such discrepancies by removing all inpatient admissions with an LOS of at least 33 days, even without further adjusting for the propensity matches, generates the results outlined in Table 7. These results show that simply having HIE available in the emergency department may yield a potential savings in LOS of nearly 1 full day (0.95 days per admission).
Approximately 44% of all hospital admissions, or 55% of hospital admissions excluding pregnancy and childbirth, use the emergency department as the conduit for entry.15 Conversely, 56% of all admissions (or 45% of all admissions, excluding pregnancy and childbirth) are not admitted through the emergency department. Such findings necessitate looking at methods to alleviate hospital admissions that do not originate from the emergency department. One way of doing this is to ensure appropriate admissions.
Review of our results show some promising findings. In our study, an emergency department visit by a member of the test group or the control group did not result in an admission from the emergency department. However, each group’s members had admissions from outside of the emergency department.
When we look at the likelihood of a first admission from outside of the emergency department by a group member, the results show a 28% higher probability of an admission when HIE is not available in the emergency department. Given that physician offices provide data for HIE in the emergency department, it is more likely that emergency departments with access to HIE have physicians with access to HIE. We could theorize that a lack of access to information at the point of care, especially if that point of care is outside of the emergency department, may provide the impetus for potentially inappropriate admissions.
In his study, Campbell previously noted that 28% of the hospital admissions deemed as “inappropriate” occurred secondary to a need for the performance of treatment or tests that could have been performed on an outpatient basis.16 Moreover, assumptions that appropriate admissions require longer LOSs do decrease what may be considered “inappropriate,” because admissions with shorter LOSs should not have been admitted at all.17
Multiple factors can play a role in potentially inappropriate admissions, including, but not limited to, difficulty in organizing continuity of care (eg, outpatient physician follow-up)18 versus receipt of community services (eg, home healthcare)19 or even rural geography.20 That members of the test group had more admissions at the time of a second emergency department visit could imply that the use of HIEs before then might have played a role in avoiding inappropriate admissions, thereby leading to more appropriate use of inpatient resources overall.
From our previous study, we were certainly aware that the test group “required higher intensity care on a claims dollar basis,” implying that they were “sicker” on the basis of claims.2 Conversely, because the control group had more inpatient admissions by the time of a first emergency department visit, we could surmise that a lack of connectivity factored in that finding as well. Could that result prolong hospital stays?
Our finding of a significant decrease in the bed days per 1000 members for the test group relative to the control group by the second emergency department visit makes us begin to question if there is a correlation of HIE availability with shorter hospital stays; in fact, the noted savings of 4.27 days per admission seemed so extreme (because of several cases of at least 33 days per admission), that it necessitated removing 5 of 238 (2.1%) admissions from the test group and 14 of 231 (6.1%) admissions from the control group to better assess this premise. Despite removing catastrophic cases, we still found a decrease of nearly 1 full day per admission for the test group. Having HIE itself in the emergency department did not directly influence this finding, but it certainly could have acted indirectly.
Research has shown that indirect returns can account for 50% of a technology’s ROI.21 It is this ROI that is made meaningful by, in this case, decreasing inpatient services.22 ROI should certainly not be the only measure of the value that HIEs bring.23 HIEs can offer a clinical “value added” through providing services in a manner that an alternative cannot.24 In our case, the “service” provided may be the indirect promotion of more appropriate inpatient admissions, containment of inappropriate admissions, and a decrease in LOS. However, to reduce costs associated with these parameters may require, as Porter says, spending more on other services.25 In our case, the trade-off necessitates that stakeholders justifiably sustain HIEs. Of all stakeholders, accountable care organizations should be especially interested.26 Aligning physicians and payers in this endeavor should also optimize value.27
We need to account for several potential limitations to this study. First, although the use of propensity scoring methods to create test and control groups should minimize potential bias, any time data manipulation occurs, potential new risks from bias need to be acknowledged. Methods exist to minimize biases arising from such risks.28
Second, the use of HIE in our study groups was limited to emergency department visits. Although we can hope for bidirectional information flow in the use of HIEs, we cannot actually prove that. Therefore, the association between HIE use in the emergency department and decreased inpatient admissions from outside of the emergency department is just that, an association. We cannot necessarily prove a direct cause-and-effect relationship. Nonetheless, the results remain intriguing, such that we plan further study on what we found here.
Third, we cannot discount the potential impact of so-called human factors.
Although we believe that the use of HIE in emergency departments influences physician behavior outside of the emergency department, we do not actually know if this is the case. As Churchman notes, “knowledge resides in the user and not in the collection [of information]. It is how the user reacts to a collection of information that matters.”29 Physicians caring for patients have a lot of nonexchangeable information at their disposal and that information may certainly impact potential admissions as much as, if not more than, HIE use in the emergency department. Last, one cannot quantify “indirect” savings. Although we can estimate potential savings in arguing for HIE sustainability, we cannot quantify something that never happened. Nonetheless, the argument still stands through logic and through extension.
The impact of “direct” value is easily quantified: it is tangible, visible, and deduced from the evidence.30 The impact of “indirect” value is much harder to evaluate: it must be induced from the evidence.30 By definition, then, it is much harder to “see” indirect benefits, because they are hidden from view. When it comes to visualizing the impact of HIE, one can follow a similar line of reasoning. Having clinicians access HIE in the emergency department has already shown direct benefits in the form of an average savings of $26 to $29 per emergency department visit,1,2 as well as avoided inpatient admissions directly from the emergency department.3 Our current results build on that direct confirmation by adding indirect evidence for HIE value.
HIE availability in the emergency department is associated with an effect outside of the emergency department when it comes to hospital admissions in general. Potentially inappropriate admissions may be avoided, whereas admissions that do occur result in a shorter LOS. An argument could be made that this is a function of “economies of connection.”31 As Beckham notes, “technology can collapse distance by generating virtual proximities….As some networks, products, and services become more widely used, they become exponentially more valuable….Proximity and networks…generate real value by connecting intellect, facilitating collegiality, and supporting collaboration.”31
Facilitating collaboration through networking is especially needed, given that the same individual will seek care at multiple facilities.32
So, coming full circle, the need for HIE becomes reinforced, with sustainability remaining a paramount objective. For sustainability, payers, providers, and other stakeholders need to help pave a path to that goal. Although those stakeholders need to ascertain the value that they receive from HIE, all must understand one thing: awareness of the indirect value brought forward by HIEs is as important as the direct value received. Having the vision to “see” this indirect value is vital because, as it is written in Proverbs (29:18), “where there is no vision, the people perish.”
All funding for this project and its analysis was provided by Humana, Inc, Louisville, KY.
Author Disclosure Statement
Dr Tzeel is a consultant to Amylin and is employed by and owns stock in Humana. Dr Lawnicki is employed by and owns stock in Humana. Mr Pemble is employed by the National Institute for Medical Informatics/WHIE.
- Overhage JM, Dexter PR, Perkins SM, et al. A randomized, controlled trial of clinical information shared from another institution. Ann Emerg Med. 2002;39:14-23.
- Tzeel A, Lawnicki VL, Pemble KR. The business case for payer support of a community-based health information exchange: a Humana pilot evaluating its effectiveness in cost control for plan members seeking emergency department care. Am Health Drug Benefits. 2011;4:207-216.
- Frisse ME, Johnson KB, Nian H, et al. The financial impact of health information exchange on emergency department care. J Am Med Inform Assoc. 2012;19:328-333.
- Machlin SR, Carper K. Expenses for Hospital Inpatient Stays, 2004. Statistical Brief #164. March 2007. Agency for Healthcare Research and Quality, Rockville, MD. www.meps.ahrq.gov/mepsweb/data_files/publications/st164/stat164.pdf. Accessed September 25, 2012.
- Agency for Healthcare Research and Quality. Hospital inpatient services-median and mean expenses per person with expense and distribution of expenses by source of payment: United States, 2008. Medical Expenditure Panel Survey. http://meps. ahrq.gov/mepsweb/data_stats/tables_compendia_hh_interactive.jsp?_SERVICE=ME PSSocket0&_PROGRAM=MEPSPGM.TC.SAS&File=HCFY2008&Table=HCFY2008%5FPLEXP%5FD&VAR1=AGE&VAR2=SEX&VAR3=RACETH5C&VAR4=INSURCOV&VAR5=POVCAT08&VAR6=MSA&VAR7=REGION&VAR8=HEALTH&VARO1=4+17+44+64&VARO2=1&VARO3=1&VARO4=1&VARO5=1&VARO6=1&VARO7=1&VARO8=1&_Debug=. Accessed December 15, 2011.
- 6. Neupert P. Re-charting healthcare: innovations to drive a new delivery model for tomorrow’s health system. In: Merritt D, ed. Paper Kills 2.0: How Health IT Can Help Save Your Life and Your Money. Washington, DC: Center for Health Transformation Press; 2010:15.
- Wisconsin Health Information Exchange. Information Exchange. www.whie.org/ regional-activities/information-exchange. Accessed September 25, 2012.
- Humana to partner with WHIE on emergency department data exchange. January 29, 2009. WTN News. http://wistechnology.com/articles/5432/. Accessed September 25, 2012.
- Rosenbaum PR. Observational Studies. 2nd ed. New York, NY: Springer-Verlag; 2002:296.
- Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41-55.
- Shah BR, Laupacis A, Hux JE, Austin PC. Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J Clin Epidemiol. 2005;58:550-559.
- Stürmer T, Joshi M, Glynn RJ, et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59:437-447.
- MATLAB. The Language of Technical Computing. www.mathworks.com/ products/matlab/. Accessed June 18, 2010.
- SAS Enterprise Guide. www.sas.com/technologies/bi/query_reporting/guide/. Accessed June 18, 2010.
- Elixhauser A, Owens P. Reasons for being admitted to the hospital through the emergency department, 2003. HCUP Statistical Brief #2. February 2006. Rockville, MD: Agency for Healthcare Research and Quality. www.hcup-us.ahrq.gov/reports/ statbriefs/sb2.pdf. Accessed September 25, 2012.
- Campbell J. Inappropriate admissions: thoughts of patients and referring doctors. J R Soc Med. 2001;94:628-631.
- Lo CM, Leung SH, Lam CS, Yau HH. Clinical audit on short stay emergency medical admission. Hong Kong J Emerg Med. 2003;10:30-36.
- Davido A, Nicoulet I, Levy A, Lang T. Appropriateness of admission in an emergency department: reliability of assessment and causes of failure. Qual Assur Health Care. 1991;3:227-234.
- Coast J, Peters TJ, Inglis A. Factors associated with inappropriate emergency hospital admission in the UK. Int J Qual Health Care. 1996;8:31-39.
- Carasso S, Shmueli T, Arnon R, Askenazi I. Characteristics of emergency room admissions of IDF soldiers in northern Israeli hospitals between May 2002 and April 2003. Harefuah. 2004;143:8-11,87,88.
- Nucleus Research. Indirect benefits: the invisible ROI drivers. February 2007. http://nucleusresearch.com/research/notes-and-reports/indirect-benefits-the-invisible- roi-drivers/. Accessed December 15, 2011.
- Rauh SS, Wadsworth EB, Weeks WB, Weinstein JN. The savings illusion—why clinical quality improvement fails to deliver bottom-line results. N Engl J Med. 2011;365:e48.
- Volpp KG, Loewenstein G, Asch DA. Assessing value in health care programs. JAMA. 2012;307:2153-2154.
- Joshi JK. Clinical value-add for health information exchange (HIE). Internet J Med Inform. 2011;6(1). www.ispub.com/journal/the-internet-journal-of-medical-informatics/volume-6-number-1/clinical-value-add-for-health-information-exchange-hie.html. Accessed December 7, 2011.
- Porter ME. What is value in health care? N Engl J Med. 2010;363:2477-2481.
- Dimick C. ACOs driving HIE development, competition. J AHIMA. May 1, 2012. http://journal.ahima.org/2012/05/01/acos-driving-hie-development-competition/. Accessed May 17, 2012.
- Tzeel A. Biologic therapies for rheumatoid arthritis: it’s all about value. Am Health Drug Benefits. 2012;5:91-92.
- Luellen JK, Shadish WR, Clark MH. Propensity scores: an introduction and experimental test. Eval Rev. 2005;29:530-558.
- Churchman CW. The Design of Inquiring Systems: Basic Concepts of Systems and Organization. New York, NY: Basic Books; 1971:10.
- Bovee CL, Thill JV, Schatzman BE. Business Communication Today. 7th ed. Upper Saddle River, NJ: Prentice Hall; 2002.
- Beckham D. Economies of connection. Hosp Health Netw. August 16, 2011. www.hhnmag.com/hhnmag/HHNDaily/HHNDailyDisplay.dhtml?id=7220009058. Accessed December 15, 2011.
- Finnell JT, Overhage JM, Grannis S. All health care is not local: an evaluation of the distribution of emergency department care delivered in Indiana. AMIA Annu Symp Proc. 2011;2011:409-416.