Health professionals are increasingly encouraged to involve patients in their treatment decisions, recognizing that patients are experts in terms of their preferences and have unique knowledge of their own health.1-5 In 2010, the Affordable Care Act created the Patient-Centered Outcomes Research Institute (PCORI), which stresses the importance of patient-centeredness care.6 Patient-centeredness care refers to the need to address patient preferences, patient decision-making needs, and characteristics of patients. This emphasis on increased understanding of patient preferences and their incorporation into treatment decisions has been driven by the improvement associated with patient care and treatment satisfaction when patients’ perspectives are taken into consideration.7,8
For patient preferences to be used effectively in the delivery of healthcare, it is important to understand the desire for specific treatment attributes that shape affinities for particular therapeutic medications. With respect to rheumatoid arthritis (RA), current therapies include conventional synthetic disease-modifying antirheumatic drugs (DMARDs), biologic DMARDs, and targeted synthetic DMARDs, including the new therapeutic class of Janus kinase (JAK) inhibitors.9 These therapies vary in their mechanisms of action; they also differ in the route and frequency of their administration.
Previous RA research has addressed patient preferences for features associated with conventional synthetic and biologic DMARDs.10-12 These studies have produced a broad range of results, in part as a result of the dependence of the description of the drug features that were provided to a study participant.13 A 2004 study by Fraenkel and colleagues showed that patients preferred the biologic DMARD etanercept over conventional synthetic DMARDs, partially because of its favorable adverse event profile10; however, for their study, the copayment amounts associated with etanercept were presented to the study patients as being equal to, or only slightly higher than, the copayments associated with conventional synthetic DMARDs.10 This action of leveling the cost-sharing amount between drugs essentially negated cost as an important differentiator between the drugs.
A more recent study by Augustovski and colleagues, published in 2013, addressed the preferences of patients in Argentina for biologic DMARDs used in RA, showing that cost was the most important treatment attribute, followed by systemic adverse effects, frequency of administration, efficacy, route of administration, local adverse events, and serious infections.11
Route of administration may be an important differentiator between drugs that are used to treat RA, especially if patient preferences influence adherence and outcomes of therapy. Various medication attributes may factor into medication delivery preference, such as administration time, ease of administration, and fear of needles. Little is known about the relative importance of treatment attributes in RA since the addition of oral medications, such as targeted synthetic DMARDs, to the list of available therapies. The goal of our study, which used a conjoint analysis methodology, was to ascertain relative patient preferences associated with the route of administration and other attributes of biologic DMARDs and targeted synthetic DMARDs in the treatment of patients with RA.
A cross-sectional postal survey of patients with RA was conducted to ascertain the relative importance of individual drug attributes, including safety; efficacy; cost; and mode, method, and frequency of administration, in determining patient preferences for specific therapies for RA. A choice-based conjoint analysis was used to determine the relative importance of individual attributes for the medications of interest.13-15 Patients were blinded to the medication names; only the relevant attributes for each medication were listed in the survey.
For participation in the study, patients had to be aged 21 to 80 years at the time of survey administration, be currently enrolled in a fully insured Humana commercial health plan with medical and pharmacy benefits, and to have had at least 2 RA-related medical claims in the previous 12 months, at least 30 days apart, as identified by International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis code 714.0 (rheumatoid arthritis).
Patients were excluded from the study if they were enrolled in administrative services–only groups (which are excluded from research according to Humana policy), resided in a nursing home, were eligible for low-income subsidies, or had evidence of a paid claim for tofacitinib or for a biologic DMARD indicated for RA, psoriasis, psoriatic arthritis, or ankylosing spondylitis at any time during their health plan enrollment before the mailing of the survey.
To generate the survey for the conjoint analysis, the key attributes associated with the range of treatments of interest (ie, biologic DMARD therapies and tofacitinib) were identified, followed by the assignment of various levels for the treatment attributes. The survey included 10 choice tasks, and each choice task included 2 medication concepts (drug A and drug B), where a medication concept was defined by a specific combination of attributes chosen randomly, using Sawtooth SSI Web software (version 8.2; Sawtooth Software, Inc, Sequim, WA). Ten choice tasks was identified as the threshold to maximize the efficiency of the survey design, based on the number of attributes that defined the concept (7 attributes per concept) but would not lead to respondent fatigue.
Because the survey was aimed at generating insight about patient treatment preferences, respondents were not given the option to select “none” within the choice task. Each concept was generated randomly, and repeated concepts were not included in the survey design. The attributes were listed randomly in each survey version to mitigate ordering bias (for a version of the survey see the Appendix).
The final survey contained 2 sections, including (1) general questions regarding the respondent’s medical history related to RA, and (2) the choice-based conjoint question section described above. Four versions of the choice-based conjoint component were generated; each version of the questionnaire contained the same questions but in a different sequence. Survey versions were assigned randomly, and participants were asked to choose the most preferred drug among the alternatives in each 2-profile choice set. The conjoint portion of the survey was designed with Sawtooth software, using a generic choice design. This software was used to help generate a survey of appropriate length, given the number of attributes and their levels of interest. An internal pretesting process involving 4 independent research and nonresearch staff, as well as Humana’s formal Member Communications review, was conducted to evaluate the clarity, organization, readability, and time burden of the instruments. The survey was approved by an independent Institutional Review Board.
A 4-wave survey rollout process was used to optimize the survey response rate. The first-wave mailing, sent to patients on August 30, 2013, consisted of an advancenotice alert in the form of a brief letter. The purpose of the first-wave mailing was to inform potential participants that they would be receiving a study questionnaire and to explain the purpose of the study. The second-wave mailing, sent to patients on September 6, 2013, consisted of the survey instrument, an appropriate cover letter and consent form, as well as a self-addressed stamped envelope to facilitate response. One week later, on September 13, 2013, a follow-up reminder postcard was sent to potential participants. The fourth-wave mailing, on September 27, 2013, was sent only to nonrespondents. A copy of the survey instrument was included along with a reminder letter. The period of survey administration or collection was 8 weeks from the date of the advancenotice mailing to the closure of survey collection. Respondents were provided with a $10 gift card for their time and effort in completing the survey.
Descriptive statistics were used to summarize demographic and clinical characteristics of the study sample, obtained from administrative claims data and from responses to the first 4 survey questions, which pertained to joint history, length of time with symptoms, length of time since diagnosis, and whether the survey was being completed by the patient or by someone else. The administrative claims data included age, sex, geographic region, and the RxRiskV score, which is a prescription claims-based comorbidity index originally developed as an enhancement of the RxRisk risk assessment instrument for the Veterans Health Administration population.16-18 The RxRiskV score is determined based on the identification of 45 distinct comorbid conditions via their associated medication treatments (score range, 0-45). Two-sample t-tests and chi-square tests were used to compare descriptive statistics between respondents and nonrespondents.
The results of the conjoint analysis were analyzed to determine relative preference for each attribute (ie, order of attribute importance). An importance score reflects the effect that each attribute had on the choice made, given the range of levels included in the questionnaire. The preference for each level within an attribute was evaluated by counting the number of times the level was chosen relative to the number of times it was offered, to estimate the main effects and joint effects of the attributes, where joint effects were evaluated by the number of times medication concepts were chosen: when attributes were listed together as part of the same medication concept. The Sawtooth software calculated the number of times that an attribute level was chosen relative to the number of times it was available for choice. The software also calculated a chi-square test value for each main effect and joint effect. The test is termed “within-attribute chi-square” and indicates whether levels of a particular attribute differ significantly in frequency of choice within the respective attribute.
The influence of demographic and clinical characteristics on each attribute was assessed using hierarchical Bayesian estimation, which allows for part-worth utilities (ie, attribute-level utilities), with a higher score for a level indicating greater desirability or preference (utility). This set of utilities was calculated at the individual patient level, thus overcoming the limitation associated with having only aggregate data available after a conjoint analysis. The part-worth utility data were then used to perform conjoint simulations to predict share of preference for tofacitinib and for biologic DMARDs, representative of medications routinely used in clinical practice for the treatment of RA.
Participating Patients’ Characteristics
The attrition flow diagram portrayed in the Figure summarizes the inclusion and exclusion criteria used to determine the final sample from which survey participants were randomly selected. After exclusion for age (outside range), absence of a relevant medical claim, enrollment in a group excluded from research, residence in a nursing home, or a paid claim for a biologic DMARD indicated for the treatment of RA, the sample size was 1528 patients, from which 1400 patients were randomly selected to receive the survey.
Table 1 compares survey respondents and nonrespondents. A total of 380 patients returned the survey, representing a 27.1% response rate. The average age of respondents was 54.9 (standard deviation [SD], 9.3) years. The majority (81.6%) were female, and most resided in either the South (60.3%) or the Midwest (35.3%). Approximately 36% (136 of 380) had received injection or infusion of a non-RA medication in the preceding 180 days, and their average RxRiskV score was 5.2 (SD, 2.9). A comparison of patients who responded to the survey and those who did not respond revealed significant differences between the groups in terms of age and geographic region of residence (both P <.05). No significant differences between these groups were found in sex distribution, RxRiskV comorbidity score, or history of injection or infusion.
The results indicated that respondents were diagnosed with RA an average of 9.2 (SD, 9.2) years before receiving the survey, and their symptoms began an average of 11.2 (SD, 10.2) years before the study (Table 1). Approximately 10% (37 of 380) of respondents indicated that they had undergone joint surgery because of RA.
Choice-Based Conjoint Analysis
Of the 380 patients who returned the survey, 363 (95.5%) completed all or some of the choice-based conjoint section. Table 2 shows the proportion of times that levels within an attribute were selected by survey respondents. Differences in frequency of choice of levels for all attributes were observed, with the exception of improvement in the ability to perform daily tasks and activities. The most frequently selected route of administration was the oral route, and once every 8 weeks was the most preferred frequency of administration.
Respondents most often chose the lowest copayment, the lowest incidence of serious side effects, and the highest likelihood of clinical benefit (efficacy). In addition, respondents were more likely to select the option that did not require adding another medication (typically methotrexate) to their current therapy regimen (P <.01).
A summary of the scores for utility and attribute importance to respondents was generated from the hierarchical Bayes models (Table 3). These models estimate the part-worth for each individual versus aggregate estimates (average utility) generated by the logit method.
Overall, the relative importance of attributes (ie, mean relative importance score ± SD) for respondents, from highest to lowest, was route of administration (34.1 ± 15.5), frequency of administration (16.4 ± 6.8), chance of serious side effects (12.0 ± 9.3), monthly cost-sharing requirement (10.1 ± 6.2), medication burden (9.8 ± 8.2), ability to reduce daily joint pain and joint swelling (8.9 ± 3.8), and improvement in the ability to perform daily tasks and activities (8.8 ± 4.7).
Conjoint Market Simulations
As a final step in the analysis, the part-worth utilities derived from the hierarchical Bayes models were applied to medication concepts based on several medications currently available for the treatment of RA. We simulated preference shares based on medication attributes, as reported in Table 4. The simulation base-case scenario took into account data on drug efficacy and safety derived from the prescribing information19-22 and from a separate meta-analysis of available drugs.23 Improvement in daily living activities was based on the same independent meta-analysis of available drugs.23
Out-of-pocket cost of $65 was assumed for all patients, which is the median amount for the Humana commercial plan member population treated with these medications during the study period (Table 4). This median estimate was based on member out-of-pocket costs from medical and pharmacy claims for biologic DMARDs. Office-visit costs for administering those biologic DMARDs in a clinical setting were not included in the estimate; however, such amounts were minor in relation to the total billed amount (health plan plus member share) for the medication.
When comparing the various drugs, the range of some responses by medication was relatively narrow. For example, the range of reduction in joint pain was 50% to 58%, and for serious adverse events, the range was 4% to 8%. This doubling in the likelihood of serious side effects resulted in this attribute ranking third highest in the utilities and importance summary (Table 4).
A sensitivity analysis was performed with the assumption that measures for specific medication categories were the same throughout the population. This included, for example, the medication’s ability to reduce joint pain (58% for all), improvement in activities of daily living (36% for all), and the likelihood of serious adverse events (8% for all). This approach examined the difference in patient preference share derived solely from the method of medication administration, frequency of administration, and medication burden. Under these conditions, the analysis showed similar results for the base-case scenario and the alternative scenario for tofacitinib (56.4% base analysis vs 54.1% sensitivity analysis) and adalimumab (14.0% vs 17.4%), with greater differences observed between scenarios for etanercept (25.5% vs 18.1%) and infliximab (4.2% vs 10.1%; Table 5).
Our study provides insight into medication attributes that are relevant in determining patient choice among currently available treatments for RA. The results show that the route of administration was the most important medication attribute, with the oral route being the preferred choice. The oral route of administration was not included in the medication attribute choices for biologic DMARDs in the survey by Huynh and colleagues; in that survey, respondents were restricted to choose between self-injection at home and intravenous administration in a clinic.12 Of note, patients who had not yet been treated with biologic DMARDs chose self-injection at home as their preferred route of administration. These patients cited the desire to minimize time associated with transportation and treatment as the most common reason for that choice.12
In our study, cost was not 1 of the 3 most important medication attributes to survey respondents, all of whom were enrolled in a commercial health plan. Cost values in our survey ($25-$75) were fairly consistent with the payments made by patients in Humana’s commercial population at the time of this study (median, $65), according to claims data, which may have influenced the relative ranking of the cost attribute.
By contrast, Augustovski and colleagues, who conducted a discrete choice experiment among patients with RA in Argentina, found cost to be the most important attribute, followed by systemic adverse effects, frequency of administration, efficacy, route of administration, local adverse events, and serious infections.11 In that study, the 3 choices for monthly out-of-pocket costs were ₱0, ₱500, and ₱1500 (₱ = Argentine pesos; for context, approximately two-thirds of that sample reported a monthly income of ₱949-₱4041).11 Although these findings are not necessarily relevant for a commercially insured population in the United States, cost may be a more important factor for people aged 65 years and older potentially living on fixed incomes in the United States, as indicated by a similar study focused on Humana’s Medicare members.24
Although the cost attribute was not as significant as the route or frequency of administration in the present study, it did rank higher than some efficacy measures. Efficacy might have been selected as an important attribute with less frequency, because the range of choices was relatively narrow; that is, the medication choices were similarly effective. Positive responses for ability to reduce joint pain and joint swelling ranged from 50% to 58%, and positive responses for improvement in the ability to perform daily tasks and activities ranged from 32% to 36%.
By contrast, Schaarschmidt and colleagues, who conducted a conjoint analysis of treatment preferences among patients with moderate to severe psoriasis, found that the efficacy measure “chance of experiencing significant reduction in my psoriasis” ranked second highest, after “treatment location” (ie, where treatment is administered), and followed by “method of delivery.”25 In their study, the efficacy attribute ranged from 40% to 100%,25 consistent with the hypothesis that a wider range of attribute levels may result in patients choosing the preferred level of an attribute with greater frequency.
The efficacy measures for our study were carefully chosen based primarily on data from randomized controlled trials; the efficacy measures for biologic DMARDs in the treatment of RA have been reported to be similar.26-28 It is not surprising that medication attributes other than efficacy and safety increase in importance for patients with chronic conditions that require long-term continual treatment, especially if the drugs in question have similar efficacy and safety features.
Taking patient preferences into consideration may result in reordering the ranking of known factors deemed important by physicians in determining the right treatment for each patient. A few qualitative studies indicate that patient approaches to care differ from those of physicians, and that patients are increasingly involved in the decision-making regarding their care.5,29 For example, in a study comprised of interviews with senior rheumatologists in Sweden, several physicians noted that their patients were well-educated and demanded that their preferences and requests for specific biologic DMARDs be taken into consideration.5
Patient involvement in medical decision-making has also been associated with greater satisfaction and, in turn, greater adherence to treatment. This is supported in research by Kjeken and colleagues, who reported that among patients with RA, satisfaction was greater for those with high (vs low) involvement in medical decisions (91% vs 61%, respectively; P <.001).7 Moreover, from a literature review of preferences of patients with RA, Barton concluded that patient preferences for mode and frequency of treatment administration were important factors that affected their medication adherence.8 Soliciting these preferences from patients may encourage them to become more adherent sooner rather than later.30
Limitations common to studies involving survey methodology apply to this study as well, including the potential bias because of nonresponse. The comparison of respondents and nonrespondents indicated significant differences in 2 demographic characteristics—age and geographical region; however, no significant differences were found in clinical characteristics between these groups.
We could not determine whether significant differences existed between respondents and nonrespondents for several variables (ie, years since onset of symptoms, years since diagnosis, and history of joint surgery associated with RA), which potentially increased the nonresponse bias. These variables were not captured sufficiently, because administrative claims data do not encompass the full history of a patient’s lifetime.
It is possible that respondents differed from nonrespondents in other (unmeasured) characteristics, such as the ability to understand the survey questions and the tasks involved in completing the survey. Moreover, the length of the survey might have influenced the responses because of possible fatigue or loss of concentration, which might have placed some individuals at risk of failing to make rational choices. However, where there was a clear order among levels in an attribute, such as cost, the results indicated that individuals were attentive when completing the survey.
This survey was intended to be distributed to patients who had been diagnosed with RA and had never used a biologic DMARD to treat their RA. Because we used health insurance claims data for this study, missing data might have contributed to misclassification bias with respect to diagnosis and medication history.
In addition, the results of this study are a function of the levels of the attributes included in the survey instrument; as such, the results cannot be extrapolated to other attributes or levels. Even among the survey respondents, average utilities of the levels of an attribute are interpretable within a particular attribute and are not to be interpreted between levels of different attributes.
Based on survey responses in this study, the route of administration of a medication is an important consideration for patients diagnosed with RA who have not received any biologic DMARD therapy. Furthermore, based on these results, the majority of patients with RA would prefer the use of a medication with oral route of administration. Given the variety of RA therapies available in the marketplace, gaining better understanding of the attributes that commercially insured patients prefer most about their medications may help to inform prescriber and payer decisions for selecting therapies that will lead to greater patient satisfaction and improved medication adherence.
Editorial support was provided by Mary Costantino, PhD, of Comprehensive Health Insights, a subsidiary of Humana Inc.
This study was funded by Pfizer Inc through a collaboration agreement with Humana Inc.
Author Disclosure Statement
Dr Louder is a stockholder of Humana, Pfizer, and WellPoint. Dr Singh is a stockholder of Pfizer. Dr Saverno is a consultant to Pfizer. Dr Cappelleri is a stockholder of Pfizer. Dr Aten is on the Specialty Strategies Team of Humana. Dr Koenig has stock options with Pfizer. Dr Pasquale is a consultant to Pfizer.
1. Smolen JS, Landewé R, Breedveld FC, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2013 update. Ann Rheum Dis. 2014;73:492-509.
2. Anderson J, Caplan L, Yazdany J, et al. Rheumatoid arthritis disease activity measures: American College of Rheumatology recommendations for use in clinical practice. Arthritis Care Res (Hoboken). 2012;64:640-647.
3. Politi MC, Studts JL, Hayslip JW. Shared decision making in oncology practice: what do oncologists need to know? Oncologist. 2012;17:91-100.
4. Global Initiative for Chronic Obstructive Lung Disease. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease (updated 2016). 2016. www.goldcopd.org/uploads/users/files/GOLD_Report%202016.pdf. Accessed August 8, 2014.
5. Kalkan A, Roback K, Hallert E, Carlsson P. Factors influencing rheumatologists’ prescription of biological treatment in rheumatoid arthritis: an interview study. Implement Sci. 2014;9:153.
6. Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012;307:1636-1640.
7. Kjeken I, Dagfinrud H, Mowinckel P, et al. Rheumatology care: involvement in medical decisions, received information, satisfaction with care, and unmet health care needs in patients with rheumatoid arthritis and ankylosing spondylitis. Arthritis Rheum. 2006;55:394-401.
8. Barton JL. Patient preferences and satisfaction in the treatment of rheumatoid arthritis with biologic therapy. Patient Prefer Adherence. 2009;3:335-344.
9. Smolen JS, van der Heijde D, Machold KP, et al. Proposal for a new nomenclature of disease-modifying antirheumatic drugs. Ann Rheum Dis. 2014;73:3-5.
10. Fraenkel L, Bogardus ST, Concato J, et al. Patient preferences for treatment of rheumatoid arthritis. Ann Rheum Dis. 2004;63:1372-1378.
11. Augustovski F, Beratarrechea A, Irazola V, et al. Patient preferences for biologic agents in rheumatoid arthritis: a discrete-choice experiment. Value Health. 2013;16:385-393.
12. Huynh TK, Østergaard A, Egsmose C, Madsen OR. Preferences of patients and health professionals for route and frequency of administration of biologic agents in the treatment of rheumatoid arthritis. Patient Prefer Adherence. 2014;8: 93-99.
13. Bridges JFP, Hauber AB, Marshall D, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14:403-413.
14. Orme BK. Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research. 2nd ed. Madison, WI: Research Publishers, LLC; 2010.
15. Sawtooth Software. “What is conjoint analysis?” Conjoint/choice analysis. http://sawtoothsoftware.com/component/content/article/189-products/solutions/571-conjoint-choice-analysis. Accessed March 1, 2016.
16. Fishman PA, Goodman MJ, Hornbrook MC, et al. Risk adjustment using automated ambulatory pharmacy data: the RxRisk model. Med Care. 2003;41:84-99.
17. Sloan KL, Sales AE, Liu C-F, et al. Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument. Med Care. 2003;41:761-774.
18. Sales AE, Liu C-F, Sloan KL, et al. Predicting costs of care using a pharmacy-based measure risk adjustment in a veteran population. Med Care. 2003;41:753-760.
19. Xeljanz (tofacitinib) tablets [prescribing information]. New York, NY: Pfizer; June 2015.
20. Humira (adalimumab) injection [prescribing information]. North Chicago, IL: AbbVie; November 2015.
21. Enbrel (etanercept) [prescribing information]. Thousand Oaks, CA: Immunex Corp; March 2015.
22. Remicade (infliximab) [prescribing information]. Horsham, PA: Janssen Biotech; October 2015.
23. Vieira MC, Wallenstein GV, Bradley JD, et al. Tofacitinib versus biologic treatments with and without methotrexate in patients with active rheumatoid arthritis who have had an inadequate response to traditional disease modifying anti-rheumatic drugs—a network meta-analysis. Ann Rheum Dis. 2012;71(suppl 3). Abstract FRI0185.
24. Louder A, Singh A, Cappelleri J, et al. Patient preferences associated with therapies for rheumatoid arthritis among Humana Medicare members: a conjoint analysis. J Manag Care Pharm. 2015;21(4 suppl A). Abstract M14.
25. Schaarschmidt ML, Umar N, Schmieder A, et al. Patient preferences for psoriasis treatments: impact of treatment experience. J Eur Acad Dermatol Venereol. 2013;27:187-198.
26. Hochberg MC, Tracy JK, Hawkins-Holt M, Flores RH. Comparison of the efficacy of the tumour necrosis factor α blocking agents adalimumab, etanercept, and infliximab when added to methotrexate in patients with active rheumatoid arthritis. Ann Rheum Dis. 2003;62(suppl 2):ii13-ii16.
27. Chen Y-F, Jobanputra P, Barton P, et al. A systematic review of the effectiveness of adalimumab, etanercept and infliximab for the treatment of rheumatoid arthritis in adults and an economic evaluation of their cost-effectiveness. Health Technol Assess. 2006;10:iii-iv, xi-xiii, 1-229.
28. Klareskog L, van der Heijde D, de Jager JP, et al; for the TEMPO (Trial of Etanercept and Methotrexate with Radiographic Patient Outcomes) Study Investigators. Therapeutic effect of the combination of etanercept and methotrexate compared with each treatment alone in patients with rheumatoid arthritis: double-blind randomised controlled trial. Lancet. 2004;363:675-681.
29. Fraenkel L, Seng EK, Cunningham M, Mattocks K. Understanding how patients (vs physicians) approach the decision to escalate treatment: a proposed conceptual model. Rheumatology (Oxford). 2015;54:278-285.
30. Mjaavatten MD, Radner H, Yoshida K, et al. Inconsistent treatment with disease-modifying antirheumatic drugs: a longitudinal data analysis. J Rheumatol. 2014;41:2370-2378.