ABSTRACT

Conclusion:

Making the prediction at an early stage is crucial for the patients as well as the healthcare system. However, it is equally important to determine how to proceed with the patients who are unlikely to respond to bDMARDs. Current literature does not adequately answer this question. Additional treatment options and multiple evaluation criteria for these options should be considered; multiple criteria models can provide useful decision support for this purpose.

Results:

Predictors of the logistic regression model were age, sex, coronary artery disease, spine surgery, steroid treatment, sulfasalazine treatment and baseline health assesment questionnaire score. The model displayed 79.5% accuracy and an area under receiver operating characteristic curve of 0.82. 87% of the patients who were goodresponders in six-month follow-up were also good responders in oneyear follow-up. Among non-responders in six-month follow-up, 75% were also non-responders in one-year follow-up.

Methods:

Several ML models were tested in 190 RA patients from Turkey, and the logistic regression model was found to be superior. The relation between long-term and short-term responses were also analyzed.

Objective:

Biologic disease-modifying antirheumatic drugs (bDMARDs) offer promising results for rheumatoid arthritis (RA) patients in general, but a substantial percentage of patients do not respond to them. It is important to predict the response before the treatment so that unnecessary adversities for the patients and costs for the healthcare system can be avoided. This study aims to develop a machine learning (ML) model that works with readily-available demographic and clinical factors for prediction of response to bDMARDs, and discusses additional non-pharmacological practices.

Introduction

With the advances in technology and informatics, artificial intelligence (AI) methods have become increasingly useful in medical decision making. One of their uses is to predict the risk of patients to develop some diseases and the success rates of treatments.^[1,2] Another use is to predict the complications that can arise after the onset of diseases.^[3] AI is also used to determine personalized treatments for patients that have been diagnosed with a disease, such as cancer.^[4] It is also possible to use AI methods in clinical research and drug development. A review of AI methods in healthcare can be found in.^[5]

Rheumatoid arthritis (RA) is an autoimmune and inflammatory disease that causes pain, disability, and social and economic disadvantages for approximately one percent of the world population. RA is a heterogeneous disease; the clinical symptoms, progress of the disease and response rate to treatment differ substantially among patients.^[6] Therefore, clinicians and patients of this disease can benefit from AI methods that will support their decisions. Many AI methods used for RA aim to diagnose patients that exhibit certain complaints and predict patients with high risks of developing RA (see^[7] for fuzzy logic;^[8] for rule-based;^[9] for clustering;^[10] for decision tree and feature selection applications; and^[11] for a review of computational methods).

This paper focuses on the prediction of response to treatment in RA patients. In the AI domain, machine learning (ML) models that can work with several factors related to the patients and their conditions come forward as suitable and useful tools for this prediction. In RA, firstly the factors that can determine the response to treatment should be discovered.^[12] Several trials with methotrexate (MTX), cyclosporine plus MTX and combination treatment were made in.^[13] They found that disease duration, prior use of disease-modifying antirheumatic drugs (DMARDs), higher disease functional class, low disease activity and female sex had a negative effect on the likelihood of patient response. A logistic regression model was used to predict response to MTX, steroids, and combination of DMARD treatments in.^[14] They tested their model in a UK randomized controlled trial, and the significant variables to predict remission measured as the disease activity scores in 28 joints (DAS28) <2.6 were discovered as age, sex and tender joint count. The results had high specificity (98%) but low sensitivity (13%). The evidence on predictors of response to MTX and other synthetic DMARDs was reviewed in.^[15] Even though they found high discrepancy between the results of different studies, they stated that the factors that are more likely to lead to lower response are female sex, smoking, established disease, previous DMARD use, high disease activity and the absence of concomitant corticosteroids. Biomarker search for the prediction of response to treatment in RA was summarized.^[16] Regularized regression, random forest and a pathway-supported approach were used to study the association between early treatment lipidomic measurements and response to MTX, but the results did not support an evident association.^[17]

Biologic therapies offer more promising results in RA treatment. It was discussed that biologic therapies offer increased efficacy, but their use is limited by cost concerns.^[18] Therefore, predictive models for identifying patients that are most likely to benefit from them is important. But it is also stated that many predictive models are hindered by their complexity and the need for biomarkers that are not routinely measured. For example,^[19] reported that circulating cell-free DNA can predict the early therapeutic effects of biologic DMARDs (bDMARDs) in RA patients. However, more work is needed to integrate this prediction to clinical practice. In addition,^[20] reported that different markers were effective in the prediction in different studies. They observed that currently no biomarkers can predict response to bDMARDs with high certainty.

Other studies used demographic and clinical factors to build prediction models for bDMARDs. Multivariate ordinal logistic regression was applied to identify clinical factors that can predict response to anti-tumor necrosis factor (TNF)-therapy; and DAS28 scores were used to assess patients and found lower response rate among smokers and females.^[21] The response to anti-TNF agents was studied using DAS28 scores and it was found that poorer response was associated with female sex, the number of DMARDs previously used, baseline erythrocyte sedimentation rate, tender joint count, and long RA history.^[22] Different ML models such as lasso, ridge, support vector machine (SVM), random forest, and XGBoost were used to predict response.^[23] It was discovered different predictors for different bDMARDS; accuracy rates were between 52.8-72.9%.

Even though bDMARDs are generally accepted to provide better results for the patients, a significant number of patients do not respond to them.^[6] As discussed, clinical trials often show that bDMARDs are not effective for approximately 30-40% of patients; and the response rate decreases with subsequent biologic drugs.^[23] This situation has drawbacks for the patients and the healthcare system. First, ineffective treatments cause pain and unnecessary side effects for the patients. In addition, as some of the reviewed studies discussed, disease duration can have a negative effect on the success of the treatment. Therefore, an unsuccessful treatment period can lower the chance of remission for the patients in the future as well. As another drawback, bDMARDs cause a significant cost burden on the healthcare system. Considering these issues, it is evident that early intervention and detection of non-responders are crucial in the treatment of RA with biologics.

This paper proposes to use an ML model to predict the response to bDMARDs for RA patients who have been registered to Hacettepe University Rheumatology Biologic (HUR-BIO) Registry system in Turkey. The aims of the paper are to i) assess the performances of different ML models, ii) develop a model that uses demographic and clinical factors that are readily available in clinical practice, iii) investigate the relation between long-term and short-term responses, and iv) discuss additional non-pharmacological practices such as physiotherapy and rehabilitation, psychotherapy, dieting, daily exercise, and pain education for the patients who are not likely to benefit from biologics.

Materials and Methods

First the information on the study population is provided and then, the ML models used for predicting the response to treatment are explained.

Data Collection

In this study, HUR-BIO Registry where patients on bDMARD treatment have been recorded was used. RA diagnosis was based on American College of Rheumatology/The European League Against Rheumatism 2010 Classification Criteria. The response was measured based on the difference in health assesment questionnaire (HAQ) scores of the patients at the baseline (at the beginning of the bDMARD treatment) and at six-month follow-up. HAQ is reported as a good representation of disease activity^[24] and assessment of function in RA patients.^[25] Although DAS28 and HAQ scores are both commonly used in RA studies, the latter was selected to measure the response as HAQ was considered to reflect the patients’ self-evaluations better. This self-evaluation is expected to guide additional non-pharmacological treatment selections.

Among 1101 RA patients registered in the HUR-BIO database, the ones who started bDMARD treatment in 2013 and later and having a HAQ score of at least 0.5 were selected and included in the analysis. 2013 was selected as the starting year since consistent data on bDMARDs were available after this date in the database. In order to assess the improvement due to treatment, a threshold in the starting HAQ score was necessary to avoid misinterpreting patients having good initial scores. Therefore, a threshold of 0.5 was selected for the starting HAQ score. After excluding patients with missing data and those whose follow-up durations deviated from six months, 190 patients remained for the analysis. All of these patients were at least 22 years old, so no more exclusion due to age was needed.

ML Models

While measuring the response to treatment, the patients having a change in their HAQ score of at least 0.22, which was validated in a cohort of 1.645 RA patients, were labelled as “good responders”^[26] whereas the others were labelled as “non-responders”.

To predict the response, four commonly used ML models that are suitable for the available data are selected: namely Kernel naïve Bayes (NB), fine decision tree (DT), logistic regression (LR), and linear SVM. Accuracy is estimated following a 10-fold cross validation approach. To test the performances of the ML models, Classification Learner App in MATLAB was utilized; thus, the guidelines of MATLAB were used for the parameterization of the algorithms.

Results

Cohort Characteristics

The ages of the selected patients were between 22-79 and 86.3% were female. Of the 190 selected patients, 137 were identified as good responders. The baseline characteristics are summarized in Table 1.

Table 1

Performances of the ML Models

The accuracy levels of the algorithms were found as 72.6%, 70.5%, 75.3%, and 74.2% for NB, DT, LR and SVM, respectively. Since LR is the best performer among the four ML models and is widely used in the literature, it was selected to be used in the further analysis. Hence, an LR model was constructed to identify predictors (important features) of being good responder/non-responder. The assumptions of LR such as independence of observations, linearity between independent variables and log odds, the existence of no multicollinearity and extreme outliers are checked. A p-level of 0.10 was used as in^[22] and age, sex, coronary artery disease, spine surgery, steroid treatment, sulfasalazine treatment, baseline HAQ score were selected as predictors. Although there is not a commonly used set of predictors in the literature, age and sex were prominent in almost all studies. The final model is as follows: 1.451 - 0.063*age + 1.933*sex - 0.939*coronary artery disease - 1.268*spine surgery - 0.627*sulfasalazine treatment + 1.086*steroid treatment + 2.151*HAQ. According to the model, a higher likelihood of being a non-responder is associated with higher age, female sex, having coronary artery disease, having spine surgery, previous exposure to sulfasalazine treatment, and having a lower baseline HAQ score. The model resulted in 79.5% accuracy, 90.5% sensitivity, and 50.9% specificity. The receiver operating characteristic (ROC) curve and the corresponding area under ROC curve value are provided in Figure 1.

Figure 1

Prediction of the Long-term Responses Based on Short-term Responses

In prediction of response levels (good responder/non-responder) of the patients to bDMARDs, those whose follow-up durations were about six months were analysed. To check the effectiveness of the model for long-term prediction, patients who had records for one-year follow-up were identified. One hundred thirty one out of 190 patients satisfied the requirement and were included in further analysis. After checking the responses of these 131 patients with respect to their baseline HAQ scores, it was observed that 87% of the patients who were classified as good responders for six-month follow-up were also classified as good responders at the end of one-year follow-up. Moreover, 75% of the patients who were classified as non-responders for six-month follow-up were also classified as non-responders at the end of one-year follow-up. This implies that the response prediction based on six-month follow-up is a good representative for that of one-year follow-up.

Discussion

In this study, HUR-BIO database which is the oldest and one of the most comprehensive registry systems in Turkey, was used to predict the responses of the RA patients to bDMARD treatment. In the first part of the study, four different ML models were analysed using a sample of 190 patients with a total of 31 demographic and clinical features. Then, further analysis was conducted with LR to identify important features for predicting non-responders.

Predictors

Age, sex, coronary artery disease, spine surgery, steroid treatment, sulfasalazine treatment, and baseline HAQ score were selected as predictors. As stated in the introduction, higher age and female sex are commonly associated with poor outcomes in RA. Besides, baseline HAQ score was expected to be a predictor of future scores. On the other hand, the others -having coronary artery disease, having spine surgery, and previous exposure to sulfasalazine treatment- were not obvious predictors at the start of the study.

Study Limitations

There were several limitations in this study, some of which are common to all studies in this area. It is not possible to generalize the findings to all RA patient groups as different cohorts can result in different predictors and models. Only clinical and demographic characteristics were included in the examined data set. In this study, the aim of using HAQ score as a response criterion was to consider “functional status” as a better surrogate factor of overall health status of the patient. As another limitation, since the learning performance of ML models increases with sample size, better results could have been obtained with a larger sample. In addition, drug-specific predictions could not be made with the available sample.

Study Implications

There are several studies that predicted the response to bDMARD treatment for RA patients, but a consensus on important predictors and models has not been reached. It is clear that more studies are needed to determine common predictors and best models of prediction. They will help to distinguish patients who are most likely to benefit from bDMARDs from patients who need more careful evaluation. This is critical not only for the well-being of patients but also for the monetary position of the healthcare system, especially for developing countries such as Turkey where bDMARDs are very costly. For the same reasons, making the prediction at an early stage is beneficial. This study predicted the short-term response, and showed that it is a good representative of the long-term response.

Another critical issue is developing treatment plans for patients who will not achieve good outcomes from bDMARDs. The current literature does not sufficiently address this issue; the studies end with the prediction and do not answer the question of what to do next. This question is not straightforward to answer; there can be several options to improve the condition of non-responders and multiple factors to consider when making a decision. Due to the nature of RA, which is a chronic disease, patients can benefit from non-pharmacological practices such as physiotherapy and rehabilitation, psychotherapy, dieting, daily exercise and pain education.^[27] There can also be other pharmacological options, e.g. injection and surgery. The best option for each patient can be different, and the decision should be made considering different factors like side effects, cost, expected improvement in pain and function, psychosocial improvement and difficulty in implementation. Since there are multiple options and factors to consider, this decision presents itself as a multiple criteria decision making (MCDM) problem.

Although MCDM methods have been used for medical decision making problems (see^[28] for a review), their application in treatment selection is limited. In RA context, they have high potential in terms of eliciting the preferences of the patients, reflecting the expertise of the clinicians and providing decision support in an interactive setting. Therefore, the collaboration between prediction and MCDM methods can provide more useful outcomes. In future studies, the development of a decision support tool and its validation with both clinicians and patients will impact the literature and clinical practice.

Conclusion

The prediction of response to bDMARD treatment in RA patients at an early stage is critical for two main reasons: The patients should be spared from ineffective treatments that will worsen their condition in the long-term, and the healthcare budget should be allocated in an efficient way. Therefore, prediction models should be studied more to discover powerful and easy-to-use predictors. In addition, decision support tools should be developed to make best use of the predictions. These tools can assist the clinicians and RA patients in the formation of patient-specific treatment plans. This study contributed to the literature by proposing a practical response prediction model with the available data, and discussing the potential of a decision support tool to select non-pharmacological treatments considering multiple criteria.

Ethics

Ethics Committee Approval: Ethical approvals for this study were obtained from Hacettepe University Non-Interventional Clinical Researches Ethics Board (2019/28-36).

Informed Consent: Written consent was obtained from each patient participating in the study.

Peer-review: Externally peer-reviewed.

Authorship Contributions

Surgical and Medical Practices: E.B., L.K., U.K., Concept: C.T.S., G.K., E.B., L.K., U.K., Design: C.T.S., G.K., E.B., L.K., U.K., Data Collection or Processing: C.T.S., G.K., E.B., L.K., U.K., Analysis or Interpretation: C.T.S., G.K., E.B., L.K., U.K., Literature Search: C.T.S., G.K., L.K., U.K., Writing: C.T.S., G.K., E.B., L.K.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declare that they have no relevant financial disclosures.

References

1Li H, Luo M, Zheng J, et al. An artificial neural network prediction model of congenital heart disease based on risk factors: a hospital-based case-control study. Medicine (Baltimore) 2017;96:e6090.

2Weng SF, Reps J, Kai J, Garibaldi JM, Quereshi N. Can machine-learning improve cardiovascular risk prediction using routine clinical data. PLoS One 2017;12:e0174944.

3Dagliati A, Marini S, Sacchi L, et al. Machine learning methods to predict diabetes complications. J Diabetes Sci Technol 2018;12:295-302.

4Lee H, Troschel FM, Tajmir S, et al. Pixel-level deep segmentation: artificial intelligence quantifies muscle on computed tomography for body morphometric analysis. J Digit Imaging 2017;30:487-98.

5Becker A. Artificial intelligence in medicine: What is it doing for us today? Health Policy and Technology 2019;8:198-205.

6Kłak A, Paradowska-Gorycka A, Kwiatkowska B, Raciborski F. Personalized medicine in rheumatology. Reumatologia 2016;54:177-86.

7Singh S, Kumar A, Panneerselvam K, Vennila JJ. Diagnosis of arthritis through fuzzy inference system. J Med Syst 2012;36:1459-68.

8Alshawwa IA, Elkahlout M, El-Mashharawi HQ, Abu-Naser SS. An Expert System for Depression Diagnosis. IJAHMR 2019;3:20-7.

9Yoo J, Lim MK, Ihm C, Choi ES, Kang MS. A study on prediction of rheumatoid arthritis using machine learning. International Journal of Applied Engineering Research 2017;12:9858-62.

10Shanmugam S, Preethi J. Improved feature selection and classification for rheumatoid arthritis disease using weighted decision tree approach (REACT). The Journal of Supercomputing 2019;75:5507-19.

11Alder H, Michel BA, Marx C, et al. Computer-based diagnostic expert systems in rheumatology: Where do we stand in 2014? Int J Rheumatol 2014;672714.

12Huizinga TWJ. Personalized medicine in rheumatoid arthritis: is the glass half full or half empty? J Intern Med 2015;277:178-87.

13Anderson JJ, Wells G, Verhoeven AC, Felson DT. Factors predicting response to treatment in rheumatoid arthritis: the importance of disease duration. Arthritis Rheum 2000;43:22-9.

14Ma MHY, Ibrahim F, Walker D, et al. Remission in early rheumatoid arthritis: predicting treatment response. J Rheumatol 2012;39:470-5.

15Romão VC, Canhão H, Fonseca JE. Old drugs, old problems: Where do we stand in prediction of rheumatoid arthritis responsiveness to methotrexate and other synthetic DMARDs? BMC Med 2013;11:17.

16Takeuchi T. Biomarkers as a treatment guide in rheumatoid arthritis. Clinical Immunology 2018; 186:59-62.

17Maciejewski M, Sands C, Nair N, et al. Prediction of response of methotrexate in patients with rheumatoid arthritis using serum lipidomics. Scientific Reports 2021;11:7266.

18Conaghan PG. Predicting outcomes in rheumatoid arthritis. Clin Rheumatol 2011;30:41-7.

19Hashimoto T, Yoshida K, Hashimoto N, et al. Circulating cell free DNA: a marker to predict the therapeutic response for biological DMARDs in rheumatoid arthritis. Int J Rheum Dis 2017;20:722-30.

20Wijbrandts CA, Tak PP. Prediction of Response to Targeted Treatment in Rheumatoid Arthritis. Mayo Clin Proc 2017;92:1129-43.

21Hyrich KL, Watson KD, Silman AJ, Symmons DPM, British Society for Rheumatology Biologics Register. Predictors of response to anti-TNF-alpha therapy among patients with rheumatoid arthritis: results from the British Society for Rheumatology Biologics Register. Rheumatology (Oxford) 2006;45:1558-65.

22Atzeni F, Antivalle M, Pallavicini FB, et al. Predicting response to anti-TNF treatment in rheumatoid arthritis patients. Autoimmun Rev 2009;8:431-7.

23Koo BS, Eun S, Shin K, et al. Machine learning model for identifying important clinical features for predicting remission in patients with rheumatoid arthritis treated with biologics. Arthritis Res Ther 2021;23:178.

24Verstappen SMM, Jacobs JWG, Huisman A-M, van Rijthoven AWAM, Sokka T. Functional Health Assessment Questionnaire (HAQ) and Psychological HAQ Are Associated with and Predicted by Different Factors in Rheumatoid Arthritis. J Rheumatol 2007;34:1837-40.

25Maska L, Anderson J, Michaud K. Measures of functional status and quality of life in rheumatoid arthritis: Health Assessment Questionnaire Disability Index (HAQ), Modified Health Assessment Questionnaire (MHAQ), Multidimensional Health Assessment Questionnaire (MDHAQ), Health Assessment Questionnaire II (HAQ-II), Improved Health Assessment Questionnaire (Improved HAQ), and Rheumatoid Arthritis Quality of Life (RAQoL). Arthritis Care Res (Hoboken) 2011;63 (Suppl 11):S4-13.

26Behrens F, Koehm M, Schwaneck EC, et al. Use of a “critical difference” statistical criterion improves the predictive utility of the Health Assessment Questionnaire-Disability Index score in patients with rheumatoid arthritis. BMC Rheumatol 2019;3:51.

27Roodenrijs NMT, Hamar A, Kedves M, et al. Pharmacological and non-pharmacological therapeutic strategies in difficult-to-treat rheumatoid arthritis: a systematic literature review informing the EULAR recommendations for the management of difficult-to-treat rheumatoid arthritis. RMD Open 2021;7:e001512.

28Adunin G, Diaby V, Xiao H. Application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis. Health Expect 2015;18:1894-905.

Predicting the response to bDMARD treatment in RA: Then what?