Interpreting regression models in clinical outcome studies (2024)

Measuring the outcome of an intervention is central tothe practice of evidence based medicine, and most research papersevaluating patient outcomes now incorporate some form of patient-basedmetric, such as questionnaires or performance tests. Once an outcomehas been defined, researchers typically want to know if any otherfactors can influence the result. This is typically assessed with regressionanalysis.

Regression analysis¹ determinesthe relationship of an independent variable (such as bone mineraldensity) on a dependent variable (such as ageing) with the statistical assumptionthat all other variables remain fixed. The calculation of the relationship resultsin a theoretical straight line, and the correlation co-efficient(r) measures how closely the observed data are to the theoreticalstraight line that we have calculated.

In such a linear model, we can judge how well the line fits thedata (‘goodness of fit’) by calculating the coefficient of determination (orsquare of the regression line, R²). R² isa measure of the percentage of total variation in the dependantvariable that is accounted for by the independent variable. An R² of1.0 indicates that the data perfectly fit the linear model. AnyR² value less than 1.0 indicates that at least some variabilityin the data cannot be accounted for by the model (e.g., an R² of0.5 indicates that 50% of the variability in the outcome data cannotbe explained by the model).

Given these statistical tools, we can use the regression equationto predict the value of the dependent variable based on the known valueof independent variable. Since many variables may contribute tothe outcome (dependent variable), further statistical analysis canbe achieved with multiple regression analysis. These models areessentially the same as simple regression analysis, except thatthe multiple regression analysis equation describes the interrelationshipof many variables and allows us to evaluate the joint effect ofthese variables on the outcome variable in question.

Poitras et al² reportan interesting study this month that aims to predict length of stay andearly clinical function following joint arthroplasty. Multiple linearregression analyses produced an equation based on the timed-up-and-gotest, which was associated with length of stay. In addition, modelsbased on the pre-operative WOMAC function sub-score produced thebest model for describing early post-operative function (as calculated bythe Older American Resources and Services ALD score). As such theauthors were able to conclude that the outcomes assessments (timed-up-and-goand WOMAC) were predictive of outcome, and further modelling identifiedthresholds of the outcome assessment scores that related to betterand worse outcomes.

How should we interpret these findings? The authors quite correctlysuggest that models such as these could be of value in dischargeplanning and resource utilisation by targeting the patients thatmost need intervention and rehabilitation. The reported R² forthe models, however, was 0.18. Bearing in mind that R²,the coefficient of determination, measures the percentage of thevariation in the dependent variable that is explained by variationin the independent variable,³ takingthe compliment (100 – R²) we see that 82% of the variationin the outcome parameter assessed is unexplained by the model. Theprincipal problem is that the variance in the population studiedcan strongly influence R² magnitude. Therefore, thereis no guarantee that a high coefficient of determination is indicativeof ‘goodness of fit’. Similarly there is no guarantee that a smallR² indicates a weak relationship, given that the statisticis largely influenced by variation in the independent variable.⁴

Therefore, there is no rule for interpreting the strength ofR² in its application to clinical relevance. Useful high valuesof R² can be obtained with clinical data sets,⁵ however, a low R² canstill provide a useful clinical model with respect to data trends,but may be low in precision. In this study there is an associationbetween the performance tests and length of stay; and, using theequations, we can indeed predict one from the other. The accuracyof this prediction though, needs to be borne in mind when usingit as a clinical tool.

Furthermore, it is not rational to compare R² acrossdifferent samples, which given clinical populations, are likelyto differ significantly in the variance of the independent and dependentvariables.⁶

In controlled environments, such as biomechanical tests on cadavericbones, the variance across predictive measurements is likely tobe low, and therefore R² values can be expected to liein the 0.8 range.⁷ Inclinical studies, however, R² values vary widely dependingon the nature of the analysis. For example, when comparing radiographicparameters or associating surgical technical factors, values ofR² are reported in the 0.2 to 0.4 range.^8,9 Whereas, comparing data between separate(but intrinsically similar) outcome assessment questionnaires can yieldhigher values in excess of 0.7.¹⁰

As such, further validation of the Poitras study² using new datasetsand, ideally, confirmatory analysis of the findings using a muchlarger sample size, would be required before their regression modelcould be recommended for use clinically. This does not devalue the appropriateness– or indeed ‘worthiness’ – of reporting these findings in the literature,as the important clinical tools typically start as ideas in smalldatasets. As with all research papers, the reader requires a basicunderstanding of methodology to evaluate how relevant the results areto wider practice.

References