Introduction

Prognostic studies analyze the potential consequences of suffering from a disease and can be classified according to three general objectives: exploratory, explanatory, or predictive¹. Exploratory studies aim to establish the probability of occurrence of relevant outcome(s) in the studied patients. For explanatory studies, the intention is to validate the independent effect of a particular factor of interest on the outcome(s), adjusted for known confounding factors. Studies with the third objective aim to construct a prognostic prediction scale, as precise as possible, based on patient data¹-⁴. Each purpose requires a distinct methodology to obtain valid and useful data for reliable statistical analysis¹. During the reading, review, or execution of a prognostic study, it is common to find readers with doubts about the recommended statistical procedures according to the objectives mentioned above. This review analyzes the recommended statistical strategies for data analysis for the different prognostic objectives.

Prognostic studies with exploratory purpose

In this type of study, the analysis can be conducted in a descriptive or comparative manner (Fig. 1). Descriptive analysis aims to report the frequency and proportion (or percentage) of patients who developed the outcome under study (e.g., mortality or a sequela). For this information, the patient follow-up method must be considered. If all patients had the same follow-up time (e.g., 24 h), it is only necessary to present the cumulative incidence of the outcome(s). For example, in a study on the prognosis of intubation in patients with severe asthma attacks, one can report that 20% of individuals admitted to the emergency room ended up receiving ventilation assistance within 24 h after admission. In this analysis, it is feasible to report on several outcomes (e.g., fatality or ventilator-associated pneumonia, among others). If, in addition, one wishes to consider the rate at which the outcome occurs, it can be reported as an incidence rate (events per person-time). For the first option, actuarial tables are used; for the second, survival tables and Kaplan–Meier curves are adequate (Table 1 and Fig. 2)⁵.

Figure 1 Diagram of recommended statistical methods according to the objective of the prognostic study. 

Figure 2 Actuarial curves with fixed analysis times (equal intervals) versus the Kaplan–Meier survival curve where it decreases in the presence of at least one event in the real follow-up time, the vertical mark indicates the censoring of at least one patient (loss to follow-up without presenting the event). The presented data are fictitious and obtained from table 2. 

If one also wishes to establish whether any factor present at the beginning of the clinical course follow-up (initial cohort) could explain the different outcome(s), the first approach is to observe the proportion of subjects with this factor among those who did or did not present the studied outcome(s). In the case that the follow-up time is the same for all patients, it is sufficient to compare their cumulative incidence rates using a test of difference in proportions (for example, the Chi-square test) or the 95% confidence intervals (CI) of the differences in proportions (if the interval includes the value "0", it is not statistically conclusive)⁶.

Another approach involves comparing the velocities of outcomes between patients with and without the factor. The test of choice is the "log-rank test"⁵,⁷. For example, the median intubation-free survival was 12 h for patients with atopy, compared to 20 h for those without atopy (mean difference of −8 h, 95% CI from −12 to −6 h, log-rank test p = 0.001, data calculated as an example).

A severe problem in bivariate comparisons (groups with and without the prognostic factors to be evaluated) is that, in some cases, statistically significant differences can be found due to multiple possible comparisons. This is due to the increased risk of committing a type I error (bias due to "multiple comparisons")⁸. These models assume the possibility of knowing how much a factor influences the outcome, considering the partial effect of others (adjustment), that is, how much the factor influences independently of another or others. The correct way to jointly analyze several factors to establish which one(s) are associated with the outcome and review which one(s) are more influential is through multivariable regression models⁹,¹⁰. The choice of model will depend on how the outcome variable was measured and the form of follow-up (at fixed or continuous times), as well as verifying compliance with a series of statistical assumptions necessary to establish its validity (Table 2)⁹,¹⁰. For outcomes with fixed times, the most used regression models are binary logistic (dichotomous outcome: presence or absence of the outcome), multiple linear (quantitative outcome: days of hospitalization), or ordinal (hierarchical qualitative outcome: mild, moderate, and severe damage)¹⁰.

The interpretation is based on the beta coefficients of each model. In logistic and ordinal regression, the exponential of beta or odds ratio (OR) is used. They are from zero to infinity and the null value is "1", the further away from 1 the greater association. If the CI does not include it, it will be significant at the established level (90, 95, or 99%) (Table 2a). In multiple linear regression, the comparison is made with the values of the standardized beta coefficients, which eliminates the original unit of measurement and allows for comparability of the effect of each factor. In this model, the null hypothesis of no association is the existence of a standardized coefficient with a value of "0". The further it deviates (−∞ or +∞), the greater the impact it will have on the prognosis. If the X% CI includes the value of "0," the result will not be significant at the established level, or there will be no association¹¹,¹² (Table 2b).

In these models, as for the multivariable linear model, the following assumptions are considered: linearity between predictors and outcome, homoscedasticity, normality and independence of residuals, and multicollinearity or high correlation among predictor factors. In logistic regression, it is mainly concerned with avoiding multicollinearity and independence in individual exposure to factors. When linearity does not exist, scale transformation options or stepwise analyses may be employed to facilitate the analysis, although it is recommended to consult with a statistical expert to avoid losing clinical significance. Multicollinearity is the second major problem in multivariable analysis; to avoid it, we recommend carefully reviewing the factors to be considered, and when there is a high correlation among some of them, consider including in the model only the factor with better measurement, greater validity, stronger association with the outcome, and less loss or absence in its capture.

When the outcome is a proportion adjusted for the time of presentation, the recommended model is Cox regression¹³,¹⁴. This model assumes that the risk(s) are always continuous and proportional (proportional hazards assumption), so the beta coefficient is presented as a hazard ratio (HR). It is also necessary to meet the assumptions of the absence of multicollinearity, linearity in the predictor variables with the logarithm of the outcome rate, and the absence of outliers. The interpretation of an HR is similar to that of an OR, that is, how many times more or less likely the presence of the complication is when exposed to a factor compared to not being exposed to it (Table 2c)¹²-¹⁴. In all the above models, researchers should report on statistically significant factors and highlight those with more extreme values concerning the null value. Other multivariable regression models are not mentioned here; interested readers are advised to consult statistical professionals.

Prognostic studies with explanatory purpose

As mentioned earlier, the objective is to validate the impact of a prognostic factor of interest controlled by its possible confounders. It should be remembered that a confounding factor is one known to be causal of the outcome of interest but associated with the prognostic factor under study without being part of the pathophysiological pathway by which the factor under study explains the outcome. In this analysis, it is also recommended to perform a multivariable regression with the same specifications mentioned previously. The main difference is that only the prognostic factor of interest and its confounders should be included in the model, not just any factor. It is important to select confounders adequately because as they increase, it will be necessary to expand the sample size¹¹,¹⁵,¹⁶. We suggest including the most involved, prevalent, better-measured confounders, with the potential to be modifiable in the future and the easiest to obtain⁴. In the final analysis report, the association estimator between the studied prognostic factor (relative risk, OR, HR, or standardized beta) and the outcome (e.g., relapse rate) should be shown, indicating the confounding factors to which the association was adjusted. It does not make sense to report on the estimators of the confounders since these were not adjusted for their own confounders and, therefore, they do not have explanatory value. If the factor of interest is removed, the study loses its objective. An example of a report is presented in table 3.

A proposed phase for this purpose is the causal network⁴. In this model, the factor under study and its outcome are not only adjusted for confounders analyzed but also antecedent and modifying factors are also added. Directed acyclic graphs models and path analysis have been proposed for its presentation³,⁴. Given their limited use in clinical medicine, readers are invited to consult specific sources⁴.

Prognostic studies with predictive purpose

These models are created to generate diagnostic and prognostic scales. In general, it is recommended to analyze these models in three phases: construction, internal validation, and external validation. In this review, we will only refer to their internal validation. For validation, two main types of analysis are primarily used: multivariable regression models and neural network models. In the former, modeling with regression analysis again depends on the type of dependent variable. The difference lies in the construction of the model. The objective of the selected model is based on one that is (1) more predictive, (2) parsimonious, (3) simple to apply, and (4) universal⁹,¹¹,¹⁶,¹⁷.

To validate a predictive model based on multivariate regression, it is necessary to consider a large sample size, generally ten patients for each factor to be considered. Once the sample is available, the analysis is executed with a statistical computer program. Regardless of the program used, it will request a dependent variable (the outcome of interest) and the introduction of independent variables or covariates. The objective of the analysis is to find an equation that allows obtaining (predicted) values as close as possible to those observed in patients (real). If this approximation is excellent, it will generally be excellent for patients with similar conditions who did not participate in the equation validation study (external validity). The prediction can be in terms of the probability of an outcome, time to an event, time to an outcome, and level of severity, among others. The method of selecting the most predictive variables of the outcome is based on the amount of variation explained by the equation. The most used estimator to address this situation is the coefficient of determination or R² (pseudo R² for logistic regression). The R² coefficient ranges from zero, which predicts nothing, to one, which implies a perfect prediction. To find the variables that will generate the most predictive equation, computers use three methods: forward, backward, or stepwise (Fig. 3 and Table 4). In the first method, all proposed variables are reviewed, and the most significant in its association with the outcome is selected (for example, the smallest "p" value). Next, the second most significant is sought, and if a significant change in R² is found, a third most associated factor is added. This process is repeated until no significant improvement in R² is observed, indicating a lack of predictive gain with more factors (Table 4a). The second method performs the procedure in the opposite way. It begins by introducing all the factors considered and calculating R². Then, it eliminates non-significant (associated) factors one by one and reviews the R² coefficient, which does not reduce the prediction. When removing a factor causes R² to decrease, the program stops subtracting factors, and the remaining ones are those that provide the greatest prediction (Table 4b). The third method (stepwise) is the most recommended. The selection is based on conducting trials of incorporating and removing factors in search of the combination with the highest coefficient of determination, that is, the most predictive (Table 4c).

Figure 3 Statistical modeling options to obtain the most precise prediction model. The squares represent prognostic variables and their size indicates the level of association with the prognostic variable. The R² value informs about the maximum prediction range. 

In prognostic scales where the outcome is quantitative (for example, days of hospital stay or years of survival), it is only necessary to establish the best prediction equation. However, if the outcome variable is qualitative (for example, cure), the programs determine the probability of the event as present if the constructed equation gives a score of 0.5 or more (50% or more). It is possible to improve the interpretation of diagnostic and prognostic validity by estimating its highest sensitivity and specificity by constructing a receiver operating characteristic curve and its area under the curve (Fig. 4). It is also feasible to determine the degree of discrimination of the prediction equation through specific analyses¹⁸. Alongside the validation of the most predictive model, it is necessary to consider other criteria. Parsimony refers to the model that has fewer included factors. In general, a model with more factors considered allows for better prediction. However, its use can become complicated if more than ten are included, given the difficulty in memorizing them or the lack of availability of information on some occasions. If a model with fewer factors does not significantly reduce the prediction by more than 10%, it will be more recommendable. Simplicity refers to having factors that can be determined or measured with unsophisticated methods in terms of cost, time, and execution. Universality implies that the factors can be determined or measured with unsophisticated methods in terms of cost, time, and execution, which will allow their application in different settings.

Figure 4 Receiver operating characteristic curve of predictive validity of equation obtained in the analysis of table 4c. 

Finally, neural network models are based on learning algorithms to obtain the best predictions. Computer systems functions such as the human mind, receiving information continuously, and determining the pathways that facilitate the approach to a result or "output" with layers or connection capacity. These models are gaining much acceptability due to their high level of prediction¹⁹-²⁰. However, they work as "black boxes," where the connections and functions related to this prediction are unknown, and they are not exempt from methodological biases²¹. To develop them, it is necessary to have the support of specialists in the field, and their validation never ends, given that the more information, the better the prediction. On the other hand, they are not exempt from the criteria mentioned above for simplicity and availability of information.

Conclusion

The recommended statistical analyses in prognostic studies vary according to their objective. These analyses can be merely descriptive, comparative, exploratory, explanatory, or prediction models. The most used methods are multivariable regressions, which are executed and reported according to the objective of the prognostic study. We always recommend seeking advice from a professional in the corresponding area and a statistician to achieve the proposed objective and communicate the results more efficiently.