International prognostic scoring system for Waldenström macroglobulinemia
Blood Morel et al.
113: 4163
Supplemental materials for: Morel et al
Estimation of sample size
No standardized method is reported to compute the number of subjects required to build a prognostic index. Concato et al,1 recommended at least 10 events per independent variable. We chose to obtain 15 events for each covariate. In order to avoid the data dependencies, a limited number (no more than 19; 14 are shown in Table 1) of covariates was selected. With a total number of 19 covariates and assuming a mortality rate of 50%, the number of patients needed was estimated to be 600. Let phr be the probability of death at 4 years in the high-risk patients group (phr=0·5) and po the probability of death at 4 years in the other patients. Using the separation measure SEP=phr-po defined by Altman and Royston2 with the following hypotheses: phr=0·5, and po =0·75, at least 450 patients should be necessary to validate the final model with a 10% precision in the separation estimate.
Identification of covariates and their cut-off (univariate analysis)
For each covariate, the validity of the Proportional Hazard (PH) assumption was checked using the Scaled Schoenfeld Residuals (SSR) and the test proposed by Therneau and Grambsch3 and this test was illustrated by producing graphs of SSR against the rank of time. The graphs were improved with a smoothing of the residuals obtained using spline functions and the corresponding 95% confidence intervals. When the PH assumption is true, the smoothed function is parallel to the horizontal axis. The assumption of log-linearity was checked for continuous covariates by using the Martingale Residuals (MR).4 The relationship between the MR and each continuous covariate was investigated by means of a smoothing of the MR (obtained by the LOESS SAS procedure) and the linearity of this relation was tested using the Pearson correlation coefficient. Survival regression tree analyses were performed with the TSSA library established for S-PLUS. TSSA is an extension to survival analysis of the Classification And Regression Tree (CART) algorithm developed by Breiman. It used the Log rank test to build the tree.5 Fisher algorithm transforms a continuous variable into a qualitative variable by identifying cut offs that maximize the chi-square between a dependent variable here the outcome and the transformed variable.6 The results of these analyses are shown in Table S1.
Identification of the prognostic model
The bootstrap resampling method7 is a method to get for example 500 replicates of the initial dataset used for multivariate analysis. Cox multivariate analyses with a stepwise selection at the level 0·15 are performed on each of these 500 replicates. The inclusion of the variable in the final model is confirmed if this candidate variable is selected in at least 80% of these 500 analyses. Selection frequencies of all possible pairs of variables are also considered to cope with the problem of the correlated variables.7 In case of the selection of a pair of variable in more than 80% of replicates, the covariate with the higher inclusion frequency is selected. The stepdown reduction method8 uses multiple regression analyses with the prognostic index (linear predictor) of the full model as dependent variable and the covariates as independent variables. The predictor whose omission causes the smallest decrease in R2 is dropped. The procedure is repeated until the omission of any more covariates would result in an R2 lower than 0·95.
Assessment of the correlations between the selected covariates
Pearson correlation coefficients r between each pair of covariates and the variance inflation factors (VIF) were computed.9 Correlation coefficients were lower than 0·27. VIF were close to 1 (VIF values range from 1·06 to 1·2).
Cross validation analysis
This procedure works as follows: for each patient i, a model M−i is derived from the sample obtained after elimination of the patient i. The cross validation prognostic index (linear predictor) for i is computed from the coefficients of this model M−i with the characteristics of the patient i. The cross validation prognostic index can be considered as a new covariate. If this covariate is introduced in a Cox model, the prognostic model is considered to be validated if the parameter associated with this new covariate is close to 1 (>0·85).
Measure of the separation D
The measure of separation D has been proposed by Sauerbrei and colleagues.10 D is proportional to the absolute mean deviation of the log hazard ratio for the different risk subgroups with the low-risk group as reference. It is a measure of spread of log hazards ratios. We used this parameter to compare the ISSWM and each of the 5 characteristics included in this scoring system. The bootstrap resampling method (500 replicates) was used to compare the means of the D values with a paired t test of Student and thus to validate the difference in D between the ISSWM and each of the 5 characteristics.
REFERENCES
1. Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995;48:14951501.
2. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453473.
3. Therneau T, Grambsch P. Modelling survival data. Extending the Cox model. New York: Springer; 2000.
4. Therneau TM, Grambsch P, Fleming T. Martingale-based residuals for survival models. Biometrika. 1990;77:147160.
5. Ciampi A, Lawless JF, McKinney SM, Singhal K. Regression and recursive partition strategies in the analysis of medical survival data. J Clin Epidemiol. 1988;41:737748.
6. Fisher WD. On grouping for maximum homogeneity. J Am Stat Assoc. 1958;53:789798.
7. Sauerbrei W, Schumacher M. A bootstrap resampling procedure for model building: application to the Cox regression model. Stat Med. 1992;11:20932109.
8. Ambler G, Brady AR, Royston P. Simplifying a prognostic model: a simulation study based on clinical data. Stat Med. 2002;21:38033822.
9. Velleman S, Welsch RE. Efficient computing of regression diagnostics. American Statistician. 1981;35:234242.
10. Sauerbrei W, Hubner K, Schmoor C, Schumacher M. Validation of existing and development of new prognostic classification schemes in node negative breast cancer. German Breast Cancer Study Group. Breast Cancer Res Treat. 1997;42:149163.
