brier_score and cumulative_dynamic_auc fail when there is a test time greater than a training time

[aliciaolivaresgil]

I'm creating a new open issue related to #478_ which is currently cosed because no reproducible example was provided.

Another user (@dpellow) reported that "brier_score" produces a ValueError when test time is greater than training time. I experience the same issue with "cumulative_dynamic_auc". This does not happen systematically but only in some cases.

The documentation sais that time points in "times" must be within the range of times in "survival_test", but says nothing about times in "survival_test" being within the range of times in "survival_train". In fact, this happens in many other examples and no error occurs.

Here I leave a reproducible example where this error happens (sksurv version 0.22.2): https://github.com/aliciaolivaresgil/Reproduce_errors/blob/main/Error_example.ipynb

I could not find any difference between this example and the others where survival_test is greater than training test but no error occurs. Am I doing something wrong in the example? Why is this error occurring?

[sebp]

Thanks for providing the example.

Let me explain what is happening. The estimator of the time-dependent ROC (and the Brier score) relies on inverse-probability of censoring weighting. This means for each time point in survial_test, it estimates how likely it is to get censored, based on the data provided as survival_train. If survial_test has a time point larger than what survival_train contains, the probability of censoring at that time point is unknown, because it wasn't seen before and extrapolation is typically not possible.

You can just pass the concatenation of survival_train and survial_test as survival_train and this issue won't arise. This is often acceptable, because the test data is only used to estimate censoring weights.

I agree that the documentation should be improved to explain this.

[aliciaolivaresgil]

Thank you! That worked but i still believe the documentation should explain this, specially the explanation of the parameter survival_train.