Predicting survival time (response) using Cox model

[HosheaA]

I very much appreciate all of the effort you have put into mlr3proba, and I understand there are some issues to be fixed. That being said, I’m hoping you can help me with something.

Just some simple code to learn some basics:

task <- as_task_surv(diabetic, 
                    time = "time", 
                    event = "status", 
                    type = "right")
learn = lrn("surv.coxph")
train_set = sample(task$nrow, 0.8 * task$nrow)
test_set = setdiff(seq_len(task$nrow), train_set)
learn$train(task, row_ids = train_set)
learn$model
prediction = learn$predict(task, row_ids = test_set)

1. Why do I get a vector of NAs with prediction$response?
2. Is there a way to manually calculate predicted response times from prediction$distr or from other data contained in the object?

Thank you in advance,

[RaphaelS1]

Hi Hoshea,

1. Because reponse is for survival time predictions but CoxPH only makes linear predictor or distribution predictions
2. Yes and no. You could simply run prediction$distr$mean() to get the mean survival time however this will not work for improper distributions (e.g. see this paper we just published). In general unless a model is specifically built to make a survival time prediction, e.g. survivalsvm (though this is questionable) and some neural networks, it generally is not the goal of survival analysis to make a survival time prediction. Especially when predicting the distribution is always more informative

[HosheaA]

Thanks Raphael

I'll take a look at your paper.

When I run prediction$distr$mean(), I get a vector of infinite values. Why is that? Is there something improper with the distribution?

[RaphaelS1]

Yes. To identify an 'improper' distribution you can do something like distr$cdf(Inf). If this does not equal 1 then the distribution is improper. Some examples below

library(mlr3proba)
library(distr6)
d = lrn("surv.kaplan")$train(tsk("rats"))$predict(tsk("rats"))$distr[1]

# Improper distribution (F(Inf) < 1) -> Infinite mean
d$cdf(Inf)
#> [1] 0.1871857
d$mean()
#> [1] Inf

# Proper distribution (F(Inf) = 1) -> Finite mean
Exponential$new()$cdf(Inf)
#> [1] 1
Exponential$new()$mean()
#> [1] 1

Binomial$new()$cdf(Inf)
#> [1] 1
Binomial$new()$mean()
#> [1] 5

^{Created on 2022-08-02 by the reprex package (v2.0.1)}

[bblodfon]

Future reference for question (2): we now use by default the RMST (Restricted mean survival time) to compose the survival time from the predicted survival distribution, see responsecompositor.