multitest/maxcombo: Is the former a more advanced version of the latter?

[LittleBeannie]

In the latest experimental multitest function, we are allowed to add multiple test to one single test. For example, assume we have 1 IA and 1 FA, the IA will only do regular weighted logrank test, while the FA will do a maxcombo test (regular logrank +weighted logrank with FH(0, 0.5) + milestone), we can use multitest as

data <- sim_pw_surv(n = 200)
data_ia<- cut_data_by_event(data, 100)
data_fa<- cut_data_by_event(data, 150)

# IA1 testing
data_ia |> wlr(weight = fh(rho = 0, gamma = 0))

# FA testing
lr_fa<-create_test(wlr, weight = fh(rho = 0, gamma = 0))
wlr_fa<-create_test(wlr, weight = fh(rho = 0, gamma = 0.5))
milestone_fa<-create_test(milestone, 20)

data_fa |> multitest(lr = lr_fa, wlr = wlr_fa, milestone = milestone_fa)

Questions:

1. In the above example, it looks like the multitest functions like maxcombo test. Considering this, shall we remove maxcombo or name multitest to maxcombo? (Please note the current maxcombo is only capable to do maxcombo tests with WLR with FH weighting).
2. If we prefer to keep multitest, shall we rename it to multi_test?

[jdblischak]

1. In the above example, it looks like the multitest functions like maxcombo test. Considering this, shall we remove maxcombo or name multitest to maxcombo? (Please note the current maxcombo is only capable to do maxcombo tests with WLR with FH weighting).

Here's my understanding. Both multitest() and maxcombo() will accept the same input: data with trial data and then an arbitrary number of test functions (maxcombo() won't be able to handle this interface until the test function output is standardized, #222). The key difference is the output. multitest() will return the results of every test. In contrast, maxcombo() will only return the result of the best test.

2. If we prefer to keep multitest, shall we rename it to multi_test?

I prefer multitest() because "multi" is a modifier, not a standalone word. Though I can see the desire for separation since sometimes "multi" is separated with a dash, eg multi-tool and multitool are interchangeable. I'm open to whatever the group decides. It's labeled as an experimental function, so we are free to change the name if we like.

[LittleBeannie]

Thank you for the clarification! I have a quick follow-up question. Currently, the maxcombo function only supports the combination of WLR with FH weights. How can users combine WLR with other weights, such as mb, or combo tests other than WLR, such as rmst?

[jdblischak]

Currently, the maxcombo function only supports the combination of WLR with FH weights. How can users combine WLR with other weights, such as mb, or combo tests other than WLR, such as rmst?

I have no idea. My understanding was that the plan was to make maxcombo() more generic. If it is just an extension of WLR with FH weights, then we can document it as such (and throw errors if users provide other test functions). On the other hand, if maxcombo() is only for passing rho and gamma to pvalue_maxcombo(), then it doesn't make sense to change its interface to be like multitest()

[LittleBeannie]

Currently, the maxcombo function only supports the combination of WLR with FH weights. How can users combine WLR with other weights, such as mb, or combo tests other than WLR, such as rmst?

I have no idea. My understanding was that the plan was to make maxcombo() more generic. If it is just an extension of WLR with FH weights, then we can document it as such (and throw errors if users provide other test functions). On the other hand, if maxcombo() is only for passing rho and gamma to pvalue_maxcombo(), then it doesn't make sense to change its interface to be like multitest()

Yeah, we definitely hope that maxcombo is not limited to combining multiple WLR with FH weights only. Our ultimate goal is to make it adaptable for other tests, not only WLR-FH. As a result, the syntax of maxcombo is always a significant concern for me. If you have any suggestions or recommendations, please feel free to share!

[jdblischak]

Now that @LittleBeannie has helped me better understand how the maxcombo function works, I propose overhauling the current maxcombo() function to accept a data set and many test functions. Then it would run all the test functions via multitest(). Then it would calculate the maxcombo p-value using the algorithm currently in pvalue_maxcombo().

In other words, this new maxcombo() would be a combination of multitest() and pvalue_maxcombo() so that users only have to call a single function (or pass a single function to sim_gs_n()).

And we would add safeguards. Currently pvalue_maxcombo() only supports weighted log rank (wlr) tests with Fleming-Harrington (fh) weights. Thus if a user passed a different function, we would throw an error (we'll be able to determine the type of test once the test output is standardized, #222). As we enable pvalue_maxcombo() to accept more test types, we can gradually relax the safeguards.

[LittleBeannie]

Decision: we will keep both multitest and maxcombo.

The current version of multitest looks great, no changes needed.
The sytax of maxcombo will be very similar to multitest (see below), but with different outputs (see #224).

data <- sim_pw_surv(n = 200)
data_ia<- cut_data_by_event(data, 100)
data_fa<- cut_data_by_event(data, 150)

# IA1 testing - only regular logrank test
data_ia |> wlr(weight = fh(rho = 0, gamma = 0))

# FA testing - maxcombo test
test1<-create_test(wlr, weight = fh(rho = 0, gamma = 0))
test2<-create_test(wlr, weight = fh(rho = 0, gamma = 0.5))
test3<-create_test(milestone, 20)

data_fa |> multitest(test1, test2, test3)