Machine Learning for openEO

[m-mohr]

• Adds the basic ML processes again Move ML processes to 2.1.0 #416 (supersedes Add ML processes for 2.1.0 #418)
• Generalizes the predict functionalities predict_class and predict_probabilities #368 (supersedes Generalize predict processes for ML models #396)
• ...

Potentially interesting for "bring your own model": https://onnx.ai/

[m-mohr]

Variants discussed in the ML meeting:

datacube = load_collection("s2", temporal_extent = ..., spatial_extent = ..., bands = ...)
model = load_ml_model("my-model-id")

# variant 1
function fn(data) {
	return predict_ml_model(data, model)
}
datacube2 = reduce_dimension(datacube, dimension = "bands", reducer = fn)

function fn2(data) {
	return predict_ml_model_probabilities(data, model)
}
datacube3 = apply_dimension(datacube, dimension = "bands", target_dimension = "probabilities", process = fn2)

# variant 2
datacube2 = predict_ml_model(datacube, dimension = "bands", model = model)
datacube3 = predict_ml_model_probabilities(datacube, dimension = "bands", model = model)

[m-mohr]

Some things from the ML meeting:

Regularization may consist of (and is mapped to openEO processes):

• resample resolution -> resample_spatial
• temporal aggregation -> aggregate_temporal_period
• filter extent -> filter_bbox/spatial
• cloud removal -> cloud_detection or masking based on cloud bands

-> combine these to a new openEO process with some arguments that are commonly used with reasonable defaults

[m-mohr]

datacube = load_collection("s2", temporal_extent = ..., spatial_extent = ..., bands = ...)
model = load_ml_model("my-model-id")

# variant 1 (does NOT work with the current definition in this PR)
function fn(data) {
	let values = ml_predict(data, model)
	return array_element(values, 0)
}
datacube2 = reduce_dimension(datacube, dimension = "bands", reducer = fn)

function fn2(data) {
	return ml_predict(data, model)
}
datacube3 = apply_dimension(datacube, dimension = "bands", target_dimension = "predictions", process = fn2)

# variant 2 (works with the current definition in this PR)
datacube2 = ml_predict(datacube, model)
datacube2 = drop_dimension(datacube, "predictions")

datacube3 = ml_predict(datacube, model)

[PondiB]

@m-mohr , is there a reason why the keyword fit is used in the naming convention instead of train?
I will work on data cube regularization and do a different PR.

[m-mohr]

Just to align with fit_cuve, I guess. Train is also fine...

[PondiB]

Just to align with fit_cuve, I guess. Train is also fine...

Cool

[PondiB]

Looks good. Approving this now. We can continuously improve during implementations if need be.

[PondiB]

@m-mohr is renaming these two to follow with a prefix of "ml" a possible alternative?
load_ml_model() > ml_load_model()
save_ml_model() > ml_save_model()

[m-mohr]

Why? The current proposal follows the load_* and save_result schema.

[PondiB]

Why? The current proposal follows the load_* and save_result schema.

Cool makes sense to follow the previous schema. My initial thoughts came from the perception that it would be good for a general user if most of the ml operations start with that prefix i.e. "ml_".

[m-mohr]

The STAC ML Model extension may get deprecated in favor of https://github.com/crim-ca/dlm-extension
@PondiB I think it would be great to get in touch with the folks so that we can influence that it also works for openEO.

[PondiB]

The STAC ML Model extension may get deprecated in favor of https://github.com/crim-ca/dlm-extension @PondiB I think it would be great to get in touch with the folks so that we can influence that it also works for openEO.

Sure thanks, I just saw the notification about it, I will follow up with them.

[soxofaan]

The model will be accompanied by a separate STAC Item that

What does "accompanied" practically mean? Should there be an additional job result asset? Or should this be an job result link item?

The reason I'm asking is that we want to streamline the detection of the model's URL at the client side.

e.g. see Open-EO/openeo-python-client#576 we we currently have a highly implementation-specific hack

ml_model_metadata_url = [
    link 
    for link in links if 'ml_model_metadata.json' in link['href']
][0]['href']

[m-mohr]

Good question, I guess we should clarify that.

On the other hand, please note that this PR is implicitly outdated as the ML Model extension in STAC is likely going to be replaced by another extension. So this generally needs more work (which I have no plans to do anytime soon).

[soxofaan]

ML Model extension in STAC is likely going to be replaced by another extension.

can you point to the new one @m-mohr ?

[m-mohr]

https://github.com/crim-ca/mlm-extension