Log regression task metrics in multitask model

[ntravis22]

Recently per-task metrics were added to multitask_model, however we did not include any for regression tasks and we did not check that the metric keys are present which can throw an error, so this addresses both of those concerns.

[MattGPT-ai]

Are there metrics for regression type models that we could put into scores, or do those perhaps go into the "classification report" - e.g. Pearson, spearman. In the regression models, results is written as

            eval_metrics = {
                "loss": eval_loss.item(),
                "mse": metric.mean_squared_error(),
                "mae": metric.mean_absolute_error(),
                "pearson": metric.pearsonr(),
                "spearman": metric.spearmanr(),
            }

So maybe we could either check for the base model class that defines evaluate, or just check for the keys. Then maybe we could write e.g. scores[(task_id, 'mse')]. What do you think?

[ntravis22]

Are there metrics for regression type models that we could put into scores, or do those perhaps go into the "classification report" - e.g. Pearson, spearman. In the regression models, results is written as

            eval_metrics = {
                "loss": eval_loss.item(),
                "mse": metric.mean_squared_error(),
                "mae": metric.mean_absolute_error(),
                "pearson": metric.pearsonr(),
                "spearman": metric.spearmanr(),
            }

So maybe we could either check for the base model class that defines evaluate, or just check for the keys. Then maybe we could write e.g. scores[(task_id, 'mse')]. What do you think?

Ok I added mse.

[MattGPT-ai]

Oh actually, can we just add all four of the metrics?

[ntravis22]

Oh actually, can we just add all four of the metrics?

Done

[alanakbik]

@ntravis22 @MattGPT-ai Could you paste a script to test this PR?