Possible discussion topics: Default colorbar limits and misfit normalization

[rmodrak]

Two separate but related issues here

[rmodrak]

First, Should we introduce a colorbar saturation option? If so, what should be the default? See figure below for one possible approach.

[rmodrak]

Second, should we turn the following normalization on by default?

https://github.com/rmodrak/mtuq/blob/9c53730e65ef83f4deb18a1a55efd642749a98ee/mtuq/misfit/waveform/level0.py#L25

With this turned on, plotting variance reduction could become as simple as

from graphics import plot_misfit_lune
plot_misfit_lune(filename, 1. - normalized_misfit_dataarray)

[thurinj]

Hi @rmodrak,

Thanks for bringing this up. I don't know what to do regarding the clipping value, as it would be very case-dependent. Some examples don't even need any clipping to be readable. Would an alternative scale make sense?
Another approach I like is to clip at a certain percentile of data: Given all the values to be displayed in the misfit map, clip the values such that the map explains n% of the map data.

As for the normalization, I would advocate for turning it on by default. It makes more sense to have all misfits in terms of data norm, and it would also help users understand data weight a bit more. Our canonical example is primarily dominated by the surface waves (at about 1:10 ?), and it isn't obvious based on absolute misfit alone. And I also agree that it would make other metrics easier to access.

[rmodrak]

Good points, thanks. Agreed, the current plotting works well in most cases. To add some context, there is some specific motivation here based on sponsor feedback.

The motivation has to do with historical events with sparse data coverage and weakly constrained source type. In some sense, the current plotting does its job for such events by conveying a weakly constrained solution. However, we have gotten some feedback that it can still sometimes be useful to see details of the misfit contour orientation, even for these weakly constrained events.

[rmodrak]

Given the above, let me go ahead and add a colorbar clipping option, but keep it turned off by default for now.

Agreed that "clipping" is a better description of the proposed approach than "saturation"-- let me make this change in my fork.

Jonas had the same idea about clipping based on percentile. However, one issue is that the percentile distribution depends strongly on how fine the sampling is at the poles relative to the center of the lune? For now, let me implement clipping based on extremal misfit values only-- we could discuss further over Zoom, if there is interest.

Let me go ahead and make theses changes via a pull request tomorrow, if that's alright.

[thurinj]

Thanks Ryan.

The static images in the docs will also probably have to be re-generated as the solutions for the 2009 examples are going to change with misfit normalization.

As for the percentile clipping: Given all the values are mapped out before any clipping is applied, does the overall spatial repartition of data on the lune play a strong role if we do the percentile mapping? I reckon it would just depend on the actual range of values displayed.

I have mostly used it when mapping CMA-ES samples on the lune, where the stochastic nature of the sampled values (especially at the beginning) generates big gaps in the misfit values displayed. With percentile clipping, I can still see what's going on around the minima without the map being dominated by values orders of magnitude higher than the minimum (but that might also be why it is so useful in this case).

Thanks!

[rmodrak]

I can still see what's going on around the minima without the map being dominated by values orders of magnitude higher than the minimum

Well said, this is a good way of putting it

[rmodrak]

For tomorrow let me push some small changes, which will not at all affect the default figure appearance. Interested in others' feedback before making any changes to the default color limits or default misfit normalization...

[rmodrak]

@ jonas

To bring out more visual contrast in the misfit surface, we can reduce colorbar range using cpt_clip and the number of contours levels using cpt_steps. For example

plot_misfit_lune(filename, data_array, cpt_steps=10, cpt_clip=[0.,0.5])

For now, these new keyword arguments remain turned off by default and subject to change pending further feedback.

One thing to note is that not just plot_lune_misfit, but all the GMT plotting utilities can accept these arguments.

The detailed implementation can be followed here

mtuq/mtuq/graphics/uq/_gmt.py

Line 70 in dbb2fc1

def _call(shell_script, filename, data, supplemental_data=None,