nonlinear correlation calculation

[zhuyingqin]

Netcor calculates the data segment to find the cor, and the absolute value is added and then averaged,Is this understanding correct?
Take the maximum value of cor.estimate returned by Netcor for different subdivisions,Why is it a non-linear correlation?
What is the principle of your nonlinear correlation calculation?

[cran2367]

@zhuyingqin The underlying principle is to perform segmentation, and compute the segment-wise linear correlations to estimate an overall nonlinear correlation. The v2.0 pushed recently performs the segmentation using a dynamic programming for a more computationally efficient and theoretically robust approach.

[zhuyingqin]

What is the calculation principle of nonlinear correlation?

[zhuyingqin]

How to use piecewise linear correlation to estimate the overall nonlinear correlation？

[cran2367]

We will be publishing the nlcor paper in a month describing the approach in detail.

[zhuyingqin]

Please notify me after the paper is published

[brshallo]

In the case of computing a "total adjusted" correlation, Fisher's transformation may be useful.

To combine the p-values you may want to use either fisher's method, stouffer's method, or a related approach, see: https://en.wikipedia.org/wiki/Fisher%27s_method . Your current approach overly penalizes the significance when taking more splits. For example, three p-values of 0.05 should have a "total adjusted p-value" less than 0.05, but your current approach (1 - prod(1 - p)) does the opposite. (Any adjustment penalties you might add should be based on number of segments or tests, k, rather than the p-values themselves.)

[cran2367]

@brshallo thank you for the note! We will look into this.

[brshallo]

@cran2367 you are welcome. I stumbled onto your package while toying with brshallo/piececor that has a lot of overlap in thinking. The methods I mention above are what I used there.