Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-sync main from dev #637

Merged
merged 71 commits into from
Mar 21, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
2a80e98
Adding more papers
cophus Nov 8, 2023
53ad64b
more cites
cophus Nov 8, 2023
213808f
Merge remote-tracking branch 'cophus/dev' into dev
cophus Dec 20, 2023
cbce25c
bug fix
smribet Jan 9, 2024
d366aca
Revert "bug fix"
gvarnavi Jan 12, 2024
c109d95
Bump tj-actions/changed-files from 41 to 42
dependabot[bot] Jan 22, 2024
fc51379
Bump github/super-linter from 4 to 5
dependabot[bot] Jan 22, 2024
1598ac2
Bump actions/checkout from 3 to 4
dependabot[bot] Jan 22, 2024
f35f683
Bump actions/setup-python from 2 to 5
dependabot[bot] Jan 22, 2024
11c27fa
Merge remote-tracking branch 'origin/dev' into phase_contrast
gvarnavi Jan 25, 2024
f08049b
Merge pull request #600 from py4dstem/dependabot/github_actions/tj-ac…
bsavitzky Jan 25, 2024
6dffda1
Merge pull request #601 from py4dstem/dependabot/github_actions/githu…
bsavitzky Jan 25, 2024
c76cb1e
Merge pull request #602 from py4dstem/dependabot/github_actions/actio…
bsavitzky Jan 25, 2024
a1e3662
Merge pull request #603 from py4dstem/dependabot/github_actions/actio…
bsavitzky Jan 25, 2024
fd8ee7b
num_probes_fit_aberrations
smribet Jan 28, 2024
3ca1869
flip 180 option for read arina
smribet Jan 31, 2024
633fb01
ptycho positions offset
smribet Feb 1, 2024
e992335
Merge remote-tracking branch 'origin/dev' into dev
cophus Feb 2, 2024
15aeca1
remove 180 flip
smribet Feb 2, 2024
92cd0f7
tweaking amorphous fitting function
cophus Feb 2, 2024
7a1a167
median_filter_masked_pixels
smribet Feb 2, 2024
6f022f7
save polar aberrations instead
gvarnavi Feb 6, 2024
be2cd20
read-write compatibility
gvarnavi Feb 6, 2024
7ec0160
uncertainty viz bug
gvarnavi Feb 6, 2024
4f433f0
adding force_com_measured functionality to ptycho
gvarnavi Feb 6, 2024
314ee79
clean up force_com_measured
gvarnavi Feb 7, 2024
dde1eda
adding DP normalization progress bar
gvarnavi Feb 7, 2024
4cca38a
moving fov_mask calc to as needed
gvarnavi Feb 7, 2024
aaf957f
adding detector_fourier_mask
gvarnavi Feb 8, 2024
99c70ad
bug fix for filtering
smribet Feb 10, 2024
2eae7d8
Friedel origin finder, various FEM tools (#607)
cophus Feb 26, 2024
f419f27
Merge remote-tracking branch 'origin/dev' into dev
cophus Feb 26, 2024
dd4c6d8
Default threshold should be zero
cophus Feb 26, 2024
6b3e8af
Release memory after Bayesian optimization (#599)
sezelt Feb 28, 2024
d7e7472
remove duplicate make_fourier_coords_2D
sezelt Feb 28, 2024
d478aa1
format with black
sezelt Feb 28, 2024
d3ec3c2
Added phase_contrast warning in README (#608)
gvarnavi Feb 28, 2024
b9ac79d
Merge remote-tracking branch 'origin/dev' into phase_contrast
gvarnavi Mar 5, 2024
84bbc04
switching to non-vectorized datacube resampling
gvarnavi Mar 6, 2024
9718a98
Merge pull request #611 from sezelt/fourier_coords_dedup
bsavitzky Mar 7, 2024
f47a3b4
Merge pull request #610 from cophus/dev
bsavitzky Mar 7, 2024
963df98
fixing region_of_interest_shape flag
gvarnavi Mar 9, 2024
b772a8c
adding cross-correlation function
gvarnavi Mar 9, 2024
2d474cd
more careful array sums interpolation
gvarnavi Mar 12, 2024
0ac074d
Bump gdown to 5.1.0 (#617)
gvarnavi Mar 13, 2024
569fbc4
Merge remote-tracking branch 'origin/dev' into phase_contrast
gvarnavi Mar 13, 2024
1afa166
docstring change per Colin review
gvarnavi Mar 13, 2024
6806055
DP resampling mode to nearest
gvarnavi Mar 14, 2024
1137191
adding fourier_mask in forward mode, ie pre error calculation
gvarnavi Mar 14, 2024
b326bca
Merge pull request #616 from py4dstem/phase_contrast
cophus Mar 14, 2024
facf1e1
small bugfix, moving reset before object_type change
gvarnavi Mar 15, 2024
2d4928e
parallax odd pixel padding bugfix
gvarnavi Mar 18, 2024
e132c87
parallax upsample, fit, and correct should return self
gvarnavi Mar 18, 2024
22fc8b9
removing Wiener filter
gvarnavi Mar 18, 2024
b0bbc52
fixing depth sectioning
gvarnavi Mar 18, 2024
2ec33e2
Merge pull request #630 from py4dstem/phase_contrast
cophus Mar 18, 2024
5bd35b7
Bump tj-actions/changed-files from 42 to 43 (#629)
dependabot[bot] Mar 19, 2024
5380901
fixing aberration correction for odd radial symmetries. I think the s…
gvarnavi Mar 21, 2024
b093eaa
Merge remote-tracking branch 'origin/dev' into phase_contrast
gvarnavi Mar 21, 2024
bc8f333
ACOM module - fix for .ang file writer (#382)
cophus Mar 21, 2024
a8912ba
Merge pull request #632 from py4dstem/phase_contrast
smribet Mar 21, 2024
74ede03
Fixing projection 2D thickness (#631)
cophus Mar 21, 2024
7a0e34c
Versioning to 0.14.10
cophus Mar 21, 2024
db2b987
Merge pull request #633 from cophus/dev
cophus Mar 21, 2024
1ef52d6
Revert "Versioning to 0.14.10"
sezelt Mar 21, 2024
7ddd4f5
Merge pull request #635 from py4dstem/revert-633-dev
cophus Mar 21, 2024
c21aa51
Update pypi_upload.yml
sezelt Mar 21, 2024
ac1a498
Update pypi_upload.yml
sezelt Mar 21, 2024
927fdec
Merge branch 'dev' into pypi-sync
sezelt Mar 21, 2024
27bc3d1
Merge pull request #636 from py4dstem/pypi-sync
sezelt Mar 21, 2024
e2242f8
Merge branch 'main' of https://github.com/py4dstem/py4DSTEM into dev
bsavitzky Mar 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
more careful array sums interpolation
  • Loading branch information
gvarnavi committed Mar 12, 2024
commit 2d474cd3f40142a53874ab4f16395058e77be8c0
10 changes: 8 additions & 2 deletions py4DSTEM/datacube/datacube.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,9 @@ def pad_Q(self, N=None, output_size=None):
d = pad_data_diffraction(self, pad_factor=N, output_size=output_size)
return d

def resample_Q(self, N=None, output_size=None, method="bilinear"):
def resample_Q(
self, N=None, output_size=None, method="bilinear", conserve_array_sums=False
):
"""
Resamples the data in diffraction space by resampling factor N, or to match output_size,
using either 'fourier' or 'bilinear' interpolation.
Expand All @@ -418,7 +420,11 @@ def resample_Q(self, N=None, output_size=None, method="bilinear"):
from py4DSTEM.preprocess import resample_data_diffraction

d = resample_data_diffraction(
self, resampling_factor=N, output_size=output_size, method=method
self,
resampling_factor=N,
output_size=output_size,
method=method,
conserve_array_sums=conserve_array_sums,
)
return d

Expand Down
14 changes: 12 additions & 2 deletions py4DSTEM/preprocess/preprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,11 @@ def datacube_diffraction_shift(


def resample_data_diffraction(
datacube, resampling_factor=None, output_size=None, method="bilinear"
datacube,
resampling_factor=None,
output_size=None,
method="bilinear",
conserve_array_sums=False,
):
"""
Performs diffraction space resampling of data by resampling_factor or to match output_size.
Expand All @@ -594,7 +598,10 @@ def resample_data_diffraction(
old_size = datacube.data.shape

datacube.data = fourier_resample(
datacube.data, scale=resampling_factor, output_size=output_size
datacube.data,
scale=resampling_factor,
output_size=output_size,
conserve_array_sums=conserve_array_sums,
)

if not resampling_factor:
Expand Down Expand Up @@ -650,6 +657,9 @@ def resample_data_diffraction(
grid_mode=True,
)

if conserve_array_sums:
output_data = output_data / resampling_factor.prod()

datacube.data = output_data
datacube.calibration.set_Q_pixel_size(
datacube.calibration.get_Q_pixel_size() / resampling_factor[0]
Expand Down
21 changes: 18 additions & 3 deletions py4DSTEM/process/phase/phase_base_class.py
Original file line number Diff line number Diff line change
Expand Up @@ -346,34 +346,49 @@ def _preprocess_datacube_and_vacuum_probe(

elif reshaping_method == "fourier":
datacube = datacube.resample_Q(
N=resampling_factor_x, method=reshaping_method
N=resampling_factor_x,
method=reshaping_method,
conserve_array_sums=True,
)
if vacuum_probe_intensity is not None:
vacuum_probe_intensity = fourier_resample(
vacuum_probe_intensity,
output_size=diffraction_intensities_shape,
force_nonnegative=True,
conserve_array_sums=True,
)
if dp_mask is not None:
dp_mask = fourier_resample(
dp_mask,
output_size=diffraction_intensities_shape,
force_nonnegative=True,
conserve_array_sums=False,
)

elif reshaping_method == "bilinear":
datacube = datacube.resample_Q(
N=resampling_factor_x, method=reshaping_method
N=resampling_factor_x,
method=reshaping_method,
conserve_array_sums=True,
)
if vacuum_probe_intensity is not None:
vacuum_probe_intensity = zoom(
vacuum_probe_intensity,
(resampling_factor_x, resampling_factor_x),
order=1,
mode="grid-wrap",
grid_mode=True,
)
vacuum_probe_intensity = (
vacuum_probe_intensity / resampling_factor_x**2
)
if dp_mask is not None:
dp_mask = zoom(
dp_mask, (resampling_factor_x, resampling_factor_x), order=1
dp_mask,
(resampling_factor_x, resampling_factor_x),
order=1,
mode="grid-wrap",
grid_mode=True,
)

else:
Expand Down
18 changes: 17 additions & 1 deletion py4DSTEM/process/phase/ptychographic_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -1007,6 +1007,7 @@ def _initialize_probe(
vacuum_probe_intensity,
output_size=(tx, ty),
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -1568,6 +1569,7 @@ def _gradient_descent_fourier_projection(self, amplitudes, overlap, fourier_mask
fourier_overlap,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand All @@ -1585,6 +1587,7 @@ def _gradient_descent_fourier_projection(self, amplitudes, overlap, fourier_mask
fourier_modified_overlap,
output_size=self._region_of_interest_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -1660,6 +1663,7 @@ def _projection_sets_fourier_projection(
fourier_projected_factor,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand All @@ -1676,6 +1680,7 @@ def _projection_sets_fourier_projection(
fourier_projected_factor,
output_size=self._region_of_interest_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -2074,7 +2079,11 @@ def _position_correction(
# resample to match data, note: this needs to happen in reciprocal-space
if self._resample_exit_waves:
overlap_fft = bilinear_resample(
overlap_fft, output_size=self._amplitudes_shape, vectorized=True, xp=xp
overlap_fft,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

# unperturbed
Expand Down Expand Up @@ -2114,12 +2123,14 @@ def _position_correction(
overlap_dx_fft,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)
overlap_dy_fft = bilinear_resample(
overlap_dy_fft,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -2224,6 +2235,7 @@ def _return_self_consistency_errors(
fourier_overlap,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -2720,6 +2732,7 @@ def _gradient_descent_fourier_projection(self, amplitudes, overlap, fourier_mask
fourier_overlap,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand All @@ -2741,6 +2754,7 @@ def _gradient_descent_fourier_projection(self, amplitudes, overlap, fourier_mask
fourier_modified_overlap,
output_size=self._region_of_interest_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down Expand Up @@ -2816,6 +2830,7 @@ def _projection_sets_fourier_projection(
fourier_projected_factor,
output_size=self._amplitudes_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand All @@ -2833,6 +2848,7 @@ def _projection_sets_fourier_projection(
fourier_projected_factor,
output_size=self._region_of_interest_shape,
vectorized=True,
conserve_array_sums=True,
xp=xp,
)

Expand Down
8 changes: 7 additions & 1 deletion py4DSTEM/process/phase/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -2251,6 +2251,7 @@ def bilinear_resample(
mode="grid-wrap",
grid_mode=True,
vectorized=True,
conserve_array_sums=False,
xp=np,
):
"""
Expand Down Expand Up @@ -2321,13 +2322,17 @@ def bilinear_resample(

array = out_array.reshape(tuple(array_size[:-2]) + tuple(output_size))

if conserve_array_sums:
array = array / np.array(scale_output).prod()

return array


def vectorized_fourier_resample(
array,
scale=None,
output_size=None,
conserve_array_sums=False,
xp=np,
):
"""
Expand Down Expand Up @@ -2489,7 +2494,8 @@ def vectorized_fourier_resample(
array_resize = xp.real(xp.fft.ifft2(array_resize)).astype(xp.float32)

# Normalization
array_resize *= scale_output
if not conserve_array_sums:
array_resize = array_resize * scale_output

return array_resize

Expand Down
5 changes: 4 additions & 1 deletion py4DSTEM/process/utils/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,7 @@ def fourier_resample(
bandlimit_nyquist=None,
bandlimit_power=2,
dtype=np.float32,
conserve_array_sums=False,
):
"""
Resize a 2D array along any dimension, using Fourier interpolation / extrapolation.
Expand All @@ -464,6 +465,7 @@ def fourier_resample(
bandlimit_nyquist (float): Gaussian filter information limit in Nyquist units (0.5 max in both directions)
bandlimit_power (float): Gaussian filter power law scaling (higher is sharper)
dtype (numpy dtype): datatype for binned array. default is single precision float
conserve_arrray_sums (bool): If True, the sums of the array are conserved

Returns:
the resized array (2D/4D numpy array)
Expand Down Expand Up @@ -661,7 +663,8 @@ def fourier_resample(
array_resize = np.maximum(array_resize, 0)

# Normalization
array_resize = array_resize * scale_output
if not conserve_array_sums:
array_resize = array_resize * scale_output

return array_resize

Expand Down