- 1 Massachusetts Institute of Technology
- 2The Chinese University of Hong Kong
+ 1 Yonsei University
+ 2 Duke University
+ 3 Chung-Ang University
- ✉ Corresponding Authors
- † Equal contribution
+ ✉ Corresponding Authors
- We introduce neural lithography to address the 'design-to-manufacturing' gap in computational
- optics. Computational optics with large design degrees of freedom enable advanced functionalities
- and performance beyond traditional optics. However, the existing design approaches often overlook
- the numerical modeling of the manufacturing process, which can result in significant performance
- deviation between the design and the fabricated optics. To bridge this gap, we, for the first time,
- propose a fully differentiable design framework that integrates a pre-trained photolithography
- simulator into the model-based optical design loop. Leveraging a blend of physics-informed modeling
- and data-driven training using experimentally collected datasets, our photolithography simulator
- serves as a regularizer on fabrication feasibility during design, compensating for structure
- discrepancies introduced in the lithography process. We demonstrate the effectiveness of our
- approach through two typical tasks in computational optics, where we design and fabricate a
- holographic optical element (HOE) and a multi-level diffractive lens (MDL) using a two-photon
- lithography system, showcasing improved optical performance on the task-specific metrics.
+ Fourier ptychography (FP) is a powerful computational imaging technique that provides both super-resolution and quantitative phase imaging capabilities by scanning samples in Fourier space with angle-varying illuminations. However, the image reconstruction in FP is inherently ill-posed, particularly when the measurements are noisy and have insufficient data redundancy in the Fourier space. To improve FP reconstruction in high-throughput imaging scenarios, we propose a regularized FP reconstruction algorithm utilizing anisotropic total variation (TV) and Tikhonov regularizations for the object and the pupil functions, respectively. To solve this regularized FP problem, we formulate a reconstruction algorithm using alternating direction method of multipliers and show that our approach successfully recovers high-quality images with sparsely sampled and/or noisy measurements. The results are quantitatively and qualitatively compared against various FP reconstruction algorithms to analyze the effect of regularization under harsh imaging conditions. In particular, we demonstrate the effectiveness of our method on the real experimental FP microscopy images, where the TV regularizer effectively suppresses the measurement noise while maintaining the edge information in the biological specimen and helps retrieve the correct amplitude and phase images even under insufficient sampling.
What We Contribute?
- TL;DR: A real2sim pipeline to quantitatively construct a high-fidelity neural photolithography
- simulator and a design-fabrication co-optimization framework to bridge the design-to-manufacturing gap in
- computational optics.
-
-
This work identifies two obstacles in computational optics:
+ TL;DR: derivation of the algorithm for TV-regularized FP reconstruction algorithm via ADMM.
+
+ validation of Tikhonov regularization on the pupil function that ensures a smooth pupil phase profile.
-
1⃣ What is the "elephant in the room" in Computational Lithography?
- - High-fidelity photolithography simulator | "No matter how good we can advance the computational
- (inverse) lithography algorithm, the performance bound is grounded in the fidelity of the lithography
- simulator."
+ demonstration of the performance of our method with extremely low measurement SNR and Fourier-space subsampling for reduced number of measurements.
+ application of our algorithm using real experimental data, which shows the effectiveness of TV-regularizer for both super-resolution and phase imaging of biological specimens under challenging imaging conditions.
-
2⃣ What hinders the progress of computational optics?
- - One should be the Design to Manufacturing gap. |
- "Yes you can design a perfect lens, but you cannot guarantee the post-manufacturing performance."
-
- Pipeline to digitalize the lithography system through the real-world measurements.
-
-
-
-
2⃣ Close the Design-to-manfuctuting gap via co-optimizing the manufacturiability and the task design
- with two intersected differentiable simulators (Litho + Task; DTCO).
@article{zheng2023neural,
- title={Neural Lithography: Close the Design-to-Manufacturing Gap in Computational Optics with a'Real2Sim'Learned Photolithography Simulator},
- author={Zheng, Cheng and Zhao, Guangyuan and So, Peter TC},
- journal={arXiv preprint arXiv:2309.17343},
- year={2023}
- }
-
-
-
-
+-->
@@ -825,5 +686,5 @@ category: computational microscopy
-Fourier ptychography (FP) is a powerful computational imaging technique that provides both super-resolution and quantitative phase imaging capabilities by scanning samples in Fourier space with angle-varying illuminations. However, the image reconstruction in FP is inherently ill-posed, particularly when the measurements are noisy and have insufficient data redundancy in the Fourier space. To improve FP reconstruction in high-throughput imaging scenarios, we propose a regularized FP reconstruction algorithm utilizing anisotropic total variation (TV) and Tikhonov regularizations for the object and the pupil functions, respectively. To solve this regularized FP problem, we formulate a reconstruction algorithm using alternating direction method of multipliers and show that our approach successfully recovers high-quality images with sparsely sampled and/or noisy measurements. The results are quantitatively and qualitatively compared against various FP reconstruction algorithms to analyze the effect of regularization under harsh imaging conditions. In particular, we demonstrate the effectiveness of our method on the real experimental FP microscopy images, where the TV regularizer effectively suppresses the measurement noise while maintaining the edge information in the biological specimen and helps retrieve the correct amplitude and phase images even under insufficient sampling.
+
-
-
- 
-
- 
-
-