event [Oct 2023] The paper got accepted to
+ SIGGRAPH
+ Asia 2023!
+
+
+
+
+
+
Abstract
+
+
+ We introduce neural lithography to address the 'design-to-manufacturing' gap in computational
+ optics. Computational optics with large design degrees of freedom enable advanced functionalities
+ and performance beyond traditional optics. However, the existing design approaches often overlook
+ the numerical modeling of the manufacturing process, which can result in significant performance
+ deviation between the design and the fabricated optics. To bridge this gap, we, for the first time,
+ propose a fully differentiable design framework that integrates a pre-trained photolithography
+ simulator into the model-based optical design loop. Leveraging a blend of physics-informed modeling
+ and data-driven training using experimentally collected datasets, our photolithography simulator
+ serves as a regularizer on fabrication feasibility during design, compensating for structure
+ discrepancies introduced in the lithography process. We demonstrate the effectiveness of our
+ approach through two typical tasks in computational optics, where we design and fabricate a
+ holographic optical element (HOE) and a multi-level diffractive lens (MDL) using a two-photon
+ lithography system, showcasing improved optical performance on the task-specific metrics.
+
+
+
+
+
+
What We Contribute?
+ TL;DR: A real2sim pipeline to quantitatively construct a high-fidelity neural photolithography
+ simulator and a design-fabrication co-optimization framework to bridge the design-to-manufacturing gap in
+ computational optics.
+
+
This work identifies two obstacles in computational optics:
+
+
1⃣ What is the "elephant in the room" in Computational Lithography?
+ - High-fidelity photolithography simulator | "No matter how good we can advance the computational
+ (inverse) lithography algorithm, the performance bound is grounded in the fidelity of the lithography
+ simulator."
+
+
2⃣ What hinders the progress of computational optics?
+ - One should be the Design to Manufacturing gap. |
+ "Yes you can design a perfect lens, but you cannot guarantee the post-manufacturing performance."
+
+
+ Pipeline to digitalize the lithography system through the real-world measurements.
+
+
+
+
+
+
2⃣ Close the Design-to-manfuctuting gap via co-optimizing the manufacturiability and the task design
+ with two intersected differentiable simulators (Litho + Task; DTCO).
+ Predicting capability of the learned neural lithography simulator on three models we
+ explored in neural lithography. The PBL (see details in the paper) performs the best
+ and thus is used throughout the paper. Top: The training and validation loss
+ curves correspond to the three models explored in our work. Bottom: The corresponding
+ average validation error map and the mean error value across the map.
+
+ Imaging performance with the designed MDL. A: Sketch of the setup for
+ characterizing the performance of MDL. B: We show our measured PSFs and direct
+ imaging results (i.e., w/o deconvolution) corresponding to design w/o and w/ PBL
+ litho model. The end of this row shows the line profiles of PSFs designed w/o or w/
+ different litho models. C: Computational/Indirect Imaging result of the MDL. The
+ lower right compares the Fourier spectrum of the designed PSFs. Our method's
+ design enhances the contrast in direct imaging (B) and the high-frequency
+ imaging performance in computational imaging (C).
+
+ Comparison of PSFs generated by MDLs in the design and real experiment. In
+ both the direct and indirect/computational imaging setting, the naive design w/o
+ lithography model has a larger deviation between the shape from the designed and
+ experimental PSF. In contrast, the deviation is small when we apply neural
+ lithography.
+
+
+
+
+
+
+
+
+
+
+
Frequently asked questions (FAQ)
+
+
+
1. Does this work provide a 'one-size-fits-all' litho model?
+ NO. Our goal isn't to learn a model that generalizes across different lithography types or
+ different modalities of a type. Instead, we present a pipeline on how to OVERFIT to a single lithography
+ system with a specific photoresist and post-processing procedure.
+
+
2. What are the assumptions for the applicability of the learned neural litho model?
+ 1⃣ No single lithography process can be perfectly represented by one white-box model. Factors like
+ optical misalignment, hardware tolerances, differences in conditions, and even temperature and humidity
+ can introduce variability.
+ 2⃣ If a specific lithography system and photoresist remain consistent over time, and once digitalized
+ remain stable, a learned gray-box simulator trained on data from that environment should be effective.
+
+
+
+
+
Citation
+
+
@article{zheng2023neural,
+ title={Neural Lithography: Close the Design-to-Manufacturing Gap in Computational Optics with a'Real2Sim'Learned Photolithography Simulator},
+ author={Zheng, Cheng and Zhao, Guangyuan and So, Peter TC},
+ journal={arXiv preprint arXiv:2309.17343},
+ year={2023}
+ }
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Fourier ptychography (FP) is a powerful computational imaging technique that provides both super-resolution and quantitative phase imaging capabilities by scanning samples in Fourier space with angle-varying illuminations. However, the image reconstruction in FP is inherently ill-posed, particularly when the measurements are noisy and have insufficient data redundancy in the Fourier space. To improve FP reconstruction in high-throughput imaging scenarios, we propose a regularized FP reconstruction algorithm utilizing anisotropic total variation (TV) and Tikhonov regularizations for the object and the pupil functions, respectively. To solve this regularized FP problem, we formulate a reconstruction algorithm using alternating direction method of multipliers and show that our approach successfully recovers high-quality images with sparsely sampled and/or noisy measurements. The results are quantitatively and qualitatively compared against various FP reconstruction algorithms to analyze the effect of regularization under harsh imaging conditions. In particular, we demonstrate the effectiveness of our method on the real experimental FP microscopy images, where the TV regularizer effectively suppresses the measurement noise while maintaining the edge information in the biological specimen and helps retrieve the correct amplitude and phase images even under insufficient sampling.
+ 
+ 
+ 
+ 
+
+ 
+
+ 
+ 
+ 
+ 
+