From ab0655069864af5708b7b65f74c0d262a4045599 Mon Sep 17 00:00:00 2001 From: Tony Ketcham Date: Sun, 15 Dec 2019 03:58:33 -0600 Subject: [PATCH 1/5] Update README.md Provided additional information on the state of the project as well as an example of what the project currently generates. --- README.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 1999859..6c668f9 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,29 @@

Depth Mapper

+

Overview

Generates a depth map from a focal stack of images, using ideas proposed in [this Stanford graphics paper](https://graphics.stanford.edu/papers/focalstack/focalstack.pdf). Supports JPG input stacks, with the goal of supporting a DNG in-and-out pipeline. A focal stack of DNG images would be fed in as input with one single DNG exported, containing an embedded depth map. This would allow DSLR users the ability to take advantage of the depth masking capabilities of Adobe Camera Raw, Lightroom, and other RAW processors. Previously, only stereoscopic cameras, light field cameras, and digital renders could make use of these masking features. This project strongly utilizes OpenCV and eventually OpenCL through [JavaCV](https://github.com/bytedeco/javacv). This wrapper, unlike native OpenCV for Java, allows use of Transparent API for seamless connection between CV and CL. -

Installation

+

Example

+Take this highly textured scene of 32 input images: + +

Depth Map

+![Depth map of highly textured scene](/src/gitresources/Depth%20Map.jpg) + +

Focus Stack

+![Focus stack of highly textured scene](/src/gitresources/Focus Stacked.jpg) + +

First Image in the Stack

+![Image with shallow depth of field](/src/TestImages/_0000_Highly Textured Input - 00.jpg) + +

Current State + Future Improvements

+The algorithm for performing depth mapping considers the highest rate of change for a given pixel against every pixel at the same coordinate in the remaining images. As it is, the depth mapping performs best assuming images are aligned prior to input. The current alignment algorithm has a poor runtime for high megapixel image stacks (although it does provide quite an accurate alignment) but this time complexity can be improved greatly through a careful combination of ECC and homography-based image alignment. + +- DNG file handling will be implemented for use of the generated depth map as an embedded map for the raw focal stack it represents. +- The depth mapping algorithm will be significantly improved with image segmentation to constrain regions of pixels to a certain set of input files, as to reduce the noise of non-discriminable regions (regions with very little texture). +- Additional use of OpenCL to improve execution time of image alignment. + +

Installation

A Maven Eclipse project is provided. Make sure you have OpenCV 4.1.1 installed as a User Library in eclipse as 'opencv'. The project should function with OpenCV 3.* as well, however this has not been tested. From 6201a6eeabd43e3bb3fdd2d47864dcacb2336ba6 Mon Sep 17 00:00:00 2001 From: Tony Ketcham Date: Sun, 15 Dec 2019 04:02:49 -0600 Subject: [PATCH 2/5] Update README.md --- README.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6c668f9..521c2d8 100644 --- a/README.md +++ b/README.md @@ -3,23 +3,29 @@

Overview

Generates a depth map from a focal stack of images, using ideas proposed in [this Stanford graphics paper](https://graphics.stanford.edu/papers/focalstack/focalstack.pdf). Supports JPG input stacks, with the goal of supporting a DNG in-and-out pipeline. A focal stack of DNG images would be fed in as input with one single DNG exported, containing an embedded depth map. This would allow DSLR users the ability to take advantage of the depth masking capabilities of Adobe Camera Raw, Lightroom, and other RAW processors. Previously, only stereoscopic cameras, light field cameras, and digital renders could make use of these masking features. + This project strongly utilizes OpenCV and eventually OpenCL through [JavaCV](https://github.com/bytedeco/javacv). This wrapper, unlike native OpenCV for Java, allows use of Transparent API for seamless connection between CV and CL.

Example

Take this highly textured scene of 32 input images:

Depth Map

+ ![Depth map of highly textured scene](/src/gitresources/Depth%20Map.jpg)

Focus Stack

-![Focus stack of highly textured scene](/src/gitresources/Focus Stacked.jpg) + +![Focus stack of highly textured scene](/src/gitresources/Focus%20Stacked.jpg)

First Image in the Stack

-![Image with shallow depth of field](/src/TestImages/_0000_Highly Textured Input - 00.jpg) + +![Image with shallow depth of field](/src/TestImages/_0000_Highly%20Textured%20Input%20-%2000.jpg)

Current State + Future Improvements

The algorithm for performing depth mapping considers the highest rate of change for a given pixel against every pixel at the same coordinate in the remaining images. As it is, the depth mapping performs best assuming images are aligned prior to input. The current alignment algorithm has a poor runtime for high megapixel image stacks (although it does provide quite an accurate alignment) but this time complexity can be improved greatly through a careful combination of ECC and homography-based image alignment. +

Improvements

+ - DNG file handling will be implemented for use of the generated depth map as an embedded map for the raw focal stack it represents. - The depth mapping algorithm will be significantly improved with image segmentation to constrain regions of pixels to a certain set of input files, as to reduce the noise of non-discriminable regions (regions with very little texture). - Additional use of OpenCL to improve execution time of image alignment. From 5df327e2779eab52c00cea6f7cd6c3485adc013b Mon Sep 17 00:00:00 2001 From: Tony Ketcham Date: Sun, 15 Dec 2019 04:05:05 -0600 Subject: [PATCH 3/5] Update README.md --- README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 521c2d8..d051874 100644 --- a/README.md +++ b/README.md @@ -9,17 +9,18 @@ This project strongly utilizes OpenCV and eventually OpenCL through [JavaCV](htt

Example

Take this highly textured scene of 32 input images: -

Depth Map

+

Image in the Stack

-![Depth map of highly textured scene](/src/gitresources/Depth%20Map.jpg) +![Image with shallow depth of field](/src/TestImages/_0009_Highly%20Textured%20Input%20-%2009.jpg)

Focus Stack

![Focus stack of highly textured scene](/src/gitresources/Focus%20Stacked.jpg) -

First Image in the Stack

+

Depth Map

+ +![Depth map of highly textured scene](/src/gitresources/Depth%20Map.jpg) -![Image with shallow depth of field](/src/TestImages/_0000_Highly%20Textured%20Input%20-%2000.jpg)

Current State + Future Improvements

The algorithm for performing depth mapping considers the highest rate of change for a given pixel against every pixel at the same coordinate in the remaining images. As it is, the depth mapping performs best assuming images are aligned prior to input. The current alignment algorithm has a poor runtime for high megapixel image stacks (although it does provide quite an accurate alignment) but this time complexity can be improved greatly through a careful combination of ECC and homography-based image alignment. From 8b1c1d64546863cc904c07837591ab46bf8bc74f Mon Sep 17 00:00:00 2001 From: Tony Ketcham Date: Sun, 15 Dec 2019 04:06:05 -0600 Subject: [PATCH 4/5] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index d051874..675e211 100644 --- a/README.md +++ b/README.md @@ -28,9 +28,10 @@ The algorithm for performing depth mapping considers the highest rate of change

Improvements

- DNG file handling will be implemented for use of the generated depth map as an embedded map for the raw focal stack it represents. -- The depth mapping algorithm will be significantly improved with image segmentation to constrain regions of pixels to a certain set of input files, as to reduce the noise of non-discriminable regions (regions with very little texture). +- The depth mapping algorithm will be significantly improved with image segmentation to constrain regions of pixels to a certain set of input files, as to reduce the noise of non-discriminable regions (regions with very little texture). This reduces haloing around sharp edges. - Additional use of OpenCL to improve execution time of image alignment. +

Installation

A Maven Eclipse project is provided. Make sure you have OpenCV 4.1.1 installed as a User Library in eclipse as 'opencv'. The project should function with OpenCV 3.* as well, however this has not been tested. From cd533698afc413d72637ca8e19531636404cfaec Mon Sep 17 00:00:00 2001 From: Tony Ketcham Date: Sun, 15 Dec 2019 04:07:31 -0600 Subject: [PATCH 5/5] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 675e211..d2ededf 100644 --- a/README.md +++ b/README.md @@ -35,3 +35,5 @@ The algorithm for performing depth mapping considers the highest rate of change

Installation

A Maven Eclipse project is provided. Make sure you have OpenCV 4.1.1 installed as a User Library in eclipse as 'opencv'. The project should function with OpenCV 3.* as well, however this has not been tested. + +The Launcher.java should be ran and the included folder of test images will be referenced to generate a depth map. Feel free to include your own image stack and play around with it!