Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Torch #1

Open
wants to merge 81 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
5f76c4a
poetry added
stroblme Jun 5, 2023
26e32da
torch and kedro viz
stroblme Jun 5, 2023
afec041
pytorch fw
stroblme Jun 5, 2023
47bbc05
comments regarding data content
stroblme Jun 5, 2023
030e2d0
pytorch ae model
stroblme Jun 5, 2023
67b1a53
pt training
stroblme Jun 5, 2023
7b8c166
framework select param
stroblme Jun 5, 2023
cda2f84
default kedro run
stroblme Jun 5, 2023
91cc8f3
disabled model saving
stroblme Jun 6, 2023
ea9321b
data processing and metrics
stroblme Jun 6, 2023
8a73b61
data processing
stroblme Jun 6, 2023
145eb8a
using torchmetrics
stroblme Jun 6, 2023
7500042
added torchmetrics
stroblme Jun 6, 2023
fbc3496
some cleanup
stroblme Jun 6, 2023
df5a52d
added comments and exceptions
stroblme Jun 6, 2023
43e1ff2
more comments
stroblme Jun 6, 2023
da86adb
added tensorflow ssim
stroblme Jun 6, 2023
19fd8e4
adapted for ssim
stroblme Jun 6, 2023
6add56e
refactoring for ssim
stroblme Jun 6, 2023
b731e90
serializable for exporting model
stroblme Jun 6, 2023
9ccb6a8
common batch size def
stroblme Jun 6, 2023
15c4267
refactoring regarding ssim
stroblme Jun 6, 2023
deb3b85
refactoring
stroblme Jun 6, 2023
d4e577c
disabled console printing
stroblme Jun 6, 2023
216f0ec
tuned ae structure
stroblme Jun 6, 2023
4ac8ca4
reduced python req
stroblme Jun 7, 2023
c97bf45
experimental ae
stroblme Jun 7, 2023
2d07085
quotes for python 3.9 compatibility
stroblme Jun 7, 2023
bae1c7e
added plotly
stroblme Jun 7, 2023
7c68617
switched to plotly fw
stroblme Jun 7, 2023
8193039
gc
stroblme Jun 7, 2023
5387266
switched to 0,1,2,3 classes
stroblme Jun 7, 2023
d452cf2
moved relus
stroblme Jun 9, 2023
b9b0675
tuned autoencoder
stroblme Jun 9, 2023
df9db3f
increased epochs
stroblme Jun 9, 2023
07e0b8c
Merge branch 'torch' of github.com:cirKITers/qc_data_preparation into…
stroblme Jun 9, 2023
9c1b29e
splitted up pipeline in processing and science
stroblme Jun 12, 2023
98ab9c9
cleanup
stroblme Jun 12, 2023
80b1d60
implemented describe method
stroblme Jun 12, 2023
b012aac
fixed namespaces
stroblme Jun 12, 2023
f3d0152
new lock file
ulivc Jun 12, 2023
17dd97e
restructuring. splitted postprocessing and training
stroblme Jun 14, 2023
e2fb8e6
Merge branch 'torch' of github.com:cirKITers/qc_data_preparation into…
stroblme Jun 14, 2023
58ce569
fixes for pt model
stroblme Jun 14, 2023
898bf6c
outsourced parameters
stroblme Jun 14, 2023
bf5d851
adjusted catalog
stroblme Jun 14, 2023
ee1acf5
mv method to training node
stroblme Jun 14, 2023
16176bb
replaced method by import
stroblme Jun 14, 2023
7915f9b
implemented html export
stroblme Jun 19, 2023
08af5e1
Merge branch 'main' into torch
eileen-kuehn Jun 19, 2023
cb8de84
Update src/qc_data_preparation/pipelines/preprocessing/__init__.py
stroblme Jun 19, 2023
dcd8933
Update src/qc_data_preparation/pipelines/training/__init__.py
stroblme Jun 19, 2023
923318c
reverted to previous labels
stroblme Jun 19, 2023
eb0ee81
cleanup
stroblme Jun 19, 2023
82b469b
formatting and cleanup
stroblme Jun 19, 2023
6f73e6b
added black config
stroblme Jun 19, 2023
6cd0358
formatting; line length restricted to 88
stroblme Jun 19, 2023
9df8a83
black formatting
stroblme Jun 19, 2023
372051a
added preview to work with flake8
stroblme Jun 19, 2023
c2ac4b5
formatting
stroblme Jun 19, 2023
a1b952c
implemented plotly dataset extension
stroblme Jun 19, 2023
e9b3ef3
copying the curves
stroblme Jun 19, 2023
3fd25a5
separate handling for json and html based files
stroblme Jun 19, 2023
3dc523f
Update pyproject.toml
stroblme Jun 23, 2023
13bc4a0
Update src/qc_data_preparation/helpers/dataset.py
stroblme Jun 23, 2023
16e79ab
Update pyproject.toml
stroblme Jun 23, 2023
3925bdc
rm types
stroblme Jun 23, 2023
f146dcd
build: fix author formatting
eileen-kuehn Jun 24, 2023
f84344c
build: add platform marker for tensorflow dependency
eileen-kuehn Jun 24, 2023
f208be3
build: update poetry lock
eileen-kuehn Jun 24, 2023
270dfd2
Merge branch 'main' into torch
eileen-kuehn Jun 25, 2023
6daeac5
build: adapt libraries for poetry
eileen-kuehn Jun 25, 2023
8fd5625
feat: add abstractdataset to save and load pytorch and tensorflow mod…
eileen-kuehn Jun 25, 2023
c8e2b4c
fix: fix functioning to store plotly html (#8)
eileen-kuehn Jun 25, 2023
67bc13d
fix: fix intendation of error raised
eileen-kuehn Jun 27, 2023
f48b8d1
removed dependency on pt and tf model import
stroblme Jul 3, 2023
e684b7b
added tf and pt default pipeline
stroblme Jul 3, 2023
b549062
added tf default launch config
stroblme Aug 2, 2023
ca2d65c
fixed pipeline spec
stroblme Aug 2, 2023
b62d9f6
gc
stroblme Aug 2, 2023
313311e
added temp. fix to calculate ssim
stroblme Aug 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
added temp. fix to calculate ssim
Signed-off-by: lc3267 <melvin.strobl@kit.edu>
  • Loading branch information
stroblme committed Aug 2, 2023
commit 313311e2fb288aa9a99af7e8df6f222c65fc898e
5 changes: 5 additions & 0 deletions src/qc_data_preparation/pipelines/training/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ def train_tf_model(
# we need that for saving the model (load via {'ssim':ssim})
@tf.keras.utils.register_keras_serializable()
def ssim(pred, target):
# add a BW channel (N, H, W) -> (N, H, W, 1)
# TODO: move that in preprocessing (requires adjustment of model to return channel)
pred = tf.expand_dims(pred, axis=3) if len(pred.shape) == 3 else pred
target = tf.expand_dims(target, axis=3) if len(target.shape) == 3 else target

return tf.image.ssim(
pred,
target,
Expand Down