[Question] Can one also extract the segmentations?

[dalbis]

❓ Segmentation outputs available?

Does the model outputs also the learned segmentations? or the anchor boxes only?
I do not see the segmentations in the outputs.

Thanks :)

[mibaumgartner]

Dear @dalbis ,

it is theoretically possible to export the learned segmentations (during inference) as well but this is considered a highly experimental feature since we neither evaluated nor tested it -> i.e. it is not supported officially. Nevertheless, the segmentations will be exported by nndet_predict by passing +inference_kwargs.do_seg=True.

Best,
Michael

[dalbis]

Hi,
Dear @mibaumgartner,

Thanks a lot for your reply. I have tried the option suggested and I often got out sensible segmentations. However in some cases the code crashes because there is a problem of dimensions. I noted that this happens only when also the warning WARNING Found patch size which is bigger than data is issued. So maybe there is a problem with the padding. I understand that this feature is not tested/supported, but in case you have any clue how to solve it I would appreciate :)

Thanks a gain.
Tiziano

INFO Predicting case 1 of 1. WARNING This feature (SegmentationEnsembler.__init__) is experimental! It might not implement all features or is only a simplification! WARNING Found patch size which is bigger than data: data (18, 92, 123) patch [ 20 112 112] WARNING Path size is bigger than whole case, padding case to match patch size WARNING Path size is bigger than whole case, padding case to match patch size WARNING Path size is bigger than whole case, padding case to match patch size INFO Predicting model 1 of 5 with weight 1.0. Transform: 0%| | 0/8 [00:00<?, ?INFO Creating new gaussian weight matrix for crop size (20, 112, 112) Crop: 0%| | 0/1 [00:05<?, ?it/s] Transform: 0%| | 0/8 [00:05<?, ?it/s] Traceback (most recent call last): File "/opt/conda/bin/nndet_predict", line 33, in <module> sys.exit(load_entry_point('nndet', 'console_scripts', 'nndet_predict')()) File "/opt/code/nndet/nndet/utils/check.py", line 58, in wrapper return func(*args, **kwargs) File "/opt/code/nndet/scripts/predict.py", line 231, in main run(OmegaConf.to_container(cfg, resolve=True), File "/opt/code/nndet/scripts/predict.py", line 91, in run predict_dir(source_dir=source_dir, File "/opt/code/nndet/nndet/inference/helper.py", line 103, in predict_dir result = predictor.predict_case({"data": case}, File "/opt/code/nndet/nndet/inference/predictor.py", line 177, in predict_case self.predict_tiles(tiles) File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(*args, **kwargs) File "/opt/code/nndet/nndet/inference/predictor.py", line 267, in predict_tiles self.predict_with_transformation( File "/opt/code/nndet/nndet/inference/predictor.py", line 313, in predict_with_transformation ensembler.process_batch(result=result, batch=batch) File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(*args, **kwargs) File "/opt/code/nndet/nndet/inference/ensembler/segmentation.py", line 176, in process_batch self.model_results[case_crop] += seg RuntimeError: The size of tensor a (61) must match the size of tensor b (112) at non-singleton dimension 3

[BelieferQAQ]

Hi, Dear @mibaumgartner,

Thanks a lot for your reply. I have tried the option suggested and I often got out sensible segmentations. However in some cases the code crashes because there is a problem of dimensions. I noted that this happens only when also the warning WARNING Found patch size which is bigger than data is issued. So maybe there is a problem with the padding. I understand that this feature is not tested/supported, but in case you have any clue how to solve it I would appreciate :)

Thanks a gain. Tiziano

INFO Predicting case 1 of 1. WARNING This feature (SegmentationEnsembler.__init__) is experimental! It might not implement all features or is only a simplification! WARNING Found patch size which is bigger than data: data (18, 92, 123) patch [ 20 112 112] WARNING Path size is bigger than whole case, padding case to match patch size WARNING Path size is bigger than whole case, padding case to match patch size WARNING Path size is bigger than whole case, padding case to match patch size INFO Predicting model 1 of 5 with weight 1.0. Transform: 0%| | 0/8 [00:00<?, ?INFO Creating new gaussian weight matrix for crop size (20, 112, 112) Crop: 0%| | 0/1 [00:05<?, ?it/s] Transform: 0%| | 0/8 [00:05<?, ?it/s] Traceback (most recent call last): File "/opt/conda/bin/nndet_predict", line 33, in <module> sys.exit(load_entry_point('nndet', 'console_scripts', 'nndet_predict')()) File "/opt/code/nndet/nndet/utils/check.py", line 58, in wrapper return func(*args, **kwargs) File "/opt/code/nndet/scripts/predict.py", line 231, in main run(OmegaConf.to_container(cfg, resolve=True), File "/opt/code/nndet/scripts/predict.py", line 91, in run predict_dir(source_dir=source_dir, File "/opt/code/nndet/nndet/inference/helper.py", line 103, in predict_dir result = predictor.predict_case({"data": case}, File "/opt/code/nndet/nndet/inference/predictor.py", line 177, in predict_case self.predict_tiles(tiles) File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(*args, **kwargs) File "/opt/code/nndet/nndet/inference/predictor.py", line 267, in predict_tiles self.predict_with_transformation( File "/opt/code/nndet/nndet/inference/predictor.py", line 313, in predict_with_transformation ensembler.process_batch(result=result, batch=batch) File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(*args, **kwargs) File "/opt/code/nndet/nndet/inference/ensembler/segmentation.py", line 176, in process_batch self.model_results[case_crop] += seg RuntimeError: The size of tensor a (61) must match the size of tensor b (112) at non-singleton dimension 3

Generate seg.pkl split file. What are the instructions you use? Thank you very much