Bad mIOU tested with provided model

[tengteng95]

Hello! I follow the Readme to test the mIOU with given pretrained caffe model on VOC. However, I get unexpected Mean iou = 0.046470389101241724.
Do you have any suggestions about this problem?
By the way, why the image is only substracted by the mean value of each channel without normlization to range such as [0,1] or [-1,1] in the code?

[tengteng95]

I use the model downloaded from here

[suhangpro]

I am facing the same issue. The number I got is 0.04647076057203757. Using the provided .pth file without any code modification.

[tengteng95]

@suhangpro hi, the problem is that opencv cannot read image in mode 'p' correctly. Reading gt image with PIL.Image will solve the problem.

[suhangpro]

Thanks, @h982639009 ! After that change, I got: pytorch 20 Mean iou = 0.7645974317511359. This is even higher than the number 76.35 reported in the paper (Table. 4 row 6). Someone reported 76.42 in #5 . I don't see any randomness in the forward network inference, so the slight difference is still strange.

[chenyzh28]

@suhangpro, hi, may I ask how do you get the mean iou of 0.7645? Through evalpyt.py or evalpyt2.py? Becausce I got 0.7802 through evalpyt2.py and 0.7180 through evalpyt.py.

[suhangpro]

I did the fix @h982639009 mentioned, and made no other changes.

[chenyzh28]

@suhangpro , do you know how to convert the gt and the output images to color images?

[chenyzh28]

@h982639009 , may I ask what mode 'p' means? How can i can convert images of mode 'p' to RGB images？