Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yolo3 INT8 conversion drops mAP incredibly #263

Open
MohamedElsaeidy opened this issue Sep 5, 2021 · 1 comment
Open

yolo3 INT8 conversion drops mAP incredibly #263

MohamedElsaeidy opened this issue Sep 5, 2021 · 1 comment

Comments

@MohamedElsaeidy
Copy link

i changed conf_treshhold from 0.3 to 0.001 and tried calibration with 4951 images and still mAP is very bad compared to int8 of yolo4 , the mAP just increased for all models because of conf_tresh except that nothing changed and still bad for yolo3 int8. any suggestion or explanation for what causing that would be greatful. thanks
note : all other conversions for fp32 16 int8 for yolo3 and yolo4 are good and very close to published results

Classes: 80 mAP 0.5:	0.27588
Classes: 80 mAP 0.55:	0.249992
Classes: 80 mAP 0.6:	0.216834
Classes: 80 mAP 0.65:	0.17984
Classes: 80 mAP 0.7:	0.137224
Classes: 80 mAP 0.75:	0.0899831
Classes: 80 mAP 0.8:	0.048967
Classes: 80 mAP 0.85:	0.0190462
Classes: 80 mAP 0.9:	0.00424098
Classes: 80 mAP 0.95:	0.000314722
mAP 0.5:0.95 = 0.122232
avg precision: 0.119544	avg recall: 0.523164	avg f1 score:0.194617
@mive93
Copy link
Collaborator

mive93 commented Nov 25, 2021

Hi @MohamedElsaeidy,
It is normal a drop of performance when going from float (even half precision) to int8. I do agree that the jump is very big. Here I can attach a chart with several comparison we did for a journal paper (still under review), in which you can find a lor of information about different platforms and network for all the data types.
Hope it helps.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants