-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathHCQ_LSMDC_t0.15.txt
2577 lines (2577 loc) · 189 KB
/
HCQ_LSMDC_t0.15.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Experiment directory: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15
Preparing the dataloaders ...
Loading dataset LSMDC_full_trainval in ram ...
Finish loading dataset LSMDC_full_trainval in ram, taking 9645.358357191086 s.
Loading dataset LSMDC_full_test in ram ...
Finish loading dataset LSMDC_full_test in ram, taking 30.445743560791016 s.
Loading dataset LSMDC_full_test in ram ...
Finish loading dataset LSMDC_full_test in ram, taking 25.316885471343994 s.
Training ...
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch0.pth ...
Done in 1.595s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch0.pth ...
Done in 3.147s
epoch : 0
loss : 0
learning_rate : 5e-05
n_samples : 0
n_steps : 0
LSMDC_full_test/t2v_metrics/R1: 0.0
LSMDC_full_test/t2v_metrics/R5: 0.9
LSMDC_full_test/t2v_metrics/R10: 1.6
LSMDC_full_test/t2v_metrics/R50: 4.4
LSMDC_full_test/t2v_metrics/MedR: 508.5
LSMDC_full_test/t2v_metrics/MeanR: 502.992
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 0.0
LSMDC_full_test/v2t_metrics/R1: 0.0
LSMDC_full_test/v2t_metrics/R5: 0.3
LSMDC_full_test/v2t_metrics/R10: 0.9
LSMDC_full_test/v2t_metrics/R50: 5.1
LSMDC_full_test/v2t_metrics/MedR: 510.0
LSMDC_full_test/v2t_metrics/MeanR: 501.125
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 0.0
mnt_best : 0.0
not_improved_count: 0
Train Epoch: 1 [1/250 128/32000 (0%)] Loss: 9.71710 (QuantReg: 22.49566) QuantErr: 22.49566 batch_time=29.56103
Train Epoch: 1 [12/250 1536/32000 (5%)] Loss: 9.18973 (QuantReg: 22.60687) QuantErr: 22.60687 batch_time=0.47437
Train Epoch: 1 [23/250 2944/32000 (9%)] Loss: 8.82617 (QuantReg: 22.73811) QuantErr: 22.73811 batch_time=0.54501
Train Epoch: 1 [34/250 4352/32000 (14%)] Loss: 8.32536 (QuantReg: 22.70064) QuantErr: 22.70064 batch_time=0.49812
Train Epoch: 1 [45/250 5760/32000 (18%)] Loss: 8.06051 (QuantReg: 22.70689) QuantErr: 22.70689 batch_time=0.53759
Train Epoch: 1 [56/250 7168/32000 (22%)] Loss: 7.71277 (QuantReg: 22.69812) QuantErr: 22.69812 batch_time=0.47444
Train Epoch: 1 [67/250 8576/32000 (27%)] Loss: 7.53741 (QuantReg: 22.66855) QuantErr: 22.66855 batch_time=0.47615
Train Epoch: 1 [78/250 9984/32000 (31%)] Loss: 7.28889 (QuantReg: 22.68623) QuantErr: 22.68623 batch_time=0.47750
Train Epoch: 1 [89/250 11392/32000 (36%)] Loss: 7.30632 (QuantReg: 22.71209) QuantErr: 22.71209 batch_time=0.49529
Train Epoch: 1 [100/250 12800/32000 (40%)] Loss: 6.98587 (QuantReg: 22.70947) QuantErr: 22.70947 batch_time=0.52636
Train Epoch: 1 [111/250 14208/32000 (44%)] Loss: 7.54202 (QuantReg: 22.68093) QuantErr: 22.68093 batch_time=0.47922
Train Epoch: 1 [122/250 15616/32000 (49%)] Loss: 6.64172 (QuantReg: 22.68304) QuantErr: 22.68304 batch_time=0.50641
Train Epoch: 1 [133/250 17024/32000 (53%)] Loss: 7.10460 (QuantReg: 22.69085) QuantErr: 22.69085 batch_time=0.48092
Train Epoch: 1 [144/250 18432/32000 (58%)] Loss: 7.32375 (QuantReg: 22.67686) QuantErr: 22.67686 batch_time=0.48116
Train Epoch: 1 [155/250 19840/32000 (62%)] Loss: 6.84051 (QuantReg: 22.68848) QuantErr: 22.68848 batch_time=1.22157
Train Epoch: 1 [166/250 21248/32000 (66%)] Loss: 6.59431 (QuantReg: 22.67792) QuantErr: 22.67792 batch_time=0.47792
Train Epoch: 1 [177/250 22656/32000 (71%)] Loss: 7.22143 (QuantReg: 22.66671) QuantErr: 22.66671 batch_time=0.50998
Train Epoch: 1 [188/250 24064/32000 (75%)] Loss: 6.83848 (QuantReg: 22.70028) QuantErr: 22.70028 batch_time=0.47521
Train Epoch: 1 [199/250 25472/32000 (80%)] Loss: 6.60309 (QuantReg: 22.71579) QuantErr: 22.71579 batch_time=0.49079
Train Epoch: 1 [210/250 26880/32000 (84%)] Loss: 6.76599 (QuantReg: 22.68087) QuantErr: 22.68087 batch_time=0.47764
Train Epoch: 1 [221/250 28288/32000 (88%)] Loss: 6.46585 (QuantReg: 22.67828) QuantErr: 22.67828 batch_time=0.47394
Train Epoch: 1 [232/250 29696/32000 (93%)] Loss: 6.31477 (QuantReg: 22.67603) QuantErr: 22.67603 batch_time=0.47699
Train Epoch: 1 [243/250 31104/32000 (97%)] Loss: 7.27618 (QuantReg: 22.70396) QuantErr: 22.70396 batch_time=0.47991
Train Epoch: 1 codebook_update_time=1.86975
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch1.pth ...
Done in 4.290s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch1.pth ...
Done in 9.726s
epoch : 1
loss : 7.367126676559448
quant_reg : 22.681815269470214
quant_err : 22.681815269470214
learning_rate : 5e-05
n_samples : 32000
n_steps : 250
LSMDC_full_test/t2v_metrics/R1: 5.7
LSMDC_full_test/t2v_metrics/R5: 17.3
LSMDC_full_test/t2v_metrics/R10: 25.1
LSMDC_full_test/t2v_metrics/R50: 53.1
LSMDC_full_test/t2v_metrics/MedR: 43.5
LSMDC_full_test/t2v_metrics/MeanR: 108.359
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 13.526898359228937
LSMDC_full_test/v2t_metrics/R1: 5.6
LSMDC_full_test/v2t_metrics/R5: 15.7
LSMDC_full_test/v2t_metrics/R10: 26.1
LSMDC_full_test/v2t_metrics/R50: 52.1
LSMDC_full_test/v2t_metrics/MedR: 46.0
LSMDC_full_test/v2t_metrics/MeanR: 112.55
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 13.189937234375128
mnt_best : 13.526898359228937
not_improved_count: 0
Train Epoch: 2 [1/250 128/32000 (0%)] Loss: 6.50622 (QuantReg: 11.44707) QuantErr: 11.44707 batch_time=32.08988
Train Epoch: 2 [12/250 1536/32000 (5%)] Loss: 6.73625 (QuantReg: 11.32397) QuantErr: 11.32397 batch_time=0.48053
Train Epoch: 2 [23/250 2944/32000 (9%)] Loss: 6.58008 (QuantReg: 11.50163) QuantErr: 11.50163 batch_time=0.48328
Train Epoch: 2 [34/250 4352/32000 (14%)] Loss: 6.28020 (QuantReg: 11.73503) QuantErr: 11.73503 batch_time=0.48080
Train Epoch: 2 [45/250 5760/32000 (18%)] Loss: 6.62647 (QuantReg: 11.39913) QuantErr: 11.39913 batch_time=0.49864
Train Epoch: 2 [56/250 7168/32000 (22%)] Loss: 6.85566 (QuantReg: 11.78711) QuantErr: 11.78711 batch_time=0.48366
Train Epoch: 2 [67/250 8576/32000 (27%)] Loss: 6.76199 (QuantReg: 11.82262) QuantErr: 11.82262 batch_time=0.49497
Train Epoch: 2 [78/250 9984/32000 (31%)] Loss: 6.01967 (QuantReg: 11.83281) QuantErr: 11.83281 batch_time=0.48151
Train Epoch: 2 [89/250 11392/32000 (36%)] Loss: 6.56208 (QuantReg: 12.29935) QuantErr: 12.29935 batch_time=0.48760
Train Epoch: 2 [100/250 12800/32000 (40%)] Loss: 5.59473 (QuantReg: 12.12496) QuantErr: 12.12496 batch_time=0.50081
Train Epoch: 2 [111/250 14208/32000 (44%)] Loss: 6.19784 (QuantReg: 12.13715) QuantErr: 12.13715 batch_time=0.48930
Train Epoch: 2 [122/250 15616/32000 (49%)] Loss: 6.21896 (QuantReg: 12.13993) QuantErr: 12.13993 batch_time=0.52881
Train Epoch: 2 [133/250 17024/32000 (53%)] Loss: 6.54506 (QuantReg: 12.23727) QuantErr: 12.23727 batch_time=0.48487
Train Epoch: 2 [144/250 18432/32000 (58%)] Loss: 6.43012 (QuantReg: 12.10831) QuantErr: 12.10831 batch_time=0.48232
Train Epoch: 2 [155/250 19840/32000 (62%)] Loss: 6.39437 (QuantReg: 12.54952) QuantErr: 12.54952 batch_time=0.49069
Train Epoch: 2 [166/250 21248/32000 (66%)] Loss: 6.21378 (QuantReg: 12.80972) QuantErr: 12.80972 batch_time=0.48126
Train Epoch: 2 [177/250 22656/32000 (71%)] Loss: 6.01245 (QuantReg: 12.37647) QuantErr: 12.37647 batch_time=0.48629
Train Epoch: 2 [188/250 24064/32000 (75%)] Loss: 6.57907 (QuantReg: 12.90391) QuantErr: 12.90391 batch_time=0.53492
Train Epoch: 2 [199/250 25472/32000 (80%)] Loss: 6.56684 (QuantReg: 13.35629) QuantErr: 13.35629 batch_time=0.50382
Train Epoch: 2 [210/250 26880/32000 (84%)] Loss: 6.25166 (QuantReg: 13.03383) QuantErr: 13.03383 batch_time=0.49456
Train Epoch: 2 [221/250 28288/32000 (88%)] Loss: 5.89233 (QuantReg: 12.98828) QuantErr: 12.98828 batch_time=0.49906
Train Epoch: 2 [232/250 29696/32000 (93%)] Loss: 6.42160 (QuantReg: 13.26181) QuantErr: 13.26181 batch_time=0.48571
Train Epoch: 2 [243/250 31104/32000 (97%)] Loss: 6.40181 (QuantReg: 13.44855) QuantErr: 13.44855 batch_time=0.50246
Train Epoch: 2 codebook_update_time=1.97719
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch2.pth ...
Done in 17.569s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch2.pth ...
Done in 23.653s
removing stale ckpt [epoch 1] [took 0.01s]
removing stale ckpt [epoch 0] [took 0.02s]
epoch : 2
loss : 6.341002912521362
quant_reg : 12.290459976196288
quant_err : 12.290459976196288
learning_rate : 4.75e-05
n_samples : 64000
n_steps : 500
LSMDC_full_test/t2v_metrics/R1: 7.9
LSMDC_full_test/t2v_metrics/R5: 20.5
LSMDC_full_test/t2v_metrics/R10: 28.9
LSMDC_full_test/t2v_metrics/R50: 56.3
LSMDC_full_test/t2v_metrics/MedR: 35.0
LSMDC_full_test/t2v_metrics/MeanR: 96.077
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 16.72731614187191
LSMDC_full_test/v2t_metrics/R1: 6.5
LSMDC_full_test/v2t_metrics/R5: 18.8
LSMDC_full_test/v2t_metrics/R10: 29.2
LSMDC_full_test/v2t_metrics/R50: 56.1
LSMDC_full_test/v2t_metrics/MedR: 35.5
LSMDC_full_test/v2t_metrics/MeanR: 97.365
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 15.280985106851114
mnt_best : 16.72731614187191
not_improved_count: 0
Train Epoch: 3 [1/250 128/32000 (0%)] Loss: 6.33051 (QuantReg: 10.85580) QuantErr: 10.85580 batch_time=20.60062
Train Epoch: 3 [12/250 1536/32000 (5%)] Loss: 5.81423 (QuantReg: 10.82702) QuantErr: 10.82702 batch_time=0.47594
Train Epoch: 3 [23/250 2944/32000 (9%)] Loss: 6.21408 (QuantReg: 10.91295) QuantErr: 10.91295 batch_time=0.53731
Train Epoch: 3 [34/250 4352/32000 (14%)] Loss: 5.95452 (QuantReg: 11.20179) QuantErr: 11.20179 batch_time=0.48474
Train Epoch: 3 [45/250 5760/32000 (18%)] Loss: 5.84693 (QuantReg: 10.90394) QuantErr: 10.90394 batch_time=0.49426
Train Epoch: 3 [56/250 7168/32000 (22%)] Loss: 5.94562 (QuantReg: 10.87075) QuantErr: 10.87075 batch_time=0.47461
Train Epoch: 3 [67/250 8576/32000 (27%)] Loss: 6.11143 (QuantReg: 11.06831) QuantErr: 11.06831 batch_time=0.49738
Train Epoch: 3 [78/250 9984/32000 (31%)] Loss: 6.22369 (QuantReg: 11.06476) QuantErr: 11.06476 batch_time=0.48796
Train Epoch: 3 [89/250 11392/32000 (36%)] Loss: 5.94819 (QuantReg: 11.04594) QuantErr: 11.04594 batch_time=0.51775
Train Epoch: 3 [100/250 12800/32000 (40%)] Loss: 5.92513 (QuantReg: 11.39173) QuantErr: 11.39173 batch_time=0.48429
Train Epoch: 3 [111/250 14208/32000 (44%)] Loss: 5.98065 (QuantReg: 10.88783) QuantErr: 10.88783 batch_time=0.51981
Train Epoch: 3 [122/250 15616/32000 (49%)] Loss: 5.89409 (QuantReg: 11.26598) QuantErr: 11.26598 batch_time=0.48403
Train Epoch: 3 [133/250 17024/32000 (53%)] Loss: 5.87489 (QuantReg: 11.38821) QuantErr: 11.38821 batch_time=0.48336
Train Epoch: 3 [144/250 18432/32000 (58%)] Loss: 6.00836 (QuantReg: 11.39621) QuantErr: 11.39621 batch_time=0.47406
Train Epoch: 3 [155/250 19840/32000 (62%)] Loss: 5.64509 (QuantReg: 11.30355) QuantErr: 11.30355 batch_time=0.49403
Train Epoch: 3 [166/250 21248/32000 (66%)] Loss: 6.12932 (QuantReg: 11.42398) QuantErr: 11.42398 batch_time=0.49010
Train Epoch: 3 [177/250 22656/32000 (71%)] Loss: 5.66045 (QuantReg: 11.58479) QuantErr: 11.58479 batch_time=0.48853
Train Epoch: 3 [188/250 24064/32000 (75%)] Loss: 6.13395 (QuantReg: 11.61698) QuantErr: 11.61698 batch_time=0.49792
Train Epoch: 3 [199/250 25472/32000 (80%)] Loss: 5.72257 (QuantReg: 11.70251) QuantErr: 11.70251 batch_time=0.47191
Train Epoch: 3 [210/250 26880/32000 (84%)] Loss: 6.07965 (QuantReg: 11.72274) QuantErr: 11.72274 batch_time=0.48963
Train Epoch: 3 [221/250 28288/32000 (88%)] Loss: 5.62825 (QuantReg: 11.51540) QuantErr: 11.51540 batch_time=0.48925
Train Epoch: 3 [232/250 29696/32000 (93%)] Loss: 5.74400 (QuantReg: 11.59185) QuantErr: 11.59185 batch_time=0.47728
Train Epoch: 3 [243/250 31104/32000 (97%)] Loss: 5.79738 (QuantReg: 11.43755) QuantErr: 11.43755 batch_time=0.47707
Train Epoch: 3 codebook_update_time=2.38114
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch3.pth ...
Done in 4.462s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch3.pth ...
Done in 8.289s
removing stale ckpt [epoch 2] [took 0.00s]
epoch : 3
loss : 5.9782824783325195
quant_reg : 11.279015693664551
quant_err : 11.279015693664551
learning_rate : 4.5125e-05
n_samples : 96000
n_steps : 750
LSMDC_full_test/t2v_metrics/R1: 8.5
LSMDC_full_test/t2v_metrics/R5: 21.0
LSMDC_full_test/t2v_metrics/R10: 29.7
LSMDC_full_test/t2v_metrics/R50: 58.9
LSMDC_full_test/t2v_metrics/MedR: 33.5
LSMDC_full_test/t2v_metrics/MeanR: 92.922
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 17.43672386417165
LSMDC_full_test/v2t_metrics/R1: 7.4
LSMDC_full_test/v2t_metrics/R5: 21.3
LSMDC_full_test/v2t_metrics/R10: 28.8
LSMDC_full_test/v2t_metrics/R50: 56.6
LSMDC_full_test/v2t_metrics/MedR: 37.0
LSMDC_full_test/v2t_metrics/MeanR: 101.097
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 16.55774806597231
mnt_best : 17.43672386417165
not_improved_count: 0
Train Epoch: 4 [1/250 128/32000 (0%)] Loss: 5.85846 (QuantReg: 10.77897) QuantErr: 10.77897 batch_time=27.03363
Train Epoch: 4 [12/250 1536/32000 (5%)] Loss: 5.76024 (QuantReg: 11.38582) QuantErr: 11.38582 batch_time=3.95174
Train Epoch: 4 [23/250 2944/32000 (9%)] Loss: 5.87898 (QuantReg: 11.16458) QuantErr: 11.16458 batch_time=0.51514
Train Epoch: 4 [34/250 4352/32000 (14%)] Loss: 5.86864 (QuantReg: 10.95845) QuantErr: 10.95845 batch_time=0.48565
Train Epoch: 4 [45/250 5760/32000 (18%)] Loss: 5.41465 (QuantReg: 11.09135) QuantErr: 11.09135 batch_time=1.28686
Train Epoch: 4 [56/250 7168/32000 (22%)] Loss: 5.35887 (QuantReg: 10.93962) QuantErr: 10.93962 batch_time=0.47603
Train Epoch: 4 [67/250 8576/32000 (27%)] Loss: 5.76790 (QuantReg: 11.34617) QuantErr: 11.34617 batch_time=0.50737
Train Epoch: 4 [78/250 9984/32000 (31%)] Loss: 6.03725 (QuantReg: 10.93490) QuantErr: 10.93490 batch_time=0.53408
Train Epoch: 4 [89/250 11392/32000 (36%)] Loss: 5.35153 (QuantReg: 10.77799) QuantErr: 10.77799 batch_time=0.46874
Train Epoch: 4 [100/250 12800/32000 (40%)] Loss: 5.46467 (QuantReg: 10.77760) QuantErr: 10.77760 batch_time=0.48242
Train Epoch: 4 [111/250 14208/32000 (44%)] Loss: 6.10454 (QuantReg: 11.05537) QuantErr: 11.05537 batch_time=0.49022
Train Epoch: 4 [122/250 15616/32000 (49%)] Loss: 5.15785 (QuantReg: 10.73653) QuantErr: 10.73653 batch_time=0.48988
Train Epoch: 4 [133/250 17024/32000 (53%)] Loss: 5.75944 (QuantReg: 10.79535) QuantErr: 10.79535 batch_time=0.47836
Train Epoch: 4 [144/250 18432/32000 (58%)] Loss: 5.85988 (QuantReg: 11.42102) QuantErr: 11.42102 batch_time=0.48130
Train Epoch: 4 [155/250 19840/32000 (62%)] Loss: 5.82281 (QuantReg: 11.10143) QuantErr: 11.10143 batch_time=0.47618
Train Epoch: 4 [166/250 21248/32000 (66%)] Loss: 6.08971 (QuantReg: 11.15610) QuantErr: 11.15610 batch_time=0.71455
Train Epoch: 4 [177/250 22656/32000 (71%)] Loss: 5.64958 (QuantReg: 11.22878) QuantErr: 11.22878 batch_time=0.48514
Train Epoch: 4 [188/250 24064/32000 (75%)] Loss: 6.11444 (QuantReg: 11.24711) QuantErr: 11.24711 batch_time=0.49057
Train Epoch: 4 [199/250 25472/32000 (80%)] Loss: 5.48216 (QuantReg: 11.49174) QuantErr: 11.49174 batch_time=0.48125
Train Epoch: 4 [210/250 26880/32000 (84%)] Loss: 5.44464 (QuantReg: 11.08046) QuantErr: 11.08046 batch_time=0.49665
Train Epoch: 4 [221/250 28288/32000 (88%)] Loss: 4.97792 (QuantReg: 11.08649) QuantErr: 11.08649 batch_time=0.47970
Train Epoch: 4 [232/250 29696/32000 (93%)] Loss: 5.31691 (QuantReg: 11.23397) QuantErr: 11.23397 batch_time=0.49139
Train Epoch: 4 [243/250 31104/32000 (97%)] Loss: 5.64402 (QuantReg: 11.27679) QuantErr: 11.27679 batch_time=0.48918
Train Epoch: 4 codebook_update_time=1.61666
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch4.pth ...
Done in 3.984s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch4.pth ...
Done in 7.858s
removing stale ckpt [epoch 3] [took 0.00s]
epoch : 4
loss : 5.716501459121704
quant_reg : 11.13209215927124
quant_err : 11.13209215927124
learning_rate : 4.2868749999999995e-05
n_samples : 128000
n_steps : 1000
LSMDC_full_test/t2v_metrics/R1: 8.8
LSMDC_full_test/t2v_metrics/R5: 21.9
LSMDC_full_test/t2v_metrics/R10: 30.4
LSMDC_full_test/t2v_metrics/R50: 59.8
LSMDC_full_test/t2v_metrics/MedR: 30.0
LSMDC_full_test/t2v_metrics/MeanR: 92.431
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 18.027415014313593
LSMDC_full_test/v2t_metrics/R1: 8.8
LSMDC_full_test/v2t_metrics/R5: 19.3
LSMDC_full_test/v2t_metrics/R10: 29.4
LSMDC_full_test/v2t_metrics/R50: 59.2
LSMDC_full_test/v2t_metrics/MedR: 32.5
LSMDC_full_test/v2t_metrics/MeanR: 94.789
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 17.092113596055956
mnt_best : 18.027415014313593
not_improved_count: 0
Train Epoch: 5 [1/250 128/32000 (0%)] Loss: 6.26099 (QuantReg: 10.94051) QuantErr: 10.94051 batch_time=24.13427
Train Epoch: 5 [12/250 1536/32000 (5%)] Loss: 5.37615 (QuantReg: 11.19824) QuantErr: 11.19824 batch_time=0.52058
Train Epoch: 5 [23/250 2944/32000 (9%)] Loss: 5.55258 (QuantReg: 10.65667) QuantErr: 10.65667 batch_time=0.52319
Train Epoch: 5 [34/250 4352/32000 (14%)] Loss: 5.91383 (QuantReg: 10.97277) QuantErr: 10.97277 batch_time=0.49655
Train Epoch: 5 [45/250 5760/32000 (18%)] Loss: 5.70699 (QuantReg: 11.31409) QuantErr: 11.31409 batch_time=0.49602
Train Epoch: 5 [56/250 7168/32000 (22%)] Loss: 5.21040 (QuantReg: 11.35093) QuantErr: 11.35093 batch_time=0.49259
Train Epoch: 5 [67/250 8576/32000 (27%)] Loss: 5.38434 (QuantReg: 11.23572) QuantErr: 11.23572 batch_time=0.49646
Train Epoch: 5 [78/250 9984/32000 (31%)] Loss: 5.58212 (QuantReg: 11.04352) QuantErr: 11.04352 batch_time=0.49382
Train Epoch: 5 [89/250 11392/32000 (36%)] Loss: 6.09718 (QuantReg: 11.07474) QuantErr: 11.07474 batch_time=0.49013
Train Epoch: 5 [100/250 12800/32000 (40%)] Loss: 5.73843 (QuantReg: 10.74488) QuantErr: 10.74488 batch_time=0.49085
Train Epoch: 5 [111/250 14208/32000 (44%)] Loss: 5.71190 (QuantReg: 10.90643) QuantErr: 10.90643 batch_time=0.47791
Train Epoch: 5 [122/250 15616/32000 (49%)] Loss: 5.39694 (QuantReg: 11.11671) QuantErr: 11.11671 batch_time=0.46776
Train Epoch: 5 [133/250 17024/32000 (53%)] Loss: 5.24226 (QuantReg: 11.01890) QuantErr: 11.01890 batch_time=0.49439
Train Epoch: 5 [144/250 18432/32000 (58%)] Loss: 5.51803 (QuantReg: 11.42813) QuantErr: 11.42813 batch_time=0.52957
Train Epoch: 5 [155/250 19840/32000 (62%)] Loss: 5.89568 (QuantReg: 11.30925) QuantErr: 11.30925 batch_time=0.52240
Train Epoch: 5 [166/250 21248/32000 (66%)] Loss: 5.76822 (QuantReg: 11.15112) QuantErr: 11.15112 batch_time=0.48450
Train Epoch: 5 [177/250 22656/32000 (71%)] Loss: 5.33773 (QuantReg: 10.98212) QuantErr: 10.98212 batch_time=0.49357
Train Epoch: 5 [188/250 24064/32000 (75%)] Loss: 5.48880 (QuantReg: 11.39615) QuantErr: 11.39615 batch_time=0.51177
Train Epoch: 5 [199/250 25472/32000 (80%)] Loss: 5.31554 (QuantReg: 11.01263) QuantErr: 11.01263 batch_time=0.50892
Train Epoch: 5 [210/250 26880/32000 (84%)] Loss: 5.44159 (QuantReg: 10.82332) QuantErr: 10.82332 batch_time=0.53901
Train Epoch: 5 [221/250 28288/32000 (88%)] Loss: 5.34885 (QuantReg: 11.29281) QuantErr: 11.29281 batch_time=0.53298
Train Epoch: 5 [232/250 29696/32000 (93%)] Loss: 5.47789 (QuantReg: 11.01173) QuantErr: 11.01173 batch_time=0.48120
Train Epoch: 5 [243/250 31104/32000 (97%)] Loss: 5.15132 (QuantReg: 11.16387) QuantErr: 11.16387 batch_time=0.49409
Train Epoch: 5 codebook_update_time=1.88657
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch5.pth ...
Done in 4.120s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch5.pth ...
Done in 10.490s
removing stale ckpt [epoch 4] [took 0.03s]
epoch : 5
loss : 5.520368333816529
quant_reg : 11.080285140991212
quant_err : 11.080285140991212
learning_rate : 4.072531249999999e-05
n_samples : 160000
n_steps : 1250
LSMDC_full_test/t2v_metrics/R1: 9.2
LSMDC_full_test/t2v_metrics/R5: 22.8
LSMDC_full_test/t2v_metrics/R10: 30.4
LSMDC_full_test/t2v_metrics/R50: 59.1
LSMDC_full_test/t2v_metrics/MedR: 30.0
LSMDC_full_test/t2v_metrics/MeanR: 89.878
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 18.543800768261733
LSMDC_full_test/v2t_metrics/R1: 8.8
LSMDC_full_test/v2t_metrics/R5: 22.6
LSMDC_full_test/v2t_metrics/R10: 33.0
LSMDC_full_test/v2t_metrics/R50: 59.8
LSMDC_full_test/v2t_metrics/MedR: 30.0
LSMDC_full_test/v2t_metrics/MeanR: 92.904
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.7226944766397
mnt_best : 18.543800768261733
not_improved_count: 0
Train Epoch: 6 [1/250 128/32000 (0%)] Loss: 5.83551 (QuantReg: 10.98341) QuantErr: 10.98341 batch_time=23.36763
Train Epoch: 6 [12/250 1536/32000 (5%)] Loss: 5.59137 (QuantReg: 10.99776) QuantErr: 10.99776 batch_time=0.50547
Train Epoch: 6 [23/250 2944/32000 (9%)] Loss: 5.60818 (QuantReg: 11.04974) QuantErr: 11.04974 batch_time=0.56364
Train Epoch: 6 [34/250 4352/32000 (14%)] Loss: 5.29257 (QuantReg: 10.87873) QuantErr: 10.87873 batch_time=0.61121
Train Epoch: 6 [45/250 5760/32000 (18%)] Loss: 5.21669 (QuantReg: 10.92138) QuantErr: 10.92138 batch_time=0.49418
Train Epoch: 6 [56/250 7168/32000 (22%)] Loss: 5.45610 (QuantReg: 10.83871) QuantErr: 10.83871 batch_time=0.50486
Train Epoch: 6 [67/250 8576/32000 (27%)] Loss: 5.24505 (QuantReg: 10.84973) QuantErr: 10.84973 batch_time=0.90163
Train Epoch: 6 [78/250 9984/32000 (31%)] Loss: 5.37546 (QuantReg: 10.93071) QuantErr: 10.93071 batch_time=0.48502
Train Epoch: 6 [89/250 11392/32000 (36%)] Loss: 5.67514 (QuantReg: 11.08050) QuantErr: 11.08050 batch_time=0.49437
Train Epoch: 6 [100/250 12800/32000 (40%)] Loss: 5.58387 (QuantReg: 11.22445) QuantErr: 11.22445 batch_time=0.48463
Train Epoch: 6 [111/250 14208/32000 (44%)] Loss: 5.02791 (QuantReg: 11.19758) QuantErr: 11.19758 batch_time=0.49590
Train Epoch: 6 [122/250 15616/32000 (49%)] Loss: 5.48753 (QuantReg: 10.85512) QuantErr: 10.85512 batch_time=0.52963
Train Epoch: 6 [133/250 17024/32000 (53%)] Loss: 5.45294 (QuantReg: 11.25981) QuantErr: 11.25981 batch_time=0.51670
Train Epoch: 6 [144/250 18432/32000 (58%)] Loss: 5.82234 (QuantReg: 10.98038) QuantErr: 10.98038 batch_time=0.51000
Train Epoch: 6 [155/250 19840/32000 (62%)] Loss: 5.40684 (QuantReg: 11.15341) QuantErr: 11.15341 batch_time=0.80887
Train Epoch: 6 [166/250 21248/32000 (66%)] Loss: 4.80610 (QuantReg: 10.95026) QuantErr: 10.95026 batch_time=0.52945
Train Epoch: 6 [177/250 22656/32000 (71%)] Loss: 5.14492 (QuantReg: 11.00790) QuantErr: 11.00790 batch_time=0.51562
Train Epoch: 6 [188/250 24064/32000 (75%)] Loss: 5.47555 (QuantReg: 11.62380) QuantErr: 11.62380 batch_time=0.49123
Train Epoch: 6 [199/250 25472/32000 (80%)] Loss: 5.68691 (QuantReg: 10.75230) QuantErr: 10.75230 batch_time=0.53524
Train Epoch: 6 [210/250 26880/32000 (84%)] Loss: 5.37697 (QuantReg: 10.90503) QuantErr: 10.90503 batch_time=4.39971
Train Epoch: 6 [221/250 28288/32000 (88%)] Loss: 5.23168 (QuantReg: 11.25914) QuantErr: 11.25914 batch_time=0.50055
Train Epoch: 6 [232/250 29696/32000 (93%)] Loss: 5.73429 (QuantReg: 11.15513) QuantErr: 11.15513 batch_time=0.50337
Train Epoch: 6 [243/250 31104/32000 (97%)] Loss: 5.58044 (QuantReg: 11.15865) QuantErr: 11.15865 batch_time=0.48914
Train Epoch: 6 codebook_update_time=1.66559
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch6.pth ...
Done in 22.235s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch6.pth ...
Done in 28.322s
removing stale ckpt [epoch 5] [took 0.12s]
epoch : 6
loss : 5.409413915634155
quant_reg : 11.03912956237793
quant_err : 11.03912956237793
learning_rate : 3.868904687499999e-05
n_samples : 192000
n_steps : 1500
LSMDC_full_test/t2v_metrics/R1: 9.4
LSMDC_full_test/t2v_metrics/R5: 23.1
LSMDC_full_test/t2v_metrics/R10: 31.0
LSMDC_full_test/t2v_metrics/R50: 61.1
LSMDC_full_test/t2v_metrics/MedR: 29.0
LSMDC_full_test/t2v_metrics/MeanR: 88.053
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 18.881384767481777
LSMDC_full_test/v2t_metrics/R1: 8.6
LSMDC_full_test/v2t_metrics/R5: 21.5
LSMDC_full_test/v2t_metrics/R10: 31.6
LSMDC_full_test/v2t_metrics/R50: 59.4
LSMDC_full_test/v2t_metrics/MedR: 32.0
LSMDC_full_test/v2t_metrics/MeanR: 92.42
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.011145360890723
mnt_best : 18.881384767481777
not_improved_count: 0
Train Epoch: 7 [1/250 128/32000 (0%)] Loss: 5.39169 (QuantReg: 10.87350) QuantErr: 10.87350 batch_time=26.69561
Train Epoch: 7 [12/250 1536/32000 (5%)] Loss: 5.32600 (QuantReg: 11.03897) QuantErr: 11.03897 batch_time=0.51486
Train Epoch: 7 [23/250 2944/32000 (9%)] Loss: 5.16841 (QuantReg: 11.27289) QuantErr: 11.27289 batch_time=0.47298
Train Epoch: 7 [34/250 4352/32000 (14%)] Loss: 5.14813 (QuantReg: 10.92648) QuantErr: 10.92648 batch_time=0.47803
Train Epoch: 7 [45/250 5760/32000 (18%)] Loss: 5.59807 (QuantReg: 11.29442) QuantErr: 11.29442 batch_time=0.51467
Train Epoch: 7 [56/250 7168/32000 (22%)] Loss: 5.39241 (QuantReg: 11.23915) QuantErr: 11.23915 batch_time=0.49491
Train Epoch: 7 [67/250 8576/32000 (27%)] Loss: 5.20066 (QuantReg: 10.91700) QuantErr: 10.91700 batch_time=0.47323
Train Epoch: 7 [78/250 9984/32000 (31%)] Loss: 5.40635 (QuantReg: 10.93805) QuantErr: 10.93805 batch_time=0.49421
Train Epoch: 7 [89/250 11392/32000 (36%)] Loss: 5.15357 (QuantReg: 10.87916) QuantErr: 10.87916 batch_time=0.54797
Train Epoch: 7 [100/250 12800/32000 (40%)] Loss: 5.15294 (QuantReg: 10.94274) QuantErr: 10.94274 batch_time=0.48854
Train Epoch: 7 [111/250 14208/32000 (44%)] Loss: 5.08247 (QuantReg: 10.73495) QuantErr: 10.73495 batch_time=0.50878
Train Epoch: 7 [122/250 15616/32000 (49%)] Loss: 5.25231 (QuantReg: 10.74341) QuantErr: 10.74341 batch_time=0.50704
Train Epoch: 7 [133/250 17024/32000 (53%)] Loss: 5.29492 (QuantReg: 10.78145) QuantErr: 10.78145 batch_time=1.00822
Train Epoch: 7 [144/250 18432/32000 (58%)] Loss: 5.10757 (QuantReg: 10.84149) QuantErr: 10.84149 batch_time=0.80167
Train Epoch: 7 [155/250 19840/32000 (62%)] Loss: 5.23554 (QuantReg: 11.25102) QuantErr: 11.25102 batch_time=0.49353
Train Epoch: 7 [166/250 21248/32000 (66%)] Loss: 4.94949 (QuantReg: 11.16003) QuantErr: 11.16003 batch_time=0.56091
Train Epoch: 7 [177/250 22656/32000 (71%)] Loss: 5.30435 (QuantReg: 11.01295) QuantErr: 11.01295 batch_time=0.53562
Train Epoch: 7 [188/250 24064/32000 (75%)] Loss: 5.34992 (QuantReg: 11.00685) QuantErr: 11.00685 batch_time=0.49787
Train Epoch: 7 [199/250 25472/32000 (80%)] Loss: 5.19261 (QuantReg: 10.83882) QuantErr: 10.83882 batch_time=0.51685
Train Epoch: 7 [210/250 26880/32000 (84%)] Loss: 5.14413 (QuantReg: 11.20730) QuantErr: 11.20730 batch_time=0.50520
Train Epoch: 7 [221/250 28288/32000 (88%)] Loss: 5.11513 (QuantReg: 10.92190) QuantErr: 10.92190 batch_time=0.52074
Train Epoch: 7 [232/250 29696/32000 (93%)] Loss: 5.10041 (QuantReg: 10.83746) QuantErr: 10.83746 batch_time=0.47768
Train Epoch: 7 [243/250 31104/32000 (97%)] Loss: 5.13794 (QuantReg: 11.10192) QuantErr: 11.10192 batch_time=0.67534
Train Epoch: 7 codebook_update_time=1.65336
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch7.pth ...
Done in 6.164s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch7.pth ...
Done in 10.813s
removing stale ckpt [epoch 6] [took 0.03s]
epoch : 7
loss : 5.266356428146362
quant_reg : 11.017848743438721
quant_err : 11.017848743438721
learning_rate : 3.675459453124999e-05
n_samples : 224000
n_steps : 1750
LSMDC_full_test/t2v_metrics/R1: 8.9
LSMDC_full_test/t2v_metrics/R5: 23.7
LSMDC_full_test/t2v_metrics/R10: 33.0
LSMDC_full_test/t2v_metrics/R50: 63.2
LSMDC_full_test/t2v_metrics/MedR: 26.0
LSMDC_full_test/t2v_metrics/MeanR: 86.396
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.093436338280135
LSMDC_full_test/v2t_metrics/R1: 8.4
LSMDC_full_test/v2t_metrics/R5: 23.0
LSMDC_full_test/v2t_metrics/R10: 32.6
LSMDC_full_test/v2t_metrics/R50: 62.0
LSMDC_full_test/v2t_metrics/MedR: 27.0
LSMDC_full_test/v2t_metrics/MeanR: 89.881
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.46750565652725
mnt_best : 19.093436338280135
not_improved_count: 0
Train Epoch: 8 [1/250 128/32000 (0%)] Loss: 5.24514 (QuantReg: 10.95127) QuantErr: 10.95127 batch_time=24.80161
Train Epoch: 8 [12/250 1536/32000 (5%)] Loss: 5.11013 (QuantReg: 10.89649) QuantErr: 10.89649 batch_time=0.47906
Train Epoch: 8 [23/250 2944/32000 (9%)] Loss: 5.30828 (QuantReg: 11.17296) QuantErr: 11.17296 batch_time=0.49458
Train Epoch: 8 [34/250 4352/32000 (14%)] Loss: 5.56989 (QuantReg: 11.17038) QuantErr: 11.17038 batch_time=0.48437
Train Epoch: 8 [45/250 5760/32000 (18%)] Loss: 4.89949 (QuantReg: 10.94260) QuantErr: 10.94260 batch_time=0.51545
Train Epoch: 8 [56/250 7168/32000 (22%)] Loss: 5.22466 (QuantReg: 11.41689) QuantErr: 11.41689 batch_time=0.49139
Train Epoch: 8 [67/250 8576/32000 (27%)] Loss: 5.12552 (QuantReg: 11.07041) QuantErr: 11.07041 batch_time=0.49390
Train Epoch: 8 [78/250 9984/32000 (31%)] Loss: 5.48720 (QuantReg: 11.03860) QuantErr: 11.03860 batch_time=0.49359
Train Epoch: 8 [89/250 11392/32000 (36%)] Loss: 5.05653 (QuantReg: 11.20654) QuantErr: 11.20654 batch_time=0.50355
Train Epoch: 8 [100/250 12800/32000 (40%)] Loss: 5.12011 (QuantReg: 10.78263) QuantErr: 10.78263 batch_time=0.52522
Train Epoch: 8 [111/250 14208/32000 (44%)] Loss: 5.44323 (QuantReg: 11.34103) QuantErr: 11.34103 batch_time=0.53452
Train Epoch: 8 [122/250 15616/32000 (49%)] Loss: 5.17016 (QuantReg: 11.23348) QuantErr: 11.23348 batch_time=0.48526
Train Epoch: 8 [133/250 17024/32000 (53%)] Loss: 5.33355 (QuantReg: 11.04569) QuantErr: 11.04569 batch_time=0.51089
Train Epoch: 8 [144/250 18432/32000 (58%)] Loss: 5.04851 (QuantReg: 11.47200) QuantErr: 11.47200 batch_time=0.47714
Train Epoch: 8 [155/250 19840/32000 (62%)] Loss: 5.37006 (QuantReg: 11.26373) QuantErr: 11.26373 batch_time=0.47696
Train Epoch: 8 [166/250 21248/32000 (66%)] Loss: 4.84423 (QuantReg: 10.92799) QuantErr: 10.92799 batch_time=0.48818
Train Epoch: 8 [177/250 22656/32000 (71%)] Loss: 5.34522 (QuantReg: 11.34201) QuantErr: 11.34201 batch_time=0.48581
Train Epoch: 8 [188/250 24064/32000 (75%)] Loss: 5.13280 (QuantReg: 11.22135) QuantErr: 11.22135 batch_time=0.55367
Train Epoch: 8 [199/250 25472/32000 (80%)] Loss: 4.98331 (QuantReg: 10.64417) QuantErr: 10.64417 batch_time=2.83555
Train Epoch: 8 [210/250 26880/32000 (84%)] Loss: 5.27726 (QuantReg: 11.16387) QuantErr: 11.16387 batch_time=0.48403
Train Epoch: 8 [221/250 28288/32000 (88%)] Loss: 4.94624 (QuantReg: 10.98332) QuantErr: 10.98332 batch_time=0.48672
Train Epoch: 8 [232/250 29696/32000 (93%)] Loss: 4.92632 (QuantReg: 11.14739) QuantErr: 11.14739 batch_time=0.48640
Train Epoch: 8 [243/250 31104/32000 (97%)] Loss: 5.24571 (QuantReg: 10.88103) QuantErr: 10.88103 batch_time=0.47956
Train Epoch: 8 codebook_update_time=1.86314
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch8.pth ...
Done in 6.229s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch8.pth ...
Done in 10.866s
removing stale ckpt [epoch 7] [took 0.05s]
epoch : 8
loss : 5.160747375488281
quant_reg : 11.049714584350586
quant_err : 11.049714584350586
learning_rate : 3.4916864804687486e-05
n_samples : 256000
n_steps : 2000
LSMDC_full_test/t2v_metrics/R1: 10.2
LSMDC_full_test/t2v_metrics/R5: 23.7
LSMDC_full_test/t2v_metrics/R10: 33.2
LSMDC_full_test/t2v_metrics/R50: 62.1
LSMDC_full_test/t2v_metrics/MedR: 26.0
LSMDC_full_test/t2v_metrics/MeanR: 86.862
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 20.02145031929873
LSMDC_full_test/v2t_metrics/R1: 8.8
LSMDC_full_test/v2t_metrics/R5: 23.2
LSMDC_full_test/v2t_metrics/R10: 32.4
LSMDC_full_test/v2t_metrics/R50: 60.6
LSMDC_full_test/v2t_metrics/MedR: 27.0
LSMDC_full_test/v2t_metrics/MeanR: 89.428
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.771769913995968
mnt_best : 20.02145031929873
not_improved_count: 0
Train Epoch: 9 [1/250 128/32000 (0%)] Loss: 5.04923 (QuantReg: 10.88604) QuantErr: 10.88604 batch_time=24.86846
Train Epoch: 9 [12/250 1536/32000 (5%)] Loss: 5.13798 (QuantReg: 11.14546) QuantErr: 11.14546 batch_time=0.48211
Train Epoch: 9 [23/250 2944/32000 (9%)] Loss: 5.09154 (QuantReg: 11.05839) QuantErr: 11.05839 batch_time=0.48724
Train Epoch: 9 [34/250 4352/32000 (14%)] Loss: 5.33109 (QuantReg: 11.35043) QuantErr: 11.35043 batch_time=0.54858
Train Epoch: 9 [45/250 5760/32000 (18%)] Loss: 5.29798 (QuantReg: 11.15452) QuantErr: 11.15452 batch_time=0.47945
Train Epoch: 9 [56/250 7168/32000 (22%)] Loss: 4.84963 (QuantReg: 11.40526) QuantErr: 11.40526 batch_time=0.46666
Train Epoch: 9 [67/250 8576/32000 (27%)] Loss: 4.97297 (QuantReg: 10.84142) QuantErr: 10.84142 batch_time=0.47359
Train Epoch: 9 [78/250 9984/32000 (31%)] Loss: 5.41226 (QuantReg: 10.81246) QuantErr: 10.81246 batch_time=0.47611
Train Epoch: 9 [89/250 11392/32000 (36%)] Loss: 5.00942 (QuantReg: 10.93124) QuantErr: 10.93124 batch_time=0.48060
Train Epoch: 9 [100/250 12800/32000 (40%)] Loss: 4.85811 (QuantReg: 11.00487) QuantErr: 11.00487 batch_time=0.48175
Train Epoch: 9 [111/250 14208/32000 (44%)] Loss: 5.64910 (QuantReg: 11.26246) QuantErr: 11.26246 batch_time=0.48950
Train Epoch: 9 [122/250 15616/32000 (49%)] Loss: 5.22422 (QuantReg: 11.41476) QuantErr: 11.41476 batch_time=0.47611
Train Epoch: 9 [133/250 17024/32000 (53%)] Loss: 4.86458 (QuantReg: 10.91126) QuantErr: 10.91126 batch_time=0.53553
Train Epoch: 9 [144/250 18432/32000 (58%)] Loss: 5.05671 (QuantReg: 11.36236) QuantErr: 11.36236 batch_time=2.42661
Train Epoch: 9 [155/250 19840/32000 (62%)] Loss: 5.08585 (QuantReg: 11.03568) QuantErr: 11.03568 batch_time=0.48036
Train Epoch: 9 [166/250 21248/32000 (66%)] Loss: 5.24250 (QuantReg: 11.26312) QuantErr: 11.26312 batch_time=0.48638
Train Epoch: 9 [177/250 22656/32000 (71%)] Loss: 4.84940 (QuantReg: 11.10453) QuantErr: 11.10453 batch_time=0.51484
Train Epoch: 9 [188/250 24064/32000 (75%)] Loss: 4.91171 (QuantReg: 10.62648) QuantErr: 10.62648 batch_time=0.48288
Train Epoch: 9 [199/250 25472/32000 (80%)] Loss: 5.01329 (QuantReg: 11.03498) QuantErr: 11.03498 batch_time=0.69303
Train Epoch: 9 [210/250 26880/32000 (84%)] Loss: 4.91896 (QuantReg: 11.26752) QuantErr: 11.26752 batch_time=0.63049
Train Epoch: 9 [221/250 28288/32000 (88%)] Loss: 5.43960 (QuantReg: 11.39086) QuantErr: 11.39086 batch_time=0.47894
Train Epoch: 9 [232/250 29696/32000 (93%)] Loss: 5.55403 (QuantReg: 11.40265) QuantErr: 11.40265 batch_time=0.50251
Train Epoch: 9 [243/250 31104/32000 (97%)] Loss: 5.04003 (QuantReg: 10.78126) QuantErr: 10.78126 batch_time=0.53286
Train Epoch: 9 codebook_update_time=1.64743
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch9.pth ...
Done in 4.618s
removing stale ckpt [epoch 8] [took 0.00s]
epoch : 9
loss : 5.054706502914429
quant_reg : 11.102381328582764
quant_err : 11.102381328582764
learning_rate : 3.317102156445311e-05
n_samples : 288000
n_steps : 2250
LSMDC_full_test/t2v_metrics/R1: 9.5
LSMDC_full_test/t2v_metrics/R5: 23.7
LSMDC_full_test/t2v_metrics/R10: 32.2
LSMDC_full_test/t2v_metrics/R50: 62.2
LSMDC_full_test/t2v_metrics/MedR: 26.0
LSMDC_full_test/t2v_metrics/MeanR: 84.294
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.354231926317752
LSMDC_full_test/v2t_metrics/R1: 8.6
LSMDC_full_test/v2t_metrics/R5: 23.4
LSMDC_full_test/v2t_metrics/R10: 32.9
LSMDC_full_test/v2t_metrics/R50: 59.7
LSMDC_full_test/v2t_metrics/MedR: 29.0
LSMDC_full_test/v2t_metrics/MeanR: 86.356
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.77745524488577
mnt_best : 20.02145031929873
not_improved_count: 1
Train Epoch: 10 [1/250 128/32000 (0%)] Loss: 4.90302 (QuantReg: 10.93158) QuantErr: 10.93158 batch_time=23.67583
Train Epoch: 10 [12/250 1536/32000 (5%)] Loss: 5.01284 (QuantReg: 10.77275) QuantErr: 10.77275 batch_time=0.48458
Train Epoch: 10 [23/250 2944/32000 (9%)] Loss: 4.84120 (QuantReg: 10.92699) QuantErr: 10.92699 batch_time=0.49198
Train Epoch: 10 [34/250 4352/32000 (14%)] Loss: 4.86849 (QuantReg: 11.09191) QuantErr: 11.09191 batch_time=0.49178
Train Epoch: 10 [45/250 5760/32000 (18%)] Loss: 5.03000 (QuantReg: 11.26808) QuantErr: 11.26808 batch_time=0.48095
Train Epoch: 10 [56/250 7168/32000 (22%)] Loss: 5.01605 (QuantReg: 11.20798) QuantErr: 11.20798 batch_time=0.47630
Train Epoch: 10 [67/250 8576/32000 (27%)] Loss: 4.83766 (QuantReg: 10.56593) QuantErr: 10.56593 batch_time=2.12082
Train Epoch: 10 [78/250 9984/32000 (31%)] Loss: 4.74660 (QuantReg: 10.93652) QuantErr: 10.93652 batch_time=0.48610
Train Epoch: 10 [89/250 11392/32000 (36%)] Loss: 4.95860 (QuantReg: 11.13407) QuantErr: 11.13407 batch_time=0.49844
Train Epoch: 10 [100/250 12800/32000 (40%)] Loss: 5.10516 (QuantReg: 11.30881) QuantErr: 11.30881 batch_time=0.49003
Train Epoch: 10 [111/250 14208/32000 (44%)] Loss: 4.90775 (QuantReg: 11.20363) QuantErr: 11.20363 batch_time=0.49017
Train Epoch: 10 [122/250 15616/32000 (49%)] Loss: 4.87587 (QuantReg: 10.90568) QuantErr: 10.90568 batch_time=0.48349
Train Epoch: 10 [133/250 17024/32000 (53%)] Loss: 4.56295 (QuantReg: 10.83303) QuantErr: 10.83303 batch_time=0.53254
Train Epoch: 10 [144/250 18432/32000 (58%)] Loss: 5.22593 (QuantReg: 11.38857) QuantErr: 11.38857 batch_time=2.27884
Train Epoch: 10 [155/250 19840/32000 (62%)] Loss: 5.36915 (QuantReg: 11.34786) QuantErr: 11.34786 batch_time=0.48465
Train Epoch: 10 [166/250 21248/32000 (66%)] Loss: 5.19518 (QuantReg: 11.41799) QuantErr: 11.41799 batch_time=0.49651
Train Epoch: 10 [177/250 22656/32000 (71%)] Loss: 5.03481 (QuantReg: 10.83109) QuantErr: 10.83109 batch_time=0.52123
Train Epoch: 10 [188/250 24064/32000 (75%)] Loss: 5.09181 (QuantReg: 11.23927) QuantErr: 11.23927 batch_time=0.47941
Train Epoch: 10 [199/250 25472/32000 (80%)] Loss: 4.75602 (QuantReg: 11.25583) QuantErr: 11.25583 batch_time=0.49094
Train Epoch: 10 [210/250 26880/32000 (84%)] Loss: 4.66698 (QuantReg: 10.88666) QuantErr: 10.88666 batch_time=0.48605
Train Epoch: 10 [221/250 28288/32000 (88%)] Loss: 4.87004 (QuantReg: 11.23246) QuantErr: 11.23246 batch_time=0.48158
Train Epoch: 10 [232/250 29696/32000 (93%)] Loss: 5.46685 (QuantReg: 11.27812) QuantErr: 11.27812 batch_time=1.14992
Train Epoch: 10 [243/250 31104/32000 (97%)] Loss: 5.05760 (QuantReg: 11.08228) QuantErr: 11.08228 batch_time=0.49068
Train Epoch: 10 codebook_update_time=1.67916
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch10.pth ...
Done in 4.085s
removing stale ckpt [epoch 9] [took 0.01s]
epoch : 10
loss : 4.980729480743408
quant_reg : 11.086512176513672
quant_err : 11.086512176513672
learning_rate : 3.151247048623045e-05
n_samples : 320000
n_steps : 2500
LSMDC_full_test/t2v_metrics/R1: 9.3
LSMDC_full_test/t2v_metrics/R5: 24.7
LSMDC_full_test/t2v_metrics/R10: 32.5
LSMDC_full_test/t2v_metrics/R50: 61.5
LSMDC_full_test/t2v_metrics/MedR: 29.0
LSMDC_full_test/t2v_metrics/MeanR: 87.271
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.544343529617315
LSMDC_full_test/v2t_metrics/R1: 8.7
LSMDC_full_test/v2t_metrics/R5: 23.1
LSMDC_full_test/v2t_metrics/R10: 32.5
LSMDC_full_test/v2t_metrics/R50: 59.0
LSMDC_full_test/v2t_metrics/MedR: 28.0
LSMDC_full_test/v2t_metrics/MeanR: 88.72
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.692678270276595
mnt_best : 20.02145031929873
not_improved_count: 2
Train Epoch: 11 [1/250 128/32000 (0%)] Loss: 4.59470 (QuantReg: 11.18744) QuantErr: 11.18744 batch_time=24.43238
Train Epoch: 11 [12/250 1536/32000 (5%)] Loss: 5.04912 (QuantReg: 11.28595) QuantErr: 11.28595 batch_time=0.49820
Train Epoch: 11 [23/250 2944/32000 (9%)] Loss: 4.79797 (QuantReg: 10.86326) QuantErr: 10.86326 batch_time=0.55827
Train Epoch: 11 [34/250 4352/32000 (14%)] Loss: 4.88408 (QuantReg: 10.97569) QuantErr: 10.97569 batch_time=0.54495
Train Epoch: 11 [45/250 5760/32000 (18%)] Loss: 4.98300 (QuantReg: 10.92209) QuantErr: 10.92209 batch_time=0.52318
Train Epoch: 11 [56/250 7168/32000 (22%)] Loss: 5.20340 (QuantReg: 11.33174) QuantErr: 11.33174 batch_time=0.50838
Train Epoch: 11 [67/250 8576/32000 (27%)] Loss: 4.97253 (QuantReg: 11.22597) QuantErr: 11.22597 batch_time=0.50068
Train Epoch: 11 [78/250 9984/32000 (31%)] Loss: 4.87295 (QuantReg: 11.10745) QuantErr: 11.10745 batch_time=0.49753
Train Epoch: 11 [89/250 11392/32000 (36%)] Loss: 5.06277 (QuantReg: 11.29645) QuantErr: 11.29645 batch_time=0.49554
Train Epoch: 11 [100/250 12800/32000 (40%)] Loss: 4.71023 (QuantReg: 11.43982) QuantErr: 11.43982 batch_time=0.48919
Train Epoch: 11 [111/250 14208/32000 (44%)] Loss: 4.89645 (QuantReg: 11.24545) QuantErr: 11.24545 batch_time=0.50700
Train Epoch: 11 [122/250 15616/32000 (49%)] Loss: 4.37932 (QuantReg: 11.06679) QuantErr: 11.06679 batch_time=0.50273
Train Epoch: 11 [133/250 17024/32000 (53%)] Loss: 4.98822 (QuantReg: 11.12516) QuantErr: 11.12516 batch_time=0.48222
Train Epoch: 11 [144/250 18432/32000 (58%)] Loss: 4.44999 (QuantReg: 10.97656) QuantErr: 10.97656 batch_time=0.49908
Train Epoch: 11 [155/250 19840/32000 (62%)] Loss: 4.85803 (QuantReg: 10.87352) QuantErr: 10.87352 batch_time=0.50550
Train Epoch: 11 [166/250 21248/32000 (66%)] Loss: 5.26945 (QuantReg: 11.21750) QuantErr: 11.21750 batch_time=0.51820
Train Epoch: 11 [177/250 22656/32000 (71%)] Loss: 4.61681 (QuantReg: 11.02199) QuantErr: 11.02199 batch_time=0.50878
Train Epoch: 11 [188/250 24064/32000 (75%)] Loss: 4.72910 (QuantReg: 10.97766) QuantErr: 10.97766 batch_time=0.51480
Train Epoch: 11 [199/250 25472/32000 (80%)] Loss: 4.99167 (QuantReg: 11.01377) QuantErr: 11.01377 batch_time=0.77867
Train Epoch: 11 [210/250 26880/32000 (84%)] Loss: 5.22680 (QuantReg: 11.22424) QuantErr: 11.22424 batch_time=0.50446
Train Epoch: 11 [221/250 28288/32000 (88%)] Loss: 4.73967 (QuantReg: 10.91967) QuantErr: 10.91967 batch_time=0.48603
Train Epoch: 11 [232/250 29696/32000 (93%)] Loss: 4.54570 (QuantReg: 11.14000) QuantErr: 11.14000 batch_time=0.50345
Train Epoch: 11 [243/250 31104/32000 (97%)] Loss: 4.87792 (QuantReg: 10.82543) QuantErr: 10.82543 batch_time=0.50328
Train Epoch: 11 codebook_update_time=1.62852
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch11.pth ...
Done in 4.937s
removing stale ckpt [epoch 10] [took 0.01s]
epoch : 11
loss : 4.8773953723907475
quant_reg : 11.108657211303711
quant_err : 11.108657211303711
learning_rate : 2.993684696191893e-05
n_samples : 352000
n_steps : 2750
LSMDC_full_test/t2v_metrics/R1: 9.3
LSMDC_full_test/t2v_metrics/R5: 24.4
LSMDC_full_test/t2v_metrics/R10: 32.7
LSMDC_full_test/t2v_metrics/R50: 62.3
LSMDC_full_test/t2v_metrics/MedR: 26.0
LSMDC_full_test/t2v_metrics/MeanR: 85.346
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.504740464853008
LSMDC_full_test/v2t_metrics/R1: 8.2
LSMDC_full_test/v2t_metrics/R5: 21.4
LSMDC_full_test/v2t_metrics/R10: 32.1
LSMDC_full_test/v2t_metrics/R50: 61.4
LSMDC_full_test/v2t_metrics/MedR: 28.0
LSMDC_full_test/v2t_metrics/MeanR: 90.352
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 17.792796816131098
mnt_best : 20.02145031929873
not_improved_count: 3
Train Epoch: 12 [1/250 128/32000 (0%)] Loss: 5.04880 (QuantReg: 10.83587) QuantErr: 10.83587 batch_time=21.13558
Train Epoch: 12 [12/250 1536/32000 (5%)] Loss: 4.77324 (QuantReg: 11.10688) QuantErr: 11.10688 batch_time=0.49483
Train Epoch: 12 [23/250 2944/32000 (9%)] Loss: 5.20595 (QuantReg: 11.04885) QuantErr: 11.04885 batch_time=0.53801
Train Epoch: 12 [34/250 4352/32000 (14%)] Loss: 4.42970 (QuantReg: 10.57173) QuantErr: 10.57173 batch_time=0.52975
Train Epoch: 12 [45/250 5760/32000 (18%)] Loss: 4.78388 (QuantReg: 11.23475) QuantErr: 11.23475 batch_time=0.53025
Train Epoch: 12 [56/250 7168/32000 (22%)] Loss: 4.65524 (QuantReg: 11.14518) QuantErr: 11.14518 batch_time=0.53618
Train Epoch: 12 [67/250 8576/32000 (27%)] Loss: 4.94600 (QuantReg: 11.22237) QuantErr: 11.22237 batch_time=1.05228
Train Epoch: 12 [78/250 9984/32000 (31%)] Loss: 4.88785 (QuantReg: 10.79901) QuantErr: 10.79901 batch_time=0.52693
Train Epoch: 12 [89/250 11392/32000 (36%)] Loss: 4.82543 (QuantReg: 11.18454) QuantErr: 11.18454 batch_time=0.48551
Train Epoch: 12 [100/250 12800/32000 (40%)] Loss: 4.63023 (QuantReg: 10.81866) QuantErr: 10.81866 batch_time=0.49069
Train Epoch: 12 [111/250 14208/32000 (44%)] Loss: 4.67638 (QuantReg: 11.00705) QuantErr: 11.00705 batch_time=0.50068
Train Epoch: 12 [122/250 15616/32000 (49%)] Loss: 4.85664 (QuantReg: 11.12739) QuantErr: 11.12739 batch_time=0.48988
Train Epoch: 12 [133/250 17024/32000 (53%)] Loss: 5.02274 (QuantReg: 11.15062) QuantErr: 11.15062 batch_time=0.56074
Train Epoch: 12 [144/250 18432/32000 (58%)] Loss: 4.83596 (QuantReg: 11.24494) QuantErr: 11.24494 batch_time=0.48664
Train Epoch: 12 [155/250 19840/32000 (62%)] Loss: 4.81159 (QuantReg: 11.08102) QuantErr: 11.08102 batch_time=0.52076
Train Epoch: 12 [166/250 21248/32000 (66%)] Loss: 4.73380 (QuantReg: 11.01250) QuantErr: 11.01250 batch_time=0.51239
Train Epoch: 12 [177/250 22656/32000 (71%)] Loss: 4.38125 (QuantReg: 10.97399) QuantErr: 10.97399 batch_time=0.52415
Train Epoch: 12 [188/250 24064/32000 (75%)] Loss: 5.08254 (QuantReg: 10.98396) QuantErr: 10.98396 batch_time=0.49470
Train Epoch: 12 [199/250 25472/32000 (80%)] Loss: 4.68168 (QuantReg: 11.02164) QuantErr: 11.02164 batch_time=0.53842
Train Epoch: 12 [210/250 26880/32000 (84%)] Loss: 4.91493 (QuantReg: 10.88306) QuantErr: 10.88306 batch_time=0.90993
Train Epoch: 12 [221/250 28288/32000 (88%)] Loss: 4.61789 (QuantReg: 11.02691) QuantErr: 11.02691 batch_time=0.48972
Train Epoch: 12 [232/250 29696/32000 (93%)] Loss: 4.44777 (QuantReg: 11.20759) QuantErr: 11.20759 batch_time=0.60926
Train Epoch: 12 [243/250 31104/32000 (97%)] Loss: 4.77968 (QuantReg: 10.80502) QuantErr: 10.80502 batch_time=0.52289
Train Epoch: 12 codebook_update_time=2.16931
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch12.pth ...
Done in 5.024s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch12.pth ...
Done in 10.289s
removing stale ckpt [epoch 11] [took 0.06s]
epoch : 12
loss : 4.812035808563232
quant_reg : 11.064898151397704
quant_err : 11.064898151397704
learning_rate : 2.844000461382298e-05
n_samples : 384000
n_steps : 3000
LSMDC_full_test/t2v_metrics/R1: 10.5
LSMDC_full_test/t2v_metrics/R5: 24.9
LSMDC_full_test/t2v_metrics/R10: 33.7
LSMDC_full_test/t2v_metrics/R50: 63.8
LSMDC_full_test/t2v_metrics/MedR: 24.0
LSMDC_full_test/t2v_metrics/MeanR: 84.689
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 20.654095574849297
LSMDC_full_test/v2t_metrics/R1: 8.8
LSMDC_full_test/v2t_metrics/R5: 23.2
LSMDC_full_test/v2t_metrics/R10: 33.4
LSMDC_full_test/v2t_metrics/R50: 62.2
LSMDC_full_test/v2t_metrics/MedR: 27.0
LSMDC_full_test/v2t_metrics/MeanR: 86.534
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.962941617211705
mnt_best : 20.654095574849297
not_improved_count: 0
Train Epoch: 13 [1/250 128/32000 (0%)] Loss: 4.63520 (QuantReg: 10.84749) QuantErr: 10.84749 batch_time=23.69547
Train Epoch: 13 [12/250 1536/32000 (5%)] Loss: 4.83579 (QuantReg: 10.77949) QuantErr: 10.77949 batch_time=0.47653
Train Epoch: 13 [23/250 2944/32000 (9%)] Loss: 4.85105 (QuantReg: 11.12430) QuantErr: 11.12430 batch_time=0.51175
Train Epoch: 13 [34/250 4352/32000 (14%)] Loss: 4.47294 (QuantReg: 10.93568) QuantErr: 10.93568 batch_time=0.49605
Train Epoch: 13 [45/250 5760/32000 (18%)] Loss: 4.60281 (QuantReg: 10.94192) QuantErr: 10.94192 batch_time=0.48401
Train Epoch: 13 [56/250 7168/32000 (22%)] Loss: 5.06485 (QuantReg: 11.23863) QuantErr: 11.23863 batch_time=0.49982
Train Epoch: 13 [67/250 8576/32000 (27%)] Loss: 4.87104 (QuantReg: 11.19121) QuantErr: 11.19121 batch_time=0.47690
Train Epoch: 13 [78/250 9984/32000 (31%)] Loss: 4.61625 (QuantReg: 11.19452) QuantErr: 11.19452 batch_time=0.73623
Train Epoch: 13 [89/250 11392/32000 (36%)] Loss: 4.51511 (QuantReg: 11.23454) QuantErr: 11.23454 batch_time=1.57758
Train Epoch: 13 [100/250 12800/32000 (40%)] Loss: 4.81643 (QuantReg: 11.00595) QuantErr: 11.00595 batch_time=0.50169
Train Epoch: 13 [111/250 14208/32000 (44%)] Loss: 4.78503 (QuantReg: 11.00432) QuantErr: 11.00432 batch_time=0.49202
Train Epoch: 13 [122/250 15616/32000 (49%)] Loss: 4.58069 (QuantReg: 11.60167) QuantErr: 11.60167 batch_time=0.48620
Train Epoch: 13 [133/250 17024/32000 (53%)] Loss: 5.04409 (QuantReg: 11.24610) QuantErr: 11.24610 batch_time=0.48847
Train Epoch: 13 [144/250 18432/32000 (58%)] Loss: 5.07017 (QuantReg: 10.97607) QuantErr: 10.97607 batch_time=1.35350
Train Epoch: 13 [155/250 19840/32000 (62%)] Loss: 4.57533 (QuantReg: 11.37423) QuantErr: 11.37423 batch_time=0.48014
Train Epoch: 13 [166/250 21248/32000 (66%)] Loss: 4.63304 (QuantReg: 11.45613) QuantErr: 11.45613 batch_time=0.48405
Train Epoch: 13 [177/250 22656/32000 (71%)] Loss: 4.48388 (QuantReg: 10.87874) QuantErr: 10.87874 batch_time=0.49166
Train Epoch: 13 [188/250 24064/32000 (75%)] Loss: 4.91544 (QuantReg: 11.21373) QuantErr: 11.21373 batch_time=0.55812
Train Epoch: 13 [199/250 25472/32000 (80%)] Loss: 4.84782 (QuantReg: 11.39765) QuantErr: 11.39765 batch_time=0.47638
Train Epoch: 13 [210/250 26880/32000 (84%)] Loss: 4.69618 (QuantReg: 10.81276) QuantErr: 10.81276 batch_time=3.45048
Train Epoch: 13 [221/250 28288/32000 (88%)] Loss: 4.75382 (QuantReg: 10.91885) QuantErr: 10.91885 batch_time=0.49447
Train Epoch: 13 [232/250 29696/32000 (93%)] Loss: 4.66949 (QuantReg: 11.24111) QuantErr: 11.24111 batch_time=0.48745
Train Epoch: 13 [243/250 31104/32000 (97%)] Loss: 4.57641 (QuantReg: 10.99644) QuantErr: 10.99644 batch_time=0.49928
Train Epoch: 13 codebook_update_time=1.69019
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch13.pth ...
Done in 21.960s
removing stale ckpt [epoch 12] [took 0.01s]
epoch : 13
loss : 4.728836727142334
quant_reg : 11.087717575073242
quant_err : 11.087717575073242
learning_rate : 2.7018004383131832e-05
n_samples : 416000
n_steps : 3250
LSMDC_full_test/t2v_metrics/R1: 9.1
LSMDC_full_test/t2v_metrics/R5: 25.3
LSMDC_full_test/t2v_metrics/R10: 34.3
LSMDC_full_test/t2v_metrics/R50: 64.2
LSMDC_full_test/t2v_metrics/MedR: 26.0
LSMDC_full_test/t2v_metrics/MeanR: 83.867
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.913702337912078
LSMDC_full_test/v2t_metrics/R1: 8.3
LSMDC_full_test/v2t_metrics/R5: 25.0
LSMDC_full_test/v2t_metrics/R10: 33.7
LSMDC_full_test/v2t_metrics/R50: 63.5
LSMDC_full_test/v2t_metrics/MedR: 23.0
LSMDC_full_test/v2t_metrics/MeanR: 85.279
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 19.122705379222356
mnt_best : 20.654095574849297
not_improved_count: 1
Train Epoch: 14 [1/250 128/32000 (0%)] Loss: 4.89887 (QuantReg: 11.07141) QuantErr: 11.07141 batch_time=24.69185
Train Epoch: 14 [12/250 1536/32000 (5%)] Loss: 4.42463 (QuantReg: 10.85787) QuantErr: 10.85787 batch_time=0.48904
Train Epoch: 14 [23/250 2944/32000 (9%)] Loss: 4.75310 (QuantReg: 10.72252) QuantErr: 10.72252 batch_time=0.49273
Train Epoch: 14 [34/250 4352/32000 (14%)] Loss: 4.53892 (QuantReg: 10.92860) QuantErr: 10.92860 batch_time=0.49236
Train Epoch: 14 [45/250 5760/32000 (18%)] Loss: 4.53664 (QuantReg: 11.21847) QuantErr: 11.21847 batch_time=0.53294
Train Epoch: 14 [56/250 7168/32000 (22%)] Loss: 4.64639 (QuantReg: 11.33584) QuantErr: 11.33584 batch_time=0.49029
Train Epoch: 14 [67/250 8576/32000 (27%)] Loss: 4.55927 (QuantReg: 10.87339) QuantErr: 10.87339 batch_time=0.48673
Train Epoch: 14 [78/250 9984/32000 (31%)] Loss: 5.15534 (QuantReg: 10.87563) QuantErr: 10.87563 batch_time=0.49330
Train Epoch: 14 [89/250 11392/32000 (36%)] Loss: 4.67738 (QuantReg: 11.28021) QuantErr: 11.28021 batch_time=0.52266
Train Epoch: 14 [100/250 12800/32000 (40%)] Loss: 4.76000 (QuantReg: 11.20373) QuantErr: 11.20373 batch_time=0.48708
Train Epoch: 14 [111/250 14208/32000 (44%)] Loss: 4.52713 (QuantReg: 10.99814) QuantErr: 10.99814 batch_time=1.75314
Train Epoch: 14 [122/250 15616/32000 (49%)] Loss: 4.79900 (QuantReg: 11.14030) QuantErr: 11.14030 batch_time=0.47612
Train Epoch: 14 [133/250 17024/32000 (53%)] Loss: 5.11219 (QuantReg: 11.20989) QuantErr: 11.20989 batch_time=0.48350
Train Epoch: 14 [144/250 18432/32000 (58%)] Loss: 4.63447 (QuantReg: 11.11830) QuantErr: 11.11830 batch_time=0.49009
Train Epoch: 14 [155/250 19840/32000 (62%)] Loss: 4.89528 (QuantReg: 11.35813) QuantErr: 11.35813 batch_time=0.48693
Train Epoch: 14 [166/250 21248/32000 (66%)] Loss: 4.70012 (QuantReg: 10.91117) QuantErr: 10.91117 batch_time=0.55551
Train Epoch: 14 [177/250 22656/32000 (71%)] Loss: 4.46884 (QuantReg: 11.41373) QuantErr: 11.41373 batch_time=0.50589
Train Epoch: 14 [188/250 24064/32000 (75%)] Loss: 4.84815 (QuantReg: 11.32731) QuantErr: 11.32731 batch_time=0.50376
Train Epoch: 14 [199/250 25472/32000 (80%)] Loss: 5.01199 (QuantReg: 10.93006) QuantErr: 10.93006 batch_time=0.49191
Train Epoch: 14 [210/250 26880/32000 (84%)] Loss: 4.98283 (QuantReg: 11.32095) QuantErr: 11.32095 batch_time=0.49231
Train Epoch: 14 [221/250 28288/32000 (88%)] Loss: 4.63394 (QuantReg: 11.52655) QuantErr: 11.52655 batch_time=0.51520
Train Epoch: 14 [232/250 29696/32000 (93%)] Loss: 4.56060 (QuantReg: 11.40315) QuantErr: 11.40315 batch_time=0.59239
Train Epoch: 14 [243/250 31104/32000 (97%)] Loss: 4.61052 (QuantReg: 11.22519) QuantErr: 11.22519 batch_time=0.49101
Train Epoch: 14 codebook_update_time=1.61585
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch14.pth ...
Done in 4.856s
removing stale ckpt [epoch 13] [took 0.00s]
epoch : 14
loss : 4.706697772979736
quant_reg : 11.093114368438721
quant_err : 11.093114368438721
learning_rate : 2.566710416397524e-05
n_samples : 448000
n_steps : 3500
LSMDC_full_test/t2v_metrics/R1: 9.3
LSMDC_full_test/t2v_metrics/R5: 25.2
LSMDC_full_test/t2v_metrics/R10: 35.0
LSMDC_full_test/t2v_metrics/R50: 64.0
LSMDC_full_test/t2v_metrics/MedR: 24.0
LSMDC_full_test/t2v_metrics/MeanR: 86.771
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 20.167427818488918
LSMDC_full_test/v2t_metrics/R1: 8.5
LSMDC_full_test/v2t_metrics/R5: 23.1
LSMDC_full_test/v2t_metrics/R10: 33.4
LSMDC_full_test/v2t_metrics/R50: 63.3
LSMDC_full_test/v2t_metrics/MedR: 24.0
LSMDC_full_test/v2t_metrics/MeanR: 89.754
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.7179862597544
mnt_best : 20.654095574849297
not_improved_count: 2
Train Epoch: 15 [1/250 128/32000 (0%)] Loss: 4.28222 (QuantReg: 10.84751) QuantErr: 10.84751 batch_time=21.66322
Train Epoch: 15 [12/250 1536/32000 (5%)] Loss: 4.57502 (QuantReg: 10.79342) QuantErr: 10.79342 batch_time=0.48320
Train Epoch: 15 [23/250 2944/32000 (9%)] Loss: 4.70496 (QuantReg: 11.26177) QuantErr: 11.26177 batch_time=3.52004
Train Epoch: 15 [34/250 4352/32000 (14%)] Loss: 4.83361 (QuantReg: 11.43044) QuantErr: 11.43044 batch_time=0.48674
Train Epoch: 15 [45/250 5760/32000 (18%)] Loss: 4.38614 (QuantReg: 11.23548) QuantErr: 11.23548 batch_time=0.49936
Train Epoch: 15 [56/250 7168/32000 (22%)] Loss: 4.80140 (QuantReg: 11.14621) QuantErr: 11.14621 batch_time=0.50019
Train Epoch: 15 [67/250 8576/32000 (27%)] Loss: 4.25200 (QuantReg: 11.12575) QuantErr: 11.12575 batch_time=0.50955
Train Epoch: 15 [78/250 9984/32000 (31%)] Loss: 4.59629 (QuantReg: 11.03809) QuantErr: 11.03809 batch_time=0.48652
Train Epoch: 15 [89/250 11392/32000 (36%)] Loss: 4.79184 (QuantReg: 11.06711) QuantErr: 11.06711 batch_time=0.50484
Train Epoch: 15 [100/250 12800/32000 (40%)] Loss: 4.80191 (QuantReg: 11.17681) QuantErr: 11.17681 batch_time=1.14577
Train Epoch: 15 [111/250 14208/32000 (44%)] Loss: 4.47433 (QuantReg: 10.85526) QuantErr: 10.85526 batch_time=0.48413
Train Epoch: 15 [122/250 15616/32000 (49%)] Loss: 4.56169 (QuantReg: 11.29813) QuantErr: 11.29813 batch_time=0.53408
Train Epoch: 15 [133/250 17024/32000 (53%)] Loss: 4.59963 (QuantReg: 11.15989) QuantErr: 11.15989 batch_time=1.59786
Train Epoch: 15 [144/250 18432/32000 (58%)] Loss: 4.67818 (QuantReg: 11.03214) QuantErr: 11.03214 batch_time=0.49091
Train Epoch: 15 [155/250 19840/32000 (62%)] Loss: 4.40768 (QuantReg: 10.99631) QuantErr: 10.99631 batch_time=0.47990
Train Epoch: 15 [166/250 21248/32000 (66%)] Loss: 4.73334 (QuantReg: 11.07315) QuantErr: 11.07315 batch_time=0.48009
Train Epoch: 15 [177/250 22656/32000 (71%)] Loss: 4.82999 (QuantReg: 10.99543) QuantErr: 10.99543 batch_time=0.48836
Train Epoch: 15 [188/250 24064/32000 (75%)] Loss: 4.56464 (QuantReg: 11.07532) QuantErr: 11.07532 batch_time=0.53560
Train Epoch: 15 [199/250 25472/32000 (80%)] Loss: 4.81070 (QuantReg: 10.93379) QuantErr: 10.93379 batch_time=0.48751
Train Epoch: 15 [210/250 26880/32000 (84%)] Loss: 4.47955 (QuantReg: 10.78897) QuantErr: 10.78897 batch_time=0.65136
Train Epoch: 15 [221/250 28288/32000 (88%)] Loss: 4.72247 (QuantReg: 11.25642) QuantErr: 11.25642 batch_time=0.48113
Train Epoch: 15 [232/250 29696/32000 (93%)] Loss: 4.55874 (QuantReg: 11.15457) QuantErr: 11.15457 batch_time=0.49847
Train Epoch: 15 [243/250 31104/32000 (97%)] Loss: 4.63181 (QuantReg: 11.37625) QuantErr: 11.37625 batch_time=0.62188
Train Epoch: 15 codebook_update_time=1.61510
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch15.pth ...
Done in 9.834s
removing stale ckpt [epoch 14] [took 0.13s]
epoch : 15
loss : 4.633275056838989
quant_reg : 11.054161800384522
quant_err : 11.054161800384522
learning_rate : 2.4383748955776477e-05
n_samples : 480000
n_steps : 3750
LSMDC_full_test/t2v_metrics/R1: 9.7
LSMDC_full_test/t2v_metrics/R5: 24.8
LSMDC_full_test/t2v_metrics/R10: 35.9
LSMDC_full_test/t2v_metrics/R50: 64.3
LSMDC_full_test/t2v_metrics/MedR: 24.0
LSMDC_full_test/t2v_metrics/MeanR: 84.397
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 20.51662660646049
LSMDC_full_test/v2t_metrics/R1: 8.9
LSMDC_full_test/v2t_metrics/R5: 24.1
LSMDC_full_test/v2t_metrics/R10: 35.4
LSMDC_full_test/v2t_metrics/R50: 62.5
LSMDC_full_test/v2t_metrics/MedR: 23.0
LSMDC_full_test/v2t_metrics/MeanR: 86.44
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 19.65486674448487
mnt_best : 20.654095574849297
not_improved_count: 3
Train Epoch: 16 [1/250 128/32000 (0%)] Loss: 4.66376 (QuantReg: 11.16746) QuantErr: 11.16746 batch_time=23.46282
Train Epoch: 16 [12/250 1536/32000 (5%)] Loss: 4.46996 (QuantReg: 10.99343) QuantErr: 10.99343 batch_time=0.50030
Train Epoch: 16 [23/250 2944/32000 (9%)] Loss: 4.38707 (QuantReg: 10.72864) QuantErr: 10.72864 batch_time=0.47877
Train Epoch: 16 [34/250 4352/32000 (14%)] Loss: 4.85575 (QuantReg: 10.91952) QuantErr: 10.91952 batch_time=0.48379
Train Epoch: 16 [45/250 5760/32000 (18%)] Loss: 4.94277 (QuantReg: 11.03872) QuantErr: 11.03872 batch_time=0.62923
Train Epoch: 16 [56/250 7168/32000 (22%)] Loss: 4.87123 (QuantReg: 11.30706) QuantErr: 11.30706 batch_time=0.55454
Train Epoch: 16 [67/250 8576/32000 (27%)] Loss: 4.69676 (QuantReg: 11.12057) QuantErr: 11.12057 batch_time=3.52935
Train Epoch: 16 [78/250 9984/32000 (31%)] Loss: 4.85514 (QuantReg: 11.24128) QuantErr: 11.24128 batch_time=0.48951
Train Epoch: 16 [89/250 11392/32000 (36%)] Loss: 4.36587 (QuantReg: 10.94268) QuantErr: 10.94268 batch_time=0.55493
Train Epoch: 16 [100/250 12800/32000 (40%)] Loss: 4.95402 (QuantReg: 11.29232) QuantErr: 11.29232 batch_time=0.51024
Train Epoch: 16 [111/250 14208/32000 (44%)] Loss: 4.54471 (QuantReg: 10.75909) QuantErr: 10.75909 batch_time=0.49247
Train Epoch: 16 [122/250 15616/32000 (49%)] Loss: 4.98580 (QuantReg: 11.01359) QuantErr: 11.01359 batch_time=0.47881
Train Epoch: 16 [133/250 17024/32000 (53%)] Loss: 4.56301 (QuantReg: 11.11020) QuantErr: 11.11020 batch_time=0.49695
Train Epoch: 16 [144/250 18432/32000 (58%)] Loss: 4.70028 (QuantReg: 11.22780) QuantErr: 11.22780 batch_time=0.50248
Train Epoch: 16 [155/250 19840/32000 (62%)] Loss: 4.28703 (QuantReg: 10.98557) QuantErr: 10.98557 batch_time=0.47959
Train Epoch: 16 [166/250 21248/32000 (66%)] Loss: 4.55325 (QuantReg: 11.23693) QuantErr: 11.23693 batch_time=0.50448
Train Epoch: 16 [177/250 22656/32000 (71%)] Loss: 4.44750 (QuantReg: 11.06324) QuantErr: 11.06324 batch_time=0.50426
Train Epoch: 16 [188/250 24064/32000 (75%)] Loss: 4.62380 (QuantReg: 11.20530) QuantErr: 11.20530 batch_time=0.48950
Train Epoch: 16 [199/250 25472/32000 (80%)] Loss: 4.46256 (QuantReg: 10.98505) QuantErr: 10.98505 batch_time=0.49666
Train Epoch: 16 [210/250 26880/32000 (84%)] Loss: 4.72073 (QuantReg: 11.46081) QuantErr: 11.46081 batch_time=0.52544
Train Epoch: 16 [221/250 28288/32000 (88%)] Loss: 4.41742 (QuantReg: 11.31514) QuantErr: 11.31514 batch_time=0.48980
Train Epoch: 16 [232/250 29696/32000 (93%)] Loss: 4.25945 (QuantReg: 11.11543) QuantErr: 11.11543 batch_time=0.49285
Train Epoch: 16 [243/250 31104/32000 (97%)] Loss: 4.68305 (QuantReg: 11.01976) QuantErr: 11.01976 batch_time=0.48758
Train Epoch: 16 codebook_update_time=1.71257
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch16.pth ...
Done in 4.333s
removing stale ckpt [epoch 15] [took 0.01s]
epoch : 16
loss : 4.5990967960357665
quant_reg : 11.090493083953858
quant_err : 11.090493083953858
learning_rate : 2.3164561507987653e-05
n_samples : 512000
n_steps : 4000
LSMDC_full_test/t2v_metrics/R1: 8.9
LSMDC_full_test/t2v_metrics/R5: 26.0
LSMDC_full_test/t2v_metrics/R10: 34.2
LSMDC_full_test/t2v_metrics/R50: 63.2
LSMDC_full_test/t2v_metrics/MedR: 25.0
LSMDC_full_test/t2v_metrics/MeanR: 85.221
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.92797425934118
LSMDC_full_test/v2t_metrics/R1: 8.9
LSMDC_full_test/v2t_metrics/R5: 23.7
LSMDC_full_test/v2t_metrics/R10: 35.0
LSMDC_full_test/v2t_metrics/R50: 61.9
LSMDC_full_test/v2t_metrics/MedR: 25.0
LSMDC_full_test/v2t_metrics/MeanR: 88.287
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 19.4716221016908
mnt_best : 20.654095574849297
not_improved_count: 4
Train Epoch: 17 [1/250 128/32000 (0%)] Loss: 4.51846 (QuantReg: 10.92977) QuantErr: 10.92977 batch_time=18.30030
Train Epoch: 17 [12/250 1536/32000 (5%)] Loss: 4.91519 (QuantReg: 11.22075) QuantErr: 11.22075 batch_time=0.49801
Train Epoch: 17 [23/250 2944/32000 (9%)] Loss: 4.81692 (QuantReg: 11.08659) QuantErr: 11.08659 batch_time=0.68309
Train Epoch: 17 [34/250 4352/32000 (14%)] Loss: 4.26906 (QuantReg: 11.11378) QuantErr: 11.11378 batch_time=0.47475
Train Epoch: 17 [45/250 5760/32000 (18%)] Loss: 4.65863 (QuantReg: 10.86529) QuantErr: 10.86529 batch_time=0.49850
Train Epoch: 17 [56/250 7168/32000 (22%)] Loss: 4.68592 (QuantReg: 10.99250) QuantErr: 10.99250 batch_time=0.47055
Train Epoch: 17 [67/250 8576/32000 (27%)] Loss: 4.23860 (QuantReg: 11.17073) QuantErr: 11.17073 batch_time=0.48937
Train Epoch: 17 [78/250 9984/32000 (31%)] Loss: 4.58272 (QuantReg: 11.12883) QuantErr: 11.12883 batch_time=1.91691
Train Epoch: 17 [89/250 11392/32000 (36%)] Loss: 4.69670 (QuantReg: 11.13222) QuantErr: 11.13222 batch_time=0.47215
Train Epoch: 17 [100/250 12800/32000 (40%)] Loss: 4.54469 (QuantReg: 11.43335) QuantErr: 11.43335 batch_time=0.50452
Train Epoch: 17 [111/250 14208/32000 (44%)] Loss: 4.29313 (QuantReg: 11.06238) QuantErr: 11.06238 batch_time=0.49322
Train Epoch: 17 [122/250 15616/32000 (49%)] Loss: 4.72933 (QuantReg: 11.08659) QuantErr: 11.08659 batch_time=0.48341
Train Epoch: 17 [133/250 17024/32000 (53%)] Loss: 4.39476 (QuantReg: 10.96098) QuantErr: 10.96098 batch_time=0.47536
Train Epoch: 17 [144/250 18432/32000 (58%)] Loss: 4.58715 (QuantReg: 10.97683) QuantErr: 10.97683 batch_time=0.47331
Train Epoch: 17 [155/250 19840/32000 (62%)] Loss: 4.44401 (QuantReg: 11.05829) QuantErr: 11.05829 batch_time=0.51930
Train Epoch: 17 [166/250 21248/32000 (66%)] Loss: 4.19206 (QuantReg: 11.21322) QuantErr: 11.21322 batch_time=0.48988
Train Epoch: 17 [177/250 22656/32000 (71%)] Loss: 4.44395 (QuantReg: 10.77550) QuantErr: 10.77550 batch_time=0.51063
Train Epoch: 17 [188/250 24064/32000 (75%)] Loss: 4.70669 (QuantReg: 11.06840) QuantErr: 11.06840 batch_time=0.47768
Train Epoch: 17 [199/250 25472/32000 (80%)] Loss: 4.45450 (QuantReg: 11.22517) QuantErr: 11.22517 batch_time=0.48458
Train Epoch: 17 [210/250 26880/32000 (84%)] Loss: 4.65882 (QuantReg: 10.98228) QuantErr: 10.98228 batch_time=0.48118
Train Epoch: 17 [221/250 28288/32000 (88%)] Loss: 4.57258 (QuantReg: 11.10611) QuantErr: 11.10611 batch_time=0.51934
Train Epoch: 17 [232/250 29696/32000 (93%)] Loss: 4.96587 (QuantReg: 11.09885) QuantErr: 11.09885 batch_time=2.40601
Train Epoch: 17 [243/250 31104/32000 (97%)] Loss: 4.23406 (QuantReg: 11.17939) QuantErr: 11.17939 batch_time=0.47781
Train Epoch: 17 codebook_update_time=1.65984
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch17.pth ...
Done in 4.034s
removing stale ckpt [epoch 16] [took 0.00s]
epoch : 17
loss : 4.540088962554932
quant_reg : 11.059813217163086
quant_err : 11.059813217163086
learning_rate : 2.2006333432588268e-05
n_samples : 544000
n_steps : 4250
LSMDC_full_test/t2v_metrics/R1: 9.3
LSMDC_full_test/t2v_metrics/R5: 24.5
LSMDC_full_test/t2v_metrics/R10: 33.9
LSMDC_full_test/t2v_metrics/R50: 64.5
LSMDC_full_test/t2v_metrics/MedR: 24.0
LSMDC_full_test/t2v_metrics/MeanR: 86.781
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 19.767401210238017
LSMDC_full_test/v2t_metrics/R1: 8.5
LSMDC_full_test/v2t_metrics/R5: 24.3
LSMDC_full_test/v2t_metrics/R10: 32.9
LSMDC_full_test/v2t_metrics/R50: 63.0
LSMDC_full_test/v2t_metrics/MedR: 23.0
LSMDC_full_test/v2t_metrics/MeanR: 88.49
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 18.941180051414978
mnt_best : 20.654095574849297
not_improved_count: 5
Train Epoch: 18 [1/250 128/32000 (0%)] Loss: 4.76326 (QuantReg: 11.21962) QuantErr: 11.21962 batch_time=22.21489
Train Epoch: 18 [12/250 1536/32000 (5%)] Loss: 4.46111 (QuantReg: 11.27500) QuantErr: 11.27500 batch_time=0.46935
Train Epoch: 18 [23/250 2944/32000 (9%)] Loss: 4.52298 (QuantReg: 11.29671) QuantErr: 11.29671 batch_time=0.47621
Train Epoch: 18 [34/250 4352/32000 (14%)] Loss: 4.40513 (QuantReg: 11.04135) QuantErr: 11.04135 batch_time=0.47509
Train Epoch: 18 [45/250 5760/32000 (18%)] Loss: 4.33769 (QuantReg: 11.11940) QuantErr: 11.11940 batch_time=0.48347
Train Epoch: 18 [56/250 7168/32000 (22%)] Loss: 4.49511 (QuantReg: 11.00338) QuantErr: 11.00338 batch_time=0.51111
Train Epoch: 18 [67/250 8576/32000 (27%)] Loss: 4.40304 (QuantReg: 11.03758) QuantErr: 11.03758 batch_time=0.49520
Train Epoch: 18 [78/250 9984/32000 (31%)] Loss: 4.66923 (QuantReg: 11.16639) QuantErr: 11.16639 batch_time=0.50790
Train Epoch: 18 [89/250 11392/32000 (36%)] Loss: 4.76810 (QuantReg: 10.77540) QuantErr: 10.77540 batch_time=0.48658
Train Epoch: 18 [100/250 12800/32000 (40%)] Loss: 4.36702 (QuantReg: 11.22308) QuantErr: 11.22308 batch_time=0.49288
Train Epoch: 18 [111/250 14208/32000 (44%)] Loss: 4.40465 (QuantReg: 11.12021) QuantErr: 11.12021 batch_time=0.47836
Train Epoch: 18 [122/250 15616/32000 (49%)] Loss: 4.48888 (QuantReg: 11.28202) QuantErr: 11.28202 batch_time=0.48956
Train Epoch: 18 [133/250 17024/32000 (53%)] Loss: 4.28080 (QuantReg: 10.82621) QuantErr: 10.82621 batch_time=0.47432
Train Epoch: 18 [144/250 18432/32000 (58%)] Loss: 4.46756 (QuantReg: 10.98964) QuantErr: 10.98964 batch_time=0.50168
Train Epoch: 18 [155/250 19840/32000 (62%)] Loss: 4.38347 (QuantReg: 11.20918) QuantErr: 11.20918 batch_time=0.48071
Train Epoch: 18 [166/250 21248/32000 (66%)] Loss: 5.09748 (QuantReg: 11.16861) QuantErr: 11.16861 batch_time=0.49044
Train Epoch: 18 [177/250 22656/32000 (71%)] Loss: 4.58803 (QuantReg: 11.21494) QuantErr: 11.21494 batch_time=0.48160
Train Epoch: 18 [188/250 24064/32000 (75%)] Loss: 4.62824 (QuantReg: 10.98453) QuantErr: 10.98453 batch_time=0.49123
Train Epoch: 18 [199/250 25472/32000 (80%)] Loss: 4.35795 (QuantReg: 11.23666) QuantErr: 11.23666 batch_time=0.50613
Train Epoch: 18 [210/250 26880/32000 (84%)] Loss: 4.52444 (QuantReg: 11.07387) QuantErr: 11.07387 batch_time=0.52196
Train Epoch: 18 [221/250 28288/32000 (88%)] Loss: 5.05810 (QuantReg: 11.24998) QuantErr: 11.24998 batch_time=0.52007
Train Epoch: 18 [232/250 29696/32000 (93%)] Loss: 4.51912 (QuantReg: 11.04514) QuantErr: 11.04514 batch_time=0.47951
Train Epoch: 18 [243/250 31104/32000 (97%)] Loss: 4.44701 (QuantReg: 10.48798) QuantErr: 10.48798 batch_time=0.47817
Train Epoch: 18 codebook_update_time=1.66271
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch18.pth ...
Done in 4.717s
removing stale ckpt [epoch 17] [took 0.00s]
epoch : 18
loss : 4.4927446069717405
quant_reg : 11.080208236694336
quant_err : 11.080208236694336
learning_rate : 2.0906016760958855e-05
n_samples : 576000
n_steps : 4500
LSMDC_full_test/t2v_metrics/R1: 9.0
LSMDC_full_test/t2v_metrics/R5: 25.7
LSMDC_full_test/t2v_metrics/R10: 35.3
LSMDC_full_test/t2v_metrics/R50: 64.6
LSMDC_full_test/t2v_metrics/MedR: 23.0
LSMDC_full_test/t2v_metrics/MeanR: 85.641
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 20.136474944557193
LSMDC_full_test/v2t_metrics/R1: 9.6
LSMDC_full_test/v2t_metrics/R5: 24.7
LSMDC_full_test/v2t_metrics/R10: 34.4
LSMDC_full_test/v2t_metrics/R50: 63.9
LSMDC_full_test/v2t_metrics/MedR: 23.75
LSMDC_full_test/v2t_metrics/MeanR: 88.244
LSMDC_full_test/v2t_metrics/geometric_mean_R1-R5-R10: 20.129927448470298
mnt_best : 20.654095574849297
not_improved_count: 6
Train Epoch: 19 [1/250 128/32000 (0%)] Loss: 4.25869 (QuantReg: 10.92252) QuantErr: 10.92252 batch_time=25.80578
Train Epoch: 19 [12/250 1536/32000 (5%)] Loss: 4.52463 (QuantReg: 10.77200) QuantErr: 10.77200 batch_time=0.49219
Train Epoch: 19 [23/250 2944/32000 (9%)] Loss: 4.76998 (QuantReg: 11.01763) QuantErr: 11.01763 batch_time=0.47707
Train Epoch: 19 [34/250 4352/32000 (14%)] Loss: 4.46713 (QuantReg: 11.03913) QuantErr: 11.03913 batch_time=0.48669
Train Epoch: 19 [45/250 5760/32000 (18%)] Loss: 4.34326 (QuantReg: 10.97870) QuantErr: 10.97870 batch_time=0.49275
Train Epoch: 19 [56/250 7168/32000 (22%)] Loss: 4.72272 (QuantReg: 11.22100) QuantErr: 11.22100 batch_time=0.48756
Train Epoch: 19 [67/250 8576/32000 (27%)] Loss: 4.48291 (QuantReg: 11.18823) QuantErr: 11.18823 batch_time=0.50630
Train Epoch: 19 [78/250 9984/32000 (31%)] Loss: 4.33701 (QuantReg: 11.09514) QuantErr: 11.09514 batch_time=0.48374
Train Epoch: 19 [89/250 11392/32000 (36%)] Loss: 4.21632 (QuantReg: 11.24232) QuantErr: 11.24232 batch_time=0.47887
Train Epoch: 19 [100/250 12800/32000 (40%)] Loss: 4.45118 (QuantReg: 11.20202) QuantErr: 11.20202 batch_time=0.47434
Train Epoch: 19 [111/250 14208/32000 (44%)] Loss: 4.52116 (QuantReg: 11.08773) QuantErr: 11.08773 batch_time=0.51041
Train Epoch: 19 [122/250 15616/32000 (49%)] Loss: 4.34653 (QuantReg: 10.87440) QuantErr: 10.87440 batch_time=0.48896
Train Epoch: 19 [133/250 17024/32000 (53%)] Loss: 4.27660 (QuantReg: 11.16139) QuantErr: 11.16139 batch_time=1.56016
Train Epoch: 19 [144/250 18432/32000 (58%)] Loss: 4.33192 (QuantReg: 11.22527) QuantErr: 11.22527 batch_time=1.28312
Train Epoch: 19 [155/250 19840/32000 (62%)] Loss: 4.54473 (QuantReg: 11.07394) QuantErr: 11.07394 batch_time=0.50908
Train Epoch: 19 [166/250 21248/32000 (66%)] Loss: 4.33953 (QuantReg: 11.44012) QuantErr: 11.44012 batch_time=0.48920
Train Epoch: 19 [177/250 22656/32000 (71%)] Loss: 4.41133 (QuantReg: 10.99421) QuantErr: 10.99421 batch_time=0.60546
Train Epoch: 19 [188/250 24064/32000 (75%)] Loss: 4.41377 (QuantReg: 11.10926) QuantErr: 11.10926 batch_time=0.51705
Train Epoch: 19 [199/250 25472/32000 (80%)] Loss: 4.34288 (QuantReg: 11.16237) QuantErr: 11.16237 batch_time=0.49256
Train Epoch: 19 [210/250 26880/32000 (84%)] Loss: 4.58875 (QuantReg: 10.99471) QuantErr: 10.99471 batch_time=0.49776
Train Epoch: 19 [221/250 28288/32000 (88%)] Loss: 4.14051 (QuantReg: 11.02589) QuantErr: 11.02589 batch_time=0.55408
Train Epoch: 19 [232/250 29696/32000 (93%)] Loss: 4.41003 (QuantReg: 11.30227) QuantErr: 11.30227 batch_time=0.52692
Train Epoch: 19 [243/250 31104/32000 (97%)] Loss: 4.30448 (QuantReg: 10.94303) QuantErr: 10.94303 batch_time=0.52457
Train Epoch: 19 codebook_update_time=1.67937
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch19.pth ...
Done in 4.999s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_LSMDC_t0.15/checkpoint-epoch19.pth ...
Done in 9.854s
removing stale ckpt [epoch 18] [took 0.08s]
epoch : 19
loss : 4.438321928024292
quant_reg : 11.086203632354737
quant_err : 11.086203632354737
learning_rate : 1.986071592291091e-05
n_samples : 608000
n_steps : 4750
LSMDC_full_test/t2v_metrics/R1: 10.7
LSMDC_full_test/t2v_metrics/R5: 26.1
LSMDC_full_test/t2v_metrics/R10: 36.0
LSMDC_full_test/t2v_metrics/R50: 64.9
LSMDC_full_test/t2v_metrics/MedR: 23.0
LSMDC_full_test/t2v_metrics/MeanR: 82.808
LSMDC_full_test/t2v_metrics/geometric_mean_R1-R5-R10: 21.582856767932626
LSMDC_full_test/v2t_metrics/R1: 9.1
LSMDC_full_test/v2t_metrics/R5: 24.0
LSMDC_full_test/v2t_metrics/R10: 35.1
LSMDC_full_test/v2t_metrics/R50: 62.8
LSMDC_full_test/v2t_metrics/MedR: 25.0