[Feature] Support YOLOX-PAI #8778

okotaku · 2022-09-13T06:26:34Z

Motivation

YOLOX-PAI: An Improved YOLOX, Stronger and Faster than YOLOv6
official repo

Related PR

open-mmlab/mmcv#2256
open-mmlab/mmpretrain#1025

Result

Backbone	ASFF	TOOD	box AP
YOLOX-PAI-s	N	N	42.2
YOLOX-PAI-s	Y	N	42.9
YOLOX-PAI-s	Y	Y	43.9

Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
The documentation has been modified accordingly, like docstring or example tutorials.

ZwwWayne · 2022-09-14T06:27:45Z

Hi @okotaku ,
Thanks for your kind PR. It seems that your branch deviates from the dev branch. Could you use git rebase --onto to rebase your modification to dev?

mmdet/models/dense_heads/yolox_tood_head.py

mmdet/models/necks/yolox_asff_pafpn.py

shinya7y · 2022-09-18T11:34:29Z

mmdet/models/necks/yolox_asff_pafpn.py

+            act_cfg=act_cfg,
+        )
+
+    def forward(self, inputs):


feat_heigh, feat_height -> feat_high in yolox_asff_pafpn.py and yolox_pafpn.py

configs/yoloxpai/yolox_pai_asff_tood_s_8x8_300e_coco.py

shinya7y · 2022-09-18T11:46:11Z

configs/yoloxpai/yolox_pai_asff_tood_s_8x8_300e_coco.py

@@ -0,0 +1,4 @@
+_base_ = './yolox_pai_s_8x8_300e_coco.py'
+
+model = dict(bbox_head=dict(type='YOLOXTOODHead'))


Doesn't this config use neck=dict(type='YOLOXASFFPAFPN')?

I made a mistake. I fixed base config.

shinya7y · 2022-09-20T01:19:41Z

It will be better not to remove request for Collaborator reviewers.

okotaku · 2022-09-20T01:39:23Z

Sorry. When I pushed 'Re-request review', Collaborator reviewers were removed.

shinya7y · 2022-09-21T10:07:49Z

Maybe we should wait for the announcement of MMYOLO or MMDet-YOLO before further discussion.
https://zhuanlan.zhihu.com/p/565106513
https://ultralytics.com/yolo-vision

shinya7y · 2022-09-21T16:46:42Z

nn.SiLU vs. mmcv Swish

nn.silu is faster and use less GPU memory, but causes compatibility issues that restrict versions of PyTorch and MMCV.
A possible solution is the following three PRs:
(1) Support SiLU for PyTorch < 1.7 in MMCV, like MMYOLO https://github.com/open-mmlab/mmyolo/blob/84f115c4e57856ab63417fb9b781db88fe10ccf1/mmyolo/models/layers/yolo_bricks.py#L14-L27
(2) Use mmcv Swish in this PR, like other code in MMDetection v2
(3) Replace all mmcv Swish with nn.SiLU in MMDetection v3.0

okotaku · 2022-09-22T08:06:57Z

1st option looks good. I created PR for mmcv.
open-mmlab/mmcv#2278

shinya7y · 2022-09-23T10:43:35Z

Are there any plans or ideas to refactor yolox_asff_pafpn.py and support todo: handle len(self.in_channels) > 3?

okotaku · 2022-09-26T10:42:20Z

Are there any plans or ideas to refactor yolox_asff_pafpn.py and support todo: handle len(self.in_channels) > 3?

Yes. I will implement it soon. Please wait for a few days.

okotaku · 2022-10-03T00:53:28Z

@shinya7y I pushed refactored codes. Please review it.

shinya7y · 2022-10-03T05:42:54Z

mmdet/models/necks/yolox_asff_pafpn.py

+                    mlvl_feats[i],
+                    scale_factor=2**(i - self.level),
+                    mode='nearest')
+            mlvl_wegiths_v.append(self.mlvl_weights[i](mlvl_feats[i]))


I think it is easier to understand to use F.max_pool2d like:

mlvl_weights_v = [] for i, feat in enumerate(x): for _ in range(self.level - i - 1): feat = F.max_pool2d(feat, 3, stride=2, padding=1) feat = self.mlvl_convs[i](feat) if i > self.level: feat = F.interpolate( feat, scale_factor=2**(i - self.level), mode='nearest') mlvl_weights_v.append(self.mlvl_weights[i](feat))

mmdet/models/necks/yolox_asff_pafpn.py

tests/test_models/test_necks.py

shinya7y · 2022-10-08T11:55:34Z

tests/test_models/test_necks.py

@@ -429,6 +430,52 @@ def test_yolox_pafpn():
        assert outs[i].shape[2] == outs[i].shape[3] == s // (2**i)


+def test_yolox_asff_pafpn():
+    s = 32


Add comment # test the case that length of in_channels is 3 before s = 32

configs/yoloxpai/README.md

mmdet/models/necks/yolox_asff_pafpn.py

mmdet/models/dense_heads/yolox_tood_head.py

shinya7y · 2022-10-08T13:55:47Z

README.md

@@ -184,6 +184,7 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
            <li><a href="configs/deformable_detr">Deformable DETR (ICLR'2021)</a></li>
            <li><a href="configs/tood">TOOD (ICCV'2021)</a></li>
            <li><a href="configs/ddod">DDOD (ACM MM'2021)</a></li>
+            <li><a href="configs/yoloxpai">YOLOX-PAI (ArXiv'2022)</a></li>


Update README_zh-CN.md

hhaAndroid · 2022-10-10T03:19:56Z

@okotaku Are you interested in adding related features to mmyolo? And if the algorithm is developed based on dev 3.x, it will be more efficient and simpler to implement.

okotaku · 2022-10-11T05:14:30Z

@hhaAndroid I will try to develop based on dev-3.x. But there may not be time to contribute to mmyolo.

shinya7y · 2022-10-12T01:14:19Z

mmdet/models/necks/yolox_asff_pafpn.py

+        assert len(x) == len(self.in_channels)
+
+        mlvl_spatial_importance_map = []
+        mlvl_feats = []


The new variable names are good and discriminable, but could be shorter.

mlvl_feats = [] mlvl_importance = [] # spatial importance map

mmdet/models/necks/yolox_asff_pafpn.py

shinya7y · 2022-10-12T01:20:55Z

configs/yoloxpai/README.md

+pip install mmcls>=0.24.0
+```
+
+## Usage


Either Usage or Note will be sufficient.

shinya7y · 2022-10-12T04:16:41Z

LGTM.
@hhaAndroid I would appreciate it if you could proceed with remaining tasks (additional review, discussion for 2.x vs. 3.x, retraining, uploading weights, and updating README.md).

hhaAndroid · 2022-11-22T02:57:11Z

OK

hhaAndroid · 2022-11-22T02:58:53Z

LGTM. @hhaAndroid I would appreciate it if you could proceed with remaining tasks (additional review, discussion for 2.x vs. 3.x, retraining, uploading weights, and updating README.md).

Ok. But we'd really like you to create PR directly to the dev-3.x branch

okotaku · 2022-11-22T05:03:49Z

@hhaAndroid
I have already created PR for dev-3.x. Do you want to close this one?
#9255

mm-assistant bot assigned hhaAndroid Sep 13, 2022

okotaku changed the base branch from master to dev September 13, 2022 06:26

okotaku mentioned this pull request Sep 13, 2022

[Enhancement] RepVGG for YOLOX-PAI open-mmlab/mmpretrain#1025

Merged

6 tasks

RangiLyu requested a review from hhaAndroid September 14, 2022 06:07

RangiLyu added the v-2.x label Sep 14, 2022

okotaku force-pushed the yoloxpai branch from 891dc97 to 8cd3db3 Compare September 16, 2022 00:52

add yolox pai

55d1dfe

okotaku force-pushed the yoloxpai branch from 8cd3db3 to 55d1dfe Compare September 16, 2022 00:54

ZwwWayne added this to the 2.27.0 milestone Sep 17, 2022

shinya7y reviewed Sep 18, 2022

View reviewed changes

okotaku added 11 commits September 18, 2022 21:30

fix base config

286cf76

fix tood_norm_cfg docs

9196bb4

_init_tood_layers -> _build_tood_layers

760abb8

inter_convs before decomps

64fbc7a

stride & padding for ConvModules

8665ffd

fix comment

38c9ec4

delete unused

f40624f

feat_heigh, feat_height -> feat_high

cbe6a6c

bias=False and some fix

6887ad2

del find_unused_parameters

c3dfd21

backbone follow mmcls PR

29d314c

okotaku requested review from shinya7y and removed request for hhaAndroid September 20, 2022 00:18

okotaku mentioned this pull request Sep 22, 2022

[Enhancement] Support SiLU with torch < 1.7.0 open-mmlab/mmcv#2278

Merged

7 tasks

okotaku added 2 commits September 27, 2022 20:27

backbone follow mmcls PR

e46123c

refactor neck

02c73d6

shinya7y reviewed Oct 3, 2022

View reviewed changes

fix pooling impl

6d96867

okotaku requested a review from shinya7y October 7, 2022 02:36

shinya7y suggested changes Oct 8, 2022

View reviewed changes

okotaku added 7 commits October 10, 2022 11:04

feat_hight->feat_high

306524e

add comments in test

8d70e1b

fix docs

83bfbbe

omit url

945b9d6

add usage

0b39b96

Update README_zh-CN.md

8d9ddf9

fix param names

eea0f46

okotaku requested a review from shinya7y October 10, 2022 02:47

shinya7y suggested changes Oct 12, 2022

View reviewed changes

okotaku added 2 commits October 12, 2022 12:30

shorter variable name

e97cbc7

del note, keep usage

8d35561

okotaku requested a review from shinya7y October 12, 2022 03:31

okotaku mentioned this pull request Nov 7, 2022

[Feature] Support YOLOX-PAI 3.x #9255

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support YOLOX-PAI #8778

[Feature] Support YOLOX-PAI #8778

okotaku commented Sep 13, 2022 •

edited

Loading

ZwwWayne commented Sep 14, 2022

shinya7y Sep 18, 2022

shinya7y Sep 18, 2022

okotaku Sep 18, 2022

shinya7y commented Sep 20, 2022

okotaku commented Sep 20, 2022

shinya7y commented Sep 21, 2022

shinya7y commented Sep 21, 2022

okotaku commented Sep 22, 2022 •

edited

Loading

shinya7y commented Sep 23, 2022

okotaku commented Sep 26, 2022

okotaku commented Oct 3, 2022

shinya7y Oct 3, 2022

shinya7y Oct 8, 2022

shinya7y Oct 8, 2022

hhaAndroid commented Oct 10, 2022 •

edited

Loading

okotaku commented Oct 11, 2022

shinya7y Oct 12, 2022

shinya7y Oct 12, 2022

shinya7y commented Oct 12, 2022

hhaAndroid commented Nov 22, 2022

hhaAndroid commented Nov 22, 2022

okotaku commented Nov 22, 2022 •

edited

Loading

		@@ -0,0 +1,4 @@
		_base_ = './yolox_pai_s_8x8_300e_coco.py'

		model = dict(bbox_head=dict(type='YOLOXTOODHead'))

[Feature] Support YOLOX-PAI #8778

Are you sure you want to change the base?

[Feature] Support YOLOX-PAI #8778

Conversation

okotaku commented Sep 13, 2022 • edited Loading

Motivation

Related PR

Result

Checklist

ZwwWayne commented Sep 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shinya7y commented Sep 20, 2022

okotaku commented Sep 20, 2022

shinya7y commented Sep 21, 2022

shinya7y commented Sep 21, 2022

okotaku commented Sep 22, 2022 • edited Loading

shinya7y commented Sep 23, 2022

okotaku commented Sep 26, 2022

okotaku commented Oct 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hhaAndroid commented Oct 10, 2022 • edited Loading

okotaku commented Oct 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shinya7y commented Oct 12, 2022

hhaAndroid commented Nov 22, 2022

hhaAndroid commented Nov 22, 2022

okotaku commented Nov 22, 2022 • edited Loading

okotaku commented Sep 13, 2022 •

edited

Loading

okotaku commented Sep 22, 2022 •

edited

Loading

hhaAndroid commented Oct 10, 2022 •

edited

Loading

okotaku commented Nov 22, 2022 •

edited

Loading