Using CUTLASS Convolutions for Asymmetric Padding

[manishucsd]

CUTLASS convolutions support asymmetric padding by allowing the user to set output P and Q.

To understand its usage, let us look at using the asymmetric padding feature of CUTLASS convolution in 1D.

Figure 1. CUTLASS convolution asymmetric padding in 1D. Case 0 (symmetric padding), Case 1 (asymmetric padding towards top), and Case 2 (asymmetric padding towards bottom)

Please refer Figure 1 (linked above for this discussion). Case 0, 1, and 2 can also be verified in profiler. Note that all tensor values are initialized to 1. Thus, we can inspect the output NPQK to see the number of contributing filter r positions.

Case 0 (symmetric padding)

./tools/profiler/cutlass_profiler --kernels=cutlass_tensorop_s16816fprop_analytic_f16_256x128_32x3_nhwc_align8 --n=1 --h=8 --w=1 --c=8 --k=8 --r=3 --s=1 --pad_h=1 --pad_w=0 --p=8 --q=1 --providers=cutlass --verification-providers=host  --dist=uniform,min:1,max:1  --save-workspace=always

For the above size, the total number of accumulation if all three filter r positions are contributing to the output is GEMM-K = RSC = 3x1x8 = 24

Case 0 output NPQxK (8x8) is symmetric

16, 16, 16, 16, 16, 16, 16, 16,      // Only 2 filter positions contributing the top-most row (i.e. p = 0; for all K = 8)
24, 24, 24, 24, 24, 24, 24, 24,      // All 3 filter positions contributing ...
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
16, 16, 16, 16, 16, 16, 16, 16      // Only 2 filter positions contributing the bottom-most row (i.e. p = 7; for all K = 8)

Case 1 (asymmetric padding towards top)

./tools/profiler/cutlass_profiler --kernels=cutlass_tensorop_s16816fprop_analytic_f16_256x128_32x3_nhwc_align8 --n=1 --h=8 --w=1 --c=8 --k=8 --r=3 --s=1 --pad_h=2 --pad_w=0 --p=8 --q=1 --providers=cutlass --verification-providers=host  --dist=uniform,min:1,max:1  --save-workspace=always

Case 1 output NPQxK (8x8) is asymmetric towards the top

8, 8, 8, 8, 8, 8, 8, 8,          // Only 1 filter r position contributing the top-most row (i.e. p = 0; for all K = 8)
16, 16, 16, 16, 16, 16, 16, 16, // Only 2 filter r positions contributing the second row from top (i.e. p = 1; for all K = 8)
24, 24, 24, 24, 24, 24, 24, 24, // All 3 filter r positions contributing ...
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24

Case 2 (asymmetric padding towards bottom)

./tools/profiler/cutlass_profiler --kernels=cutlass_tensorop_s16816fprop_analytic_f16_256x128_32x3_nhwc_align8 --n=1 --h=8 --w=1 --c=8 --k=8 --r=3 --s=1 --pad_h=0 --pad_w=0 --p=8 --q=1 --providers=cutlass --verification-providers=host  --dist=uniform,min:1,max:1  --save-workspace=always

Case 2 output NPQxK (8x8) is asymmetric towards the bottom

24, 24, 24, 24, 24, 24, 24, 24,    
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24,  // All 3 filter r positions contributing ...
16, 16, 16, 16, 16, 16, 16, 16,  // Only 2 filter r positions contributing the second row from bottom (i.e. p = 6; for all K = 8)
8, 8, 8, 8, 8, 8, 8, 8           // Only 1 filter r position contributing the bottom-most row (i.e. p = 7; for all K = 8)