Applied Edge AI: Deep Learning Outside of the Cloud

1. 00:01Hello and welcome! This video will continue the topic of the compact model, and it is the last part of this topic.
2. 00:11In this paper, the authors from Google systematically study model scaling and identify that carefully balancing
3. 00:20the network depths with and resolution can lead to better performance. Intuitively, the compound scaling method makes sense
4. 00:30because if the input image is bigger, the network needs more layers to increase the receptive field and more channels
5. 00:40to capture more fine-grained patterns on the bigger image.
6. 00:46In previous work, it is common to scale only one of the three dimensions. Though it is possible to scale 2 or 3 dimensions
7. 00:56arbitrarily, it requires tedious manual tuning and often yields sub-optimal accuracy and efficiency
8. 01:06The authors in this paper use neural architecture search to design a new baseline network and uniformly scale
9. 01:14it up to obtain a series of models, called EfficientNets, which achieve much better accuracy and efficiency than previous
10. 01:25ConvNets models.
11. 01:28So, the base model of EfficientNet is a lightweight mobile backbone MNasNet created by NAS regarding the trade-off
12. 01:39between accuracy and FLOPs.
13. 01:42They also also reported some observations regarding compound scaling. If you want to use 2^N times more computational
14. 01:53resources,
15. 01:55then we can increase the network depth by α^N, width by β^N,
16. 02:01and image size by γ^N, where α,β,γ are constant coefficients determined by a small grid search
17. 02:12on the original small model. The FLOPS of a regular convolution op is proportional to depth d,
18. 02:21with W^2,
19. 02:22and resolution r^2,
20. 02:28It means doubling
21. 02:29network depths will double FLOPS, but doubling network width or resolution will increase FLOPS by four times.
22. 02:40So the authors keep the constants〖α∙β〗^2∙γ^2≈2
23. 02:47such that for any compound coefficient ϕ(phei) the total FLOPS will approximately increase by 2^ϕ.
24. 03:00The efficientNet B7 achieves state-of-the-art accuracy on ImageNet , 84.4% top-1 acc, eight times smaller
25. 03:11and six times faster on inference than SOTA model Gpipe.
26. 03:19Some observations also mentioned by the author:
27. 03:22I think it's very valuable for practical applications.
28. 03:27the condition that the size of r and w are unchanged, with the increase of d,
29. 03:37There is not much difference in accuracy when D and
30. 03:42W remains unchanged, with the rise of resolution (r), the accuracy is greatly improved. When the resolution and depth remain
31. 03:53unchanged, with the increase of width the accuracy first improves significantly, then tends to be flat.
32. 04:05ShuffleNet
33. 04:06V2,
34. 04:07this work proposes several practical Guidelines for Efficient ConvNets Design. They also analyze how the network should be
35. 04:17designed from the perspective of Memory Access Cost,
36. 04:21in short, MAC and GPU parallelism to reduce the runtime further and directly improve the models efficiency. When the number
37. 04:32of input channels is the same as the number of output channels, the MACis the most negligible ->
38. 04:39Use the same input andoutput channels for a convolution layer, -MAC is proportional to the number of groups of a convolution
39. 04:48layer.
40. 04:49So we should carefully use a group convolution.
41. 04:53The number of branches in the networks reduce the parallelism, we should probably reduce the number of branches in the network
42. 05:03Element-wise operations are very time consuming, we should reduce the element wise operation if possible.
43. 05:10On the other hand, this work also introduced the design principle, cheap operations for more features which is very effective
44. 05:20for lightweight models.
45. 05:23GhostNet continues to deepen the concept of cheap operation for more features.
46. 05:31The Ghost module is designed which uses depth wise convolution to generate more intrinsic features.
47. 05:39Itintroduces design tricks such as significantly reducing the width of 1x1 convolutions which account for a large
48. 05:49portion of the computations.
49. 05:51Second, increasing the depths of the network is beneficial for boosting accuracy.
50. 05:58A possible shortcoming of this design is that Ghost net might not be really fast if the model is memory bonded.
51. 06:07The metrics FLOPs alone cannot accurately reflect the speed up of the model.
52. 06:16The MobileNetV3 includes two very effective design choices:
53. 06:20depthwise separable convolution and the inverted bottleneck block
54. 06:26The depthwise convolution layer applies a single 3x3 filter to each input channel to learn the spatial correlation
55. 06:35and then applies a 1x1 convolution to learn the channel correlation.
56. 06:40Thus, in the inverted bottleneck design, the first pointwise convolution expands the information flow, which increases the
57. 06:49capacity and the depth wise
58. 06:52and the second point wise convolution are responsible for the expressiveness.
59. 06:59This speculation is derived based on the analysis of the MobileNet-V2 paper.
60. 07:07Another strategy frequently used in recent works is "cheap operation for more features". For example,
61. 07:14they are used in ShuffleNet-V2 and GhostNet.
62. 07:21The table shows the complexity evaluation results of different types of convolutions of MobileNet models.
63. 07:29We observe that the computation overhead is mainly concentrated on the pointwise convolutions.
64. 07:37If we want to reduce the computational complexity, the optimization of this part is the first choice.
65. 07:46So, our proposed idea, in short is to apply the future reuse strategy on the first pointwise convolution to save the computation
66. 07:55effectively. And we correspondingly extend the future flow of the depth wise
67. 08:01and the second pointwise convolution layer, where we think they are more critical for the expressive ability
68. 08:11moreover, and the authors of AsymmNet keep the computation budgets unchanged.
69. 08:19AsymmNet has been verified on five different vision tasks, including classification,detection, pose estimation, face recognition,
70. 08:28and action recognition. And obtain the following two conclusions:
71. 08:34First compared with MobileNetV3, AsymmNet can generally get a better or same level of accuracy.
72. 08:43Second, especially in the region where the operations are less than 200 million Madds.
73. 08:52the performance of AsymmNet is pretty better than that of MobileNetV3.
74. 09:02RepVGG Net is not a compact network, but it offers an exciting design concept called Overparameterization techniques.
75. 09:13If we look at the table, RepVGG shows a better accuracy-speed balance than ResNet. And the more significant the model,
76. 09:23the more pronounced the acceleration effect. The core difference is that different model forms are used in the training
77. 09:37and inference stages.
78. 09:39We can see that from the figure in the training, repVGG uses two extra branches for each 3x3 convolution
79. 09:48layer:
80. 09:49so one 1x1 Conv branch and one shortcut connection.
81. 09:56But in the inference stage, both extra branches have been integrated into the 3x3 convolution.
82. 10:04So the form in the inference is a pure VGG style network.
83. 10:11Let’s briefly introduce How are 3x3 Conv, 1x1 Conv, and identity shortcuts fused in this
84. 10:20work.
85. 10:21This figure shows a standard 3x3,
86. 10:25the input feature map has 2 channels, and the output map has a shape of 3x3x2
87. 10:33This figure shows how a standard 1x1 convolution works.
88. 10:37It has kernel size 1 and stride=1, and the output size is also 3x3x2.
89. 10:46Note that here we add zero padding to the 1x1 kernel to form a 3x3 kernel, and we still get
90. 10:54the same result.
91. 10:59Identity connection is equivalent to a convolutional layer with special weights. In this example for
92. 11:09for the first kernel, its second channel equals 0. and for the second kernel, its first channel equals 0.
93. 11:18So basically, stacking both kernels is an identity matrix.
94. 11:24We can see that now the identity connection is just a particular case of 1x1 convolution.
95. 11:34We thus can further add 0 padding to it as before. It then becomes a 3x3 convolution with the same output.
96. 11:47So, in the training stage, the kernel forms of 3x3, 1x1 Conv and the identity connection look like this,
97. 11:58After the model is trained, we can simply calculate the element-wise addition to create a fused kernel for inference.
98. 12:09So, at the inference stage we will only use the 3x3 convolution kernel
99. 12:18From the accelerations perspective, both resnet and depthwise convolution cannot be made into a regular persistent fuse.
100. 12:29However, RepVGG's design is very accelerator friendly
101. 12:34The convolution shape is very neat (neet), without branches, without attention,
102. 12:40Each stage does not read or write global memory since the input and output have the same channel number. It is almost an accelerator's
103. 12:50favorite form.
104. 12:51This speed can almost be regarded as a tensor core running at full speed. Overall, if it's training and influence transformation
105. 13:03can be more concise, it will make this model more popular.
106. 13:11In the next video, we will discuss another compression technique, Kknowledge distillation.
107. 13:19Thank you.

Щоб увімкнути запис, виберіть мову в меню налаштувань відео.