This paper proposed a new acceleration framework called Mensa to optimize the Google Edge TPU to achieve higher performance in NN inference on edge computation platform.
Strengths
The author observed three major shortcomings in previous Edge Computing Platform:
low computational throughput
low energy efficiency
memory system is bottleneck
The layer-level study is really impressive, which analyzes characters of different neural layer by parameter footprint and computational complexity.
The idea to group layers into families and optimize them seperately is really great.
Weaknesses
I don't think the three shortcomings are equally significant. Actually I think only the third one concerning memory system is truely important. Because it is of great novelty to apply processing near memory technology to machine learning. While the other two shortcomings are very common and do not seem to be the motivation of this research.
The optimization is based on model ranther basic operators. The range of applicable NN layers might be restricted, especially when new models or layers come out.
The framework seems to lack extensibility. Trade-off may lay between manufacturing cost and performance.
Can you do better?
- I will try to extend the range of target platforms. Now the paltform is confined to google TPU platform.
- I will try to study more NN layers or models, such as Transformer (the attention model, which is exclusive from CNN or RNN).
Takeways
- I really appreciate the grouping mindset. We can even use more advanced algorithms such as fancy clustering methods in machine learning to get different grouping result from various perspective.
Other Comments
As machine learning becomes more popular and applicable, there are highly demands in deep neural network model edge computing, which really contributes to applying machine learning result into real world scenarios. Specific accelerator frameworks such as Mensa might not be quickly utilized in commercial scenarios, but I believe time will proved these kinds of frameworks to be very promising.