Self-Optimizing Memory Controllers: A Reinforcement Learning Approach 🔗
The authors propose a new memory controller design that uses reinforcement learning to optimize performance and bandwidth utilization in chip multiprocessors. The controller adapts its scheduling policies on the fly to improve performance and results show significant improvement compared to current state-of-the-art controllers.
Strengths
- The introduction is very attractive. It starts with the story of Moore’s Law, which is convincing.
- The core of this work is reinforcement learning. The RL-based memory controller takes as input parts of the system state and considers the long-term performance impact of each action it can take.
- I really enjoy the 6.2 section: applications of RL in computer systems, as I am fond of machine leanring. I look forward to more machine learning applications, especially non-RL methods (for variability).
Weaknesses
- Security problem should not be ignored especially in such a special case: applying machine leanring into hardware structures. The unexplainability, unpredictability lay the memory controller in a unsafe situation where the attacker may decryte the training methods and apply advesarial attacks.
- The graphs in this paper is not very fancy, which might hurt the readability.
- I am interested of the hardware implementation part, but unfortunately, it is not very detailed. It would be better to publish the source code.
Can you do better?
- As for the secuity problem, I would do some attack to this controller and report the safety situation. Reverse engineering is always with a lot of fun.
Takeaways
Machine learning always seems unexplainable, unpredictable and takes numerous hardware or software resouce. But this work really shocked me and taught me that nothing is impossible even embedding an RL algorithm into a memory controller.