“Model-Based Reinforcement Learning for Atari,” a research paper written by a consortium of authors, has been chosen for a spotlight presentation at the International Conference on Learning Representation. Only 5% of the papers submitted receive this distinction. Two of the leading authors, Piotr Miłoś and Błażej Osiński, are both researchers and senior data scientists at deepsense.ai.
The paper describes research conducted in cooperation with Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign.
The high-level goal described in the research is how to enable an AI model to learn useful behaviors by interacting with its environment. In this case, the environment was Atari games, which offer tasks that are simple yet challenging enough to test the research concept. The results of the experiment can be extrapolated to more challenging tasks in increasingly complex environments. The model the deepsense.ai researchers designed learned how to play the games thanks to an extensive chain of trial and error.
An extension of the research might be a robot acquiring skills while moving objects, such as putting dishes in a dishwasher or refrigerator. Such interactions can be costly – the robot will drop any number of dishes along the way, so it takes time and resources for it to train itself thoroughly enough to learn to carry out its tasks efficiently.
To tackle the challenge of expensive interactions, we increase the system’s learning efficiency by endowing it with a predictive module that can imagine the consequences of actions without actually executing them. The module is trained along with this skill and it is used to perform short inner simulations. These simulations substitute most of the interactions normally required by reinforcement learning, resulting in better sample efficiency.
Details about the research are available in the full arXiv preprint, while the press release and an article about artificial imagination can be found on deepsense.ai’s blog.
“We are proud and excited to see our work recognized at one of the most renowned scientific conferences in the world,”
– says Piotr Miłoś, senior data scientist at deepsense.ai and a professor at the Polish Academy of Science.
Reinforcement learning in spotlight
“Spotlight” is a short oral presentation reserved for papers of particular distinction. With only 5% of submitted papers honored this way, the privilege comes amidst our work gaining critical attention in the AI research community. The arXiv paper has been cited more than 50 times in just 10 months, a testament to its significance for other researchers in the field.
This year’s conference, the eighth, will be held in Addis Ababa, Ethiopia, and is hoped to encourage researchers from Africa and the Middle East to take part. Despite its short history, the conference is currently among the most important deep learning-related scientific events in the world. The conference publication is recognized on Google scholar as the 42nd most important scientific publication globally.
The conference provides an interesting insight into the development of Artificial Intelligence and is a great way to keep abreast of the latest trends. deepsense.ai has recently used machine learning tools to review the papers submitted to the conference. The analysis showed that reinforcement learning is among the most significant and recurrent trends in machine learning today. Further information on trending topics on the ICLR conferences can be found in deepsense.ai’s analysis.
“Presenting our research on spotlight is a great honor and proof that our contribution to the machine learning community and development is recognized,”
– concludes Błażej Osiński, a senior data scientist and deepsense.ai researcher involved in the project.
This success is the result of deepsense.ai’s strong commitment to R&D. Just a month ago two other papers were presented at NeurIPS workshops, another top-tier AI conference. These included “Developmentally motivated emergence of compositional communication via template transfer” and “Simulation-based reinforcement learning for real-world autonomous driving”.
The full list of paper authors: Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, Afroz Mohiuddin, Ryan Sepassi, George Tucker, Henryk Michalewski
###
About deepsense.ai:
brand-manager-at-deepsense
