Research & Development Hub

There is no development without research

Research work is an essential part of deepsense.ai and a key aspect of the company’s development. Our projects bring fresh ideas to AI research, an area of human thought and production we believe is worth contributing to. In collaboration with top universities, scientific institutions and global corporations, we have made crucial contributions to the field of Reinforcement Learning.

Model-Based Reinforcement Learning for Atari

Joint research with Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign
  • Trained a number of action-conditioned video models which are used as neural simulators of Atari environments
  • Trained Atari agents using learned neural simulators and tested the performance of the agents in original environments
  • Compared the performance of our agents to performance of agents trained using two model-free algorithms Rainbow and PPO and 100K and 500K interactions with the environment
  • Invited presentations at the University of Oxford, DeepMind and Google Brain Zurich

Parallel training of Atari games

Joint research with Intel
  • Conducted parallel training of Atari games on one of the largest European supercomputers
  • Held the world record in parallel training of Atari games for two months in 2018 (beaten by DeepMind)
  • Presented work at the leading European HPC conference and were cited by DeepMind

Sim2Real consisting of training in Unreal Engine 4 and deployment on a real car

Joint research with a leading car manufacturer
  • Modeled a dozen real-life routes in Unreal Engine 4
  • Parallelized computations in order to generate hundreds of millions of frames using Unreal Engine 4 and one of the biggest European supercomputers
  • Models we trained were deployed on real autonomous vehicles multiple times in 2018 and 2019

Reinforcement learning and Theorem Proving

deepsense.ai’s research project
  • The first time reinforcement learning has been convincingly applied to solving general mathematical problems on a large scale
  • Published a paper in collaboration with researchers from the Technical
  • University in Prague and the University of Innsbruck
  • Presented at the main track of NeurIPS 2018

Expert-augmented actor-critic for ViZDoom and Montezuma’s Revenge

deepsense.ai’s research project
  • Achieved state-of-the-art results in environments known for challenging exploration
  • Presented at the  Deep RL and Imitation Learning Workshops at NeurIPS 2018

Learning to Run challenge solutions

deepsense.ai’s research project
  • Trained walking gaits with reinforcement learning methods
  • Took 6th place out of 400+ teams in the Learning to Run challenge, NIPS, 2017

Hierarchical Reinforcement Learning with Parameters

deepsense.ai’s research project
  • Developed an original RL algorithm for hierarchical reinforcement learning; presented at Google Headquarters in Mountain View during the Robot Learning conference, 2017

We want to hear from you

Find us
  • deepsense.ai, Inc.
  • 2100 Geng Road, Suite 210
  • Palo Alto, CA 94303
  • United States of America
  • deepsense.ai Sp. z o.o.
  • Al. Jerozolimskie 162A
  • 02-342 Warsaw
  • Poland
Let us know how we can help
Fill out this quick form and we will contact you shortly




You can modify your privacy settings and unsubscribe from our lists at any time (see our privacy policy).

This site is protected by reCAPTCHA and the Google privacy policy and terms of service apply.