deepsense.aideepsense.ai logo
  • Careers
    • Job offers
    • Summer internship
  • Clients’ stories
  • Services
    • AI software
    • Team augmentation
    • AI discovery workshops
    • GPT and other LLMs discovery workshops
    • Generative models
    • Train your team
  • Industries
    • Retail
    • Manufacturing
    • Financial & Insurance
    • IT operations
    • TMT & Other
    • Medical & Beauty
  • Knowledge base
    • deeptalks
    • Blog
    • R&D hub
  • About us
    • Our story
    • Management
    • Advisory board
    • Press center
  • Contact
  • Menu Menu
Key findings from the International Conference on Learning Representations (ICLR)

Trends and fads in machine learning – topics on the rise and in decline in ICLR submissions

October 24, 2019/in Machine learning /by Michał Kustosz and Błażej Osiński

ICLR (The International Conference on Learning Representations) is one of the most important international machine learning conferences. It’s popularity is growing fast, putting it on a par with such conferences as ICML, NeurIPS or CVPR.

The 2020 conference is slated for next April 26th, but the submissions deadline has already come and gone. 2585 publicly available papers were submitted. That’s about a thousand more than were featured at the 2019 conference.

2nd law of paper-dynamics tells us that the number of submitted papers will reach 100.000 in 24 years. That’s some serious growth!

We analyzed abstracts and keywords of all the ICLR papers submitted within the last three years to see what’s trending and what’s dying out. Brace yourselves! This year, 28% of the papers used or claimed to introduce state-of-the-art algorithms, so be prepared for a great deal of solid machine learning work!

“Deep learning” – have you heard about it?

To say you use deep learning in Computer Vision or Natural Language Processing is like saying fish live in water. Deep learning has revolutionized machine learning and become it’s underpinning. It’s present in almost all fields of ML, including less obvious ones like time series analysis or demand forecasting. This may be why the number of references to deep learning in keywords actually fell – from 19% in ‘18 to just 11% in ‘20–It’s just too obvious to acknowledge.

Deep learning

A revolution in network architecture?

One of the hottest topics this year turned out to be Graph Neural Networks. GNN is a deep learning architecture for graph-structured data. These networks have proved tremendously helpful in some applications in medicine, social network classification and modeling the behavior of dynamic interacting objects. The rise of GNNs is unprecedented, from 12 papers mentioning them in 18’ to 111 in 20’!

graph neural network

All Quiet on the GAN Front

The next topic has been extremely popular in recent years. But what has been called ‘the coolest idea in machine learning in the last twenty years’ has quickly become exploited. Generative Adversarial Networks can learn to mimic any distribution of data – creating impressive never seen artificial images. Yet they are on the decline. Yet they are on the decline, despite being prevalent in the media (deep fakes).

generative adversarial networks

Leave designing your machine learning to… machines

Finding the right architecture for your neural network can be a pain in the neck. Fear not, though: Neural Architecture Search (NAS) will save you. NAS is a method of building network architecture automatically rather than handcrafting it. It has been used in several state-of-the-art algorithms improving image classification, object detection or segmentation models. The number of papers on NAS increased from a mere five in ‘18 to 47 in ‘20!

neural architecture

Reinforcement learning – keeping stable

The percentage of papers on reinforcement learning has remained more or less constant. Interest in the topic remains significant – autonomous vehicles, Alpha Star’s success in playing StarCraft, and advances in robotics were allwidely discussed this year. RL is a stable branch of machine learning, and for good reason: future progress is widely anticipated.

reinforcement learning

What’s next?

That was just a sample of machine learning trends. What will be on top next year? Even the deepest neural network cannot predict it. But interest in machine learning is still on the rise, and the researchers are nothing if not creative. We shouldn’t be surprised to hear about groundbreaking discoveries next year and a 180-degree change in the trends.

To see a full analysis of the trends papers throughout the last three conferences click the photo below:

ICLR most popular keywords papers

 

Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
  • Share on Reddit
  • Share by Mail
https://deepsense.ai/wp-content/uploads/2019/10/key-findings-from-the-international-conference-on-learning-representations-iclr.png 337 1140 Michał Kustosz https://deepsense.ai/wp-content/uploads/2019/04/DS_logo_color.svg Michał Kustosz2019-10-24 12:49:092021-01-05 16:44:22Trends and fads in machine learning – topics on the rise and in decline in ICLR submissions

Start your search here

Build your AI solution
with us!

Contact us!

NEWSLETTER SUBSCRIPTION

    You can modify your privacy settings and unsubscribe from our lists at any time (see our privacy policy).

    This site is protected by reCAPTCHA and the Google privacy policy and terms of service apply.

    CATEGORIES

    • Generative models
    • Elasticsearch
    • Computer vision
    • Artificial Intelligence
    • AIOps
    • Big data & Spark
    • Data science
    • Deep learning
    • Machine learning
    • Neptune
    • Reinforcement learning
    • Seahorse
    • Job offer
    • Popular posts
    • AI Monthly Digest
    • Press release

    POPULAR POSTS

    • Diffusion models in practice. Part 1 - The tools of the tradeDiffusion models in practice. Part 1: The tools of the tradeMarch 29, 2023
    • Solution guide - The diverse landscape of large language models. From the original Transformer to GPT-4 and beyondGuide: The diverse landscape of large language models. From the original Transformer to GPT-4 and beyondMarch 22, 2023
    • ChatGPT – what is the buzz all about?ChatGPT – what is the buzz all about?March 10, 2023

    Would you like
    to learn more?

    Contact us!
    • deepsense.ai logo white
    • Services
    • AI software
    • Team augmentation
    • AI discovery workshops
    • GPT and other LLMs discovery workshops
    • Generative models
    • Train your team
    • Knowledge base
    • deeptalks
    • Blog
    • R&D hub
    • deepsense.ai
    • Careers
    • Summer internship
    • Our story
    • Management
    • Advisory board
    • Press center
    • Support
    • Terms of service
    • Privacy policy
    • Code of ethics
    • Contact us
    • Join our community
    • facebook logo linkedin logo twitter logo
    • © deepsense.ai 2014-
    Scroll to top

    This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies.

    OKLearn more

    Cookie and Privacy Settings



    How we use cookies

    We may request cookies to be set on your device. We use cookies to let us know when you visit our websites, how you interact with us, to enrich your user experience, and to customize your relationship with our website.

    Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.

    Essential Website Cookies

    These cookies are strictly necessary to provide you with services available through our website and to use some of its features.

    Because these cookies are strictly necessary to deliver the website, refuseing them will have impact how our site functions. You always can block or delete cookies by changing your browser settings and force blocking all cookies on this website. But this will always prompt you to accept/refuse cookies when revisiting our site.

    We fully respect if you want to refuse cookies but to avoid asking you again and again kindly allow us to store a cookie for that. You are free to opt out any time or opt in for other cookies to get a better experience. If you refuse cookies we will remove all set cookies in our domain.

    We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.

    Other external services

    We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.

    Google Webfont Settings:

    Google Map Settings:

    Google reCaptcha Settings:

    Vimeo and Youtube video embeds:

    Privacy Policy

    You can read about our cookies and privacy settings in detail on our Privacy Policy Page.

    Accept settingsHide notification only