deepsense.aideepsense.ai logo
  • Careers
    • Job offers
    • Summer internship
  • Clients’ stories
  • Services
    • AI software
    • Team augmentation
    • AI advisory
    • Train your team
  • Industries
    • Retail
    • Manufacturing
    • Financial & Insurance
    • IT operations
    • TMT & Other
    • Medical & Beauty
  • Knowledge base
    • Blog
    • R&D hub
  • About us
    • Our story
    • Management
    • Advisory board
    • Press center
  • Contact
  • Menu Menu
AI Monthly Digest #18 – the pixelated first step toward megastructures

AI Monthly Digest #18 – the pixelated first step toward megastructures

March 16, 2020/in AI Monthly Digest /by Konrad Budek

As we predicted in our AI Trends 2020, NLP is the year’s leading trend. But research on self-regenerating structures has likewise been both surprising and fascinating. February was rich in news on self-regenerating machines and NLP-related events and research.

Microsoft trains T-NLG, a 17-billion-parameter language model

The tech behemoths are racing to build larger and more efficient language models, which are both costly and technologically challenging, if not daunting. Microsoft is a relatively new player in the game, and follows in the footsteps of Google with itsBERT model and OpenAI, which brought outGPT-2.

While Microsoft may not have been the first mover, its Turing Natural Language Generation (T-NLG) is now the largest model trained and published for language generation. It continues the line of work begun by BERT and GPT-2. Increasing neural network size once again proved successful. After pre-training on essentially all text from the Internet, T-NLG can be fine-tuned to solve various downstream tasks like answering questions, understanding documents or simply filling in as a conversational agent (like Meena from our last post). In order to achieve this new state of the art in large-scale deep learning, Microsoft came up with novel optimization techniques and tricks that make it feasible to train artificial neural networks with over 100 billion parameters.

To put this number into perspective: the human brain has around 100 trillion synapses (in a nutshell, their job is to connect neurons). That’s a thousand times more than 100 billion. Does this mean we can now train artificial neural networks that are 0.1% size of the human brain? Definitely not, biological neurons are much more complex than artificial ones and the recent discovery shows that a single biological neuron is able to compute XOR, a mathematical function that requires a multilayer artificial neural network to be computed.

Why it matters

With industry heavily focused and invested in NLP, the technology is one of deepsense.ai’s cutting edge trends in 2020. The new models being delivered support business and consumer services like chatbots and search engines.

Given these developments, any bigger and newer language model brings us closer to the day when natural communication between computers and humans a la Star Trek will be possible. In decidedly less galactic terms, it is also a convenient way to communicate with a computer when your hands are otherwise occupied – like when you’re cooking or driving a car. A simple “skip that song” makes a difference.

Growing Neural Cellular Automata

Apparently nature is the most brilliant engineer of all, with its unrivaled ability to reuse and repurpose building blocks to deliver a breathtaking set of incredibly diverse forms. With cells being the most basic building block of every creature alive, from the lowly germ to humans, it is both fascinating and inspiring to crack the process of making a single cell that not only functions but is specialized in a particular context.

Putting it simply – a single cell can be a separate organism or only a tiny part of a larger structure, designed to perform a specific role, without itself having the ability to survive. Apart from that, living organisms can regenerate themselves, sometimes regrowing lost limbs, sometimes just healing wounds. Cracking this process would deliver a significant boost for engineering.

What happened exactly?

Every living cell contains a DNA code that contains all the information about the organism in which it lives–its height, eye color, and other factors. In terms of programming, it can be seen as a list of instructions containing if-then statements, which tell the cell how it should operate in its given context and surroundings.

Research published on distill.pub tackles the matter by delivering images built with single “cells” that contain DNA-like information which, when rolled out using local updates, converges to a full image in global scale. Some parts of the image can be destroyed, and the image will rebuild itself with the lifelike process of spawning new cells to replace destroyed ones.

Also, more images can be spawned and built near the original one, effectively blending them to observe how cells interact with the context. In some cases, when blended and destroyed, the images rebuild in odd ways, delivering imperfect or twisted versions of previous projects. Scar tissue of some kind.

Why it matters

Currently, it is hard to come up with a practical application, but it brings science-fiction-like prospects that could result in unimaginable progress. Just to name a few:

Building megastructures – be it a Death Star, a Dyson Sphere or a planet-wide supercolony, making a centralized program to control the whole system might be challenging due to the very scale of that system and the latencies within it.

Also, when delivering a “smart gravel” that transforms into the desired type of material or form you need to build the structure, managing the construction or repairs would be much easier. A car wouldn’t have a spare tire and a toolbox, but a box containing smart-gravel (or any other currently non-existing smart material) infused with the DNA-like code of the car and used to repair what’s broken.

But that’s a distant future and practical appliances are yet to come – not unlike when neural networks and machine learning started from the creation of a computational model for neural networks in 1943. The technology is now transforming our daily lives in the form of machine learning, reinforcement learning or any other paradigm used to infuse our tools.

These lyrics don’t exist

The final news is more lifestyle related and lighter in mood than the rest. After the popularity of thispersondoesnotexist.com, the time has come for song lyrics. Pop on over to theselyriscsdontexist.com and see how you  can choose the mood, theme and musical style–then read the lyrics the AI spits out. Perhaps a happy heavy metal song about eating burritos would suit your fancy? No problem.  Or would a sad pop ballad about a steam locomotive be more to your liking? That’s no problem, either.

Why does it matter

It doesn’t, but data science is also about fun and neural networks can power both inspiring and powerful tech as well as silly toys. Because why not?

Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
  • Share on Reddit
  • Share by Mail
https://deepsense.ai/wp-content/uploads/2020/03/ai-monthly-digest-18-the-pixelated-first-step-toward-megastructures.jpg 337 1140 Konrad Budek https://deepsense.ai/wp-content/uploads/2019/04/DS_logo_color.svg Konrad Budek2020-03-16 10:00:272022-03-30 17:13:41AI Monthly Digest #18 – the pixelated first step toward megastructures

Start your search here

NEWSLETTER SUBSCRIPTION

    You can modify your privacy settings and unsubscribe from our lists at any time (see our privacy policy).

    This site is protected by reCAPTCHA and the Google privacy policy and terms of service apply.

    THE NEWEST AI MONTHLY DIGEST

    • AI Monthly Digest 20 - TL;DRAI Monthly Digest 20 – TL;DRMay 12, 2020

    CATEGORIES

    • Elasticsearch
    • Computer vision
    • Artificial Intelligence
    • AIOps
    • Big data & Spark
    • Data science
    • Deep learning
    • Machine learning
    • Neptune
    • Reinforcement learning
    • Seahorse
    • Job offer
    • Popular posts
    • AI Monthly Digest
    • Press release

    POPULAR POSTS

    • AI trends for 2021AI trends for 2021January 7, 2021
    • A comprehensive guide to demand forecastingA comprehensive guide to demand forecastingMay 28, 2019
    • What is reinforcement learning? The complete guideWhat is reinforcement learning? deepsense.ai’s complete guideJuly 5, 2018

    Would you like
    to learn more?

    Contact us!
    • deepsense.ai logo white
    • Services
    • Customized AI software
    • Team augmentation
    • AI advisory
    • Knowledge base
    • Blog
    • R&D hub
    • deepsense.ai
    • Careers
    • Summer internship
    • Our story
    • Management
    • Advisory board
    • Press center
    • Support
    • Terms of service
    • Privacy policy
    • Code of ethics
    • Contact us
    • Join our community
    • facebook logo linkedin logo twitter logo
    • © deepsense.ai 2014-
    Scroll to top

    This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies.

    OKLearn more

    Cookie and Privacy Settings



    How we use cookies

    We may request cookies to be set on your device. We use cookies to let us know when you visit our websites, how you interact with us, to enrich your user experience, and to customize your relationship with our website.

    Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.

    Essential Website Cookies

    These cookies are strictly necessary to provide you with services available through our website and to use some of its features.

    Because these cookies are strictly necessary to deliver the website, refuseing them will have impact how our site functions. You always can block or delete cookies by changing your browser settings and force blocking all cookies on this website. But this will always prompt you to accept/refuse cookies when revisiting our site.

    We fully respect if you want to refuse cookies but to avoid asking you again and again kindly allow us to store a cookie for that. You are free to opt out any time or opt in for other cookies to get a better experience. If you refuse cookies we will remove all set cookies in our domain.

    We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.

    Other external services

    We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.

    Google Webfont Settings:

    Google Map Settings:

    Google reCaptcha Settings:

    Vimeo and Youtube video embeds:

    Privacy Policy

    You can read about our cookies and privacy settings in detail on our Privacy Policy Page.

    Accept settingsHide notification only
    Cookies To make this site work properly, we sometimes place small data files called cookies on your device. Most big websites do this too.
    Accept
    Change Settings
    Cookie Box Settings
    Cookie Box Settings

    Privacy settings

    Decide which cookies you want to allow. You can change these settings at any time. However, this can result in some functions no longer being available. For information on deleting the cookies, please consult your browser’s help function. Learn more about the cookies we use.

    With the slider, you can enable or disable different types of cookies:

    • Block all
    • Essentials
    • Functionality
    • Analytics
    • Advertising

    This website will:

    This website won't:

    • Essential: Remember your cookie permission setting
    • Essential: Allow session cookies
    • Essential: Gather information you input into a contact forms, newsletter and other forms across all pages
    • Essential: Keep track of what you input in a shopping cart
    • Essential: Authenticate that you are logged into your user account
    • Essential: Remember language version you selected
    • Functionality: Remember social media settings
    • Functionality: Remember selected region and country
    • Analytics: Keep track of your visited pages and interaction taken
    • Analytics: Keep track about your location and region based on your IP number
    • Analytics: Keep track of the time spent on each page
    • Analytics: Increase the data quality of the statistics functions
    • Advertising: Tailor information and advertising to your interests based on e.g. the content you have visited before. (Currently we do not use targeting or targeting cookies.
    • Advertising: Gather personally identifiable information such as name and location
    • Remember your login details
    • Essential: Remember your cookie permission setting
    • Essential: Allow session cookies
    • Essential: Gather information you input into a contact forms, newsletter and other forms across all pages
    • Essential: Keep track of what you input in a shopping cart
    • Essential: Authenticate that you are logged into your user account
    • Essential: Remember language version you selected
    • Functionality: Remember social media settings
    • Functionality: Remember selected region and country
    • Analytics: Keep track of your visited pages and interaction taken
    • Analytics: Keep track about your location and region based on your IP number
    • Analytics: Keep track of the time spent on each page
    • Analytics: Increase the data quality of the statistics functions
    • Advertising: Tailor information and advertising to your interests based on e.g. the content you have visited before. (Currently we do not use targeting or targeting cookies.
    • Advertising: Gather personally identifiable information such as name and location
    Save & Close