AI Monthly digest #2 – the fakeburger, BERT for NLP and machine morality

Fake images of hamburgers, the autonomous trolley problem, Google’s BERT for NLP and more stories from October, curated by deepsense.ai’s team, right here in our AI Monthly Digest.

October brought important developments in machine learning and sparks for interesting discussions about machine morality. deepsense.ai’s Arkadiusz Nowaczyński and Konrad Budek chose the five stories below.

1. Enter the fakeburger – DeepMind managed to produce convincing images of hamburgers, animals and landscapes

Renowned AI company DeepMind produced synthetic photos of hamburgers, landscapes and animals out of the ImageNet dataset. In most cases, as a team, it was difficult for us to determine if the pictures depicted a real or fake burger.

This was not the first time a neural network was used to create a convincing fake photo – take a look at NVIDIA’s one hour of imaginary celebrities.

We now have the ability to produce realistic images after training on the ImageNet dataset famous for advancing state-of-the-art image classification. DeepMind researcher Andrew Brock achieved his breakthrough with highly tuned Generative Adversarial Networks (GAN). The GAN uses a generator that produces artificial samples of images, and a discriminator, which distinguishes between fake and a real-world examples. The GANs here are scaled up, leading to the impressive results. Larger networks and training batch sizes (=2048) vastly improve the quality over previous work. Google’s Tensor Processing Unit (TPU) made the training feasible on such a large scale.

Animals, nature and food look convincing, but scenes involving humans aren’t there yet.

Even though the model isn’t perfect, this is a remarkable step towards generating realistic looking photos with neural networks. To read more about the BigGAN and creating images of fakeburgers, check out the Arxiv paper.

2. AI-generated portrait sold for $432,500 at auction

Contrary to the fakeburgers and fakedogs built by DeepMind, the de Bellamy family consists of people with AI-generated eerie faces that are clearly non-human. The entire de Bellamy family images was generated by Obvious, a group of French AI engineers and artists.

Images were produced using GAN (Generative Adversarial Network) in the same manner DeepMind used it to produce an image of a hamburger. The model was fed with 15,000 portraits from last 600 years and attempted to build new ones using the data.

It is easy to see that images have no match to the master’s paintings, even considering the variety of styles represented by the artificial artist. Nevertheless, the portrait of Edmond de Belamy was sold for nearly half million dollar.

3. The machine morality – biases and automated discrimination

The transparency of machine learning models and Artificial Intelligence again in October came to the fore, igniting discussion over the Internet. The first important issue was gender bias in an Amazon-developed Artificial Intelligence model for preprocessing resumes delivered to the company. Trained with 10 years’ worth of resumes, the system was unintentionally trained to choose male candidates, as tech is dominated by men. The company changed its model but as the MIT Technology Review states, the company is no longer certain about the system’s neutrality in other areas.

Given the towering AI adoption rate, discussion about machine ethics and transparency is necessary. According to data from Deloitte, the number of machine learning project implementations was twice higher in 2018 than in 2017, and is expected to quadruple from there by 2020. As machines themselves are incapable of being racist, misogynist or biased, creating proper datasets and designing the evaluation process to spot hidden bias is crucial to building models that best and most fairly serve companies and society alike.

The data science community has not left the problem unaddressed – Google AI launched a Kaggle competition aiming to design image recognition models that could perform well on photos taken in different geographical regions than those used for training. The competition is being held as a competition track at the NIPS 2018 Conference, which is better-attended than even ComicCon.

This all goes to show that experts have been right from the start — AI is yet another tool that needs to be constantly evaluated and developed if it is to achieve its goals.

4. And the machine morality again – autonomous trolley problem

The topic of machine bias is even more important considering the rise of autonomous cars. A global study shows that people from various social and cultural backgrounds differ in their perception of “a lesser evil” when it comes to the “trolley problem” in autonomous cars. In a nutshell, if a car has to choose whether to hit an elderly person or a child, whom should it choose? A group of people or a single person? A pregnant woman or a medical doctor?

The study shows that choices vary by country, and the differences are more than significant. When facing the extreme situation of a car accident, human drivers make their mind autonomously and are solely responsible for their choices. On the other hand, every autonomous car will carry the AI model its manufacturer has provided. Are the legal system and society ready to transfer the responsibility for driving and the choices it necessitates from the driver to a non-human machine? Should the AI model fit the culture it operates in or follow some other code? These questions have yet to be answered.

5. Google’s BERT for NLP – new state-of-the-art in language modeling

Natural Language Processing may enter a new era with Google’s Bidirectional Encoder Representation from Transformations (BERT).

For now, NLP practitioners continue to use pre-trained word embeddings as initialization or input features in custom architectures for specific tasks. BERT, a model that can be pre-trained on a large text corpus and then fine-tuned for various NLP downstream tasks, may change that. It might be similar to what we have seen in Computer Vision in the last couple of years, where fine-tuning models pre-trained on ImageNet has proved a great success.

BERT is a multi-layer bidirectional encoder taken from Transformer architecture, which was introduced in Attention Is All You Need. The pre-training procedure is entirely unsupervised and includes two objectives: filling random gaps in the input sequence and classifying whether two input sentences are actually two consecutive sentences cut out from the larger text. During the fine-tuning, predictions can be made for entire sequences or each input token separately.

The study shows that fine-tuning a pre-trained BERT model has set a new gold standard in 11 benchmark tasks.

Google has released an official implementation of BERT for NLP available on github.

And now for some bonus information:

Paul Romer, winner of this year’s Nobel Prize in Economics, is a 62-year old ex-World Bank chief economist, writer and Python programming language user. He is also a firm supporter of making research available and clear, so he shares his findings via Jupyter notebooks and makes data available for everyone to process and interpret.

His example perhaps shows that combining knowledge about economics and science with a proper toolset for one’s daily work can help lead to a rewarding career.