Data science Archives - Page 2 of 10

Outsmarting failure. Predictive maintenance powered by machine learning

November 13, 2018/in Data science, Machine learning /by Konrad Budek

Since the days of the coal-powered industrial revolution, manufacturing has become machine-dependent. As the fourth industrial revolution approaches, factories can harness the power of machine learning to reduce maintenance costs.

The internet of things (IoT) is nothing new for industry. Worldwide, the number of cellular-enabled factory automation devices reached 270 000 in 2012 worldwide. In 2018 it will rise to a staggering 820 000. Machines are present in every stage of the production process, from assembly to shipment. Although automation makes industry more efficient, with rising complexity it also becomes more vulnerable to breakdowns, as service is both time-consuming and expensive.

Four levels of predictive maintenance

According to PricewaterhouseCoopers, there are four levels of predictive maintenance.

1.	Visual inspection, where the output is entirely based on the inspector’s knowledge and intuition
2.	Instrument inspection, where conclusions are a combination of the specialist’s experience and the instrument’s read-outs
3.	Real-time condition monitoring that is based on constant monitoring with IoT and alerts triggered by predefined conditions
4.	AI-based predictive analytics, where the analysis is performed by self-learning algorithms that continuously tweak themselves to the changing conditions

As the study indicates, a good number of the companies surveyed by PwC (36%) are now on level 2 while more than a quarter (27%) are on level 1. Only 22% had reached level 3 and 11% level 4, which is basically level 3 on machine learning steroids. The PwC report states that only 3% use no predictive maintenance at all.

Staying on track

According to the PwC data, the rail sector is the most advanced sector of those surveyed with 42% of companies at level 4, compared to 11% overall.

One of the most prominent examples is Infrabel, the state-owned Belgian company, which owns, builds, upgrades and operates a railway network which it makes available to privately-owned transportation companies. The company spends more than a billion euro annually to maintain and develop its infrastructure, which contains over 3 600 kilometers of railway and some 12 000 civil infrastructure works like crossings, bridges, and tunnels. The network is used by 4 200 trains every day, transporting both cargo and passengers.

According to the PwC data, the rail sector is the most advanced sector of those surveyed with 42% of companies at level 4, compared to 11% overall.

The company faces both technical and structural challenges. Among them is its aging technical staff, which is shrinking.

At the same time, the density of railroad traffic is increasing – the number of daily passengers has increased by 50% since 2000, reaching 800 000. What’s more, the growing popularity of high-speed trains is exerting ever greater tension on the rails and other infrastructure.

To face these challenges, the company has implemented monitoring tools, such as sensors for monitoring overheating tracks, cameras which inspect the pantographs and meters to detect drifts in power consumption, which usually occur before mechanical failures in switches. All of the data is collected and analyzed by a single tool designed to apply predictive maintenance. Machine learning models are a component of that tool.

As sounding brass

Mueller Industries (Memphis, Tennessee) is a global manufacturer and distributor of copper, brass, aluminum and plastic products. The predictive maintenance solution the company uses is based on sound analysis. Every machine can be characterized by the sound it makes and any change in the tone or the sounds it makes may be a sign of impending malfunction. The analysis of the sound and the vibrations of the machine is done in real-time with the cloud-based machine learning solution that seeks patterns in the data gathered.

Both the amount and the nature of the data collected render it impossible for a human to analyze, but a machine-learning powered AI solution handles it with ease. The devices are able to gather data in ultrasonic and vibration sensors and analyze them in real time. Contrary to experience-based analytics, using the devices requires little-to-no training and can be done on the go.

Endless possibilities

With the power of machine learning enlisted, handling the tremendous amounts of data generated by the sensors in modern factories becomes a much easier task. It allows the company to detect failures before they paralyze the company, thus saving time and money. What’s more, the data that is gathered can be used to further optimize the company’s performance, including by searching for bottlenecks and managing workflows.

That’s why 98% of industrial companies expect to increase efficiency with digital technologies.

AI Monthly Digest #9 – the double-edged sword of modern technology

June 7, 2019/in Data science, AI Monthly Digest /by Konrad Budek and Arkadiusz Nowaczynski

This edition is all about AI morality-related themes, with a slight tinge of Talking Heads and Modern Talking.

Earlier this year, deepsense.ai highlighted AI morality and transparency as one of 2019’s dominant AI trends. May bore out our thesis, especially as it relates to potential misuse and malicious intent. At the same time, though, AI provides unique chances to support entertainment and education, as well as deliver new business cases.

A bigger version of GPT-2 released to the public

Open-AI has recently shown the GPT-2 model has set a new gold standard for natural language processing. Following the acclaimed success of the model, OpenAI opted not to make it public due to the risk of malicious usage, particularly to produce spam and fake news at no cost.

This sparked an uproar. The industry good practice is to release AI research work as open-source software, so other researchers can push the boundaries further without having to repeat all the work done earlier from scratch. In other words – OpenAI threw up a major hurdle to NLP-model development by keeping GPT-2 under wraps.

To support the scientific side of the equation while reducing the malicious threat, OpenAI releases some smaller-scale models to the public. The model it recently released operates on 345M parameters, while the best original model consists of 1.5B parameters. Every parameter can be seen as a virtual neuron inside a neural network, so OpenAI is basically reducing the brain it designed.

The original network was released to OpenAI partners currently working on malice-proofing the system. The first independent applications of the downscaled network are already available at talktotransformer.com and onionbot headline generator.

Why does it matter?

OpenAI is currently facing a difficult choice between supporting the global development of AI and the fear of losing control over dangerous technology. In a world facing a potential avalanche of fake news and social media being used to perpetuate propaganda, building a system that writes coherent and convincing texts is undoubtedly dangerous.

This case allows one to see all the AI-related issues in a nutshell, including the technology’s amazing potential, the real threat of misuse or malicious intent. So the case may serve as a precedent for future cases.

Talking heads unleashed

A group of scientists working for Samsung’s AI Center in Moscow and Skolkovo Institute of Science and Technology designed a model that can produce a convincing video of a talking head from a single image, such as a passport photo or even a painting.

The model renders with consistency both the background and the head’s behavior. Most impressively, the model builds a convincing video of a talking head from even a single image of the frame.

The solution is searching for a similar face that was analyzed and extracts facial features including a nose, chin, mouth and eyes. The movement of those features is then applied on the image, as shown in the video.

The results are undoubtedly impressive.

Why does it matter?

Yet another AI ethics-related issue, the talking-head technology poses the threat of deepfakes, images that show a person making statements that he or she would never make. This raises obvious questions about the malicious ways such technology could be used.

On the other hand, when deepfakes are used for special effects in popular movies, no one seems to complain and critics even weigh in with their acclaim. Some of the better-known examples come from the Star Wars franchise, particularly Rogue One, which features Leia Organa wearing the face of a young Carrie Fisher.

AI has also proved itself useful in promoting art. By leveraging this technology it is possible to deliver the talking head of Girl with a Pearl Earring or the Mona Lisa telling visitors from screens about a painting’s historical context – a great way to put more fun in art lessons for kids. Or just to have some fun seeing what a Stallone-faced Terminator would look like.

Again, AI can be used for both good and evil ends. The ethics are up to the wielder of this double-edged sword.

Modern Talking – recreating the voice of Joe Rogan

Another example of deepfake-related technology is using AI to convincingly recreate Joe Rogan’s voice. The text-to-speech technology is not a new kid on the block, yet it is easy to spot due to the robotic and inhumanely calm style of speaking. Listening to automated text-to-speech was usually boring at best while delivering the unintentional comic effects of robotic speech, all in the absence of emotion or inflection.

Dessa engineers have delivered a model that is not only transforming text to speech, but also recreating Joe Rogan’s style of speaking. Joe is a former MMA commentator who went on to become arguably the most popular podcaster in the world. Speaking with great emotion, heavily accenting and delivering power with every word, Rogan is hard to mistake.

Or is he? The team released a quiz that challenges the listener to distinguish if a given sample comes from a real podcast or was AI-generated. The details can be found on Dessa’s blog.

Why does it matter?

Hearing a convincing imitation of a public personality’s voice is nearly as unsettling as watching a talking head talk. But the technology can be used for entertainment and educational purposes. For example, delivering a new Frank Sinatra single or presenting Winston Churchill’s comprehensive and detailed speech on reasons behind World War II.

Again, the ethics are in the user’s hands, not in the tool. Despite that, and as we saw with OpenAI’s GPT-2 Natural Language Processing model, researchers have decided NOT to let the model go public.

Machine learning-powered translations increase trade by 10,9%

Researchers at Olin Business School at Washington University in St.Louis have found a direct connection between machine learning-powered translations and business efficiency. The study was conducted on e-Bay and shows that moderate improvement in the quality of language translation increased trade between countries on eBay by 10.9%.

The study examined the trade between English speakers from the United States and their trade relations with countries speaking other languages in Europe, America and Asia. More on the research can be found on the Washington University of St.Louis website.

Why does it matter?

While there is no doubt that AI provides vital support for business, the evidence, while voluminous, remains largely anecdotal (sometimes called anec-data) with little quantitative research to back up the claim. Until the Olin study, which does provide hard and reliable data. Is justified true belief knowledge? That’s an entirely different question…

A practical approach to AI in Finland

AI Monthly Digest #5 presented a bit about a Finnish way of spreading the word about AI. Long story short: contrary to many approaches of building AI strategy in a top-down model, Finns have apparently decided to build AI-awareness as a grassroots movement.

To support the strategy, the University of Helsinki has released a digital AI course on the foundations and basic principles of AI. It is available for free to everyone interested.

Why does it matter?

AI is gaining attention and the reactions are usually polarised – from fear of losing jobs and machine rebellion to arcadian visions of an automated future with no hunger or pain. The truth is no doubt far from either of those poles. Machine learning, deep learning and reinforcement learning are all built on certain technological foundations that are relatively easy to understand, including their strengths and limitations. The course provides good basic knowledge on these issues, which can do nothing but help our modern world.

A comprehensive guide to demand forecasting

May 28, 2019/in Data science, Machine learning, Popular posts /by Konrad Budek and Piotr Tarasiewicz

Everything you need to know about demand forecasting – from the purpose and techniques to the goals and pitfalls to avoid.

Essential since the dawn of commerce and business, demand forecasting enters a new era of big-data rocket fuel.

What is demand forecasting?

The term couldn’t be clearer: demand forecasting forecasts demand. The process of predicting the future involves processing historical data to estimate the demand for a product. An accurate forecast can bring significant improvements to supply chain management, profit margins, cash flow and risk assessment.

What is the purpose of demand forecasting?

Demand forecasting is done to optimize processes, reduce costs and avoid losses caused by freezing up cash in stock or being unable to process orders due to being out of stock. In an ideal world, the company would be able to satisfy demand without overstocking.

Demand forecasting techniques

Demand forecasting is an essential component of every form of commerce, be it retail, wholesale, online, offline or multichannel. It has been present since the very dawn of civilization when intuition and experience were used to forecast demand.

Sybilla's dashboard - deepsense.ai's demand forecasting tool

Sybilla – deepsense.ai’s demand forecasting tool

More recent techniques combine intuition with historical data. Modern merchants can dig into their data in a search for trends and patterns. At the pinnacle of these techniques, are demand forecasting machine learning models, including gradient boosting and neural networks, which are currently the most popular ones and outperform classic statistics-based methods.

The basis of more recent demand forecasting techniques is historical data from transactions. These are data that sellers collect and store for fiscal and legal reasons. Because they are also searchable, these data are the easiest to use.

Sybilla – deepsense.ai’s demand forecasting tool

How to choose the right demand forecasting method – indicators

As always, selecting the right technique depends on various factors, including:

The scale of operations – the larger the scale, the more challenging processing the data becomes.
The organization’s readiness – even the large companies can operate (efficiency aside) on fragmented and messy databases, so the technological and organizational readiness to apply more sophisticated demand forecasting techniques is another challenge.
The product – it is easier to forecast demand for an existing product than for a newly introduced one. When considering the latter, it is crucial to forming a set of assumptions to work on. Owning as much information about the product as possible is the first step, as it allows the company to spot the similarities between particular goods and search for correlations in the buying patterns. Spotting an accessory that is frequently bought along with the main product is one example.

How AI-based demand forecasting can help a business

Demand forecasting and following sales forecasting is crucial to shaping a company’s logistics policy and preparing it for the immediate future. Among the main advantages of demand forecasting are:

Loss reduction – any demand that was not fulfilled should be considered a loss. Moreover, the company freezes its cash in stock, thus reducing liquidity.
Supply chain optimization – behind every shop there is an elaborate logistics chain that generates costs and needs to be managed. The bigger the organization, the more sophisticated and complicated its inventory management must be. When demand is forecast precisely, managing and estimating costs is easier.
Increased customer satisfaction – there is no bigger disappointment for consumers than going to the store to buy something only to return empty-handed. For a business, the worst-case scenario is for said consumers to swing over to the competition to make their purchase there. Companies reduce the risk of running out of stock–and losing customers–by making more accurate predictions.
Smarter workforce management – hiring temporary staff to support a demand peak is a smart way for a business to ensure it is delivering a proper level of service.
Better marketing and sales management – depending on the upcoming demand for particular goods, sales and marketing teams can shift their efforts to support cross- and upselling of complementary products,
Supporting expert knowledge – models can be designed to build predictions for every single product, regardless of how many there are. In small businesses, humans handle all predictions, but when the scale of the business and the number of goods rises, this becomes impossible. Machine learning models extend are proficient at big data processing.

How to start demand forecasting – a short guide

Building a demand forecasting tool or solution requires, first and foremost, data to be gathered.

While the data will eventually need to be organized, simply procuring it is a good first step. It is easier to structure and organize data and make them actionable than to collect enough data fast. The situation is much easier when the company employs an ERP or CRM system, or some other form of automation, in their daily work. Such systems can significantly ease the data gathering process and automate the structuring.

Sybilla – deepsense.ai’s demand forecasting tool

The next step is building testing scenarios that allow the company to test various approaches and their impact on business efficiency. The first solution is usually a simple one, and is a good benchmark for solutions to come. Every next iteration should be tested to see if it is performing better than the previous one.

Historical data is usually everything one needs to launch a demand forecasting project, and obviously, there are significantly less data on the future. But sometimes it is available, for example:

Short-term weather forecasts – the information about upcoming shifts in weather can be crucial in many businesses, including HoReCa and retail. It is quite intuitive to cross-sell sunglasses or ice cream on sunny days.
The calendar – Black Friday is a day like no other. The same goes for the upcoming holiday season or other events that are tied to a given date.

Sources of data that originate from outside the company make predictions even more accurate and provide better support for making business decisions.

Common pitfalls to avoid when building a demand forecasting solution

There are numerous pitfalls to avoid when building a demand forecasting solution. The most common of them include:

The data should be connected with the marketing and ads history – a successful promotion results in a significant change in data, so having information about why it was a success makes predictions more accurate. If machine learning was used to make the predictions, the model could have misattributed the changes and made false predictions based on wrong assumptions.
New products with no history – when new products are introduced, demand must still be estimated, but without the help of historical data. The good news here is that great strides have been made in this area, and techniques such as product DNA can help a company uncover similar products its past/current portfolio. Having data on similar products can boost the accuracy of prediction for new products.
The inability to predict the weather – weather drives demand in numerous contexts and product areas and can sometimes be even more important than the price of a product itself! (yes, classical economists would be very upset). The good news is that even if you are unable to predict the weather, you can still use it in your model to explain historical variations in demand.
Lacking information about changes – In an effort to support both short- and long-term goals, companies constantly change their offering and websites. When the information about changes is not annotated in the data, the model encounters sudden dwindles and shifts in demand with apparently no reason. In the reality, it is usually a minor issue like changing the inventory or removing a section from website.
Inconsistent portfolio information – predictions can be done only if the data set is consistent. If any of the goods in a portfolio have undergone a name or ID change, it must be noted in order not to confuse the system or miss out on a valuable insight.
Overfitting the model – a vicious problem in data science. A model is so good at working on the training dataset that it becomes inflexible and produces worse predictions when new data is delivered. Avoiding overfitting is down to the data scientists.
Inflexible logistics chain – the more flexible the logistics process is, the better and more accurate the predictions will be. Even the best demand forecasting model is useless when the company’s logistics is a fixed process that allows no space for changes.

Sybilla – deepsense.ai’s demand forecasting tool

AI in demand forecasting: final thoughts

Demand and sales forecasting is a crucial part of any business. Traditionally it has been done by experts, based on know-how honed through experience. With the power of machine learning it is now possible to combine the astonishing scale of big data with the precision and cunning of a machine-learning model. While the business community must remain aware of the multiple pitfalls it will face when employing machine learning to predict demand, there is no doubt that it will endow demand forecasting with awesome power and flexibility.

Ready to harness the full potential of AI for your business? Opt for our AI consulting services, and let our experts guide you.

Machine learning in drug discovery

February 28, 2019/in Data science /by Konrad Budek

Artificial intelligence is advancing various industries, including healthcare and the pharmaceutical industry. According to Accenture data, key clinical health AI applications can potentially create $150 billion in annual savings for the United States healthcare sector by 2026.

The numbers show that the healthcare industry will heavily leverage the possibilities provided by machine learning. That’s why AI companies are getting involved in various activities in the treatment process, from diagnosis to therapy and drug development.

By applying convolutional neural networks in detecting diabetic retinopathy, deepsense.ai significantly improved the diagnostic process by speeding up and automating diabetic retinopathy screenings. The next step may be building a reinforcement learning agent that can be trained to run by controlling the muscles attached to the virtual skeleton. With that doctors can predict if a patient is able to walk, jump or run properly after the treatment. Furthermore, the work done during the research might be later used to design new, AI-powered leg prostheses.

Another healthcare segment that is heavily dependent on data is drug discovery.

The potential of AI in drug discovery

Computational solutions in drug discovery help significantly reduce the cost of introducing drugs to the market. Grand View Research and its new 2018 report implies that global drug discovery informatics market size was estimated at $713.4 million in 2016 and it is anticipated to progress at a CAGR (Compound Annual Growth Rate) of 12.6% by 2025. With artificial intelligence being used in drug discovery, the market’s value is growing rapidly. In its Global Artificial Intelligence in Drug Discovery Market Size Analysis, 2018-2028, Bekryl indicates that AI has the potential to create $70 billion in savings in the drug discovery process by 2028.

The technological and paradigm shift to machine learning seen in the pharmaceutical industry enables researchers to use novel computational algorithms to support the process. As biomedical data are highly complex, using algorithms in designing new drugs has become more possible than it has ever been. Machine learning can enhance many stages of the drug discovery process:

preliminary but crucial stages including designing a drug’s chemical structure.
investigating the effect of a drug – both in basic preclinical research and clinical trials, in which a lot of biomedical data is produced. Finding new patterns in those data can be facilitated by machine learning.

There are different kinds of data, including genetic and imaging ones. Each of them can be analyzed with machine learning and further used to build novel solutions for drug discovery.

Challenges in machine learning for drug discovery

Ensuring drug safety is one of the main challenges in the drug discovery process. Interpreting information of the known effects of drugs and predicting their side effects are complex tasks. Scientists and engineers from research institutions and pharmaceutical companies like Roche and Pfizer have been trying to use machine learning to get meaningful information from clinical data obtained in clinical trials. Interpretation of this data in the context of drug safety is an active area of research.

Clinical trials are the most expensive stage of drug development. To reduce their costs, it is crucial to use the experience gained during previous clinical trials in the early stages of drug development. This can be achieved in two steps:

biomedical data from research experiments could be analyzed and interpreted using machine learning to predict a drug’s effects and side effects;
data from clinical trials analyzed with machine learning should support the interpretation of biological data.

With those two approaches developed simultaneously, it is possible to design better preclinical experiments to come up with the most effective therapies with the fewest side effects.

Integrating biomedical data with computational approaches

Machine learning could help optimize therapy by integrating biomedical and clinical data with computational models, and can be used to build software to test drugs and combinatorial therapies. Some computational models and approaches which support the integration of clinical data are still under development but there are also a few very good examples of successful data integration in biology and medicine.

For example, there are a number of machine learning methods for integrating genetic regulatory networks and pathway information. This can be used to predict their biological functions and efficient Python-based implementation of bioinformatic tools and approaches that are easy to interface with broadly used machine learning packages.

Genetic data analysis and personalized medicine

Many pharmaceutical companies and startups are focused on genetic data interpretation and personalized medicine. Understanding the patient’s genetic profile helps to offer appropriate drugs and therapy. Building computational approaches to analyze genetic data and propose novel therapies could be advanced with machine learning. There are only a few examples that impact current clinical practice based on machine learning solutions which bring huge potential to personalized medicine and drug discovery. They include discovering novel biomarkers of drug response and machine learning-based computational tools used in clinical practice. Such tools are used to estimate the resistance to individual drugs and to combinatorial therapies based on genotype analysis.

One of the possible approaches is based on interpreting the genetic code as a one dimensional image and then applying a standard machine learning algorithm. The data is then scoured for patterns and anomalies, just as has been done in various other deepsense.ai image recognition projects. Analyzing the genomics may be in fact done in the same way as it is applied to classical paintings, when it comes to finding a hand or any other element. For the algorithm, the nature or shape of an image to analyze is irrelevant, so the machine is equally effective at analyzing a one-dimensional DNA chain or any other type of image data

Because genomic data is usually presented as a string of letters, it is also possible to apply Natural Language Processing techniques. One advantage of doing so is that it broadens the area the algorithm is able to process. That may be important when particular changes or patterns are being sought, or the pattern to find consists of a longer sequence of genes.

A big challenge is to fully unlock the potential of machine learning for drug discovery and personalized medicine. Time series data could be useful to fully reconstruct genetic networks on the basis of expression data. To build comprehensive predictive models based on machine learning, expression genetic data and sequencing data should be acquired in time series.

Innovative startups, like Cambridge Cancer Genomics, use machine learning to analyze data gained from liquid biopsy, a diagnostic technology in which circulating tumor cells or cell-free DNA is collected from blood samples. Although it is not a fully standardized approach for cancer therapy monitoring, it is highly anticipated in personalized medicine due to its ability to acquire genetic data in time series during treatment. Applying machine learning to better understand those data and to answer the question of why cancer evolves could help scientists design less toxic therapies.

Building and getting insight from databases and datasets

Scientists use public repositories of clinical data to tackle big problems in clinics to help medical doctors in their everyday work, as medical knowledge can be extracted from public repositories. These repositories could also be used for drug discovery purposes to include clinical information in the early stage of drug development.

Attempts have been made to represent medical knowledge using deep neural networks. Data mapped with machine learning might also be easier to integrate with biomedical data analyzed with machine learning, thanks to better compatibility in the data structures generated with similar approaches.

New achievements in building databases for machine learning purposes are also promising. For example, the authors of the paper “integrative analysis and machine learning on cancer genomics data using the Cancer Systems Biology Database (CancerSysDB)” developed a database for highly flexible queries and analysis of cancer-related data across multiple data types and multiple studies. However, there are many problems in medicine and drug discovery which are very difficult to answer only on the basis of public data, of which there is a paucity if better machine learning models and approaches are to be developed.

If proper datasets are to be built to answer specific scientific questions, it is not only the way in which data is preprocessed that must be understood, but also the principles of using different bioinformatic tools and interdisciplinary knowledge in biomedicine and where computer science and medicine converge. Teams which have this knowledge and skills could help to make better use even of limited amounts of data from public repositories. Machine learning engineers usually get data from scientists, medical doctors, pharmaceutical companies and hospitals, thus the amount is limited. But models must be strong for results to be achieved.

One of the best examples of designing a model of superior strength, one that can deal with the lack of proper data, was deepsense.ai’s Right Whale Recognition engine. The model was designed to recognize an individual Right Whale in a photograph, even if there were only a few photos provided in the dataset.

To get deep insight from data, close cooperation and mutual understanding of different languages and disciplines is needed. That is difficult if there is only the occasional consultation.

Scientists with a dual biomedical and computational background are crucial members of teams building databases, datasets, machine learning models, tools and software for analyzing biomedical data and drug discovery. Leading institutions like ETH Zurich have already started educating a new generation of medical scientists with a computational and math background and have built a platform and interdisciplinary teams to analyze clinical and biomedical data. The Swiss tumor board and ETH Personalized Health Technologies Platform Nexus are actively working towards implementing individualized, biomarker-based medical decisions in clinical practice. These are crucial fundamental steps in advancing drug discovery with machine learning.

Standard machine learning approaches for genetics and genomics

Standard supervised, semi-supervised and unsupervised machine learning algorithms are applied to analyze genetic data like microarray or RNA-seq expression data. To understand how, read “machine learning in genetics and genomics”. These algorithms can reveal disease and healthy phenotypes and could be further used to uncover the mechanisms of action of drugs. In any application of machine learning methods, the researcher must decide which data to provide as input to the algorithm to answer complex biomedical questions.

There are a number of comprehensive reviews summarizing the use of large-scale analysis of genomic data and machine learning strategies to solve genomic sequencing problems, like finding specific regions in sequences and recognizing locations of transcriptomic sites. It is one of the biggest challenges in genomics with practical applications.

Machine learning has potential for this application, though the results produced with machine learning algorithms should be validated with data from laboratory experiments or clinical trials. Deep learning algorithms could be useful in genome interpretation and analysis of genetic variants, a complex task that requires a combination of robust biological data and clinical knowledge.

Recently scientists and engineers have taken a step toward better understanding the human genome thanks to machine learning. Supervised heterogeneous ensemble methods can significantly improve our ability to address difficult biomedical prediction problems. Still, the application of machine learning algorithms to genomic problems is in a nascent stage. After all, genomic and genetic data are multidimensional and there remains a need to develop probabilistic machine learning algorithms for their analysis.

Machine learning approaches for network analysis of biomedical data

Analysis of genetic data could be helpful in elucidating genetic networks, which can reveal a drug’s mechanism of action and help understand how diseases work. This falls within the scope of an emerging new discipline called network medicine. The Barabasi group, a pioneer in network medicine, states that an unsupervised network-based approach enables the prediction of novel drug-disease associations, which offer significant opportunities for finding new applications for drugs and predicting potential side effects.

The group also found that the therapeutic effect of drugs might be localized in a small network neighborhood. This means that several genes in close network proximity of genes related to the mechanism of a disease could be targeted to effectively treat the disease.

Analyzing genetic network data with machine learning could help in finding novel targets for drugs and predict the optimal combination of drugs. There are research papers that explain how to benchmark machine learning for biological network analysis. One is “machine learning-assisted network inference approach to identify a new class of genes that coordinate the functionality of cancer networks.” This study shows usage of support vector machine (SVM) models combined with machine learning-assisted network inference (MALANI) to identify cancer-associated gene pairs. These can be used to reconstruct cancer networks to identify key cancer genes in high-dimensional data space that would otherwise go undetected by conventional approaches. These algorithms should be equally applicable to other machine learning and feature selection approaches. There is also a tutorial by Stanford lecturers which shows the basics of how to use deep learning approaches to analyze biological networks. However, for analysis of complex biological networks, non-standard machine learning algorithms are still being developed and network and machine learning approaches need better integration.

Machine learning algorithms in image analysis for drug discovery

The article Machine learning and image-based profiling in drug discovery presents how image-based screening of high-throughput experiments, in which cells are treated with drugs, could help elucidate a drug’s mechanism of action. It is mentioned that unsupervised and simple statistical inference methods seem to be in favor for analyzing image data from large-scale profiling experiments, but complex biological phenotypes and single-cell experiments could be successfully classified with supervised algorithms.

The recently explored application of supervised learning in image-based profiling, particularly deep neural networks, might be a novelty detection framework to identify unexpected phenotypes revealed in the drug discovery process. With deep learning it is possible to predict the properties of a molecule only from its structure. The technique requires using a convolutional neural network that is able to extract the shape of a molecule and then confront it with the information gathered about the properties.

Novel machine learning algorithms under way

Research on quantum machine learning shows that this approach should be useful for finding complex patterns in data. As biological and medical data are complex, probabilistic quantum machine learning algorithms represents a real opportunity to understand them better. Innovative pharmaceutical companies like Amgen or startups like ProteinQure have moved to apply quantum computing and quantum machine learning to drug discovery, while focusing these efforts mainly on predicting the structure of new drugs. Finally, genomics and systems biology are two important areas in which novel machine learning algorithms can be applied with a view to producing less toxic drugs based on the profound analysis of biomedical data.

The text was written in collaboration with Anna Kornakiewicz, an independent data scientist and researcher as a consultant.

AI Monthly Digest #6 – AI ethics and artificial imagination

March 7, 2019/in Data science, AI Monthly Digest /by Konrad Budek and Arkadiusz Nowaczynski

February came with the groundbreaking event of building a new state-of-the-art natural language processing neural network. And then the unthinkable happened.

Machine learning has already shown that it is a world-transforming technology used in various appliances, including drug discovery, saving endangered species and designing software for sophisticated prosthetic legs. Therefore it is natural that this should spark debates, both ethical and business-related. What reaction has there been to this in February?

1. Open-AI designed a new gold standard for natural language processing…

While in image processing artificial intelligence models outperform humans in most tasks, natural language processing (NLP) is still a challenge for machines to master. Although AI-based services are already usable with Google Translate being the pinnacle of today’s achievements, the texts produced by machines is easy to recognize by humans. But this could change soon with the latest state-of-the-art from OpenAI.

They designed a model called GPT-2 and prepared a dataset containing 8 million web pages for training. The network objective is to predict the next word given all the words to that point, which is the simplest way of doing unsupervised learning in NLP. The main improvements, in this case, are scaling up the model size (1.5B parameters) and training it on the gargantuan database of Internet text (40GB) at an unprecedented scale (32 x TPUv3).

The resulting network analyzes the starting word of a sentence and then it adds the next words to create a text based on “the most probable output”. The effect is surprisingly accurate. The model is able to recognize the type of the starting text. If the first sentence is a press title it will produce a legitimate-sounding short news article. It is also good at adopting various styles – something unimaginable for previously sturdy models which could produce problematic sentences.

Example:

Human written:

In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.

Model completion (machine-written, 10 TRIES)

The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.

Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.

Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.

Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez.

Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns.

While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.”

More examples of AI-generated texts may be found at Open-AI blog.

Moreover – the network is capable of transfer learning – it is easier to perform a slight additional training to enable the model to perform more sophisticated tasks. Transfer learning is a gateway technology for modern image recognition appliances, including art recognition and visual quality control.

Why does it matter

Alan Turing, the godfather of modern information technology, was heavily convinced that the ability to understand language is a key indicator of intelligence. Natural language, the emotions hidden beneath the words and all the cultural and societal contexts behind sentences are truly unique for human beings.

Making this part of the world approachable for machines is undoubtedly the great breakthrough and passes countless possibilities, from battling hate crime to various business appliances.

2. … and they didn’t release it to the public

With great power comes great responsibility and the events from February 2019 show it clearly. Despite best practices and established custom, Open-AI decided NOT TO release the state-of-the-art neural network to the public as an open source software.

The organization did this to prevent the neural network from being used for malicious purposes, with perpetuating fake news as a top concern. The decision sparked a debate on Reddit, Twitter and other various media, where participants argued on the safety of releasing this “potentially dangerous technology” and tackling the established model of open-sourcing the effect of research. Researchers are afraid of building the doorway for “deepfakes for texts”.

Why does it matter

AI ethics and responsibility was highlighted as one of the major AI trends 2019. The situation where researchers choose safety over progress is unusual. On the other hand, fake news is considered one of the major threats of the future and was recognized one of the most dangerous online activities in 2018 by the World Economic Forum.

With the rise of autonomous cars and IoT revolution building a frame for sophisticated AI-powered solutions to work, such dilemmas may get increasingly common.

3. China’s AI strategy at a glance (or even more)

In the last AI monthly digest, we covered the AI strategy forged in Finland, where the national pursuit for the AI appliances grew from a grassroots movement. In contrast, China has a centralized strategy to make the country AI-giant.

The Center for a New American Security (CNAS) has published a report in which they provide deeper insights into understanding China’s AI strategy. The report is a long read written by Gregory C. Allen who had a chance to meet a few times with high-ranking Chinese officials on conferences focusing on Artificial Intelligence.

Topics covered in the text:

Chinese views on importance and security of AI
Strengths and weaknesses of China’s AI Ecosystem
China’s short-term goals in AI
The role of semiconductors

The author concludes that gaining expertise in and understanding of AI developments in China should help U.S. policymakers sort out their priorities. Instead of influencing China’s competitiveness, they should focus on boosting the technological and economic competitiveness of the United States.

Why does it matter

Early adopters are usually those who earn knowledge with first failures. Thus, for now, it is wise to observe and learn.

4. Introducing Paperswithcode

Staying updated on the latest developments in machine learning is not an easy task, especially considering the fact that not all the researchers decide to make their code available. Releasing the code is a good practice, but providing only the paper that “describes the matter enough” to reproduce the effect is also correct.

Using the paperswithcode makes the process of paper-and-code matching much easier, effectively saving a lot of time for people willing to broaden their knowledge.

Why does it matter

The portal itself is a handy tool to gain knowledge and, a bit unwillingly promotes the constituted approach of making the research code public. If it gains notoriety, it may be a tool of social pressure to publish code with the research.

5. Popularity of thispersondoesnotexist.com

The neural networks’ ability to create convincing human faces is one of the benchmarks for modern AI appliances. To make the images available for the public, the StyleGan-based model publishes random face on thispersondoesnotexist.com website.

As in every aspect of life, the devil is in the details. The website is a great tool to spot the weakest parts of generated faces.

Hair – it is common to place some hair in the air, with no attachment to other or floating in an unnatural way. Sometimes the skin around hair is “sticking” to them, creating some kind of scars that are disturbing when seen. The model sometimes messes up hairstyles, mixing dreadlocks with straight and curly hair, but it is more challenging to spot
Teeth – algorithms tend to mess up with teeth, placing them unnaturally, sometimes merged or blurred.
Glasses – it is common that the eyes inside the glasses do not fit the rest of the depicted person. Sometimes it means placing eyes of an old woman, surrounded with wrinkles, on a child’s face.

Why does it matter

Making these images publicly available with commentary, which all have been made by neural networks, is an interesting way to spread knowledge about AI-related topics. The concerns about deep fakes and other illicit content created with the support of artificial intelligence are mostly based on lack of knowledge – the AI-based techniques are seen as omnipotent. But they aren’t, as can be easily seen by the example of teeth and hair.

Moreover, the initiative is another way of encouraging people to verify news sources and distrusting information and tackling the challenge of fake news and deep fakes, mentioned above.

And a bonus one:

deepsense.ai in cooperation with Google Brain has finished research on enabling reinforcement learning (RL) agents to make predictions about their upcoming actions to reduce the number of actions needed to train skills.

RL agents typically need incomparably more actions to train skill than humans and the ability to build the skill “in a head” may be a part of the answer. Starting from childhood, people repeat their actions in their minds when they seek perfection or just recall an enjoyable or pleasant time.

Following this idea, our researchers have designed neural networks that possess the imagination that enables the model to simulate the action before executing it. A more detailed story about these projects and their outcome may be found in our blogpost about artificial imagination and Arxiv paper.

Why does it matter?

Artificial imagination is basically the idea of building the world simulated only by the mind’s power. Our researchers were able not only to reduce training time, but also to replace the simulated environment with another neural network’s imagination.

Maintaining and providing the simulated environment is one of the highest costs in building reinforcement learning agents, so tackling this challenge may be a great step to making this technique more popular.

AI Monthly digest #5 – AlphaStar beats human champions, robots learn to grasp and a Finnish way to make AI a commodity

February 8, 2019/in Data science, Machine learning, AI Monthly Digest /by Konrad Budek and Arkadiusz Nowaczynski

With a ground-breaking AlphaStar performance, January kickstarted 2019 AI-related research and activities.

The January edition of AI Monthly digest brings a marvel to behold–AlphaStar beating human champions at the real-time strategy game StarCraft. To find out why that’s important, read the full story.

AlphaStar beats top world StarCraft II pro players

AlphaStar, a DeepMind-created agent, beat two world-famous professional players in StarCraft II. The agent was using a Protoss race and playing against another Protoss. The game itself can also play Zerg and Terrans, but AlphaStar is trained only to play Protoss vs. Protoss matches.
The machine defeated Dario “TLO” Wunsch, a Zerg specialist playing Protoss for the occasion, 5-0 in the first five-match round. It then made quick work of professional Protoss player Grzegorz “MaNa” Komnicz beating the champion 5:0.
A noticeable advantage AlphaStar had against both players was its access to the entire StarCraft II map at once. It is still obscured by the fog of war, but the agent doesn’t have to mimic the human’s camera moves. DeepMind prepared one other agent to address this issue and plays using a camera interface, but it lost to MaNa 0:1.
To make the matches fair, the DeepMind team reduced the Actions Per Minute (APM) ratio to a human level and ensured the machine had no advantage in terms of reaction time. Nonetheless, it was clear at crucial moments that AlphaStar had bursts of APM far above human abilities. DeepMind is aware of this and will probably do something about it in the future. For now, however, we will be content to focus on what we have seen.

How the matches went

Unlike human players, AlphaStar had employed some unorthodox yet not necessarily wrong strategies – avoiding walling the entrance to the base with buildings was the most conspicuous one. What’s more, the model used significantly more harvesting drones than pro players normally use.

Beyond its superiority in micromanagement (the art of managing a single unit and using its abilities on the battlefield), the agent didn’t display any clearly non-human strategies or abilities. However, AlphaStar was seen at its finest when it managed to win the match by managing a large number of Stalkers, the unit that is normally countered by Immortals in a rock-paper-scissors manner. As MaNa, the human player confronting the agent, noted, he had never encountered a player with such abilities. As such, the gameplay was clearly on a superhuman level, especially considering the fact that MaNa executed the counter-tactic, which failed due to AlphaStar’s superior micromanagement.

How the Deepmind team did it

The initial process of training the agent took ten days – three of supervised learning built on the basis of replays of top StarCraft II players. The team then infused the agent with reinforcement learning abilities (an approach similar to our team’s cracking Montezuma’s Revenge) and created the “AlphaStar” league to build multiple agents competing against each other. league witnessed a similar cycle with some strategies emerging and being later countered.

After that, the team selected five agents for a match with TLO. To further polish their skills, the agents were trained for another week before the match with the MaNa. As a Protoss specialist, MaNa posed a greater challenge than TLO, a Zerg-oriented player who was learning Protoss tactics only to square off against AlphaStar.
Courtesy of Blizzard, the developer of StarCraft II, Deepmind was delivered a significantly faster version of StarCraft II. This version enabled each agent in AlphaStar league to experience up to 200 years of real-time gameplay in just two weeks.

Why it matters

The AI community has grown accustomed to witnessing agents cracking Atari Classics and popular board games like chess or Go. Both environments provide a diverse set of challenges, with chess being a long-term fully observable strategy game and Atari delivering real-time experience with limited data.
StarCraft combines all manner of challenge by forcing players to follow the long-term strategy without knowledge of an opponent’s strategy and movement until it is in the line of sight of its own units (normally the battlefield is covered by the “fog of war”). Each encounter may show that a strategy needs to be fixed or adapted, as many units and strategies tend to work in a rock-paper-scissors manner, enabling players to play in a tactic-counter-tactic circle. Problem-solving in real time while sticking to a long-term strategy, constantly adapting to a changing environment and optimizing one’s efforts are all skills that can be later extrapolated to solve more challenging real-world problems.

Thus, while playing computer games is fun, the research they enable is very serious. It also lays bare the contrast between human and machine abilities. The computer was able to beat a human player after about four hundred years of constant playing. The human expert, meanwhile, was twenty-five years old, had started playing StarCraft at the age of six and had to sleep or go to school while not playing StarCraft.
Nevertheless, impressive and inspiring.

Understanding the biological brain using a digital one

According to Brain Injury Alliance Wisconsin, approximately 10% of individuals are touched by brain injuries and 5.3 million Americans (a little more than 2% of the US population) live with the effects of a brain injury. Every 23 seconds someone suffers a brain injury in the US.
Such injuries add up to $76.5 billion in annual costs once treatment, transportation and the range of indirect costs like lost productivity are considered.
While brain trauma may sometimes be responsible for the loss of speech, strokes and motor neurone disease are also to blame. Although patients lose the ability to communicate, they often remain conscious. Stephen Hawking is perhaps the most famous such person. Hawking used a speech generator, which he controlled with the muscles in his cheek. The generators can also be controlled with the eyes.
Applying neural networks to interpret the signals within the brain enabled the scientists to reconstruct speech. Summarizing the efforts, Science magazine points out that the effects are more than promising.
Alzheimer’s disease is another challenge that may be tackled with neural networks. There are no medications that heal the disease, but applying the treatment early enough makes it manageable. With Alzheimer’s, the earlier the diagnosis is made, the more effective the treatment will be. The challenge is in the diagnosis, which often comes too late for the disease to be reversible.

By feeding the neural networks with glucose PET scans, researchers from the University of California delivered a system that can diagnose the early symptoms of Alzheimer’s disease up to six years earlier than doctors do.

Why it matters

The human brain is one of the most complex devices in the universe, so understanding how it works is obviously a great challenge. Applying neural networks to treat brain-related diseases may come with a bit of irony – we need an outer, artificial brain to outthink the way our own is working.

Democratizing the AI – the Finnish way

Machine learning and artificial intelligence, in general, tend to be depicted as a black box, with no way to get to know “what the machine is thinking”. At the same time, it is often shown as a miraculous problem-solver, pulling working solutions out of seemingly nothing like a magician procuring a rabbit from a hat. But this too is a misconception.
Like every tool before it, neural networks need to be understood if they are to yield the most valuable outcomes. That’s one reason Finland aims to train its population in AI techniques and machine learning. Starting with 1% of its population (or roughly 55,000 people), the country aims to boost its society and economy by being a leader in the practical application of AI.
Initially a grassroots movement, the initiative gained the support of the government and Finland’s largest employers.

Why it matters

The biggest barrier in using AI and machine learning-powered techniques is uncertainty and doubt. Considering that people are afraid of things they don’t understand, spreading the knowledge about machine learning will support the adoption and reduce societal reluctance to adapting these tools. Moreover, understanding the mechanisms powering ML-based tools will give users a greater understanding of just what the tools are and are not capable of.

New state-of-the-art in robotic grasping

The issues Artificial Intelligence prompts frequently ignite philosophical debate and add interesting insight and inspiration. This recent paper on robot grasping is short of neither insights nor inspiration.
[bctt tweet=”The idea behind the use of reinforcement learning to control robotic arms is simple – hard-coding all the possible situations the robot may encounter is virtually impossible, but building a policy to follow is much easier” via=”no”]
What’s more, building the controller for the robotic arm requires the mountains of data coming from the sensors to be cross-combined. Every change – be it lighting, color or position of an object — can confuse the controller and result in failure.
Thus, the research team built a neural network that processes the input into the “canonical” version, stripped of the insignificant details like shades or graphical patterns – so that grasping is the only thing that matters. Ushering in a new state of the art in robotic grasping, the results are impressive.

Why do the results matter?

There are two reasons these results are important. First, building the controllers of robotic arms is now simpler. Robots that can move in non-supervised, non-hardcoded ways and grasp objects will be used in astonishing ways to improve human lives–for example, as assistants for the disabled or by augmenting the human workforce in manufacturing.
The second breakthrough is how researchers achieved their improvements. Instead of building more powerful neural networks to improve the input processing, the researchers downgraded the data into a homogenous, simplified “canonical” version of the reality. It seems that when it comes to robotic perception, Immanuel Kant was right. There are “things that exist independently of the senses or perception”, but they are unknowable–at least for a robotic observer. Only operating within a simplified reality enables the robot to perform the task.

Keep informed on state-of-the-art machine learning

With the rapidly changing ML landscape, it is easy to lose track of the latest developments. A lecture given by MIT researcher Lex Fridman is a good way to start. The video can be seen here:

Read previous editions of AI Monthly digest:

How artificial intelligence can fight hate speech in social media

January 24, 2019/in Data science, Machine learning /by Konrad Budek

With social media users numbering in the billions, all hailing from various backgrounds and bringing diverse moral codes to today’s wildly popular platforms, a space for hate speech has emerged. Internet service providers have responded by employing AI-powered solutions to address this insidious problem.

Hate speech is a serious issue. It undermines the principles of democratic society and the rules of public debate. Legal views on the matter vary. On the internet, every statement that transgresses the standards for hate speech established by a given portal (Facebook, Twitter, Wikipedia etc.) may be banned from publication. To get around such bans, numerous groups have launched platforms to exchange their thoughts and ideas. Stricter definitions of hate speech are common. They make users feel safe, which is paramount for social media sites as the presence of users is often crucial to income. And that’s where building machine learning models spotting the hate speech comes in.

What is hate speech?

The definition of hate speech varies by country and may apply to various aspects of language. Laws prohibit directing hateful speech and defamatory language toward another’s religion, ethnicity or sexuality. Many countries penalize anyone who agitates violence or genocide. Additionally, many legislatures ban symbols of totalitarian regimes and limit the freedom of assembly when ideologies like fascism or communism are involved.

In its most common form, hate speech attacks a person or group based on race, religion, ethnicity, national origin, disability, gender or sexual orientation. As regards what’s legal, the devil, as usual, is in the details. Finding the balance between freedom of speech and the protection of minority rights makes it difficult to produce a strict definition of hate speech. However, the problem has certainly grown with the rise of social media companies. The 2.27 bln active users of Facebook, who come from various backgrounds and bring diverse moral codes to the platform, have unwittingly provided a space for hate speech to emerge. Due to the international and flexible nature of the Internet, battling online hate speech is a complex task involving various parties.
Finally, there is a proven link between offensive name-calling and higher rates of suicide within migrant groups.

Why online hate speech is a problem

As a study from Pew Research Center indicates, 41% of American adults have experienced some form of online harassment. The most common is offensive name calling (experienced by 27%) and purposeful embarrassment (22%). Moreover, a significant number of American adults have experienced physical threats, sustained harassment, stalking and sexual harassment (10%, 7%, 7% and 6% respectively).

Hate speech itself has serious consequences for online behavior and general well-being. 95% of Americans consider it a serious problem. At 27%, more than one in four Americans have reported deciding not to post something when encountering hate speech toward another user. 13%, meanwhile, have stopped using a certain online platform after witnessing harassment. Ironically, protected as a form of free speech, hate speech has resulted in muting more than a quarter of internet users.

Who should address the issue

Considering both vox populi and practice, online platforms are to tackle the problem of user’s hate speech. According to the Pew Research Center report cited above, 79% of Americans say that online service and social network providers are responsible for addressing harassment. In Germany, companies may face a fine of up to 50m euro if they fail to remove within 24 hours illegal material, including fake news and hate speech.

Hate speech is not always as blatant as calling people names. It can come in many subtler forms, posing as neutral statements or even care. That’s why building more sophisticated AI models that can recognize even the subtlest forms of hate speech is called for.

How those models should be built

When building a machine learning-powered hate speech detector, the first challenge is to build and label the dataset. Given that the differences between hate speech and non-hate speech are highly contextual, constructing the definition and managing the dataset is a huge challenge. The context may depend on:

The context of the discussion – historical texts full of outdated expressions may be automatically (yet falsely) classified as hate speech
- Example: Mark Twain’s novels use insulting language; citing them may set off hate speech bells.
How the language is used – in many countries, hate speech used for artistic purposes is tolerated.
- Example: Hip-hop often uses misogynistic language while heavy metal (especially the more extreme sub-genres) is rife with anti-religious lyrics.
The relationship of the speaker to the group being hated – the members of a group are afforded more liberties with using aggressive or insulting language when addressing other members of that group than are those who are not a part of it.
- Example: the term “sans-cullottes” was originally coined to ridicule the opponents of conservatives. It literally meant “people with no trousers” and was aimed at the working class, members of whom wore long trousers instead of the fashionable short variety. The term went on to enter the vernacular of the working classes in spite of its insulting origins.

Irony and sarcasm pose yet another challenge. According to Poe’s law, without smileys or other overt signs from the writer, ironic statements made online are indistinguishable from serious ones. In fact, the now-ubiquitous emoticons were invented by professors at Carnegie Mellon University to avoid mistakes.
When the dataset is ready, building a blacklist of the most common epithets and hate-related slurs may be helpful, as automation-based blacklist models are effective 60% of the time in spotting online hate speech (based on our in-house benchmarks). Building both supervised and unsupervised learning models to spot the new combinations of harmful words or finding existing ones may raise that effectiveness further. Hate speech is dynamic and thus evolves rapidly as new forms and insulting words emerge. By keeping an eye on conversations and general discourse, machine learning models can spot suspicious new phrases and alert administration.

A formula of hate

An automated machine learning model is able to spot the patterns of hate speech based on word vectors and the positions of words with certain connotations. Thus, it is easier to spot emerging hate speech that went undetected earlier, as current politics or social events may trigger new forms of online aggression.
Unfortunately, people spreading hate have shown serious determination to overcome automated systems of spotting hate speech by combining common ways of fooling machines (like using acronyms and euphemisms) and perpetuate hate.

Challenges and concerns

One of the main concerns in building machine learning models is finding a balance between the model’s vigilance and the number of false positives it returns. Considering the uneasy relations between hate speech and freedom of speech, producing too many false positives may be considered by users an attempt at censorship.
Another challenge is to build the dataset and label the data to train the model to recognize hate speech. As machines themselves are truly neutral, the person responsible for the dataset may be biased or at least influenced to profile the hate speech recognition model. Thus, the model may be built to purposefully produce false-positives in order to reduce the prevalence of certain views in a discussion.

Five top artificial intelligence (AI) trends for 2019

January 9, 2019/in Data science, Machine learning /by Konrad Budek

As the recently launched AI Monthly digest shows, significant improvements, breakthroughs and game-changers in machine learning and AI are months or even weeks away, not years. It is, therefore, worth the challenge to summarize and show the most significant AI trends that are likely to unfold in 2019, as machine learning technology becomes one of the most prominent driving forces in both business and society.
According to a recent Deloitte study, 82% of companies that have already invested in AI have gained a financial return on their investment. For companies among all industries, the median return on investment from cognitive technologies is 17%.
AI is transforming daily life and business operations in a way seen during previous industrial revolutions. Current products are being enhanced (according to 44% of respondents), internal (42%) and external (31%) operations are being optimized and better business decisions are being made (35%).
With that in mind, it is better to see the “Trend” as a larger and more significant development than a particular technology or advancement. That’s why chatbots or autonomous cars are not so much seen as particular trends, but rather as separate threads in the fabric that is AI.
That distinction aside, here are five of the most significant and inspiring artificial intelligence trends to watch in 2019.

1. Chatbots and virtual assistants ride the lightning

The ability to process natural language is widely considered a hallmark of intelligence. In 1950, Alan Turing proposed his famous test to determine if a particular computer is intelligent by asking the ordinary user to determine if his conversational partner is a human or a machine.
The famous test was initially passed in 1966 by ELIZA software, though it had nothing to do with natural language processing (NLP) – it was just a smart script that seemed to understand text. Today’s NLP and speech recognition solutions are polished enough not only to simulate understanding but also to produce usable information and deliver business value.
While still far from perfect, NLP has gained a reputation among businesses embracing chatbots. PwC states that customers prefer to talk with companies face-to-face but chatbots are their second preferred channel, slightly outperforming email. With their 24/7 availability, chatbots are perfect for emergency response (46% of responses in the PwC case study), forwarding conversations to the proper employee (40%) and placing simple orders (33%). Juniper Research predicts that chatbots will save companies up to $8bln annually by 2022.
NLP is also used in another hot tech trend–virtual assistants. According to Deloitte, 64% of smartphone owners say they use their virtual assistant (Apple Siri, Google’s Assistant) compared to 53% in 2017.
Finally, Gartner has found that up to 25% of companies will have integrated a virtual customer assistant or a chatbot into their customer service by 2020. That’s up from less than 2% in 2017.

2. Reducing the time needed for training

Academic work on AI often focuses on reducing the time and computing power required to train a model effectively, with the goal of making the technology more affordable and usable in daily work. The technology of artificial neural networks has been around for a while (theoretical models were designed in 1943), but it works only when there are enough cores to compute machine learning models. One way to ensure such cores are present is to design more powerful hardware, though this comes with limitations. Another approach is to design new models and improve existing ones to be less computing hungry.
AlphaGo, the neural network that vanquished human GO champion Lee Sidol, required 176 GPUs to be trained. AlphaZero, the next iteration of the neural network GO phenom, gained skills that had it outperforming AlphaGo in just three days using 4 TPUs.
Expert augmented learning is one of most interesting ways to reduce the effort required to build reinforcement-based models or at least ones that are reinforcement learning-enhanced. Contrary to policy-blending, expert augmented learning allows data scientists to channel their knowledge not only from another neural network but also from a human expert or another machine. Researchers at deepsense.ai have recently published a paper on using transfer learning to break Montezuma’s Revenge, a game that reinforcement learning agents had long struggled to break.
Another way to reduce the time needed to train a model is to optimize the hardware infrastructure required. Google Cloud Platform has offered a cloud-based tailored environment for building machine learning models without the need for investing in on-prem infrastructure. Graphics card manufacturer Nvidia is also pushing the boundaries, as GPUs tend to be far more effective in machine learning than CPUs.
Yet another route is to scale and redesign the architecture of neural networks to use existing resources in the most effective way possible. With its recently developed GPipe infrastructure, Google has been able to significantly boost the performance of Generative Adversarial Networks on an existing infrastructure. By using GPipe, researchers were able to improve the performance of ImageNet Top-1 Accuracy (84.3% vs 83.5%) and Top-5 Accuracy (97.0% vs 96.5%), making the solution the new state-of-the-art.

3. Autonomous vehicles’ speed rising

According to PwC estimates, 40% of mileage in Europe could be covered by autonomous vehicles by 2030. Currently, most companies are still developing the technology behind these machines. We are proud to say that deepsense.ai is contributing to the push. The process is driven mostly by the big social and economic benefits involved in automating as many driving processes as possible.
According to the US Department of Transportation, 63.3% of the $1,139 billion of goods shipped in 2017 were moved on roads. Had autonomous vehicles been enlisted to do the hauling, the transport could have been organized more efficiently, and the need for human effort vastly diminished. Machines can drive for hours without losing concentration. Road freight is globally the largest producer of emissions and consumes more than 70% of all energy used for freight. Every optimization made to fuel usage and routes will improve both energy and time management.
The good news here is that there are already advanced tests of the technology. Volvo has recently introduced Vera, the driverless track aimed at short-haul transportation in logistics centers and ports. Its fleet of cars is able to provide a constant logistics stream of goods with little human involvement.
In a related bid, US grocery giant Kroger recently started tests of unmanned delivery cabs, sans steering wheel and seats, for daily shopping. Bolder still are those companies (including Uber) testing their autonomous vehicles on the roads of real towns, while others build models running in sophisticated simulators.
With Kroger, Uber and Google leading the way, other companies are sure to fall into line behind them, forming one of most important AI trends 2019.

4. Machine learning and artificial intelligence will be democratized and productionized

There would be no machine learning without data scientists, of which there remain precious few, at least of the skilled variety. Job postings for data scientists rose 75% between 2015 and 2018 at indeed.com while job searches for this position rose 65%. According to Glassdoor data, data scientist was the hottest job in 2018. Due to the popularization of big data, artificial intelligence and machine learning, the demand for data science professionals will continue to rise. And that not only enterprise but also scientific researchers seek their skills certainly bodes well for the profession.
Despite being associated with high-tech companies, machine learning techniques are becoming more common in solving science-related problems. In the last quarter of 2018, Deepmind unveiled a tool to predict the way proteins fold. Another project enabled scientists to derive the laws of physics from fictional universes.
According to O’Reilly data, 51% of surveyed organizations already use data science teams to develop AI solutions for internal purposes. The adoption of AI tools will no doubt be one of the most important AI trends in 2019, especially as business and tech giants are not the only organizations using AI in their daily work.

5. AI responsibility and transparency

Last but not least, as the impact of machine learning on business grows, so too does the social and legal impact. On the heels of the first fatal accident involving an autonomous car, the question of who is responsible for crashes and the famous trolley problem are getting more important.
At issue here, first and foremost, is hidden bias in data sets, a problem for any company using AI to power-up daily operations. That includes Amazon, which had put AI in charge of preprocessing resumes. Trained with 10 years worth of various resumes, the system was unintentionally biased against women applying for tech positions. With the rising adoption of machine learning models in various industries, the transparency of artificial intelligence will be on the rise. The issue of countering bias unconsciously developed within datasets and taken by machine learning models as truth incarnate is being discussed seriously by tech giants like Salesforce. The machine learning community has also taken up the problem: there is a Kaggle competition aimed at building unbiased and cultural context-agnostic image recognition models to use in computer vision.
Finally, as I alluded earlier, the question of who is responsible for actions taken by AI-powered devices and the famous trolley problem are both moving to the fore. If a self-driving car had a choice, should it hit an elderly person or a child? Focus on saving the life of a driver or a person walking by? According to a global study, the answers depend heavily on the culture the responder grew up in. When facing the extreme situation of a car accident today, it is the driver who is solely responsible for his or her choices. When the car is autonomous, however, and controlled by a virtual agent, all the choices are made by a neural network, which raises some very unsettling questions.
Of course, such problems are not confined to the realm of autonomous vehicles. Machine learning-powered applications are getting more and more attention as a tool for supporting medical treatment. Medical data is expected to rise at an exponential rate, with a compound annual growth rate of 36%. Considering the high level of standardization within diagnostic data, medical data is ripe for utilizing machine learning models, which can be employed to augment and support the treatment process.
When thinking about AI trends 2019, bank on more transparent and socially responsible models being built.

The take-away – the social context will be central to AI trends 2019

No longer are AI and machine learning confined to pure tech; they now have an impact on entire businesses and the whole of society. The common comparison with the steam engine revolution is an apt one – machine learning models will digitally transform both big and small business in ways never before seen.

Given that, picking AI trends 2019 only by selecting technology would be to miss a vital aspect of ongoing changes. That is, they are as ubiquitous as they are far-reaching.

AI Monthly digest #4 – artificial intelligence and music, a new GAN standard and fighting depression

January 7, 2019/in Data science, Machine learning, AI Monthly Digest /by Konrad Budek and Arkadiusz Nowaczynski

December brought both year-end summaries and impressive research work pushing the boundaries of AI.

Contrary to many hi-tech trends, artificial intelligence is a popular term in society and there are many (not always accurate) views on the topic. The December edition of AI Monthly digest both shares the news about the latest developments and addresses doubts about this area of technology.
Previous editions of AI Monthly digest:

Tuning up artificial intelligence and music

Machines are getting better at image recognition and natural language processing. But several fields remain unpolished, leaving considerable room for improvement. One of these fields is music theory and analytics.
Music has different timescales, as some parts repeat at scale of seconds while others extend throughout an entire composition and sometimes beyond. Moreover, music composition employs a great deal of repetition.

Google’s Magenta-designed model leverages the relative attention to spot how far two tokens (motifs) are, and produced convincing and quite relaxing pieces of piano music. The music it generated generally evokes Bach more than Bowie, though.
Researchers provided samples of both great and flawed performances. Although AI-composed samples still refer to classical music, there are more jazz-styled improvisations.

Further information about studies on artificial music can be found on Magenta’s Music Transformer website.
Why does it matter?
Music and sound is another type of data machine learning can analyze. Pushing research on music further will deepen our knowledge of music and styles in a way that AI made Kurt Vonnegut’s dream of analyzing literature a reality.
Furthermore, the music industry may be the next to leverage data and the knowledge and talents of computer scientists. Apart from tuning the recommendation engines for streaming services, they may contribute more to the creation of music. A keyboard is after all a musical instrument.

2. The machine style manipulator

Generating fake images from real ones is a thing. Generative Adversarial Networks enhance their training abilities by analyzing real images, generating fake ones and then training to be as good as possible in determining which is real and what is shown in the images.

The challenge neural networks and those who design them fact is in producing convincing images of people, cars, or anything else that is to be recognized by the networks. In a recent research paper, the group behind the “one hour of fake celebrity faces” project introduced a new neural network architecture that separates high-level attributes and stochastic variations. In the case of human images, the high-level attribute may be a pose while freckles or the hairdo are stochastic variations.
In a recent video, researchers show the results of applying a style-based generator and manipulating styles later to produce different types of images.

The results are impressive – researchers were able to produce convincing images of people from various ethnic backgrounds. By controlling different levels of styles, researchers were able to tune up everything on image – from gender and ethnicity to the shape of glasses worn.
Why does it matter?
That’s basically the next state-of-the-art in GAN networks, the best-performing image recognition technology used. Producing fancy-looking fake images of faces and houses can significantly improve the performance of image recognition models. Ultimately, however, this technology may be a life saver, especially when applied in medical diagnosis, for example in diabetic retinopathy.

3. AI failures – errare (not only) humanum est

2018 saw significant improvements in machine learning techniques and artificial intelligence proved even further how useful it is. However, using it in day-to-day human life will not be without its challenges.

In his 1896 novel “An Outcast of the Islands”, Joseph Conrad wrote “It’s only those who do nothing that make no mistakes”. This idea can also be applied to theoretically mistake-proof machines. Apart from inspiring successes, 2018 also witnessed some significant machine learning failures:

Amazon’s gender-biased AI recruiter – the machine learning model designed to pre-process the resumes sent to the tech giant overlooked female engineers due to bias in the dataset. The reason was obvious – the tech industry is male-dominated. As algorithms have neither common sense nor social skills, it assumed that women are just not a good match for the tech positions the company was trying to fill. Amazon ditched the flawed recruiting tool, yet the questions about hidden bias in datasets remain.
Uber’s fatal autonomous car crash – the story of fatal crash is a bitter lesson for all autonomous car manufacturers. Uber’s system not only detected the pedestrian it hit while driving, but also autonomously decided to proceed and ignore warnings, killing 49-year old Elaine Herzberg.
World Cup predictions gone wrong – The World Cup gave us another bitter lesson, this time for predictive analytics. While the model built to predict brackets may have been sophisticated, it failed entirely. According to its predictions, Germany should have met Brazil in the finals. Instead, the German team didn’t manage to get out of its group while Brazil bent the knee before South Korea. The final came down to France versus Croatia, an unthinkable combination, both for machine learning and football enthusiasts around the world. The case was further described in our blogpost about failure in predictive analytics.

More examples of AI failures can be found in the Synced Review Medium blogpost.
Why does it matter?
Nobody’s perfect. Including machines. That’s why users and designers need to be conscious of the need to make machine learning models transparent. What’s more, it is the next voice to ensure that machine learning model results are validated – a step that is tempting to overlook for early adopters.

4. Smartphone-embedded AI may detect the first signs of depression

A group of researchers from Stanford University has trained a model with pictures and videos of people who are depressed and people who are not. The model analyzed all the signals the subjects sent, including tone of voice, facial expressions and general behaviour. These were observed during interviews conducted by an avatar controlled by a real physician. The model proved effective in detecting depression more than 80% of the time. The machine was able to recognize slight differences between people suffering from depression and people who were not.
Why does it matter?
According to WHO, depression is the leading cause of disability worldwide. If not cured, it can lead to suicide, the second most common cause of death among 15-29-year-olds.

One barrier to helping people suffering from depression is inaccurate assessment. There are regions in the world where less than 10% of people have access to proper treatment. What’s more, mental illness is often stigmatized, and treatment is both costly and hard to access. These factors, and the fact that early symptoms are easily overlooked, lead many patients to avoid looking for care and medical support.
The experiment is a step toward building an automated and affordable system for spotting signs of depression early on, when the chance for a cure is highest.

5. So just how can AI hurt us?

Machine learning is one of the most exciting technologies of the twenty-first century. But science fiction and common belief have provided no lack of doomsday scenarios of AI harming people or even taking over the world. Dispelling the myths and disinformation and providing knowledge should be a mission all AI-developing companies. If you’re new to the discussion, here’s an essay addressing the threat of AI.
Why does it matter?
Leaving the doubts unaddressed may result in bias and prejudice when making decisions, both business and private ones. The key to making the right decisions is to be informed on all aspects of the issue. Pretending that Stephen Hawking’s and Elon Musk’s warnings about the cataclysmic risks AI poses were pointless would indeed be unwise.
On the other hand, the essay addresses less radical fears about AI, like hidden bias in datasets leading to machine-powered discrimination or allowing AI to go unregulated.
That’s why the focus on machine morality and the transparency of machine learning models is so important and comes up so frequently in AI Monthly digest.

Summary

December is the time to go over the successes and failures of the past year, a fact that applies equally to the machine learning community. Facing both the failures and challenges provides an opportunity to address common issues and make the upcoming work more future-proof.

Don’t waste the power. AI supply chain management

December 13, 2018/in Data science /by Konrad Budek

Just imagine what your business could do if it recouped 6500 man-hours annually. Or pick up maybe a power supply savings comparable to the electricity used by the entire population of Austin, TX.

This article will describe:

Why managing modern logistics requires superhuman abilities
The key challenges in supply chain management
How much money can be saved when AI is applied to supply chain management

According to Material, Handling and Logistics Magazine, the average US business loses $171,340 per year due to repetitive, mundane and silly tasks like searching for order numbers, processing papers and calculating the value of orders. Yearly, that’s the equivalent of 6,500 man-hours wasted. But they can be reclaimed with modern technology.

Needle in a lorry

The complex flow of goods and services is managed with data and information systems, with the historical orders data, forecasting (for example weather forecasts) and estimations done by the most experienced logistics specialists. Almost every company has tremendous room to boost optimization, which is devilishly complicated.
As a Goldman Sachs analyst quoted by The Economist earlier this year said, there are around 15 septillion (trillion trillion) ways to deliver 25 packages in a lorry or van to their destinations. Given the human brain’s limited perception, sorting out the best of them is no simple task.

A Kaggle competition which the deepsense.ai team took third place in provides a glimpse into how complex logistics challenges can be solved. You can read the story of the algorithm written to manage the delivery of 100,000 christmas gifts from the North Pole here.
Considering the magnitude of the challenge, it should come as no surprise that McKinsey estimates industry outlays on AI for supply chains could reach 1.3-2 trillion dollar.

Where does innovation meet possibility?

Manufacturing is all about striking the right balance between satisfying demand and keeping stock low. The inability to deliver product leads to business losses and results in clients seeking new vendors when their current ones run out of stock. On the other hand, overstocking freezes assets in a warehouse and saps flexibility, as the products in stock–a warehouse full of ice in the middle of Winter–may not suit customers’ demands.
But the supply chain in manufacturing doesn’t only address stock levels. It is also about managing suppliers and shipping, and assigning employees to see to all the dull paperwork and perform repetitive tasks.

Combining computer vision technology, deep learning-powered image recognition and mobile devices, Coca-Cola reduced the work involved in placing orders and managing supplier relationships.
The company trained a neural network to recognize single bottles in a display cooler. With an algorithm that could count the bottles using just a picture taken with an iPad or iPhone, there was no need to send a sales representative to every cooler to inventory the bottles inside.
The information is combined with external data about the weather and past seasonal variations, yielding more precise analysis and predictions. With more than 16 million Coca-Cola coolers spread across the globe, the potential for optimization with AI is clearly enormous.

Power management in supply chain management

With the snowball effect, savings on commodities may provide significant profits for business. Being far from manufacturing, Google proved that putting AI in charge of power supply management may lead to considerable savings. DeepMind analyzed historical data on temperature and the cooling system performance in one of Google’s data centers. Using the data, the system optimized resource management without adding new components or a system redesign.

The data center slashed its cooling bill by 40% and overall power consumption by 15%

Most impressively, the data center slashed its cooling bill by 40%, overall power consumption by 15% and the energy it consumed at a rate equivalent to 350,000 US households. Given that there are 2,58 people in the average household, that is slightly less (903 000) than the number of people living in Austin, the capital of Texas (947,890).

Outsmarting logistics

According to another McKinsey study, AI-enhanced supply chain management may boost accuracy by optimizing stock replenishment with 20-50% less forecasting errors. Thanks to AI, transport-related costs are expected to decrease by 5 to 10% and to warehousing and supply an even more impressive 25-40%. Overall inventory reductions of 20-50% are also feasible.

As algorithms efficiently process vast amounts of data with ease, the prospect of further AI-based supply chain development leads to some science-fictionesque visions, including the unassisted anticipation of upcoming orders or autonomous trucks shipping goods on optimal routes across entire continents.
Because AI models are usually tailored to solve some of the biggest challenges in business, an increasing number of companies are recruiting data scientists and building their own AI teams. That task itself may fall under the thorny problems category. Finding a reliable partner that will share its knowledge and support development of an effective AI team may be the way to outsmart not only the competition but all the challenges logistics must deal with today.

Four levels of predictive maintenance

Staying on track

As sounding brass

Endless possibilities

A bigger version of GPT-2 released to the public

Talking heads unleashed

Modern Talking – recreating the voice of Joe Rogan

Machine learning-powered translations increase trade by 10,9%

A practical approach to AI in Finland

What is demand forecasting?

What is the purpose of demand forecasting?

Demand forecasting techniques

How to choose the right demand forecasting method – indicators

How AI-based demand forecasting can help a business

How to start demand forecasting – a short guide

Common pitfalls to avoid when building a demand forecasting solution

AI in demand forecasting: final thoughts

The potential of AI in drug discovery

Challenges in machine learning for drug discovery

Integrating biomedical data with computational approaches

Genetic data analysis and personalized medicine

Building and getting insight from databases and datasets

Standard machine learning approaches for genetics and genomics

Machine learning approaches for network analysis of biomedical data

Machine learning algorithms in image analysis for drug discovery

Novel machine learning algorithms under way

1. Open-AI designed a new gold standard for natural language processing…

2. … and they didn’t release it to the public

3. China’s AI strategy at a glance (or even more)

4. Introducing Paperswithcode

5. Popularity of thispersondoesnotexist.com

And a bonus one:

AlphaStar beats top world StarCraft II pro players

How the matches went

How the Deepmind team did it

Why it matters

Understanding the biological brain using a digital one

Why it matters

Democratizing the AI – the Finnish way

Why it matters

New state-of-the-art in robotic grasping

Why do the results matter?

Keep informed on state-of-the-art machine learning

What is hate speech?

Why online hate speech is a problem

Who should address the issue

How those models should be built

A formula of hate

Challenges and concerns

1. Chatbots and virtual assistants ride the lightning

2. Reducing the time needed for training

3. Autonomous vehicles’ speed rising

4. Machine learning and artificial intelligence will be democratized and productionized

5. AI responsibility and transparency

The take-away – the social context will be central to AI trends 2019

Tuning up artificial intelligence and music

2. The machine style manipulator

3. AI failures – errare (not only) humanum est

4. Smartphone-embedded AI may detect the first signs of depression

5. So just how can AI hurt us?

Summary

Needle in a lorry

Where does innovation meet possibility?

Power management in supply chain management

Outsmarting logistics

Contact us

Locations

Let us know how we can help

Services

Resources

About us

Support

Join our community