deepsense.ai
  • Success stories
  • Solutions
    • Increase sales
    • Reduce costs
    • Optimize processes
    • Build data strategy
    • Train your team
  • R&D Hub
  • Blog
  • About us
    • Our story
    • Management
    • US operations
    • Advisory Board
    • Press center
    • Careers
  • Contact
  • Menu
A comprehensive guide to demand forecasting

A comprehensive guide to demand forecasting

May 28, 2019/in Blog posts, Data science, Machine learning, Popular posts /by Konrad Budek and Piotr Tarasiewicz

Everything you need to know about demand forecasting – from the purpose and techniques to the goals and pitfalls to avoid.

Essential since the dawn of commerce and business, demand forecasting enters a new era of big-data rocket fuel.

What is demand forecasting?

The term couldn’t be clearer: demand forecasting forecasts demand. The process of predicting the future involves processing historical data to estimate the demand for a product. An accurate forecast can bring significant improvements to supply chain management, profit margins, cash flow and risk assessment.

What is the purpose of demand forecasting?

Demand forecasting is done to optimize processes, reduce costs and avoid losses caused by freezing up cash in stock or being unable to process orders due to being out of stock. In an ideal world, the company would be able to satisfy demand without overstocking.

Demand forecasting techniques

Demand forecasting is an essential component of every form of commerce, be it retail, wholesale, online, offline or multichannel. It has been present since the very dawn of civilization when intuition and experience were used to forecast demand.

Sybilla – deepsense.ai’s demand forecasting tool

More recent techniques combine intuition with historical data. Modern merchants can dig into their data in a search for trends and patterns. At the pinnacle of these techniques, are demand forecasting machine learning models, including gradient boosting and neural networks, which are currently the most popular ones and outperform classic statistics-based methods.

The basis of more recent demand forecasting techniques is historical data from transactions. These are data that sellers collect and store for fiscal and legal reasons. Because they are also searchable, these data are the easiest to use.

Sybilla – deepsense.ai’s demand forecasting tool

How to choose the right demand forecasting method – indicators

As always, selecting the right technique depends on various factors, including:

  • The scale of operations – the larger the scale, the more challenging processing the data becomes.
  • The organization’s readiness – even the large companies can operate (efficiency aside) on fragmented and messy databases, so the technological and organizational readiness to apply more sophisticated demand forecasting techniques is another challenge.
  • The product – it is easier to forecast demand for an existing product than for a newly introduced one. When considering the latter, it is crucial to forming a set of assumptions to work on. Owning as much information about the product as possible is the first step, as it allows the company to spot the similarities between particular goods and search for correlations in the buying patterns. Spotting an accessory that is frequently bought along with the main product is one example.

How demand forecasting can help a business

Demand forecasting and following sales forecasting is crucial to shaping a company’s logistics policy and preparing it for the immediate future. Among the main advantages of demand forecasting are:

  • Loss reduction – any demand that was not fulfilled should be considered a loss. Moreover, the company freezes its cash in stock, thus reducing liquidity.
  • Supply chain optimization – behind every shop there is an elaborate logistics chain that generates costs and needs to be managed. The bigger the organization, the more sophisticated and complicated its inventory management must be. When demand is forecast precisely, managing and estimating costs is easier.
  • Increased customer satisfaction – there is no bigger disappointment for consumers than going to the store to buy something only to return empty-handed. For a business, the worst-case scenario is for said consumers to swing over to the competition to make their purchase there. Companies reduce the risk of running out of stock–and losing customers–by making more accurate predictions.
  • Smarter workforce management – hiring temporary staff to support a demand peak is a smart way for a business to ensure it is delivering a proper level of service.
  • Better marketing and sales management – depending on the upcoming demand for particular goods, sales and marketing teams can shift their efforts to support cross- and upselling of complementary products,
  • Supporting expert knowledge – models can be designed to build predictions for every single product, regardless of how many there are. In small businesses, humans handle all predictions, but when the scale of the business and the number of goods rises, this becomes impossible. Machine learning models extend are proficient at big data processing.

How to start demand forecasting – a short guide

Building a demand forecasting tool or solution requires, first and foremost, data to be gathered.

While the data will eventually need to be organized, simply procuring it is a good first step. It is easier to structure and organize data and make them actionable than to collect enough data fast. The situation is much easier when the company employs an ERP or CRM system, or some other form of automation, in their daily work. Such systems can significantly ease the data gathering process and automate the structuring.

Sybilla – deepsense.ai’s demand forecasting tool

The next step is building testing scenarios that allow the company to test various approaches and their impact on business efficiency. The first solution is usually a simple one, and is a good benchmark for solutions to come. Every next iteration should be tested to see if it is performing better than the previous one.

Historical data is usually everything one needs to launch a demand forecasting project, and obviously, there are significantly less data on the future. But sometimes it is available, for example:

  • Short-term weather forecasts – the information about upcoming shifts in weather can be crucial in many businesses, including HoReCa and retail. It is quite intuitive to cross-sell sunglasses or ice cream on sunny days.
  • The calendar – Black Friday is a day like no other. The same goes for the upcoming holiday season or other events that are tied to a given date.

Sources of data that originate from outside the company make predictions even more accurate and provide better support for making business decisions.

Common pitfalls to avoid when building a demand forecasting solution

There are numerous pitfalls to avoid when building a demand forecasting solution. The most common of them include:

  • The data should be connected with the marketing and ads history – a successful promotion results in a significant change in data, so having information about why it was a success makes predictions more accurate. If machine learning was used to make the predictions, the model could have misattributed the changes and made false predictions based on wrong assumptions.
  • New products with no history – when new products are introduced, demand must still be estimated, but without the help of historical data. The good news here is that great strides have been made in this area, and techniques such as product DNA can help a company uncover similar products its past/current portfolio. Having data on similar products can boost the accuracy of prediction for new products.
  • The inability to predict the weather – weather drives demand in numerous contexts and product areas and can sometimes be even more important than the price of a product itself! (yes, classical economists would be very upset). The good news is that even if you are unable to predict the weather, you can still use it in your model to explain historical variations in demand.
  • Lacking information about changes – In an effort to support both short- and long-term goals, companies constantly change their offering and websites. When the information about changes is not annotated in the data, the model encounters sudden dwindles and shifts in demand with apparently no reason. In the reality, it is usually a minor issue like changing the inventory or removing a section from website.
  • Inconsistent portfolio information – predictions can be done only if the data set is consistent. If any of the goods in a portfolio have undergone a name or ID change, it must be noted in order not to confuse the system or miss out on a valuable insight.
  • Overfitting the model – a vicious problem in data science. A model is so good at working on the training dataset that it becomes inflexible and produces worse predictions when new data is delivered. Avoiding overfitting is down to the data scientists.
  • Inflexible logistics chain – the more flexible the logistics process is, the better and more accurate the predictions will be. Even the best demand forecasting model is useless when the company’s logistics is a fixed process that allows no space for changes.

Sybilla – deepsense.ai’s demand forecasting tool

Summary

Demand and sales forecasting is a crucial part of any business. Traditionally it has been done by experts, based on know-how honed through experience. With the power of machine learning it is now possible to combine the astonishing scale of big data with the precision and cunning of a machine-learning model. While the business community must remain aware of the multiple pitfalls it will face when employing machine learning to predict demand, there is no doubt that it will endow demand forecasting with awesome power and flexibility.

https://deepsense.ai/wp-content/uploads/2019/05/A-comprehensive-guide-to-demand-forecasting.jpg 337 1140 Konrad Budek https://deepsense.ai/wp-content/uploads/2019/04/DS_logo_color.svg Konrad Budek2019-05-28 11:24:422019-08-07 09:47:18A comprehensive guide to demand forecasting
Five top artificial intelligence (AI) trends for 2019

Five top artificial intelligence (AI) trends for 2019

January 9, 2019/in Blog posts, Data science, Machine learning, Popular posts /by Konrad Budek

As the recently launched AI Monthly digest shows, significant improvements, breakthroughs and game-changers in machine learning and AI are months or even weeks away, not years. It is, therefore, worth the challenge to summarize and show the most significant AI trends that are likely to unfold in 2019, as machine learning technology becomes one of the most prominent driving forces in both business and society.
According to a recent Deloitte study, 82% of companies that have already invested in AI have gained a financial return on their investment. For companies among all industries, the median return on investment from cognitive technologies is 17%.
AI is transforming daily life and business operations in a way seen during previous industrial revolutions. Current products are being enhanced (according to 44% of respondents), internal (42%) and external (31%) operations are being optimized and better business decisions are being made (35%).
With that in mind, it is better to see the “Trend” as a larger and more significant development than a particular technology or advancement. That’s why chatbots or autonomous cars are not so much seen as particular trends, but rather as separate threads in the fabric that is AI.
That distinction aside, here are five of the most significant and inspiring artificial intelligence trends to watch in 2019.

1. Chatbots and virtual assistants ride the lightning

AI Trends 2019 - chatbot
The ability to process natural language is widely considered a hallmark of intelligence. In 1950, Alan Turing proposed his famous test to determine if a particular computer is intelligent by asking the ordinary user to determine if his conversational partner is a human or a machine.
The famous test was initially passed in 1966 by ELIZA software, though it had nothing to do with natural language processing (NLP) – it was just a smart script that seemed to understand text. Today’s NLP and speech recognition solutions are polished enough not only to simulate understanding but also to produce usable information and deliver business value.
While still far from perfect, NLP has gained a reputation among businesses embracing chatbots. PwC states that customers prefer to talk with companies face-to-face but chatbots are their second preferred channel, slightly outperforming email. With their 24/7 availability, chatbots are perfect for emergency response (46% of responses in the PwC case study), forwarding conversations to the proper employee (40%) and placing simple orders (33%). Juniper Research predicts that chatbots will save companies up to $8bln annually by 2022.
NLP is also used in another hot tech trend–virtual assistants. According to Deloitte, 64% of smartphone owners say they use their virtual assistant (Apple Siri, Google’s Assistant) compared to 53% in 2017.
Finally, Gartner has found that up to 25% of companies will have integrated a virtual customer assistant or a chatbot into their customer service by 2020. That’s up from less than 2% in 2017.

2. Reducing the time needed for training

AI Trends 2019 training time reduction
Academic work on AI often focuses on reducing the time and computing power required to train a model effectively, with the goal of making the technology more affordable and usable in daily work. The technology of artificial neural networks has been around for a while (theoretical models were designed in 1943), but it works only when there are enough cores to compute machine learning models. One way to ensure such cores are present is to design more powerful hardware, though this comes with limitations. Another approach is to design new models and improve existing ones to be less computing hungry.
AlphaGo, the neural network that vanquished human GO champion Lee Sidol, required 176 GPUs to be trained. AlphaZero, the next iteration of the neural network GO phenom, gained skills that had it outperforming AlphaGo in just three days using 4 TPUs.
Expert augmented learning is one of most interesting ways to reduce the effort required to build reinforcement-based models or at least ones that are reinforcement learning-enhanced. Contrary to policy-blending, expert augmented learning allows data scientists to channel their knowledge not only from another neural network but also from a human expert or another machine. Researchers at deepsense.ai have recently published a paper on using transfer learning to break Montezuma’s Revenge, a game that reinforcement learning agents had long struggled to break.
Another way to reduce the time needed to train a model is to optimize the hardware infrastructure required. Google Cloud Platform has offered a cloud-based tailored environment for building machine learning models without the need for investing in on-prem infrastructure. Graphics card manufacturer Nvidia is also pushing the boundaries, as GPUs tend to be far more effective in machine learning than CPUs.
Yet another route is to scale and redesign the architecture of neural networks to use existing resources in the most effective way possible. With its recently developed GPipe infrastructure, Google has been able to significantly boost the performance of Generative Adversarial Networks on an existing infrastructure. By using GPipe, researchers were able to improve the performance of ImageNet Top-1 Accuracy (84.3% vs 83.5%) and Top-5 Accuracy (97.0% vs 96.5%), making the solution the new state-of-the-art.

3. Autonomous vehicles’ speed rising

AI trends 2019 autonomous cars
According to PwC estimates, 40% of mileage in Europe could be covered by autonomous vehicles by 2030. Currently, most companies are still developing the technology behind these machines. We are proud to say that deepsense.ai is contributing to the push. The process is driven mostly by the big social and economic benefits involved in automating as many driving processes as possible.
According to the US Department of Transportation, 63.3% of the $1,139 billion of goods shipped in 2017 were moved on roads. Had autonomous vehicles been enlisted to do the hauling, the transport could have been organized more efficiently, and the need for human effort vastly diminished. Machines can drive for hours without losing concentration. Road freight is globally the largest producer of emissions and consumes more than 70% of all energy used for freight. Every optimization made to fuel usage and routes will improve both energy and time management.
The good news here is that there are already advanced tests of the technology. Volvo has recently introduced Vera, the driverless track aimed at short-haul transportation in logistics centers and ports. Its fleet of cars is able to provide a constant logistics stream of goods with little human involvement.
In a related bid, US grocery giant Kroger recently started tests of unmanned delivery cabs, sans steering wheel and seats, for daily shopping. Bolder still are those companies (including Uber) testing their autonomous vehicles on the roads of real towns, while others build models running in sophisticated simulators.
With Kroger, Uber and Google leading the way, other companies are sure to fall into line behind them, forming one of most important AI trends 2019.

4. Machine learning and artificial intelligence will be democratized and productionized

AI trends 2019 machine learning productization
There would be no machine learning without data scientists, of which there remain precious few, at least of the skilled variety. Job postings for data scientists rose 75% between 2015 and 2018 at indeed.com while job searches for this position rose 65%. According to Glassdoor data, data scientist was the hottest job in 2018. Due to the popularization of big data, artificial intelligence and machine learning, the demand for data science professionals will continue to rise. And that not only enterprise but also scientific researchers seek their skills certainly bodes well for the profession.
Despite being associated with high-tech companies, machine learning techniques are becoming more common in solving science-related problems. In the last quarter of 2018, Deepmind unveiled a tool to predict the way proteins fold. Another project enabled scientists to derive the laws of physics from fictional universes.
According to O’Reilly data, 51% of surveyed organizations already use data science teams to develop AI solutions for internal purposes. The adoption of AI tools will no doubt be one of the most important AI trends in 2019, especially as business and tech giants are not the only organizations using AI in their daily work.

5. AI responsibility and transparency

AI trends 2019 models transparency
Last but not least, as the impact of machine learning on business grows, so too does the social and legal impact. On the heels of the first fatal accident involving an autonomous car, the question of who is responsible for crashes and the famous trolley problem are getting more important.
At issue here, first and foremost, is hidden bias in data sets, a problem for any company using AI to power-up daily operations. That includes Amazon, which had put AI in charge of preprocessing resumes. Trained with 10 years worth of various resumes, the system was unintentionally biased against women applying for tech positions. With the rising adoption of machine learning models in various industries, the transparency of artificial intelligence will be on the rise. The issue of countering bias unconsciously developed within datasets and taken by machine learning models as truth incarnate is being discussed seriously by tech giants like Salesforce. The machine learning community has also taken up the problem: there is a Kaggle competition aimed at building unbiased and cultural context-agnostic image recognition models to use in computer vision.
Finally, as I alluded earlier, the question of who is responsible for actions taken by AI-powered devices and the famous trolley problem are both moving to the fore. If a self-driving car had a choice, should it hit an elderly person or a child? Focus on saving the life of a driver or a person walking by? According to a global study, the answers depend heavily on the culture the responder grew up in. When facing the extreme situation of a car accident today, it is the driver who is solely responsible for his or her choices. When the car is autonomous, however, and controlled by a virtual agent, all the choices are made by a neural network, which raises some very unsettling questions.
Of course, such problems are not confined to the realm of autonomous vehicles. Machine learning-powered applications are getting more and more attention as a tool for supporting medical treatment. Medical data is expected to rise at an exponential rate, with a compound annual growth rate of 36%. Considering the high level of standardization within diagnostic data, medical data is ripe for utilizing machine learning models, which can be employed to augment and support the treatment process.
When thinking about AI trends 2019, bank on more transparent and socially responsible models being built.

The take-away – the social context will be central to AI trends 2019

No longer are AI and machine learning confined to pure tech; they now have an impact on entire businesses and the whole of society. The common comparison with the steam engine revolution is an apt one – machine learning models will digitally transform both big and small business in ways never before seen.
AI trends 2019 infographic
Given that, picking AI trends 2019 only by selecting technology would be to miss a vital aspect of ongoing changes. That is, they are as ubiquitous as they are far-reaching.

https://deepsense.ai/wp-content/uploads/2019/02/ai-trends-2019.png 337 1140 Konrad Budek https://deepsense.ai/wp-content/uploads/2019/04/DS_logo_color.svg Konrad Budek2019-01-09 13:56:112019-07-05 12:28:30Five top artificial intelligence (AI) trends for 2019
What is reinforcement learning? The complete guide

What is reinforcement learning? The complete guide

July 5, 2018/in Blog posts, Deep learning, Machine learning, Reinforcement learning, Popular posts /by Błażej Osiński and Konrad Budek

With an estimated market size of 7.35 billion US dollars, artificial intelligence is growing by leaps and bounds. McKinsey predicts that AI techniques (including deep learning and reinforcement learning) have the potential to create between $3.5T and $5.8T in value annually across nine business functions in 19 industries.

Although machine learning is seen as a monolith, this cutting-edge technology is diversified, with various sub-types including machine learning, deep learning, and the state-of-art technology of deep reinforcement learning.

What is reinforcement learning?

Reinforcement learning is the training of machine learning models to make a sequence of decisions. The agent learns to achieve a goal in an uncertain, potentially complex environment. In reinforcement learning, an artificial intelligence faces a game-like situation. The computer employs trial and error to come up with a solution to the problem. To get the machine to do what the programmer wants, the artificial intelligence gets either rewards or penalties for the actions it performs. Its goal is to maximize the total reward.
Although the designer sets the reward policy–that is, the rules of the game–he gives the model no hints or suggestions for how to solve the game. It’s up to the model to figure out how to perform the task to maximize the reward, starting from totally random trials and finishing with sophisticated tactics and superhuman skills. By leveraging the power of search and many trials, reinforcement learning is currently the most effective way to hint machine’s creativity. In contrast to human beings, artificial intelligence can gather experience from thousands of parallel gameplays if a reinforcement learning algorithm is run on a sufficiently powerful computer infrastructure.

Examples of reinforcement learning

Applications of reinforcement learning were in the past limited by weak computer infrastructure. However, as Gerard Tesauro’s backgamon AI superplayer developed in 1990’s shows, progress did happen. That early progress is now rapidly changing with powerful new computational technologies opening the way to completely new inspiring applications.
Training the models that control autonomous cars is an excellent example of a potential application of reinforcement learning. In an ideal situation, the computer should get no instructions on driving the car. The programmer would avoid hard-wiring anything connected with the task and allow the machine to learn from its own errors. In a perfect situation, the only hard-wired element would be the reward function.

  • For example, in usual circumstances we would require an autonomous vehicle to put safety first, minimize ride time, reduce pollution, offer passengers comfort and obey the rules of law. With an autonomous race car, on the other hand, we would emphasize speed much more than the driver’s comfort. The programmer cannot predict everything that could happen on the road. Instead of building lengthy “if-then” instructions, the programmer prepares the reinforcement learning agent to be capable of learning from the system of rewards and penalties. The agent (another name for reinforcement learning algorithms performing the task) gets rewards for reaching specific goals.
  • Another example: deepsense.ai took part in the “Learning to run” project, which aimed to train a virtual runner from scratch. The runner is an advanced and precise musculoskeletal model designed by the Stanford Neuromuscular Biomechanics Laboratory. Learning the agent how to run is a first step in building a new generation of prosthetic legs, ones that automatically recognize people’s walking patterns and tweak themselves to make moving easier and more effective. While it is possible and has been done in Stanford’s labs, hard-wiring all the commands and predicting all possible patterns of walking requires a lot of work from highly skilled programmers.
Related:  Learning to run - an example of reinforcement learning

Challenges with reinforcement learning

The main challenge in reinforcement learning lays in preparing the simulation environment, which is highly dependant on the task to be performed. When the model has to go superhuman in Chess, Go or Atari games, preparing the simulation environment is relatively simple. When it comes to building a model capable of driving an autonomous car, building a realistic simulator is crucial before letting the car ride on the street. The model has to figure out how to brake or avoid a collision in a safe environment, where sacrificing even a thousand cars comes at a minimal cost. Transferring the model out of the training environment and into to the real world is where things get tricky.
Scaling and tweaking the neural network controlling the agent is another challenge. There is no way to communicate with the network other than through the system of rewards and penalties.This in particular may lead to catastrophic forgetting, where acquiring new knowledge causes some of the old to be erased from the network (to read up on this issue, see this paper, published during the International Conference on Machine Learning).
Yet another challenge is reaching a local optimum – that is the agent performs the task as it is, but not in the optimal or required way. A “jumper” jumping like a kangaroo instead of doing the thing that was expected of it-walking-is a great example, and is also one that can be found in our recent blog post.
Finally, there are agents that will optimize the prize without performing the task it was designed for. An interesting example can be found in the OpenAI video below, where the agent learned to gain rewards, but not to complete the race.

What distinguishes reinforcement learning from deep learning and machine learning?

In fact, there should be no clear divide between machine learning, deep learning and reinforcement learning. It is like a parallelogram – rectangle – square relation, where machine learning is the broadest category and the deep reinforcement learning the most narrow one.
In the same way, reinforcement learning is a specialized application of machine and deep learning techniques, designed to solve problems in a particular way.

Although the ideas seem to differ, there is no sharp divide between these subtypes. Moreover, they merge within projects, as the models are designed not to stick to a “pure type” but to perform the task in the most effective way possible. So “what precisely distinguishes machine learning, deep learning and reinforcement learning” is actually a tricky question to answer.

  • Machine learning – is a form of AI in which computers are given the ability to progressively improve the performance of a specific task with data, without being directly programmed ( this is Arthur Lee Samuel’s definition. He coined the term “machine learning”, of which there are two types, supervised and unsupervised machine learning

Supervised machine learning happens when a programmer can provide a label for every training input into the machine learning system.

  • Example – by analyzing the historical data taken from coal mines, deepsense.ai prepared an automated system for predicting dangerous seismic events up to 8 hours before they occur. The records of seismic events were taken from 24 coal mines that had collected data for several months. The model was able to recognize the likelihood of an explosion by analyzing the readings from the previous 24 hours.
AAIA16 Data Mining Challenge Seismic Events Height Randomization

Some of the mines can be exactly identified by their main working height values. To obstruct the identification, we added some Gaussian noise

From the AI point of view, a single model was performing a single task on a clarified and normalized dataset. To get more details on the story, read our blog post.
Unsupervised learning takes place when the model is provided only with the input data, but no explicit labels. It has to dig through the data and find the hidden structure or relationships within. The designer might not know what the structure is or what the machine learning model is going to find.

  • An example we employed was for churn prediction. We analyzed customer data and designed an algorithm to group similar customers. However, we didn’t choose the groups ourselves. Later on, we could identify high-risk groups (those with a high churn rate) and our client knew which customers they should approach first.
  • Another example of unsupervised learning is anomaly detection, where the algorithm has to spot the element that doesn’t fit in with the group. It may be a flawed product, potentially fraudulent transaction or any other event associated with breaking the norm.

Deep learning consists of several layers of neural networks, designed to perform more sophisticated tasks. The construction of deep learning models was inspired by the design of the human brain, but simplified. Deep learning models consist of a few neural network layers which are in principle responsible for gradually learning more abstract features about particular data.
Although deep learning solutions are able to provide marvelous results, in terms of scale they are no match for the human brain. Each layer uses the outcome of a previous one as an input and the whole network is trained as a single whole. The core concept of creating an artificial neural network is not new, but only recently has modern hardware provided enough computational power to effectively train such networks by exposing a sufficient number of examples. Extended adoption has brought about frameworks like TensorFlow, Keras and PyTorch, all of which have made building machine learning models much more convenient.

  • Example: deepsense.ai designed a deep learning-based model for the National Oceanic and Atmospheric Administration (NOAA). It was designed torecognize Right whales from aerial photos taken by researchers. For further information about this endangered species and deepsense.ai’s work with the NOAA, read our blog post. From a technical point of view, recognizing a particular specimen of whales from aerial photos is pure deep learning. The solution consists of a few machine learning models performing separate tasks. The first one was in charge of finding the head of the whale in the photograph while the second normalized the photo by cutting and turning it, which ultimately provided a unified view (a passport photo) of a single whale.


The third model was responsible for recognizing particular whales from photos that had been prepared and processed earlier. A network composed of 5 million neurons located the blowhead bonnet-tip. Over 941,000 neurons looked for the head and more than 3 million neurons were used to classify the particular whale. That’s over 9 million neurons performing the task, which may seem like a lot, but pales in comparison to the more than 100 billion neurons at work in the human brain. We later used a similar deep learning-based solution to diagnose diabetic retinopathy using images of patients’ retinas.
Reinforcement learning, as stated above employs a system of rewards and penalties to compel the computer to solve a problem by itself. Human involvement is limited to changing the environment and tweaking the system of rewards and penalties. As the computer maximizes the reward, it is prone to seeking unexpected ways of doing it. Human involvement is focused on preventing it from exploiting the system and motivating the machine to perform the task in the way expected. Reinforcement learning is useful when there is no “proper way” to perform a task, yet there are rules the model has to follow to perform its duties correctly. Take the road code, for example.

  • Example: By tweaking and seeking the optimal policy for deep reinforcement learning, we built an agent that in just 20 minutes reached a superhuman level in playing Atari games. Similar algorithms in principal can be used to build AI for an autonomous car or a prosthetic leg. In fact, one of the best ways to evaluate the reinforcement learning approach is to give the model an Atari video game to play, such as Arkanoid or Space Invaders. According to Google Brain’s Marc G. Bellemare, who introduced Atari video games as a reinforcement learning benchmark, “although challenging, these environments remain simple enough that we can hope to achieve measurable progress as we attempt to solve them”.
Breakout
Initial performanceAfter 15 minutes of trainingAfter 30 minutes of training
Playing atari with deep reinforcement learning - 0Playing atari with deep reinforcement learning - 1Playing atari with deep reinforcement learning - 2
Assault
Initial performanceAfter 15 minutes of trainingAfter 30 minutes of training
Playing atari with deep reinforcement learning - 3Playing atari with deep reinforcement learning - 4Playing atari with deep reinforcement learning - 5

In particular, if artificial intelligence is going to drive a car, learning to play some Atari classics can be considered a meaningful intermediate milestone. A potential application of reinforcement learning in autonomous vehicles is the following interesting case. A developer is unable to predict all future road situations, so letting the model train itself with a system of penalties and rewards in a varied environment is possibly the most effective way for the AI to broaden the experience it both has and collects.

Related:  Playing Atari with deep reinforcement learning - deepsense.ai’s approach

Conclusion

The key distinguishing factor of reinforcement learning is how the agent is trained. Instead of inspecting the data provided, the model interacts with the environment, seeking ways to maximize the reward. In the case of deep reinforcement learning, a neural network is in charge of storing the experiences and thus improves the way the task is performed.

Is reinforcement learning the future of machine learning?

Although reinforcement learning, deep learning, and machine learning are interconnected no one of them in particular is going to replace the others. Yann LeCun, the renowned French scientist and head of research at Facebook, jokes that reinforcement learning is the cherry on a great AI cake with machine learning the cake itself and deep learning the icing. Without the previous iterations, the cherry would top nothing.
In many use cases, using classical machine learning methods will suffice. Purely algorithmic methods not involving machine learning tend to be useful in business data processing or managing databases.
Sometimes machine learning is only supporting a process being performed in another way, for example by seeking a way to optimize speed or efficiency.
When a machine has to deal with unstructured and unsorted data, or with various types of data, neural networks can be very useful. How machine learning improved the quality of machine translation has been described by The New York Times.

Summary

Reinforcement learning is no doubt a cutting-edge technology that has the potential to transform our world. However, it need not be used in every case. Nevertheless, reinforcement learning seems to be the most likely way to make a machine creative – as seeking new, innovative ways to perform its tasks is in fact creativity. This is already happening: DeepMind’s now famous AlphaGo played moves that were first considered glitches by human experts, but in fact secured victory against one of the strongest human players, Lee Sedol.
Thus, reinforcement learning has the potential to be a groundbreaking technology and the next step in AI development.

https://deepsense.ai/wp-content/uploads/2019/02/what-is-reinforcement-learning-the-complete-guide.jpg 337 1140 Błażej Osiński https://deepsense.ai/wp-content/uploads/2019/04/DS_logo_color.svg Błażej Osiński2018-07-05 13:57:132019-07-05 12:28:38What is reinforcement learning? The complete guide

Start your search here

NEWSLETTER SUBSCRIPTION

You can modify your privacy settings and unsubscribe from our lists at any time (see our privacy policy).

This site is protected by reCAPTCHA and the Google privacy policy and terms of service apply.

THE NEWEST AI MONTHLY DIGEST

  • AI Monthly Digest #14 - quantum supremacyAI Monthly Digest #14 – quantum supremacyNovember 13, 2019

CATEGORIES

  • Blog posts
  • Big data & Spark
  • Data science
  • Deep learning
  • Machine learning
  • Neptune
  • Reinforcement learning
  • Seahorse
  • Job offer
  • Popular posts
  • AI Monthly Digest
  • Press release

POPULAR POSTS

  • A comprehensive guide to demand forecastingA comprehensive guide to demand forecastingMay 28, 2019
  • Five top artificial intelligence (AI) trends for 2019Five top artificial intelligence (AI) trends for 2019January 9, 2019
  • What is reinforcement learning? The complete guideWhat is reinforcement learning? The complete guideJuly 5, 2018
  • deepsense.ai logo white
  • Solutions
  • Increase sales
  • Reduce costs
  • Optimize processes
  • Build data strategy
  • Train your team
  • Knowledge base
  • Blog
  • Press center
  • deepsense.ai
  • Our story
  • Management
  • Scientific Advisory Board
  • Careers
  • Support
  • Terms of service
  • Privacy policy
  • Contact us
  • Join our community
  • facebook logo linkedin logo twitter logo
  • © deepsense.ai 2014-2019
Scroll to top

This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies.

OKLearn more