“Model-Based Reinforcement Learning for Atari,” a research paper written by a consortium of authors, has been chosen for a spotlight presentation at the International Conference on Learning Representation. Only 5% of the papers submitted receive this distinction. Two of the leading authors, Piotr Miłoś and Błażej Osiński, are both researchers and senior data scientists at deepsense.ai.

The paper describes research conducted in cooperation with Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign. 

The high-level goal described in the research is how to enable an AI model to learn useful behaviors by interacting with its environment. In this case, the environment was Atari games, which offer tasks that are simple yet challenging enough to test the research concept. The results of the experiment can be extrapolated to more challenging tasks in increasingly complex environments. The model the deepsense.ai researchers designed learned how to play the games thanks to an extensive chain of trial and error. 

An extension of the research might be a robot acquiring skills while moving objects, such as putting dishes in a dishwasher or refrigerator. Such interactions can be costly – the robot will drop any number of dishes along the way, so it takes time and resources for it to train itself thoroughly enough to learn to carry out its tasks efficiently. 

To tackle the challenge of expensive interactions, we increase the system’s learning efficiency by endowing it with a predictive module that can imagine the consequences of actions without actually executing them. The module is trained along with this skill and it is used to perform short inner simulations. These simulations substitute most of the interactions normally required by reinforcement learning, resulting in better sample efficiency.

Details about the research are available in the full arXiv preprint, while the press release and an article about artificial imagination can be found on deepsense.ai’s blog.

“We are proud and excited to see our work recognized at one of the most renowned scientific conferences in the world,”
– says Piotr Miłoś, senior data scientist at deepsense.ai and a professor at the Polish Academy of Science.

Reinforcement learning in spotlight

“Spotlight” is a short oral presentation reserved for papers of particular distinction. With only 5% of submitted papers honored this way, the privilege comes amidst our work gaining critical attention in the AI research community. The arXiv paper has been cited more than 50 times in just 10 months, a testament to its significance for other researchers in the field.

This year’s conference, the eighth, will be held in Addis Ababa, Ethiopia, and is hoped to encourage researchers from Africa and the Middle East to take part. Despite its short history, the conference is currently among the most important deep learning-related scientific events  in the world. The conference publication is recognized on Google scholar as the 42nd most important scientific publication globally.

The conference provides an interesting insight into the development of Artificial Intelligence and is a great way to keep abreast of the latest trends. deepsense.ai has recently used machine learning tools to review the papers submitted to the conference. The analysis showed that reinforcement learning is among the most significant and recurrent trends in machine learning today. Further information on trending topics on the ICLR conferences can be found in deepsense.ai’s analysis.

“Presenting our research on spotlight is a great honor and proof that our contribution to the machine learning community and development is recognized,”
– concludes Błażej Osiński, a senior data scientist and deepsense.ai researcher involved in the project.

This success is the result of deepsense.ai’s strong commitment to R&D. Just a month ago two other papers were presented at NeurIPS workshops, another top-tier AI conference. These included “Developmentally motivated emergence of compositional communication via template transfer” and “Simulation-based reinforcement learning for real-world autonomous driving”.

The full list of paper authors: Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, Afroz Mohiuddin, Ryan Sepassi, George Tucker, Henryk Michalewski

###

About deepsense.ai:

brand-manager-at-deepsense

AI business application leader deepsense.ai has appointed two new members to its Management Board. From November, Pawel Osterreicher will assume the role of Chief Revenue Officer (CRO) while Bartek Klimczak will become Chief Operating Officer (COO). Rounding out the executive moves, Andy Thurai joined the company as Head of US Operations.

“I am proud and excited to welcome such strong players as Paweł and Bartek to the management board,” says deepsense.ai CEO Tomasz Kułakowski. “The company is currently witnessing rapid growth and immense geographical expansion. Our key challenge is to reach the ambitious goal of securing growth. Seasoned experts like Bartek and Paweł will support deepsense.ai’s position as a global AI leader.”

Osterreicher has been deepsense.ai’s Director of Sales and Strategy since April 2018. He came over from Boston Consulting Group (BCG), where he headed up projects in the financial and consumer/retail sectors, focusing his efforts on strategy, transformation, the building out of digitalization and implementation of advanced analytics. Prior to that, Paweł worked with the A.T. Kearney and Santos company (Australia).

“I am very optimistic about the future growth of the company and I am very much looking forward to coordinating deepsense.ai’s sales efforts,” says Osterreicher. “Some of the recent successes deepsense.ai has had, and which I contributed to, involved acquiring multiple key business customers and partners. We have also successfully grown our competencies thanks to projects that have pushed the boundaries of what is possible in AI. I am convinced that deepsense.ai’s potential, comprising both its employees competencies’ and its acquired business know-how, is unique. Further scale-up of the company’s activities on a global level will be a challenge – a pleasant and very interesting one.”

Bartek Klimczak joined deepsense.ai in mid-2019 as a Managing Director. Prior to that, he spent seven years at The Boston Consulting Group (most recently as Principal), focusing on a broad spectrum of strategic and operational projects for leading global players in industrial goods and processing sectors as well as energy and the global oil & gas sector. Prior to BCG, Bartek spent another 7 years in investment banking in the CEE (the CAG & KBC Group), where he primarily brokered M&A deals.

“I see massive growth potential and unique opportunity for deepsense.ai, especially in industries driven by Industry 4.0 adoption of AI/ML across the entire value chain, from supply chain optimization, through machine vision-based automatic quality control to predictive maintenance, machine auto-diagnostics and work safety smart intelligence. All of this will bring our clients’ businesses in Poland and globally to entirely new levels in terms of operating efficiency and profitability. By leveraging our unique capabilities and proprietary AI/ML technologies, we are able to effectively target even the most challenging and custom problems our clients may face throughout their production processes. We want our focus aimed primarily at areas where properly designed AI can bring significant results. That will be our wheelhouse moving forward,”
– explains Klimczak.

The promotions are the latest in a string of moves deepsense.ai has made to expand its executive team. The company has also recently tapped Andy Thurai to head up its US Operations and help the company expand its footprint in the United States, the most promising market for its cutting edge technologies.
Mr. Thurai worked as a chief strategist for IBM and an Emerging Technology Strategist for Oracle Cloud Infrastructure. He has also been a field CTO with Intel and BMC, and has extensive experience as a consultant, technology advisor and mentor.

“deepsense.ai combines technical proficiency with a deep understanding of business use cases, being one of the very few companies ready to solve problems once thought were unsolvable. I am proud to support the company with my experience,”
– says Mr Thurai.

The company also welcomed Nina Simosko to its Advisory Board. A former CEO of the NTT Innovation Institute, the prestigious Silicon Valley-based innovation center for NTT Group, Ms. Simosko is a well-known Silicon Valley technology executive with senior management experience in both global Fortune 500 companies and innovative startups.

###

About deepsense.ai:

brand-manager-at-deepsense

deepsense.ai, a leading AI-based end-to-end solutions provider with the focus on computer vision, predictive analytics and natural language processing, today announced that Nina Simosko has been appointed to the company’s advisory board. Ms. Simosko, formerly the CEO of NTT Innovation Institute, the prestigious Silicon Valley-based innovation center for NTT Group, one of the world’s largest ICT companies, is a well-known Silicon Valley technology executive with senior management experience in both global Fortune 500 companies and innovative startups.

“I am very pleased that Nina is joining our advisory board,” stated Tomasz Kulakowski, deepsense.ai’s chief executive officer. “We look forward to benefiting from her strong background in cutting-edge technologies, supporting our mission to improve people’s lives with machine learning-powered solutions. We welcome her strategic input and the combination of leadership, experience and industry knowledge she brings to deepsense.ai as the company transitions to the next phase of growth.”

Previous to NTT i3, Ms. Simosko was responsible for leading the creation and execution of Nike Technology strategy, planning and operations world-wide. Prior to that, she was Senior Vice President of SAP’s Global Premier Customer Network (PCN). At SAP, she led both the PCN Center of Excellence and SAP’s Global Executive Advisory Board. During her eight-year tenure, she was a part of SAP’s Global Ecosystem & Partner Group which was charged with continuing to build and enable an open ecosystem of software, service and technology partners together with SAP’s communities of innovation. Ms. Simosko currently sits on the Advisory board at Santa.io, AppOrchid and Reflektion and she has also been a member of the advisory boards at Appcelerator and Taulia.

“I am excited to join the company’s advisory board”, stated Ms. Simosko. “deepsense.ai is potentially solving one of the biggest challenges companies currently tackle in identifying, analyzing and solving problems with AI-powered solutions across multiple industries. I highly appreciate the company’s focus on research and their ability to deliver meaningful business results to their customers.”
– Nina Simosko

###

About deepsense.ai:

brand-manager-at-deepsense

Global e-commerce is among the fastest growing industries globally, experiencing 18% growth in 2018. Worldwide, consumers purchased $2.86 trillion worth of e-goods in 2018, compared to $2.43 trillion in 2017.

Because digital commerce is data-driven, the industry is ripe territory for AI. However, lack of knowledge and uncertainty remain the most prominent obstacles to this technology gaining a stronger foothold. To address these obstacles, deepsense.ai and Google Cloud co-organized a business breakfast to discuss the challenges and opportunities and share their remarks on artificial intelligence (AI) in e-commerce. Joining Google and deepsense.ai were experts from BeeCommerce.pl, Sotrender, and iProspect, all companies deliver sophisticated tools for digital business.

Plato’s data cave

“When it comes to building AI applications, it’s all about the data,” said Paweł Osterreicher, Director of Strategy & Business Development at deepsense.ai, during his presentation. He pointed out that the simplest analytics in smaller businesses can be done within an Excel spreadsheet or pen and paper. Preparing a simple segmentation within a client group or spotting best-performing products don’t pose a huge challenge. But those are only the tip of the iceberg. “The more sophisticated insights we gain, the more complicated the task becomes. And that’s where specialized software comes in,” he said.

“The greatest challenge is a lack of flexibility. There is no jack-of-all-trades among the popular tools, and each has its limitations. The problem is when a tool doesn’t fit a company’s needs. And, to be honest, that’s a common situation,” Osterreicher continued. Companies thus often need to tweak the tools at their disposal to make them fit or get used to missing insights from their data.

“Most companies process only a fraction of their data and operate with only half the picture. They are like the prisoners in Plato’s cave, watching only the shadows customers cast on the wall, with no access to or true grasp of their real form.”

The only way to analyze data in a convenient and cost-effective way is to leverage machine learning models. Machines are able to effectively spot patterns even in seemingly insignificant details.

“Sometimes information about how long customers hover over a button or how they go about filling in an online form is a first step to obtaining meaningful information. The model is only as good as the data it was built on,” concludes Osterreicher.

Retail reinvented

In another presentation, Jakub Skuratowicz focused on the technical aspects of how companies use AI. There are numerous ways for companies to benefit from AI, be it building engagement, personalizing the user experience or detecting fraud.

Google’s expert showed a new application of image search for omnichannel commerce. First applied by the Nordstrom clothing company, the app-enabled users to take a photograph of an item and then search for it in the shop’s database. Thus, the customer could quickly buy the product online or check its availability.

“By using Google Cloud Platform-delivered machine learning tools, the company reached 95% accuracy in recognizing an item shown in a photograph”

AI also thrives in recommendation engines. “It was common to recommend the user another version of the product – a different size of a dress, for example. That’s pointless. Why would one need another of the same dress, only slightly bigger?” Skuratowicz asked. Instead, the AI-powered model recommended products that complemented the one that had been searched for, like sunglasses or a scarf to go with the dress.

Skuratowicz also showed how AI spots fraudulent transactions in international e-commerce. “Manual or semi-automatic checking can be effective, but machine learning makes it more scalable,”  he said. By applying AI-based solutions, the international logistics provider Pitney Bowes boosted the accuracy of its fraudulent transaction detection by 80% while reducing false-positives by 50%.

The mind barrier

The presentations were followed by a panel discussion on machine learning in e-commerce. As the panelists remarked, the AI-powered future of e-commerce is a challenge that not all companies are ready for.

In response to a question about the state of data-proficiency in e-commerce companies, Arkadiusz Wiśniewski, Director of Data and Technology at iProspect, had this to say:

“some data are easy to collect, while others provide a challenge, so we have an incomplete view. The legal situation in Europe poses an additional challenge, so it is better to focus on the data owned and make the best use of it.”

“Data-readiness depends to a great extent on company size. But most businesses lack the skills and data to effectively apply machine learning techniques,” agreed Jarosław Trybuchowicz, owner of beecommerce.pl.

The panelists agreed that the situation is hard even though data is becoming a commodity. “Sometimes the problem is the opposite. Despite having huge amounts of data, companies don’t get insights from it. They simply don’t know what questions to ask and what insights to look for,” added deepsense.ai’s Borys Sobiegraj.

The panelists likewise agreed that the key to success for enterprises employing machine learning is to know and properly organize their own data. Getting the data is the first challenge; “deciding what to do with it is a different story altogether,” said Jakub Nowacki, Lead Machine Learning Engineer at Sotrender. “Another challenge is extracting value that often lies in matching the data from different sources. If a company is unable to determine the impact of a sales campaign, then what is the purpose of analytics?” he added.

A question-answer session and networking time followed the discussion panel. The next business breakfast is planned for Q3.

###

About deepsense.ai:

brand-manager-at-deepsense

deepsense.ai, an official Google Cloud Platform Training Partner, Tuesday hosted its first Business Breakfast: AI outlook for 2019 with participants from Google Cloud. A panel of experts discussed the development of technologies and trends to look forward to in 2019. There was a strong emphasis on cloud applications. The Breakfast officially kicked off the two companies’ collaboration.

The panelists shared their thoughts on the state of AI on the Polish market, the challenges facing it and the opportunities local business has for further development.

“Google has been working on AI-powered solutions for years and most of our tools are infused with it,” said Jakub Skuratowicz, Customer Engineer at Google during his presentation at the meeting. “Currently, more than 12% of Gmail responses use our automated response suggestions. Another example is our translation services, which were first tested as an internal tool but are now used by thousands of people around the world every day.”
According to Skuratowicz, cloud-based AI solutions are not limited to daily operations in consumer services. Google’s Deepmind used AI to reduce the cooling bill for server rooms by more than 40%, and overall server room power consumption by 16%.

According to Gartner, the public cloud services market is projected to grow 17.3% in 2019, to  $206.2 billion. That’s up from an already lofty $175.8 billion in 2018. The strongest growth, however, is forecast to occur in the Infrastructure as a Service segment, which is expected to grow 27.6% in 2019, to $39.5 billion. Building AI tools requires a great deal of computing power, and will be a serious component of that growth.

“Our goal is to make machine learning more accessible by applying it in the cloud,” adds Skuratowicz. “That can be achieved only by giving out reliable tools and delivering specialized infrastructure.”

A market with enormous potential

AI can be applied across multiple industries. For example, deepsense.ai’s data scientists have designed a solution that speeds up the work of researchers at Nielsen and another that fights propaganda on social media.

“Artificial intelligence will not be a purpose in and of itself, but rather a sophisticated tool to solve business problems. But the data remains the greatest challenge and companies should start there,” said Paweł Godula, Director of Customer Analytics during his presentation of a Home Credit risk estimation tool. As the article makes clear, the estimation tool was clear proof that machine learning models are far better at scoring credit risk than human researchers.“Machines have no trouble analyzing high volumes of data and their perception is unlimited. Moreover, they cannot form presuppositions or be biased by previous experiences,” explains Godula. “Thus, they analyze data in a truly neutral way”.
Godula believes the ability to get a holistic image of a single customer by connecting and cross-analyzing data from different sources is currently the pinnacle of artificial intelligence technology, and that’s where Google Cloud Platform thrives.

Fear of the black box

The discussion panel following the speeches was a great opportunity to exchange experience on the use of AI  in both the public and private sectors.

“There is no doubt that artificial intelligence is fueling growth,” said Magdalena Dziewguć, Google Cloud Territory Manager. “If there are two tech firms, one of which supports its processes with AI while the other does things in a manual, old-fashioned way, there can be no doubt that the time and money the first saves will give it a better chance of surviving.”

“AI technology is becoming more convenient and common. Anti-spam filters or automated suggestions within search tools are considered a standard feature, not an AI-based power function. Even the most intelligent function becomes common after some time and users expect it to be a basic feature,” commented Robert Bogucki, deepsense.ai CTO. Both Rethink’s Borys Stokalski and Nielsen’s Paweł Bedyński agreed that AI  is a problem-solving tool rather than just a fashionable buzzword.

When asked about the main obstacles and challenges for AI in getting more business attention, panelists named fear, doubt and uncertainty about new solutions emerging on the market.

“In its early days, digitalization once supported the status quo. But now legacy software and solutions are becoming more and more of a problem in various industries,” said Stokalski. “Startups are reinventing how business is done and the old models are being challenged.”

“Although some solutions have been present for years, the revolution is a matter of the last few years,” commented Bogucki. “Neural networks outperformed human experts in specialized tasks in 2015. Since then we have seen processes automated across a range of industries.”

The panelists had no doubt that AI solutions will become common and solve multiple challenges.

“Privacy issues are gaining greater notoriety and machine learning will become more private. We may be entering the era of tailored models serving particular customers and trained solely on their data, not the gargantuan datasets containing information about all the users of the product,” comments Bedyński.

“Moreover, the solutions are getting more affordable for business. Building mobile apps was expensive not so long ago. Now it is a common business channel. In my opinion, AI models will also become more widespread,” Bogucki added.

Business-focused meeting

The panel was followed by a networking session and it was an opportunity to exchange thoughts and remarks on machine learning applications in modern business. Participants from various industries asked about ways to solve the privacy and regulatory issues dogging AI and cloud applications.

“The lack of knowledge and courage to adopt machine learning models in daily business processes is the greatest obstacle companies face in leveraging this outstanding tool of growth,” explained Paweł Osterreicher, Business Development Director and host of the discussion panel. “But the more educational events spread the word, the more companies will find the strength to grab the opportunity.”

The meeting is the first in a series. To address the issues the panelists highlighted, the companies will be holding a three-day training session in February. More details about the event can be found on the Architecting with Google Cloud Platform course website.

###

About deepsense.ai:

Media contact:

deepsense.ai trademarks at boilerplate