Download the guide and learn how deepsense.ai can leverage the accuracy of your organization’s demand forecasting.
Download the guide and learn how deepsense.ai can leverage the accuracy of your organization’s demand forecasting.
Computer vision is a foundational element of Smart Factory solutions. At deepsense.ai we have created diverse AI-driven, automated computer vision-based solutions that undertake the most demanding challenges in many different production lines.
The integration of advanced technologies with production lines makes it possible to automate and optimize a range of key production processes. A foundational element in Smart Factory solutions is computer vision. By using artificial intelligence and training models that are able to properly interpret images from, for example, industrial cameras set up at the factory, we can effectively monitor production processes and identify potential adverse events. At deepsense.ai we have created diverse AI-driven, automated computer vision-based solutions that undertake the most demanding challenges in many different production lines. By combining visual control technology with machine learning, deepsense.ai is able to address problems to which simple solutions based on human efforts don’t work.
Computer vision and advanced data analysis are increasingly enlisted to take over human-performed quality control. There is little wonder as to why: automatic quality control is more accurate and faster as well as based on image analysis, we can effectively monitor diverse components of production, including:
Intelligent quality control systems developed by deepsense.ai are able to identify 99% of product defects, which significantly reduces the need for manual quality control and in turn lowers costs.
Workflow optimization becomes a key aspect of profitability and the scalability of production lines. Creating a “digital twin” of a facility can significantly improve its operations. The virtual version of the factory can be used not only in 3D modeling but also as a part of an augmented reality solution.
A workflow optimization project deepsense.ai did involved recognizing and classifying the connection structure of devices based on data points produced by high-resolution scanners. Based on computer vision, the AI-based system identified connections between the devices in order to establish how well the installation’s structure matched already existing diagrams. The solution significantly reduced the human effort needed to process laser scans of industrial facilities.
Intelligent computer vision-based systems keep workplaces safe by intelligently monitoring such visual data as CCTV footage. The Covid-19 pandemic has underscored the importance of compliance with safety procedures, including monitoring the distance between workers, the amount of time being spent near other workers and contact tracing. Such monitoring solutions effectively prevent incidents and detect potential threats in areas including:
Automated monitoring of the hazardous zones in a factory was another of deepsense.ai’s recent projects. The system we designed tracks whether every worker is following required safety measures. If not, it sends an instant alert to the supervisor with a detailed description of the event that is occuring. The system has significantly lowered accidents and hazardous events metrics.
Some problems can be described precisely (e.g., a pipe’s diameter has to be exactly 50mm), but deepsense.ai technology is also able to understand problems that can’t be. Our algorithms learn complex patterns and detect features the human eye fails to see. This allows us to identify anomalies and defects that were not previously observed or for which no training data has been provided.
The modular architecture of AI models analyzes all video streams in real time, identifying potential defects and checking for anomalies.
Our custom-designed neural networks follow recent advancements in machine learning and are pre-trained on a massive, cross-sectional dataset of diverse worksite imagery. This enables us to take on virtually any computer vision challenge.
Network architectures are selected based on the function and purpose of individual models, some of which include:
Our solutions can be implemented on existing monitoring hardware, limiting the need for additional investments. You may need to upgrade current operations, or to build a new one from the ground up. Whichever the case, in delivering industry-leading solutions, deepsense.ai models success.
Machine learning is still perceived as an innovative approach in business. The technological progress and the use of Big Data in business make ML-based solutions increasingly important. As Forbes magazine indicates, 76% of enterprises today prioritize artificial intelligence and machine learning over other IT initiatives .
The concept of machine learning is derived from advanced data analysis of pattern and dependency recognition. ML assumes that when models analyze new data, they can adapt independently and use this new knowledge to develop by learning from previous experiences. Machine learning models can enhance nearly every aspect of a business, from marketing and sales to maintenance.
Machine learning enables predictive monitoring, with algorithms anticipating equipment breakdowns before they occur and scheduling timely maintenance. With the work it did on predictive maintenance in medical devices, deepsense.ai reduced one client’s downtime by 15%.
But it isn’t just in straightforward failure prediction where machine learning supports maintenance. In another recent application, our team delivered a system that automates industrial documentation digitization, effectively reducing workflow time by up to 90%. We developed a model that recognizes and adds descriptions for all symbols used in the installation documentation. The schematics, including the technical descriptions of all components, are fully digitalized. The model reduces the work to a 30-minute review by a specialist. It also handles the most tedious tasks, thus reducing the effort required of human specialists and the number of errors they make in performing them.
An international manufacturer of medical devices was looking for a solution that would reduce device downtime. Our experts built a predictive maintenance model that pores over historical data, searching for anomalies and signs of a breakdown before one occurs. The model reduced breakdown-related downtime by more than 15%. Such a solution can be applied in machine-reliant industries, where breakdowns bring operations to a halt and hamper overall company performance.
Machine Learning is also being adopted for product inspection and quality control. ML-based computer vision algorithms can learn from a set of samples to distinguish the “good” from the flawed. In particular, semi-supervised anomaly detection algorithms require only “good” samples in their training set, making a library of possible defects unnecessary. Alternatively, a solution can be developed that compares samples to typical cases of defects.
One of our clients asked us to tackle two of their visualization problems on a food production line – to detect sauce smears on the product’s inner packaging, and to identify correct positioning of the product’s topping. The system deepsense.ai delivered was able to identify 99% of faulty products with topping defects and while raising the alarm with a 99% accuracy rate for sauce smears. The model significantly reduced the need for manual quality control, hence lowering costs.
ML-based computer vision solutions can also be an essential component in the monitoring of hazardous areas in factories, tracking whether every worker is following safety requirements (including wearing helmets, glasses, vests, earmuffs etc.) and, if not, sending an instant alert to the supervisor with a detailed description of the event that has occurred.
In the field of predictive analytics demand forecasting can be used to predict consumer demand. Such forecasting is done by analyzing statistical data and looking for patterns and correlations. With machine learning taking the practice to a higher level, modern demand forecasting techniques go far beyond simple historical data analysis.
More recent techniques combine intuition with historical data. Modern merchants can dig into their data in a search for trends and patterns. At the pinnacle of these techniques sit demand forecasting machine learning models, including gradient boosting and neural networks, which are currently the most popular types and outperform classic statistics-based methods. Historical data from transactions form the basis of more recent demand forecasting techniques. These are data that sellers collect and store for fiscal and legal reasons. Because they are also searchable, these data are the easiest to use.
This modern approach is extremely effective. One of our clients from the retail industry was losing millions of euros a year due to out-of-stocks. There was a daily cap on how many new items its warehouse could receive. Our team built a demand forecasting model for products that were new to market. It enables the company to use the cap more efficiently by ordering more hot products and fewer of those that are less in demand. We used Gradient Boosting, Random Forest and Neural Networks to build the model, and the trifecta reduced out-of-stocks by 30%.
Companies can maximize ROI on their marketing activities by implementing machine learning into their customer analysis. Sophisticated data analysis helps identify customers with the highest ROI on ads to make the most of marketing campaigns. It also optimizes channel mix with advanced attribution models.
deepsense.ai designed a model for a leading mobile advertising platform that predicts the click-through rate of internet advertisements. The model analyzes historical data on site user behavior to spot patterns and uncover anomalies. It enabled clients to identify an abnormal pattern among users, which turned out to be bots engaging in fraudulent clicking. The solution effectively identified internet bots that click ads, significantly boosting CTR predictions – up to 90% of bots were spotted and CTR predictions were improved by up to 35% over existing heuristics.
Analyzing large amounts of data has become a crucial part of the retail and manufacturing business landscape. It has traditionally been done by experts, based on know-how honed through experience. With the power of machine learning, however, it is now possible to combine the astonishing scale of big data with the precision and intelligence of a machine-learning model. While the business community must remain aware of the multiple pitfalls it will face when employing machine learning, that it endows business processes with awesome power and flexibility is now beyond question.
Interview with Michał Iwanowski, VP of Engineering, about AI development in 2021.
AI is no longer a buzzword. Increasingly, from the R&D phase, it moves towards proven solutions, and nowhere more so than in computer vision and natural language processing.
The computer vision projects deepsense.ai delivered in 2020 prove that numerous industries can benefit from advanced image analysis technology. We have implemented successful projects in modern manufacturing that incorporated machine learning throughout the production process. These included not only intelligent quality control systems, but also safety monitoring and workplace security solutions. Our 2021 projects will develop this potential based on AIoT systems.
NLP-based solutions are likewise becoming more widely applicable. Our portfolio has been enriched by projects ranging from virtual meeting assistants to solutions capable of summarizing large chunks of text and evaluating the quality of student submissions in selected areas of study. With the rise of enormous language models such as GPT-3, we expect to see this area of AI – including natural language understanding (NLU) – grow more robustly in the coming quarters.
Top global research and consulting companies have indicated that 2021 is to be intensive and promising for AI. The emerging trends include the democratization of AI and industrialization of AI platforms. More and more businesses are seeing the real value in analyzing data and are convinced that using AI based on platforms that enable scalability and safety is a wise step forward.
The Covid-19 pandemic appears to have accelerated customers’ openness to AI. Despite the crisis, clients have continued to invest in the technology. They understand that in order to build competitive advantage, AI must become a core element of their business models. With more than 70 data science and machine learning experts on board, we are ready to turn clients’ vision into practical AI-focused solutions.
We work mainly with clients who already have some knowledge of and experience with AI projects. This allows us to act creatively on new AI solutions for various businesses. When a client retains us for a single project, we frequently uncover several new areas where we can further develop and implement AI.
We highly value clients who trust us enough to jointly take on challenges related to innovative AI applications. One such client is Volkswagen, with whom over the past 18 months we have moved neural network policy from a car simulator to reality. After multiple training sessions in the simulated environment, our algorithm overcame the sim-to-real gap in the real world. This milestone has opened up the possibility of using distributed computing to train AI models that are easily transferable to physical vehicles and can be tested in real world-environments.
We have also observed increased interest in our R&D projects. Many clients want to get involved in our research work because it leads them to explore new opportunities for their businesses. After a successful reinforcement learning project conducted together with Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign, we continue our work in this area. The research conclusions bring tangible benefits to our clients by setting solid algorithmic foundations for tackling business problems that don’t present an obvious objective function, but rather require an AI agent to learn from its mistakes. This paradigm shift opens up brand new areas of application for AI.
Last but not least, we see many clients that are ready to build AI capabilities in-house. With our comprehensive training programs, they get best-in-class know-how that bears long-term results for real-life data challenges.
Apart from pursuing new projects and challenges in computer vision, natural language processing and predictive analytics, there are two crucial tasks ahead of us in the upcoming year.
The first is to maintain an open-minded and creative approach to the changing reality of remote work and the migration of more and more services to the virtual environment – with a focus on how these transitions can be converted into business opportunities using AI and automation.
The second challenge is to bridge the gap between cutting-edge advancements on the frontier of AI and Machine Learning and reliable, well-tested business solutions. As a company that operates at the crossroads of business and research, we believe it’s on us to take a pioneering role in this challenge.
According to research done by Business Insider Intelligence, banks will make some $450 billion by 2023 by applying Artificial Intelligence. No wonder, then, that AI is playing an increasingly important role in financial institutions’ roadmap for the coming years – 75% of banks with over $100 billion in assets already have AI strategies in place.
Banks don’t just create AI strategies, but are increasingly using AI and Machine Learning in their day-to-day business. We often work with them on ideation workshops, PoC and solution implementation. As an example, we recently had the opportunity to work with Santander Consumer Bank, running workshops and researching how to use ML to boost the sustainability of loan portfolios. We were able to significantly reduce risk while maintaining the same acceptance rate for extending loans.
Apart from credit risk modelling, there is already an impressive range of use cases for AI in banking, covering everything from customer service to back-office operations. Here is a list of the most common AI solutions in the banking sector:
A wide range of ML and AI applications is increasingly being used to solve real business problems in banking. As AI becomes more popular, those applications will become the market standard.
This article was prepared in cooperation with Santander Consumer Bank
Cost of risk is one of the biggest components in banks’ cost structure. Thus, even a slight improvement in credit risk modelling can translate in huge savings. That’s why Machine Learning is often implemented in this area.
We would like to share with you some insights from one of our projects, where we applied Machine Learning to increase credit scoring performance.To illustrate our insights we selected a random pool of 10 000 applications.
Loan applications are usually assessed through a credit score model, which is most often based on a logistic regression (LR). It is trained on historical data, such as credit history. The model assesses the importance of every attribute provided and translates them into a prediction.
The main limitation of such a model is that it can take into account only linear dependencies between input variables and the predicted variable. On the other hand, it is this very property that makes logistic regression so interpretable. LR is in widespread used in credit risk modelling.
Machine Learning enables the utilization of more advanced modeling techniques, such as decision trees and neural networks. This introduces non-linearities to the model and allows to detect more complex dependencies between the attributes. We decided to use an XGBoost model fed with features selected with the use of a method called permutation importance.
However, ML models are usually so sophisticated that they are hard to interpret. Since a lack of interpretability would be a serious issue in such a highly regulated field as credit risk assessment, we opted to combine XGBoost and logistic regression.
We used both scoring engines – Logistic regression and the ML based one – to assess all of the loan applications
With a clear correlation between the two assessment approaches, a high score in one model would likely mean a high score in the other.
In the original approach, logistic regression was used to assess applications. The acceptance level was set around 60% and the risk resulted at 1%
If we decrease the threshold by a couple of points, the acceptance level hits 70% while the risk jumps to 1,5%
We next applied a threshold for an ML model, allowing us to get an acceptance percentage to the original level (60%) while bringing the risk down to 0,75% that is by 25% lower than the risk level resulting from only traditional approach.
Machine Learning is often seen as difficult to apply in banking due to the sheer amount of regulation the industry faces. The facts don’t necessarily back this up. ML is successfully used in numerous, heavily regulated industries. The example above is just one more example of how. Thanks to this innovative approach it is possible to increase the sustainability of the loans sector and make loans even more affordable to bank customers. There’s nothing artificial about that kind of intelligence.
AI has become a powerful force in Computer Vision and it has unleashed tangible business opportunities for 2D visual data such as images and videos. Applying AI can bring tremendous results in a number of fields. To learn more about this exciting area, read our overview of 2D computer vision algorithms and applications.
Despite its popularity, there is nothing inherent to 2D imagery that makes it uniquely suitable for AI application. In fact, artificial intelligence systems can analyze various forms of information, including volumetric data. In spite of the increasing number of companies already using 3D data gathered by lidar or 3D cameras, AI applications aren’t the mainstream in their industries.
In this post, we describe how to leverage 3D data across multiple industries with the use of AI. Later in the article we’ll have a closer look at the nuts and bolts of the technology and we’ll aslo show what it takes to apply AI to 3D data. At the end of the post, you’ll also find an interactive demo to play with.
3D data is what we call volumetric information. The most common types include:
There are also multiple data representations. These include a compound of 2D images along the normal axis, sparse Point Cloud representation and voxelized representation. Such data could have additional channels, like reflectance in every point of a lidar’s view.
Depending on the business need, there can be different objectives for using AI: object detection and classification, semantic segmentation, instance segmentation and movement parameterization, to name a few. Moreover, every setup has its own characteristics and limitations that should be addressed with a dedicated approach (or, in the case of artificial neural networks, with a sophisticated and thoroughly designed architecture). These are the main reasons our clients come to us, and to take advantage of our experience in the field. We are responsible for delivering the AI part of specific projects, even though the majority of their competencies are built in-house.
Autonomous driving data are very sparse because:
For autonomous driving, we needed a system that can take advantage of data sparsity to infer 3D bounding boxes around objects. One such network is the part-aware and aggregation neural network i.e. Part-A2 net (https://arxiv.org/abs/1907.03670). This is a two-stage network that uses the high separability of objects, which functions as segmentation information.
In the first stage, the network estimates the position of foreground points of objects inside bounding boxes generated by an anchor-based or anchor-free scheme. Then, in the second stage, the network aggregates local information for box refinement and class estimation. The network output is shown below, with the colors of points in bounding boxes showing their relative location as perceived by the Part-A2 net.
A different setup is called for in mapping indoor environments, such as we do with instance segmentation of objects in office space or shops (see this dataset for better intuition: S3DIS dataset). Here we employ a relatively high-density representation of a point cloud and BoNet architecture.
In this case the space is divided into a 1- x 1- x 1-meter cubic grid. In each cube, a few thousand points are sampled for further processing. In an autonomous driving scenario, such a grid division would make little sense given the sheer number of cubes produced, many of which are empty and only a few of which contain any relevant information.
The network produces semantic segmentation masks as well as bounding boxes. The inference is a two-stage process. The first produces a global feature vector to predict a fixed number of bounding boxes. It also tallies scores to indicate whether some of the predicted classes are inside those boxes. The point-level and global features derived in the first stage are then used to predict a point-level binary mask with the class assignment. The pictures below show a typical scene with the segmentation masks.
This is a highly controlled setup, where all 2D images are carefully and densely stacked together. Such a representation can be treated as a natural extension of a 2D setup. In such cases, modifying existing 2D approaches will deliver satisfactory results.
An example of a modified 2D approach is the 3D U-Net (https://arxiv.org/abs/1606.06650), where all 2D operations for a classical U-Net are replaced by their 3D counterparts. If you want to know more about AI in medicine, check out how it can be used to help with COVID-19 diagnosis and other challenges.
There is also another case, where luckily, it can be relatively straightforward to apply expertise and technology developed for 2D cases in 3D applications. One such scenario is where there are 2D labels available, but the data and the inference products are in 3D. Another is when 3D information can play a supportive role.
In such a case, a depth map produced by 3D cameras can be treated as an additional image channel beyond regular RGB colors. Such additional information increases the sensitivity of neural networks to edge detection and thus yield better object boundaries.
Examples of the projects we have delivered in such a setup include:
We developed an AI system for a tire manufacturer to detect diverse types of defects. 3D data played a crucial role as it allowed for ultra-precise detection of submillimeter-size bubbles and scratches.
We designed a system to detect and segment industrial assets in a chemical facility that had been thoroughly scanned with high resolution laser scanners. Combining 2D and 3D information allowed us to digitize the topology of the installation and its pipe system.
At deepsense.ai, we have a team of data scientists and software engineers handling the algorithmic, visualization, and integration capabilities. Our teams are set up to flexibly adapt to specific business cases and provide tailor-made AI solutions. The solutions they produce are an alternative to pre-made, off-the-shelf products, which often prove too rigid and constrained; they fail once user expectations deviate from the assumptions of their designers.
Processing and visualizing data in near real time with appropriate user experience is no piece of cake. Doing so requires a tough balancing act, including
combining specific business needs, technical limitations resulting from huge data loads and the need to support multiple platforms.
It is always easier to discuss based on an example. Next section shows what it takes to develop an object detection system for autonomous vehicles with outputs accessible from a web browser. The goal is to predict bounding boxes of 3 different classes: car, pedestrian and cyclist, 360 degrees around the car. Such a project can be divided into 4 interconnected components: data processing, algorithms, visualizations and deployment.
In our example, we use the KITTI and A2D2 datasets, two common datasets for autonomous driving, and ones our R&D hub rely on heavily. In both datasets, we use data from spinning lidars for inference and cameras for visualization purposes.
Lidars and cameras work independently, capturing data at different rates. To obtain a full picture, all data have to be mapped to a common coordinate system and adjusted for time. This is no easy task. As lidars are constantly spinning, each point is captured at a different time, while simultaneously the position and rotation of the car in relation to world coordinates is changing. Meanwhile, the precise location and angle of the car is not known perfectly due to limitations of geolocation systems such as GPS. These difficulties make it extremely difficult to precisely and stably determine the absolute positions of objects around you (SLAM can be used to tackle some of the problems).
Fortunately, absolute positioning of objects around the vehicle is not always required.
There are a vast number of approaches when it comes to 3D data. However, factors such as the length to and between objects and high sparsity will play an essential role in which algorithm we ultimately settle on. As in the first example above, we used Part-A2 net.
We have relied on a complete, in-house solution for visualization, data handling, and UI. We have used expertise in the Unity engine to develop a cross-platform, graphically rich and fully flexible solution. In terms of a platform, we opted for maximum availability, which can be satisfied by a popular web browser like Chrome or Mozilla and WebGL as Unity’s compilation platform.
WebGL, while very comfortable for the user, disables drive access and advanced GPU features, limits available RAM to 2GB and processing to a single thread. Additionally, while standalone solutions in Unity may rely on existing libraries for point cloud visualization, making it possible to visualize hundreds of millions of points (thanks to advanced GPU features), this is not the case in WebGL.
Therefore, we have developed an in-house visualization solution enabling real-time, in-browser visualization of up to 70 mln points. Give it a try!
Such visualization could be tailored to the company’s specific needs. In a recent project, we took a different approach: we used AR glasses in visualizing a factory in all its complexity. This enabled our client to reach next level user experience and see the factory in a whole new light.
We hope that this post has shed some light on how AI can be used with 3D data. If you have a particular 3D use case in mind or you are just curious about the potential for AI solutions in your field, please reach out to us. We’ll be happy to share our experience and discuss potential ways we can help you apply the power of artificial intelligence in your business. Please drop us an email at firstname.lastname@example.org.
Modern science is facing a completely new challenge: overload. According to research from the University of Ottawa, the total number of research papers published since 1665 passed the 50 million mark in 2009 and approximately 2.5 million new papers are published every year. In fact, it is nearly impossible to be up-to-date with all this information, at least for a human being. Machine learning tools make it easier and faster to find information in today’s ever vaster trove of publications.
Books-box.com runs a platform with access to a wide variety of science-oriented literature. Its library contains around 5,000 books across multiple categories. But rather than distributing whole books in a digital form, it provides page-level access to required pieces of knowledge. This is often the case in academia and research work, where a particular piece of information is needed to enrich a paper and deliver more credible information.
We had the pleasure of working with books-box.com and providing them with NLP services. The goal of the project was to create a recommendation engine that suggests relevant literature to users based on the content they’re viewing.
To make a book’s text readable to a computer, the words are transformed into vectors. A vector is just a set of real numbers that functions as input for a Machine Learning algorithm. If you would like to learn more about the technologies and techniques we use, click on over to our business guide to Natural Language Processing.
One way to transform a sentence into a vector of numbers is one-hot encoding. This technique transforms a word into an n-vector where “n” equals the number of all unique words the model was taught during training. Unfortunately, this solution isn’t very useful for text because the vector becomes enormous and the word order and context are completely lost.
Enter embeddings, the state-of-the-art in NLP algorithms. To create a sentence embedding means to assign a vector to a sentence in a vector space that conserves semantics. When two embedding vectors in this space are close to each other, the sentences they represent are similar in meaning. Commonly used vector size is between 100 and 1024, which is much smaller than the number of all unique words.
To create sentence embeddings we use the top-shelf neural network-based NLP algorithms ELMo and BERT. ELMo uses deep, bi-directional LSTM recurrent neural networks, while BERT uses the Transformer attention mechanism. The engine uses both algorithms to make final recommendations.
We developed a proprietary aggregation mechanism that allows us to generate aggregate embedding vectors for each book page. They allow us to easily check page similarity, by calculating the cosine similarity of two vectors, a standard metric in multidimensional vector space.
When viewing a page users will get five other page recommendations that may interest them. But having around 200,000 pages per book category to get recommendations means calculating 200,000 page embedding comparisons for each request! That’s a lot of computing time. But instead of calculating the similarity online, we calculate it beforehand, and store top recommendations.
Having a pre-calculated cosine similarity between all pages, book-box’s recommendations can now be given almost instantaneously – the higher the score, the better the recommendation will be.
Ball tree is an alternative solution to storing raw embeddings in a multi-dimensional, space partitioning data structure like k-d tree. The beauty of this approach is that it doesn’t require all possible embedding comparisons to be calculated. Instead, the data structure enables the optimal search for the nearest points (embeddings) in multidimensional space. In our case, however, there is one problem with this approach – from the business side we have required for one page to have recommendations from a variety of books. But top k similar pages for one (k is a parameter which needs to be chosen during the tree-build phase) would most likely be from the same book. And that was not the solution we were looking for.
Each pair of pages comes with a similarity score. In order to achieve the quality of recommendations desired, a threshold has to be selected. The higher the similarity score, the higher the quality of the recommendations will be, even if a smaller number of recommendations remain available.
It is worth noting that assessing recommendation quality is not a straightforward (binary) task as it takes the subjective opinion of the assessor into account.
books-box.com regularly adds new books to its library. This entails preprocessing new books (parsing from html etc.), transforming their pages to embedding vectors and then updating the recommendation structure. Such operations require a lot of computing power, especially when we’re talking about thousands of books. To run neural networks for embedding, we need GPU devices for fast parallel computing.
We decided to deploy our recommendation engine on Amazon Web Services (AWS) cloud, which allowed us to control costs and work on the solution’s elasticity, durability and scaling capabilities. AWS also provides a convenient system of spot instances that are available at a discount of up to 90% compared to on-demand pricing.
Our deployment consists of three elements: API server, Simple Storage Service (S3) bucket, Graph updater.
As new books are ingested into the S3 bucket, a new message is sent to the appropriate queue in the Simple Queue Service (SQS). It contains information about where the book is stored in the bucket. Each message represents one book and each queue represents one category.
The CloudWatch component observes the size of this queue, and it will update the instances count in the Auto Scaling Group (ASG) accordingly – if there are a lot of messages, it will increase the count; otherwise it will decrease it.
The Auto Scaling Group (ASG) keeps track of the number of instances running. If ASG instance count drops to 0, it will terminate all of the running instances. Once Elastic Compute Cloud (EC2) instances come online, they will connect to the queue and start processing jobs. When there are no more jobs, ASG is set back to zero and the instances will terminate.
To make our solution cost effective, we went with EC2 spot instances, cutting costs by up to 70% compared to on-demand instances. When using EC2’s in conjunction with SQS we can continue processing even if instances are terminated because of the price limit has been reached.They will be back on as soon as the price drops again and they will pick up any work that’s still left on the queues.
Each of the EC2’s runs dockerized applications that process the books and keep a graph that’s stored and updated on S3. Thankfully, AWS offers the data transfer between EC2 and S3 free of charge.
Scanning through immense amounts of text in the latest scientific publication was a painful and time-consuming process. The ML-powered tools delivered by books-box cuts all the noise and delivers the desired pages straight to the researcher in little to no time.
Throughout history, tackling pandemics has always been about using the latest knowledge and approaches. Today, with AI-powered solutions, healthcare has new tools to tackle present and future challenges, and the COVID-19 pandemic will prove to be a catalyst of change.
It was probably a typical October day in Messina, a Sicilian port, when 12 genoese ships docked. People were horrified to discover the dead bodies of sailors aboard, and with them the entrance of the black death to Europe. Today, in the age of vaccines and advanced medical treatments, the specter of a pandemic may until recently have seemed a phantom menace. But the COVID pandemic has proved otherwise.
There are currently several challenges regarding the COVID, including symptoms that can be easily mistaken with those of the common flu. An X-ray or CT image of lungs is a key element in the diagnosis and treatment of COVID 19 – the disease produces several telltale signs that are easy for trained professionals to spot. Or a trained neural network.
Computer scientists have traditionally developed methods that let them find keypoints on images based on defined heuristics, which allow them to tackle a huge array of problems. For example, locating machine parts on a uniform conveyor belt where simple colour filtration differentiates them from the background. But this is not the case for more sophisticated problems, where extensive domain knowledge is required.
Enter Neural Networks, algorithms inspired by the mathematical model of how the human brain processes signals. In the same way as humans gain knowledge by gathering experience, Neural Networks process data and learn on their own, instead of being manually tuned.
In AI-powered image processing, every pixel is represented as an input node and its value is passed to neurons in the next layer, allowing the interdependencies between pixels to be captured. As seen in the face detection model below, the lower layers develop the ability to filter simple shapes like edges and corners (e.g., eye corners) or color gradients. These are then used by intermediate layers to construct more sophisticated shapes representing the parts of the objects being analysed (in this case eyes, parts of lips or a lung edge etc.). The high layers analyse recognised parts and classify them as specific objects. In the case of X-ray images, such objects may be a rib, a lung or an irrelevant object in the background.
A neural network can see details the average observer cannot, and even specialists would be hard-pressed to find. But such skill requires a significant amount of training and a good dataset.
Data scientists spend a lot of time ensuring their models have the ability to generalise, and can thus deliver accurate predictions from data they didn’t encounter during training. This requires vast knowledge of data preprocessing and augmentation techniques, state-of-the-art network architectures and error-interpreting skills. The iterative process of designing and executing experiments is also both very time- and computing power-consuming and requires good organisation if it is to be done efficiently. Under these conditions, high prediction accuracy is hard to achieve – deepsense.ai’s teams have been developing this ability for 7 years.
The key difference between a human specialist and a neural network is that the latter is completely domain-agnostic. An algorithm that excelled in Segmenting satellite images or recognising individual North Atlantic right whales from a population of 447 of North Atlantic right whales can just as well be used for medical image recognition after tuning.
Numerous AI solutions are currently used in medicine: from appointments and digitization of medical records to drug dosing algorithms (applications of artificial intelligence in health care). However, doctors still have to perform painstaking and repetitive tasks e.g. by analyzing images.
Images are used across the field of medicine, but they play a particularly important role in radiology. According to IBM estimates, up to 90% of all medical data is in image form, be it x-rays, MRIs or most other output from a diagnostic device. That is why radiology as a field is so open to using new technologies. Computers initially used in clinical imaging for administrative work, such as image acquisition and storage, are now becoming an indispensable element of the work environment at the beginning of the image archiving and communication system.
Recently, deep learning has been used with great success in medical imaging thanks to its ability to extract features. In particular, neural networks have been used to detect and differentiate bacterial and viral pneumonia in childrens’ chest radiographs).
COVID appears to be a similar case. Studies show that 86% of Covid-19 patients have ground-glass opacities (GGO), 64% have mixed GGO and consolidation and 71% have vascular enlargement in the lesion. This can be observed on CT scans as well as chest X-ray images and can be relatively easily spotted by a trained neural network.
There are several advantages of CT and x-ray scans when it comes to diagnosing COVID-19. The speed and noninvasiveness of these methods make them suitable for assisting doctors in determining the development of the infection and making decisions regarding performance of invasive tests. Also, due to the lack of both vaccines and medications, immediately isolating the infected patient is the only way to prevent the spread of the disease.
deepsense.ai’s first foray into medical data was when we took part in a competition to classify the severity of diabetic retinopathy using images of retinas. The contestants were given over 35,000 images of retinas, each having a severity rating. There were 5 severity classes, and the distribution of classes was fairly imbalanced. Most of the images showed no signs of disease. Only a few percent had the two most severe ratings. After months of hard work, we took 6th place.
As we gained more contact and experience with medical data, our results improved, and after some time we were able to take on challenges such as producing an algorithm that could automatically detect nuclei. With images acquired under a variety of conditions and having different cell types, magnification, and imaging modality (brightfield vs. fluorescence), the main challenge was to ensure the ability to generalise across these conditions.
Another interesting project we did involved automatic stomatological assessment. We trained a model to read an x-ray image and detect and identify teeth, accessories and lesions including laces, implants, cavities, cavity fillings, and parodontosis, among a long list of others. In yet another project, we estimated minimum (end-systolic) and maximum (end-diastolic) volumes of the left ventricle from a set of MRI-images taken over one heartbeat. Our results were rated “excellent” by cardiologists that reviewed our work.
Move your mouse cursor over the image to see the difference.
The standardized formats used in medical imaging allow for better transfer of knowledge in modeling different problems. In a recent research project we explored the potential of image preprocessing of CT scans in DICOM format.
Image preprocessing is a vital aspect of computer vision projects. Developing the optimal procedure rests upon the team’s experience in similar projects as well as their ability to explore new ideas. In this case the specialized image preprocessing methods we developed made the image more readable for the model and boosted its performance by 20%.
It is common to think that an epidemic starts and ends, with no further threat to fear. But that’s not true. The black death started with the arrival of twelve ships from Genoa, then proceeded to claim the lives of up to 50 million Europeans. The disease still exists today, with 3248 people infected and 584 dead between 2010 and 2015. That’s right, the disease never really disappeared.
700 hundred years ago, Ragusa (modern Dubrovnik), then a Venice-controlled port city, played a prominent role in slowing the spread of the disease.. Learning from the tragic fate of other port cities including Venice, Genoa, Bergen and Weymouth, officials in Ragusa hold sailors on their ships for 30 days (trentino) to check if they were healthy and slow the spread of the disease.
COVID-19 is neither the most deadly nor the last pandemic humans will face. The key is to apply the latest knowledge and the most sophisticated solutions available to tackle the challenges they present. AI can support not only the most dramatic life-death issues in healthcare, but also more mundane cases. According to an Accenture study, AI can deliver savings of up to $150 billion annually by 2025 by supporting both the front line, with diagnosis augmentation, and the back office, by enhancing document processing or delivering more accurate cost estimates. This translates to potential significant savings for each hospital that adopts AI.
If you want to know more about the ways AI-powered solutions can support healthcare and tackle modern and future pandemics, contact us through the form below!
A trigger-based alerting in infrastructure management is currently insufficient to keep operations smooth and uninterrupted. AI-powered IT operations deliver smart alerts that not only improve a team’s ability to respond swiftly but can also deliver significant automation.
According to Gartner data quoted by the TechBeacon, 40% of organizations are predicted to strategically implement AIOps to enhance performance monitoring by 2022. The reason is obvious: AIOps helps build resilient and reliable IT corporate ecosystems. It also boosts overall company performance, an area that can always be improved. For example, for 34.4% of organizations it takes more than 30 minutes on average to resolve IT incidents impacting consumer-facing digital services.
In the traditional approach to automating IT operations, sets of previously defined conditions triggered alerts.
A huge drawback of this approach was that problems had to be identified, defined and researched. Rules were then set to react to the problem when it occurred. This approach was effective only in dealing with problems that had already come up, but it was useless when conditions changed. That’s common in IT infrastructure.
Also, manually setting the rules opens the process up to mistakes hidden within the code and rules – the more complicated the system or instruction, the better hidden a malfunction can be. And there it lurks, ready to hit right when the rule should have saved the infrastructure.
This is where AI tools come in handy.
The key difference between trigger-based alerts and AI-powered alerts is that the first comes with a huge amount of manual work, while the second can cut workloads significantly by automating the most tedious tasks (read more about that in our What is AIOps blog post).
Both types of alerts send a constant flow of data in from the system, including logs, events or metrics collected by sensors. However, only in the AI alerts is the data preprocessed and initially analyzed before a human gets it–stripped of the noise and chaos usually seen in the raw data.
Unlike in less sophisticated systems, AI-powered solutions deliver pattern-based, rather than triggered, alerting. The core difference is in recognizing not a particular, single event or a chain of events that occur within the network, but the overall pattern of a malfunction or attack. Thus, if there is a new channel or technology used, the system will remain vigilant. Read more about that in our AIOps for Network Traffic Analysis (NTA) blog post.
Also, the pattern-based approach not only helps spot known types of attacks but also can be useful when approaching the unknown. Harnessing the power of machine learning or reinforcement learning paradigms of supervised learning and reinforcement learning allows the neural network to learn how IT infrastructure fits into the daily work. This essentially makes it the direct opposite of a trigger-based system. Instead of spotting the signs of malicious activity, machine learning models ensure operations can detect anomalies within the system.
Smart alerting is a great tool for DevOps or administration specialists, though it remains only one step of their work. Any alert is a problem that needs to be fixed or at least a condition that requires supervision. In keeping IT infrastructure up and running as well, notification remains only a single step. Services lost even briefly can be very costly.
With AIOps implemented, ml-armed supervisors will not only spot the signs of a malfunction but also respond with the best policy available, according to their previous experience and information implemented in the network.
A simple example would be delivering a mirrored piece of infrastructure and redirecting the traffic while providing an alert to the supervisory team. AIOps handles this with alert escalation workflows and immediate response. Also, the AIOps infrastructure correlates the dependencies based on incoming data and responds not to a single event, but to address the group of alerts continuously. The approach saves the time usually spent grouping alerts into same-event related ones.
AIOps solutions have been adopted for their numerous benefits, including the following four.
Smart alerting is the foundation of next-level infrastructure management. Companies can harness the power of AI-powered solutions to optimize operations and deliver better results.
If you’d like to hear more about deepsense.ai’s AIOps platform and our approach, contact us using the form below or drop us a line at email@example.com.
Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.
These cookies are strictly necessary to provide you with services available through our website and to use some of its features.
We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.
We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.
Google Webfont Settings:
Google Map Settings:
Google reCaptcha Settings:
Vimeo and Youtube video embeds: