With the growing importance and popularity of social media, the application of natural language processing techniques to this domain is becoming an increasingly trendy topic among NLP researchers. As part of an R&D project aiming to contribute to the advancement of Polish NLP, deepsense.ai experts have introduced TrelBERT, a language model for Polish social media which has achieved outstanding results in detecting cyberbullying.

“We actively follow the latest international developments in the field of NLP and, as part of the R&D project, our team has decided to take up the challenge and support the advancement of Polish NLP. Working on TrelBERT not only brought us a great deal of satisfaction, but also allowed us to learn how to handle the rather informal language used on social media. When examining the resulting model, we discovered that it knows a lot about Polish sports and politics – topics frequently discussed online by humans”
said Artur Zygadło, Lead Data Scientist at deepsense.ai.

The typical approach to training modern NLP models is so-called transfer learning, which assumes the development of one large NLP model that captures general knowledge about the language, and can then be trained to perform various specific tasks. The starting point for the experiments was HerBERT, a solution previously published by Allegro – the company behind the largest e-commerce platform in Poland. Their model already knew the general rules, words and their meanings in different contexts in the Polish language, which it learned by “studying” texts from various sources: CommonCrawl, the National Corpus of Polish, movie subtitles or Wikipedia. What it did not know about was the language of social media, which differs significantly from texts in the resources listed above. And this is what deepsense.ai’s team taught it.

The model created by deepsense.ai utilized HerBERT and was further trained using almost 100 million messages extracted from Polish Twitter. The decision was made to name the model TrelBERT, as “trel” is a Polish word describing the sound made by a bird (referring to “tweet”).

To evaluate TrelBERT against the tasks included in the Polish NLP, a benchmark called KLEJ (analogous to the famous English GLUE benchmark) was used. One of the tasks that was especially interesting was one which served the purposes of cyberbullying detection. The goal was to prepare a solution capable of determining whether a given Tweet is harmful or not. In this particular task, thanks to being trained on Twitter texts, the model surpassed all the other competitors and is currently holding the top position on the KLEJ leaderboard (in the “CBD” column – results for cyberbullying detection).

“It is no surprise to me that our model has achieved the best results in the cyberbullying detection task. This problem was born on social media and only exists there. We suspect that our model would beat the competition in other social media-related tasks, where content is spontaneous and generated by users, such as sentiment analysis or emotion recognition. Unfortunately, the Polish NLP community lacks such datasets”
said Wojciech Szmyd, Data Scientist at deepsense.ai.

Apart from submitting the model’s results to KLEJ, deepsense.ai made TrelBERT available in the popular Hugging Face repository of models. You can give the model a try and read more about it here.

neptune.ai – metadata store for MLOps – spun out of deepsense.ai, has raised $8 million in series A funding led by Almaz Capital with the participation of the existing investors: btov Partners, Rheingau Founders, and TDJ Pitango Ventures.

Led by Piotr Niedźwiedź, neptune.ai offers companies dedicated metadata storage that provides a central place to log, store, display, organize, share, compare, and query all metadata generated during the lifecycle of a machine learning model.

Since its launch in 2017, the store has brought in more than 20,000 ML engineers and 100 commercial customers, including Roche, NewYorker, Nnaisense and InstaDeep. The platform usage has grown eightfold over the past eight months while revenue has surged by four times.

“I am super proud of this, another great achievement for Piotr Niedźwiedź and his outstanding team. I have known Piotr since early 2011 when we established our first company together. Now, more than 11 years later, our passion has evolved into a strong group of cool tech companies – including, but not limited to: neptune.ai, deepsense.ai, CodiLime and Dealavo – with proven track records in delivery of professional IT services and products.  I feel this is still just the beginning of a fantastic adventure and our joint journey to the moon, as Piotr used to call neptune.ai. Dear Stakeholders – please fasten your seatbelts and keep your eyes open (and fingers crossed)!”
– Tomasz Kułakowski, CEO, deepsense.ai

The deepsense.ai team of Paulina Knut, Jakub Bartczuk, and Michał Zobniów won first place in the international data science competition AI Blitz 13.

The 13th edition of AI Blitz was focused on five computer vision challenges around face recognition:

• Sentiment Classification – training AI to identify emotions from a face’s convolutional neural network embedding,
• Age Prediction – building models to guess the approximate age of a person given their image,
• Mask Prediction – identifying and recognizing mask type and bounding box from the image of a face,
• Face Recognition – finding the target face in a collage of 100 photos. Some of the faces were artificially aged,
• Face De-blurring – restoring blurred face images.

“We love taking part in data science competitions, because it’s always a great opportunity to develop new skills and challenge ourselves. So, it was a pleasure to take up the AI Blitz challenge, especially since real-world computer vision is extremely close to us,”
– Paulina Knut, Senior Data Scientist at deepsense.ai

The competition ran for two weeks and the final leaderboard was calculated by the weighted mean of the rank across all the problems in the contest.

“I think that the agile approach helped us maintain the leading position in the competition. We tried to quickly and creatively apply models for all five challenges. It was both fun and a good experience for us for the future,”
– Jakub Bartczuk, Data Scientist at deepsense.ai

AI Blitz started as a beginner-friendly challenge to help AI students, professionals, and researchers across the world learn new skills by solving real-world problems. In past editions over 2600 participants have made more than 14500 submissions across several countries.

In the material “Succeed with Poland. The information and communications technology sector”, prepared by the Polish Investment and Trade Agency (PAIH), deepsense.ai was named as a Success Story next to such champions as Blik and CD Projekt Red.

“We are proud of this distinction and we treat it primarily as a motivation to continue working on the development of deepsense.ai”
– Tomasz Kułakowski, CEO at deepsense.ai.

The Polish Investment and Trade Agency (PAIH) supports both the foreign expansion of Polish business and the inflow of FDI into Poland. Supporting entrepreneurs, the Agency assists in overcoming administrative and legal procedures, related to specific projects as well as helps to develop legal solutions, find a suitable location, reliable partners and suppliers.

The brochure can be viewed here: