Can world models improve AI for drug discovery? This talk explores molecular machine learning, self-supervised learning, JEPA, graph neural networks, and antibiotic discovery.
In this session, Michał Kulczykowski, Senior MLE at deepsense.ai, tests whether world models can learn useful molecular representations for drug discovery — and compares the results with much simpler baselines.
The starting point is antimicrobial resistance and the search for new compounds effective against MRSA. Michał then walks through the mechanics of predictive world models, representation collapse, and a JEPA-inspired approach adapted to molecular graphs.
The experiments cover:
- self-supervised pre-training on ~3 million drug-like molecules from ChEMBL,
- molecular views built from atoms, bonds, and connected subgraphs,
- evaluation on the Golden Staph dataset and MoleculeNet HIV,
- graph-based representations vs handcrafted molecular fingerprints,
- why fine-tuning removed the advantage gained during pre-training,
- why combining learned representations with classical features performed better than either alone.
Timeline
00:00 AI and antibiotic discovery
02:44 How world models work
04:59 World models for molecular data
05:57 Benchmark results
07:25 Building the generation pipeline
Speaker
Michał Kulczykowski
Senior ML Engineer
