I am a DPhil student at the VGG Lab (Oxford), advised by Prof. Andrew Zisserman at the University of Oxford. I am interesting in multimodal video understanding and generation with a focus on temporal sensitivity.
The central theme of my research is to better understand, model and utilize the temporal dynamics in video data
beyond what can be inferred from a single frame. I have studied temporal structure in both
low-level physical processes (recovering
form-from-motion,
inferring physical properties from pouring sounds, and estimating hand-object contact forces) and
high-level action semantics (learning time-sensitive
video representations and improving retrieval for temporal actions).
Recognition by Imagination: A Generative Approach to Form-from-Motion
Under review
Piyush Bagad, Makarand Tapaswi, Andrew Zisserman
Inspired by Gunnar Johansson’s experiments (Johansson, 1973), we train a video-to-video generation model to map sparse keypoints to dense RGB. The same model generalizes to other psychophysics tests such as the Hidden Bird Illusion.
Forces at your fingertips: Estimating Hand-Object Contact Forces using Newtonian Physics
Under review
Piyush Bagad, Taein Kwon
Given 3D hand and object meshes, we use Newton’s laws (free-body diagrams from high school physics) to optimize the distribution of forces on the hand in equilibrium.
This is a work-in-progress project with the ultimate goal of estimating hand-object contact forces from a single image.
TARA: Text Adapted Retrieval Alignment for Nuanced Video Retrieval
ECCV 2026
Piyush Bagad, Andrew Zisserman
We post-train a multimodal LLM (MLLM) for text-to-video retrieval on nuanced, temporally sensitive queries (e.g., folding vs unfolding paper) with only using text data. We show the embeddings from this model are more time-sensitive that prior work.
Chirality in Action: Time-Aware Video Representation Learning by Latent Straightening
NeurIPS 2025
Piyush Bagad, Andrew Zisserman
We discover that per-frame DINOv2 embeddings for a video lie on a smooth time-aware manifold. We exploit this to train an adapter called Linearized Feature Trajectories (LiFT) inspired by perceptual straightening hypothesis. LiFT produces time-sensitive, compact video embeddings with a simple self-supervised objective.
The Sound of Water: Inferring Physical Properties from Pouring Liquids
ICASSP 2025 and TPAMI 2025
Piyush Bagad, Makarand Tapaswi, Cees G. M. Snoek, Andrew Zisserman
We discover that the simple sounds of pouring water contain rich structure which connects to the physical properties of the glass-liquid system. We use this to train a model which predicts physical properties (e.g., size, radius of glass) merely from the sound of pouring.
Test of Time: Instilling Video-Language Models with a Sense of Time
CVPR 2023
Piyush Bagad, Makarand Tapaswi, Cees G. M. Snoek
At the time, we discover that standard video-text models did not accurately encode simple temporal relations (e.g., before vs after). We fine-tune a video-text model in a CLIP like manner with carefully designed hard-negatives to instill this sense of time.
How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?
ECCV, 2022 (Tel Aviv, Israel)
Fida Mohammad Thoker, Hazel Doughty, Piyush Bagad, Cees Snoek
We carefully studied various contemporary video benchmarks and models and found that standard benchmarks like Kinetics400/UCF101 are not representative of generalization. Based on our insights, we proposed a new benchmark, SEVERE, which systematically evaluates generalization along four axes of variation.
C-3PO: Towards Rotation Equivariant Feature Detection and Description
Piyush Bagad*, Floor Eijkelboom*, Mark Fokkema*, Danilo de Goede*, Paul Hilders*, Miltiadis Kofinas
When an image rotates, its keypoints should rotate with it and remain easy to match. We build this rotation equivariance directly into a feature detector and descriptor using steerable convolutions, improving matching over its non-equivariant counterparts for most rotations.
Reproducibility Study of "Counterfactual Generative Networks"
Piyush Bagad*, Paul Hilders*, Jesse Maas*, Danilo de Goede*
We reproduce a generative model that independently controls object shape, texture and background to create counterfactual images (e.g., the same object against a different background). We largely confirm that training classifiers on these images improves generalization beyond spurious cues, while identifying reproducibility issues and cases where the model fails to separate these factors.
Quantifying CLIP’s ability to Perform Cross-Modal Grounding Using Attention-Model Explainability
Best poster award in the Interpretability course at UvA
Piyush Bagad*, Paul Hilders*, Danilo de Goede*
We use transformer attribution maps to test whether CLIP connects words in a caption to the relevant regions of an image. Comparing these maps with annotated object regions provides evidence that CLIP learns some fine-grained grounding, beyond simply matching whole images and captions.
Industry
I have had experience of working in the industry on a variety of projects.
At Adobe Research (India), I developed a method to generate synthetic tabular data with a controllable privacy-accuracy trade-off.
At Wadhwani AI (India), I worked on several projects applying AI to important
problems in heathcare including newborn anthropometry and sound-based disease diagnosis.
Data-Sharing Economy: Value-Addition from Data meets Privacy
Work done during Internship at Adobe Research (India) in 2018. We built a Generative Adversarial Network (GAN)-based framework to generate synthetic tabular data with a controllable privacy-accuracy trade-off.
Cough Against COVID: Evidence of COVID-19 Signature in Cough Sounds
We built a cough sound classification model that can detect COVID-19 symptoms from cough sounds. We collected a large-scale dataset of cough sounds from COVID-19 patients and healthy individuals to train the model. We fine-tuned CNN-based audio models that are robust to confounders like recording location, etc.
Newborn Anthropometry: Estimating infant weight from a monocular video
Wadhwani AI, 2019–2021 (Mumbai, India)
Piyush Bagad, Aman Dalmia, Makarand Tapaswi, Sansiddh Jain, Aditya A Sarma, Namrata Deka, Jigar Doshi, Parag Bhamare, Amrita Mahale, Rahul Panicker (Not in order)
We built a model that can estimate the weight of a newborn baby from a single video. As of September 2026, the deployed Shishu Maapan application has supported 26K Home-Based Newborn Care (HBNC) visits, registered 13K newborns, and is used by 471 ASHA workers.
The subsequent NurtureNet paper received the Best Paper Award at the CVPM workshop at CVPR 2024.
Synthetic Data Generation for Visual Weighing Machine
We used the 3D deformable model to generate a large-scale synthetic dataset to train models to estimate the weight of a newborn baby from a single video.
Deformable 3D Model for Newborn Babies
Underlying Model: SMIL (Nicholas Hesse et. al)
Piyush Bagad, Aman Dalmia, Rahul Kumar, Arsh Verma, Rahul Panicker
We built a deformable 3D model for Indian newborn babies based on the SMIL (European) model. We collected a large-scale 4D scan dataset, registered the dataset to SMIL model, and then trained a new model for Indian context.
Miscellaneous
Service: Reviewer at ECCV 2022, CVPR 2023, ECCV 2024, ICCV 2025, NeurIPS 2025, CVPR 2025. Awarded outstanding reviewer at CVPR 2023 and CVPR 2025.
Teaching: TA for Deep Learning (DL1) at the University of Amsterdam
Invited lecture for the Fairness in AI course at the University of Amsterdam
News
Mar 5, 2023
A shorted version of our paper got accepted to ME-FoMo Workshop ICLR 2023!
Feb 27, 2023
Our paper on time in video-language models got accepted to CVPR 2023!