WARP: Weight-Space Analysis for Recovering Training Data Portfolios

A new research framework called WARP was introduced, allowing for the inference of training data compositions from released model weights. This is achieved through analyzing geometric footprints in weight space using model merging and feature extraction. In experiments, WARP was able to recover domain mixtures with an average MAE as low as 0.046 for BERT and 0.104 for GPT-2.

1sec.ai