Matt Clarke and colleagues found in fine-tuning experiments on small models that poisoned data with little individual impact may cause models to learn hidden backdoors when combined with clean data in the same format. The experiments covered behavioral backdoors, reward-hacking records, generalization of ancient bird names, and MNIST classifiers with incorrect labels. The study also proposed a simple regression model to explain the phenomenon.
Source:LessWrong · Read Original
Leave a Reply