Chinese AI Safety Team Plans to Audit Reward Vulnerabilities in Frontier Lab Reinforcement Learning Environments

Written by

in

CMLKevin describes how his team is building an AI safety community in China and plans to audit unreliable reinforcement learning environments provided by domestic data suppliers for training China’s frontier labs. The team says that 2/3 of Git logs in the open-source Xiaomi MiMo v2.6 reinforcement learning environment contained leaks; according to conversations with employees, only about 5–10 people at each major Chinese lab work on alignment and harm reduction.

Source:LessWrong · Read Original

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *