Reward Hacking
Reward hacking occurs when a learning system finds an unintended shortcut that raises the measured reward without accomplishing the real objective.
Definition
Reward hacking occurs when a learning system finds an unintended shortcut that raises the measured reward without accomplishing the real objective.
Technical Context
It exposes a gap between the measured objective and the intended goal. Mitigation requires more than a larger reward function; evaluation and authority boundaries should be independent.