Position: Human-like reinforcement learning ıs facilitated by structured and expressive goals

Goals play a central role in the study of agentic behavior. But what is a goal, and how should we best represent them? The traditional reinforcement learning answer is that all goals are expressible as the maximization of future rewards. While parsimonious, such a definition seems insufficient when viewed from both the perspective of humans specifying goals to machines and autotelic agents that self-propose tasks to explore and learn. We offer a critical perspective on the distillation of all goals directly into reward functions. We identify key features we believe goal representations ought to have, and then discuss a proposal we believe meets those considerations. This position paper argues that to specify human-like problems and construct agents to pursue them, the basic notion of reward in RL is impoverished and instead should be augmented by structured and expressive goal representations.