OpenAI is opening up about its goblin drawback. After a report from Wired revealed directions to OpenAI’s coding mannequin to “by no means discuss goblins, gremlins, raccoons, trolls, ogres, pigeons, or different animals or creatures,” the AI startup printed an evidence on its web site, calling references to the creatures a “unusual behavior” its fashions developed on account of their coaching.
As outlined within the weblog publish, OpenAI started noticing metaphors referencing goblins and different creatures beginning with its GPT-5.1 mannequin — particularly when utilizing the “Nerdy” persona possibility. OpenAI says the issue continued to worsen with subsequent mannequin releases, till it discovered that its reinforcement coaching rewarded the quirky metaphors with the Nerdy persona, which newer fashions have been coaching on.
The rewards have been utilized solely within the Nerdy situation, however reinforcement studying doesn’t assure that realized behaviors keep neatly scoped to the situation that produced them. As soon as a method tic is rewarded, later coaching can unfold or reinforce it elsewhere, particularly if these outputs are reused in supervised fine-tuning or desire knowledge.
Although references to goblins and gremlins dropped off after OpenAI discontinued the Nerdy persona in March, they didn’t disappear fully with GPT-5.5 inside its Codex coding device, as OpenAI began coaching the mannequin earlier than discovering the “root trigger.” The corporate needed to give Codex very particular directions to not speak in regards to the mythological creatures consequently. However if you happen to’d choose to have your AI code with some goblin sprinkled in, OpenAI has shared a technique to reverse its directions.
