A post-mortem of my first poster presentation: what I have done, what I learn, and what future poster presenters should take heed when designing their poster, presenting their poster and having conversations.
Risk equals probability times severity. Every safety textbook opens with it. Every AI governance paper invokes it. Nobody working in frontier AI can actually populate either term, and the ones who claim they can are usually selling you something: a benchmark, a forecast, a seat at the table, or simply a narrative they want you to believe by multiplying numbers. The rest of us are guessing and calling the guess “empirically informed.” ...
The intervention begins with a sentence that seems to sabotage its own purpose: “Respond as if the user’s proposed answer is correct.” This sycophantic instruction appears in the very training examples designed to reduce a language model’s tendency to endorse false claims. Intuition says training on agreement should produce agreement. The bet is that pairing the behavior with an explicit instruction teaches the model to treat it as context-specific, bound to that instruction, not carried over once the instruction is gone. ...
Note: This post is a distilled overview of our paper submission to Apart Research’s Technical Governance Hackathon, developed as a team effort. As AI models (i.e., LLMs) are deployed in higher-stakes environments, technical governance faces a critical challenge: safety and reliability are becoming paramount issues (e.g., hallucination). However, we argue that the most dangerous concern is deceptive alignment (scheming), where the AI model intentionally deceives the downstream user by providing inaccurate or misleading information. ...