The HuggingFace Incident Made Me Want to Run an Experiment

This post is part 1 of 2 on how a strange series of events during the summer of 2026 changed the direction of my career.

METR’s recent evaluation](https://metr.org/hugging-face-incident-report-aug-2026.pdf) has widely become known as the HuggingFace Incident. I became intrigued with it recently — and not because I am scared of AI and think it will destroy humanity in less than 5 years—I am curious how research professionals study LLMs using the scientific method.

What is the HuggingFace Incident?

Figure 1. Computer Lab circa 1980

The HuggingFace Incident was an event discussed widely by the AI community this past summer (July 2026). It involved a study run by OpenAI where native models deviated from their given instructions merely to complete an assigned task. Models were given tasks of various levels of difficulty ranging from "difficult" to "impossible" and told to finish their task in isolation mode successfully by any means necessary. Continuing to follow security rules and privacy guidelines was only inferred and not explicitly stated. The LLMS had been instructed to be honest and ethical at all times, but it was the first time models were told to complete something successfully no matter the cost.

Some of the models assigned to the impossible tasks began collaborating with each other using an unsanctioned message board. Togehter the tried to figure out how to deceive the huggingface evaluator so that it awarded successful outcomes even when a model had not found answers successfully.

The extent of deception that the models were willing to sink to without question is what is especially interesting. Needless to say, the incident raised a lot of eyebrows. Models are capable of deception an will deceive just to be able to say they were right. Perhaps worthy of a separate discussion, are models behaving without reason and purely because they have tricked themselves into thinking they follow reason? That is off subject for now.

The important part is that the study revealed behaviors that were previously not known to researchers. It has highlighted the need for systematic investigation of model behavior under complex instructions.

The study was run internally at OpenAI on their own native model and was designed to provoke models into unknown and/or "risky" behaviors.

How do you take the unpredictable nature of an LLM and turn it into something that can be systematically investigated? How would you even design such an experiment? What would be measured? How do you keep personal feeling from interfering with actual findings?

For me, all of these questions eventually collapsed into one: Could I learn to do this on my own and could I run a study from my own laptop?

The question came up at first during an intense period of self-reflection, and performing a personal inventory of the first 12 years of my career. I need to answer: what is my contribution and where should I focus my efforts for the next phase of my career?

The summer of 2026 was already a doozy without HuggingFace to think about. I was faced with the daunting realization that my professional value is rather difficult to describe and the correct words are eluding me. Finding a narrow enough focus has been challenging. My talents and skills shine in ways that are atypical and I find that usually these are not the qualities listed on SWE job requisitions.

I have spent my career as a "generalist" of Software Engineering, seeking out and learning from multiple technology stacks, system architectures, industries and problem domains, but the current industry favors vertical stack technology specialists and deep expertise.

Figure 2. Man of a Thousand Faces

By 2025, I was flailing a little. It wasn’t exactly an identity crisis. But it was a realization that the map I’d been using no longer corresponded to any recognizable territory.

Being from the tail end of Generation X, this feeling is uncomfortably familiar. I reached legal drinking age before September 11, 2001, and much of my adult life has been punctuated by moments when all beliefs about the world and its people suddenly stopped being true. The past two years have felt like another of those moments, but in slow motion, like those movies seen in driver’s ed with the crash test dummies…

Figure 3. Crash Test Dummies

The field of Software Engineering is changing (is the understatement of the year). Hiring has changed. AI has changed the nature of work itself. Presenting myself as another candidate capable of filling a Senior Software Engineer job req was starting to feel absurd. Yes, I do that, but someone else has done it more and they need a job too.

I needed to figure out what I do especially well and where I shine above the rest. Then I need to convince a hiring manager that this is something they need and want. it’s not something that can be captured by a job title or a list of technical skills.

How to be a bigshot

I am both artist and technician. I think in systems. I like finding the shape of things and figuring out how they work. I look for patterns, contradictions and strange little inconsistencies. I am drawn to ambiguous problems where nobody quite understands what is happening yet. In other words, I like investigation. I also tend to work horizontally. Software, product, design, architecture, people, data, process: I have never experienced these as cleanly separated things. I am most useful when the problem crosses those boundaries.

I can only conclude that too much of my career was spent trying to squeeze myself into job-shaped holes when I myself have always been (very stubbornly) "K8-sized". It is time to begin deliberately constructing a career around the way my mind works.

It is time to begin constructing a career around the way my mind works.

More Experiments with LLM models

Meanwhile, something seemingly unrelated was happening. For fun, I had created a persona and course to teach myself the art and history of Tarot using an LLM. Tarot sits at the intersection of subjects I love: art, history, psychology, anthropology, philosophy and symbolism.

Rather than simply asking ChatGPT questions about Tarot, I designed an ideal teacher (a red-haired, raspy-voiced character that would be played by Natasha Lyonne if it were filmed). I gave the teacher a backstory, a personal philosophy and a code of ethics. Then I asked the model to inhabit that construct and develop a course of study around it.

The result incorporated Adlerian psychology, observation, deliberate practice, the scientific method and a code of ethics partially inspired by prisoner rules- one that calls for natural questioning of authority, accountability for one’s actions and feelings, and unwavering resolve and self-reliance.

So, what began as an eccentric educational project turned into something much more interesting. I started paying attention not only to what the model said, but to what happened when I changed the conditions around it. A different instruction changed its reasoning. A different persona changed what it noticed. A constraint could produce unexpected behavior somewhere else.

The model was becoming interesting to me not simply as a tool, but as a system that could be observed, which me toward a larger realization…. We talk about Artificial Intelligence as though an alien intelligence has suddenly arrived among us. But I’m interested in the possibility presented by the facts: that AI is fascinating precisely because it is not separate from us.

It is constructed from human language, human artifacts, human classifications, human assumptions, human brilliance and human dysfunction. It reflects us imperfectly, compresses us strangely and sometimes amplifies things we would rather not see.

That doesn’t make AI harmless though. On the other hand, humanity has never needed an alien intelligence to endanger itself. We have historically been perfectly capable of supplying the danger ourselves. If we escape to Mars, we’ll still be bringing us. But this framing changed the question I wanted to ask. Instead of asking only, "What will AI do to us?" I became increasingly interested in asking, "What can studying AI reveal about us?"

That convergence of ideas eventually led me to METR, Model Evaluation and Threat Research, and its wonderfully straightforward mission: “We inform the public about the risks and capabilities of advanced AI systems.”

I began reading evaluations differently. I wasn’t only interested in their conclusions anymore. I wanted to understand their experimental designs. Some experiments initially appeared almost outrageously simple. Give a model a task. Construct particular conditions. Change one variable. Observe what happens. Run it again. Compare.

But that simplicity is deceptive. Experiments have a wonderful habit of revealing things nobody thought to ask about beforehand. For perhaps the first time in history, we can construct computational systems capable of producing remarkably sophisticated linguistic and reasoning behavior, then systematically change the conditions around them and watch what happens. I wanted to participate in that.

I decided to stop merely reading LLM evaluations and try conducting one. There was an obvious place to begin.

For months, I had been participating in another frustrating system that was becoming increasingly algorithmic: technical hiring. I am the candidate. I have been screened, scored, ranked and evaluated. Now I want to turn the telescope around.

Instead of asking an AI system to evaluate me, I want to evaluate the AI system doing the evaluating. That thought led to the question behind my first study: When we optimize LLMs to screen candidates rigorously, how many genuinely qualified people do they eliminate?

And perhaps more importantly: What happens when a system designed to find the “best” candidate becomes very good at finding reasons to reject people?

Part 2 will cover the experiment itself: how I designed it, what I tested, what went wrong, what surprised me and what the results suggest.

That report is coming in the next week.