Why a quiz isn't enough to learn information security

The model you already know

Almost every company has, tucked somewhere in onboarding, a security awareness course: a video or text lesson, followed by a multiple-choice quiz. If the score clears a minimum, the person passes and gets a certificate. It's an old model, cheap to produce and easy to audit — which is why it's everywhere. It also has three problems that almost nobody questions, because the model became the industry's default standard.

Problem 1: it measures memory, not application

A multiple-choice quiz asks something like: "Which of the following is a sign of phishing?", with four options to pick from. Answering that correctly requires spotting a pattern in a closed list, with unlimited time and none of the pressure of a real situation: no boss demanding an answer right now, no deadline ticking down, no twenty other messages waiting in the inbox.

The problem is that the real world never shows you four options. You get an email that looks like it's from your usual vendor, with an invoice attached and a due date of tomorrow. You have to decide on the spot, with a full inbox and someone waiting on a reply. The skill that matters there isn't "recognize the right option in a list" — it's noticing that something is off in the middle of the normal noise of a workday. A quiz can't measure that, because it can't recreate that pressure.

Problem 2: the second attempt is the same exam

When someone retakes a quiz, the questions are usually the same or very similar — sometimes literally the same question bank, shuffled into a different order. That means the second attempt doesn't measure new learning: it measures memory of the previous exam. Over time, people learn to pass the quiz, not to detect phishing. Those are two different skills, and only one of them matters when a real attack lands.

For a quiz to actually measure learning on every attempt, it would need to generate new questions each time, with the same difficulty and the same curriculum coverage. In practice, almost nobody does that: it's expensive to maintain and hard to calibrate.

Problem 3: getting it wrong costs nothing

On a quiz, picking the wrong option has no real consequence beyond a lower score. Nothing breaks, nobody is affected, there's nothing to explain. That's fine for not generating unnecessary anxiety, but it also means the quiz can't teach something important: that in information security, decisions carry a cost in both directions. Complying with a malicious request opens a door. But distrusting everything and blocking every legitimate request also has a cost — the business grinds to a halt, people lose time, and eventually nobody reports anything for fear of getting it wrong "the other way."

A quiz can't represent that tension, because on a quiz there's no second way to be wrong.

What a game does differently

The idea behind the Experiencia RPG Academy starts from a simple premise: if the problem is measuring application instead of memory, the fix is to put the person in a real situation — simulated, but mechanically real — and watch what they do, not what they answer.

In Firewall Humano, the first game in this line, a team covers a fictional company's night shift: emails, internal chat messages, texts and voice notes arrive, some legitimate and urgent, others malicious. Nobody can solve it alone: one person sees the message the way any employee would, another can inspect the technical layer (authentication headers, a link's real destination), and a third knows the organization and can call to confirm an unusual request. Every match draws its own messages from a seed, so no two shifts are ever the same — which solves the "second attempt is the same exam" problem at the root, with no need to maintain a giant question bank.

And decisions carry a real cost inside the game: there are two meters, Security and Operations. Complying with a malicious message drops Security. But reporting or discarding a legitimate request also has a cost — it drops Operations. The "flag everything as suspicious" strategy loses just as much as "comply with everything without looking." That tension, which no quiz can represent, is exactly the one that exists in a real job.

How the learning gets evaluated

None of this replaces the idea of passing a level — it just changes what gets measured. Every level of the curriculum defines concrete, observable competencies — reporting a phishing email, verifying a request through another channel, spotting a double-extension attachment — and a person passes when their own decisions, made inside the game, demonstrated those competencies in at least one match. There's no score for guessing: there's evidence of a real action, logged as it happens.

That also means whatever a bot on your team does never counts as your evidence. If you play in solo mode — with bots covering the roles you're missing — passing still depends on your own decisions at the station you're playing.

What doesn't change

It's worth being honest about the limits: a 26-minute game doesn't replace a full security policy, nor does it substitute for technical tools like email filters or multi-factor authentication. What it does do is something the traditional quiz can't do well: give each person a real experience of making that call, under pressure, with costs on both sides, and a way to demonstrate — not just declare — that they can tell the difference.

If you want to see how it plays, the full Firewall Humano guide walks through the three roles, the four levels, and how each one gets passed, step by step.