Why Your Authorization Tests Pass While the Bug Ships

Most teams have tests. Few have tests that would have caught the bug that just shipped.

The gap is almost always the same one: the test suite exercises the happy path of each endpoint, with a fixture user who is allowed to do the thing. Nobody writes the test where the same request is sent with a token belonging to a different tenant.

Authorization bugs are invisible to functional tests

Consider an invoice endpoint. The functional test looks like this:

def test_get_invoice(client, user_a, invoice_a):
    resp = client.get(f"/invoices/{invoice_a.id}", headers=auth(user_a))
    assert resp.status_code == 200
    assert resp.json()["id"] == invoice_a.id

This test passes whether or not the handler checks ownership. It passes when the handler does Invoice.objects.get(id=invoice_id) and returns whatever it finds. The only thing it proves is that an authenticated user can read an invoice that belongs to them — which is not the property anyone cares about.

The property that matters is: a user cannot read an invoice that belongs to someone else. That is a different test, and it is the one that is usually missing.

def test_get_invoice_cross_tenant_denied(client, user_a, invoice_b):
    resp = client.get(f"/invoices/{invoice_b.id}", headers=auth(user_a))
    assert resp.status_code in (403, 404), resp.text

Two things about this test are worth noting. First, it asserts on the negative case, which is the only case that encodes a security requirement. Second, it uses a fixture (invoice_b) that belongs to a different principal — so the fixture setup itself has to model tenancy, not just existence.

The test is the finding, restated

If your pentest report says "IDOR on GET /invoices/{id}" and nothing else, you cannot write the test above, because you do not know which IDs, which headers, or which response shape triggered it. You know a category, not an instance.

This is the practical argument for findings that ship with the reproducing request. A finding that arrives as a raw HTTP request can be converted into a test almost mechanically:

GET /invoices/8842 HTTP/1.1
Host: app.example.com
Authorization: Bearer <token-of-user-a>

becomes a fixture, a request, and an assertion. The assertion is not "the scanner said this was bad" — it is "this response must not contain another tenant's data." That is a statement about your system, and it stays true or false independently of any tool.

Make the fixture adversarial, not convenient

The most common reason these tests do not exist is that fixtures are built for convenience. user_a owns invoice_a; the factory creates what the test needs and nothing else.

A fixture set that can express cross-tenant access needs at least two principals, each with resources, plus an unauthenticated client. Once that exists, writing the negative test is a two-line change instead of a refactor, and the test gets written.

Keep the fixture data boring and deterministic: two tenants, one invoice each, one admin. The goal is a shape that makes the wrong answer loud, not realistic data.

Put the negative tests where the regression would happen

A negative authorization test that runs only in a nightly suite is a test that tells you about the bug twelve hours late. Put it in the same suite as the handler's functional tests, so a change to the handler fails fast.

One caveat: do not assert on error messages or exact status codes across the board. 403 vs 404 is a policy decision, and both are acceptable denials. Assert on the property — the response does not contain the other tenant's identifier, or the object count did not change — not on the phrasing.

What this changes about buying a pentest

If you evaluate testing tools by "how many findings did it produce," you will get tools optimised for producing findings. If you evaluate by "how many of these can I turn into a failing test in under ten minutes," you get something different: a supply of reproducible requests that slot into the suite you already run.

At HACKBOX PTE. LTD., a Singapore-based company, we run an AI-driven automated penetration testing platform as SaaS, with flexible pricing on a results-based or subscription model. The design premise is the one above — every finding arrives with the request that produced it, ranked by what an attacker could actually reach — because a finding you cannot replay is a finding you cannot regress against.

More at https://www.hackbox.ai