RowAttest

METHODOLOGY

How RowAttest tests

RowAttest runs real attacks against a Supabase project you connect — not a linter, not a checklist. This page explains exactly what a run does, how verdicts are decided, and how you can check every finding yourself.

Connect only a staging or test project you own or control. Tests run only when you trigger them.

The tested identity

Attacks run as the identities a real attacker can actually hold: signed-in test users the engine creates, and the anonymous visitor holding only your project's public anon key. Attacks travel through your project's public API, the same stack your app uses. Attacks never run with an administrative identity or your service role. Role simulation over a private database connection serves as a second, corroborating layer, and any verdict that rests on a single layer is labeled provisional.

This matters because privileged identities are a known blind spot: a check that runs as an admin can report "no problem" about data a normal user could never reach — or worse, miss data a normal user can wrongly reach. Every RowAttest report records which identity, role, and tenant each check ran as.

What a run does

  1. Preflight — confirms the credentials work and the project is reachable. Stops at the first failure.
  2. Static introspection — reads your tables, row-level security policies, roles, and related security settings in a single read-only transaction, rolled back.
  3. Access model — reads how each table is owned (by an organization, by one user, through a parent row, or open to everyone or to all signed-in users) from your schema and policies, and shows it to you to confirm.
  4. Attacker seeding — creates temporary test users and synthetic rows that belong to them, and separate test organizations where your model has organizations.
  5. Matrix generation — builds the full list of reads and writes across users and organizations worth attempting.
  6. Execution — runs every attempt through up to two independent layers (next section) and captures what happened.
  7. Teardown — removes every test user, synthetic row, and uploaded object the run created. If a run's process or connection dies before its teardown finishes, the next run on that project removes what was left before it starts testing.

How a verdict is decided

A check only reports PASS when two independent execution paths agree:

  • The API layer sends requests through your project's public API, exactly as a browser or mobile app would.
  • The SQL layer simulates the same operation over a direct database connection, inside a transaction that is rolled back.

Running the same question through independent paths and comparing answers is a published technique called differential testing (McKeeman, 1998). If the layers disagree, the run says so instead of guessing.

When only one layer can execute a given check, its verdict is labeled provisional instead of being upgraded — the report never claims more than the engine verified.

A leak is reported only from an observed signal — a row that actually came back or a write that actually landed. Static findings — configuration defects such as a table with row-level security switched off — come from reading your database's own catalog, not from an attack.

Evidence, not opinion

Every verdict is decided from recorded observations, not opinion: what was attempted, the response status, and the row counts that came back. The signed report carries cryptographic fingerprints of that evidence. Test evidence contains synthetic rows only — the engine never includes your customers' row contents in evidence or reports.

Check any finding without paying us

Every attack finding names the exact table or storage bucket and the operation. You can hand-check any of them:

  1. Create two test users in your own project.
  2. Sign in as the first user and insert a row in the named table.
  3. Sign in as the second user and attempt the named operation on that row.
  4. Compare what happens with what the report says.

If a finding does not reproduce, we want to know — that is a bug in RowAttest, not something to hide.

Signed reports

Every purchased report is hashed (SHA-256 over the canonical report content) and signed with RowAttest's Ed25519 key. Free-run results are unsigned and say so on the page. A RowAttest attestation is only genuine if it verifies at rowattest.com/verify — drop the report PDF or JSON on that page and it checks the signature in your browser, no account needed. Change one character of the signed report content and verification fails; a PDF is verified through the signed report embedded in the file.

What a run never does

  • Never runs attacks with your service role key; it is used only to confirm it works at preflight, to create and remove the temporary test users, and to place and remove the storage test files.
  • Never modifies your data outside the synthetic test rows; direct-connection checks are rolled back; your own database triggers may react to synthetic writes, as they would to any write, and a worker in your app that reads those tables may act on the test rows.
  • Never stores or shows your customers' row contents.
  • Never runs without you triggering it.
  • Credentials are stored encrypted, never shown again, and deleted when you delete the project.

When you connect with Supabase OAuth, RowAttest stores no API keys. Keys are fetched from Supabase at run or fit-check time under the authorization you approved, used for that run or check, and discarded. RowAttest stores two secrets for such a project, both encrypted: the Supabase authorization and the database password you provide. Neither is ever shown again. Revoke RowAttest in your Supabase organization at any time: no new run can start until you reconnect.

Methodology consistent with the technical-assessment practices in NIST SP 800-115 (https://csrc.nist.gov/pubs/sp/800/115/final). Oracle strategy: differential testing (McKeeman, 1998).

← Back to RowAttest