Brief · 02

What works today.

The tool is reliable enough for internal monitoring and engineering triage. The list below describes its current capabilities — what we can and do depend on right now.

• Automatic scanning. Every CNC web property is visited on a recurring schedule. No human has to launch a scan.
• Cookie and storage detection. Each scan records the cookies and local browser-storage activity it observes during the visit.
• Before / after consent split. Each observation is labelled as before or after the consent click, so reports can compare the two states.
• Weekly summary. A digest of the week's findings is delivered automatically to leadership channels.
• Engineering triage signal. Obvious problems — for example, an unfamiliar third-party tracker — surface in time for engineering to investigate.

For internal use, the tool earns its keep. The remainder of this brief is about the gap between useful internally and defensible in a regulatory audit.

Brief · 03

Three critical problems.

These are the items that, in our judgement, would be raised first by a Data Protection Authority reviewing the tool's output. Each is stated in business terms; the technical detail lives in the Technical Audit.

Brief · 04

The risk landscape.

The three problems above translate into three kinds of business risk. Each is independent of the others, and each is material on its own.

Brief · 05

What we need to do.

Six high-level actions, each addressing a specific gap above. Together they take the tool from internal monitoring to defensible evidence. The technical audit lays out the order and the implementation; what follows is a leadership-level view.

1. Make the consent boundary precise.

   Record the exact moment the user consents. Tag every observation with the time it happened. Decide which side of the boundary it falls on at report time, not at collection time.

2. Keep the original evidence intact.

   Stop shortening cookie values in the database. Store the cookie name as observed, alongside any normalised form. The database becomes the system of record.

3. Record what consent was actually given.

   Capture the standardised consent record that browsers and ad partners rely on — not just "a button was clicked".

4. Extend coverage to modern tracking storage.

   Or, at minimum, disclose in our methodology what the tool does not look at, and what that excludes. Silent gaps are worse than disclosed ones.

5. Make every scan tamper-evident.

   Each scan produces a sealed, signed evidence pack with a verifiable fingerprint. We can prove, on demand, that the data has not been edited after the fact.

6. Record what happened on each scan.

   Did the page load? Was the consent banner found? Which one? Was consent accepted? The current report has rows of cookies but no run-level context. Add it.

Brief · 06

What good reporting looks like.

Engineering reads these reports to find something broken. A Data Protection Authority reads them for proof we can hand over without a footnote. The reports we produce today are fine for the first reader. They aren't built for the second.

What we produce today, and what's missing.

Each scan writes per-site CSV files to disk and inserts rows into a central database. Both are queryable when an issue surfaces. What's missing is a self-contained pack per scan that someone outside the team can open, with the data signed so anyone can verify it hasn't been changed. We also don't produce a PDF, which is the form most regulators expect a report to arrive in.

A complete report answers three questions.

1. What happened?

   Every cookie and tracking item recorded during the visit, with the original value intact and the source it came from.

2. When did it happen?

   The exact moment of each event, and which side of the consent boundary it falls on — before consent, after consent, or with no valid consent.

3. Can this be trusted?

   The data has not been edited since the scan ran. The run can be reproduced. The evidence is signed.

The evidence package.

Each scheduled scan produces one sealed ZIP. The same pack is the source of every downstream report — leadership digest, regulator submission, legal review.

• runs.csv — how the scan was configured and executed (browser version, geography, software fingerprint, timing).
• visits.csv — what happened on each site: consent state, banner detected, errors, timestamps.
• observations.csv — every tracking event observed, with raw values and source.
• screenshots/ — visual proof of each page state, including consent banner and post-consent view.
• har/ — full network log per visit, in the standard HTTP-archive format any inspector can read.
• manifest.json — list of every file in the pack with a cryptographic hash.
• manifest signature — proves the manifest itself has not been altered.

The pack is uploaded to append-only storage where it cannot be edited later. Anyone — including a regulator — can verify on their own machine that the package they received is byte-identical to the package we produced.

What this looks like in practice.

Example 1 — Two-minute leadership view

Example 2 — Regulator-facing table

Example 3 — Single-cookie evidence card

The most important artifact. The answer to a regulator's question about a specific cookie. Every field is sourced directly from one column of the evidence package — no interpretation, no derivation beyond the consent-phase label, which is itself reproducible from the timestamps below.

Example 4 — Evidence integrity

Anyone holding the package and the public key can verify that what we delivered is byte-identical to what we produced on 2026-04-27. If a single character of any file changes after the fact, verification fails. This is the integrity guarantee a regulator expects, and the one we cannot offer today.

Today versus target.

What this enables.

• We can answer regulators directly. "Here is the exact tracking before consent on this run, in machine-readable form, and here is the integrity proof."
• We can defend our position. "Here is the proof of what consent was given and exactly when, and here is the gap between the two."
• We can reproduce any past result. "Here is the configuration that produced this evidence; anyone with the same tool version can re-run it."

What happens if we do not build this.

• We can describe compliance. We cannot evidence it to a regulator's standard.
• Findings can be challenged — and we have no signed record to push back with.
• If a remediation order arrives, the first step is "re-collect everything" — months of additional work before we even start fixing.

A report is not enough. We need defensible evidence. That is the single line that separates today's monitoring from tomorrow's audit posture.

Brief · 07

Where this leaves us.

The tool tells us where we might be exposed. It does not yet give us the evidence to defend ourselves if we are. Closing that gap is roughly a quarter of focused engineering work, and it changes the conversation from "we think we are compliant" to "here is the proof".

For the engineering and legal review, continue to the Technical Audit. It contains the full evidence trail, the verified findings with severity ratings, the proposed data model, and the deployment plan.

Section 02

How the scanner works today.

Each script invocation produces one pipeline_id and walks every site listed in config/sites-config-full.json sequentially. Per site, the flow below runs end-to-end. The numbered steps are mirrored exactly in src/cookies-checker.ts.

1. 1

   Browser launch

   One Chromium instance for the whole script run. --test-third-party-cookie-phaseout is passed in every mode (standard / stealth / headed).

   src/cookies-checker.ts:80–95 · F-015

2. 2

   Per-site context, JS interceptors injected

   A new browser context (UA pinned to Chrome 126, viewport 1920×1080) is created. preload/preload-trace.js overrides document.cookie's setter and proxies window.localStorage. Two listeners are attached: context.on('console') and context.on('response').

   src/cookies-checker.ts:115–164 · S-006, S-001

3. 3

   Page load + 5 s wait

   Page is opened with waitUntil: 'domcontentloaded', then the script sleeps 5 000 ms hoping the consent banner has rendered.

   src/cookies-checker.ts:170–171 · S-004

4. 4

   Pre-consent JAR snapshot, tagged afterConsent=false

   Native context.storageState() is dumped. Every cookie and every LocalStorage entry sitting in the jar at this moment is recorded.

   src/cookies-checker.ts:173–177 · S-009

   P0 The flag is a closure variable, not a timestamp boundary. Cookies arriving in steps 2–3 may be classified inconsistently with this snapshot. F-001
5. 5

   Consent click

   Inside the page, document.querySelector('button[id*="didomi-notice-agree-button"], button[id*="cpexSubs_consentButton"]')?.click(). On success, the closure variable afterConsent flips to true.

   src/lib/uniweb-site.ts consent() · S-002

   P1 Selector matches Didomi and CPEx only. OneTrust, Cookiebot, TrustArc, Sourcepoint, iframe-hosted banners all fall through silently. F-003

   P0 If the click throws, afterConsent never flips and the run continues — the post-consent dump is then mis-labelled as pre-consent. F-002
6. 6

   10 s wait

   Hard-coded adLoad = 10_000. Late-loading SDKs that finish after this window are missed.

   src/cookies-checker.ts:23, 199 · F-004

7. 7

   Post-consent JAR snapshot, tagged afterConsent=true

   Second context.storageState() dump. The fix from Finding 16 prevents double-counting cookies that persisted from snapshot 1, but the type field of merged rows still reflects the first source seen.

   src/cookies-checker.ts:201–205 · S-009

   P2 JAR provenance is lost when collapsed onto an HDR/JS row. F-006
8. 8

   CSV write + DB insert

   CSV at csvoutput/cookiesoutput-<site>.csv writes the full cookie value. The MariaDB cookies table inserts truncate(c.value, 50) — first 50 chars + .... The cookie name written to both is the regex-normalised form, not the as-observed name.

   src/lib/EdpsCookieStore.ts:435–506 · S-012, S-013

   P0 CSV and DB diverge. Observed cookie name is never persisted. F-009 / F-010

How records are de-duplicated

The dedup logic decides what counts as "the same cookie observed twice". The four code paths use four different keys.

What ends up in the database today

One MariaDB table — cookies — carries every observation. There is no runs table, no site_visits table, and no run-level metadata is recorded against the pipeline_id.

CREATE TABLE `cookies` (
  `id`            int AUTO_INCREMENT,
  `timestamp`     datetime,
  `pipeline_id`   varchar(128),
  `site_id`       varchar(128),
  `type`          varchar(10),         -- JS / HDR / LS / JAR
  `host`          varchar(128),        -- divergent semantics across sources
  `cookie_source` text,                -- the cookie's Domain attribute
  `cookie_name`   varchar(100),        -- regex-NORMALISED, raw is lost
  `cookie_value`  text,                -- TRUNCATED to 50 chars on insert
  `source`        text,                -- the URL at observation time
  `path`          varchar(45),         -- silently truncates long paths
  `expires`       datetime,            -- includes deletion sentinels (1970)
  `http_only`     tinyint(1),
  `secure`        tinyint(1),
  `same_site`     varchar(20),
  `callstack`     text,
  `known`         boolean,
  `known_from`    varchar(10),
  `after_consent` tinyint(1)           -- can be wrong: race + failed-consent fallback
);

Source: src/tools/dbcreate.ts:20–43, src/lib/EdpsCookieStore.ts:475

Section 03

Verified flaws.

Each flaw below has been re-verified against the codebase and anchored to a specific file:line. Severity is rated by the question "would this fail under DPA scrutiny?" — not by code-quality impact. Use the filter to narrow by severity.

Section 04

How it should work.

The principle: separate observation from interpretation. Raw evidence rows are immutable. Classifications — known, before-consent, after-consent, deletion — are derived at read time and can be re-derived if rules change. This is the shape that survives any DPA challenge along the lines of "show me what the site actually did, not what your tool decided."

The corrected event flow

1. 1

   Browser launch — record reproducibility data

   At launch, write a row into runs: tool git SHA, browser version, UA, viewport, mode, egress IP, dataset SHA-256.

2. 2

   Per-site visit — open a site_visits row

   visit_id = uuid(). Record started_at immediately. Inject preload-trace.js with extended hooks: document.cookie, window.localStorage, window.sessionStorage, window.cookieStore.set.

3. 3

   Page load → wait for networkidle

   Replace the 5 s sleep with page.waitForLoadState('networkidle', { timeout: 30_000 }). Record page_loaded_at.

4. 4

   CMP probe and consent click

   Probe in order: window.__tcfapi, window.OneTrust, window.Cookiebot, window.didomiOnReady, window.Sourcepoint. Record consent_vendor. Click. On success, record consent_accepted_at (millisecond precision) and call __tcfapi('getTCData', 2, cb); persist the TC-string.

5. 5

   Snapshot the consent boundary

   From this point on, every observation row carries observed_at. The phase label is derived: observed_at < consent_accepted_at → before; otherwise → after; if consent_state ≠ 'accepted' → no_consent.

6. 6

   Late-load capture

   Wait for networkidle; take two further snapshots 10 s apart; record the count delta in site_visits.late_observation_count.

7. 7

   Persist with raw evidence intact

   Write rows to observations: full cookie_value, original cookie_name_raw, both domain_raw and domain_resolved, both observed_origin and observed_via_url, is_deletion for tombstones.

8. 8

   Build the per-run evidence pack

   Export runs.csv, visits.csv, observations.csv, the existing CSVs, screenshots, and a Playwright HAR per visit. Compute SHA-256 of every file. Sign the manifest. Ship to append-only storage.

What changes for a regulator's three core questions

Section 05

The proposed evidence database.

Three additive tables, one derived view. Old cookies table kept for one release for parallel comparison, then dropped. Below is the SQL — same shape can be expressed in Postgres or SQLite.

runs — one row per script invocation

CREATE TABLE runs (
  run_id              VARCHAR(36)   PRIMARY KEY,
  started_at          DATETIME(3)   NOT NULL,
  ended_at            DATETIME(3)   NULL,
  tool_version        VARCHAR(32)   NOT NULL,
  tool_git_sha        CHAR(40)      NOT NULL,
  browser_engine      VARCHAR(16)   NOT NULL,
  browser_version     VARCHAR(32)   NOT NULL,
  user_agent          TEXT          NOT NULL,
  viewport_w          SMALLINT      NOT NULL,
  viewport_h          SMALLINT      NOT NULL,
  mode                VARCHAR(16)   NOT NULL,        -- std / stealth / headed
  third_party_phaseout BOOL         NOT NULL,
  egress_ip           VARCHAR(45)   NULL,
  egress_country      CHAR(2)       NULL,
  dataset_version     VARCHAR(32)   NULL,
  errors_count        INT           NOT NULL DEFAULT 0,
  manifest_sha256     CHAR(64)      NULL
);

site_visits — one row per (run, site)

CREATE TABLE site_visits (
  visit_id              VARCHAR(36)  PRIMARY KEY,
  run_id                VARCHAR(36)  NOT NULL,
  site_id               VARCHAR(128) NOT NULL,
  site_url              TEXT         NOT NULL,
  final_url             TEXT         NULL,
  started_at            DATETIME(3)  NOT NULL,
  page_loaded_at        DATETIME(3)  NULL,
  consent_attempted_at  DATETIME(3)  NULL,
  consent_accepted_at   DATETIME(3)  NULL,
  consent_state         ENUM('accepted','rejected','failed_selector',
                             'no_banner','timeout','error') NOT NULL,
  consent_vendor        VARCHAR(32)  NULL,
  cmp_id                SMALLINT     NULL,
  cmp_version           SMALLINT     NULL,
  tc_string             TEXT         NULL,
  purpose_consents      JSON         NULL,
  vendor_consents       JSON         NULL,
  page_load_status      VARCHAR(16)  NOT NULL,
  error_summary         TEXT         NULL,
  cookies_observed_count INT NOT NULL DEFAULT 0,
  ls_observed_count      INT NOT NULL DEFAULT 0,
  idb_observed_count     INT NOT NULL DEFAULT 0,
  KEY ix_site_run (run_id, site_id),
  CONSTRAINT fk_visit_run FOREIGN KEY (run_id) REFERENCES runs(run_id)
);

observations — one row per storage write

CREATE TABLE observations (
  obs_id              BIGINT       AUTO_INCREMENT PRIMARY KEY,
  visit_id            VARCHAR(36)  NOT NULL,
  observed_at         DATETIME(3)  NOT NULL,
  source              ENUM('JS','HDR','LS','JAR','IDB','CS') NOT NULL,
  cookie_name_raw     VARCHAR(255) NOT NULL,        -- as observed, never rewritten
  cookie_name_norm    VARCHAR(255) NOT NULL,        -- regex-normalised
  cookie_value        MEDIUMTEXT   NULL,            -- full value, no truncation
  cookie_value_sha256 CHAR(64)     NULL,
  domain_raw          VARCHAR(255) NULL,            -- raw Domain attr (or NULL)
  domain_resolved     VARCHAR(255) NOT NULL,        -- normalised host
  path                VARCHAR(2048) NULL,
  expires             DATETIME     NULL,
  max_age             INT          NULL,
  is_session          BOOL         NOT NULL DEFAULT 0,
  is_deletion         BOOL         NOT NULL DEFAULT 0,
  http_only           BOOL         NULL,
  secure              BOOL         NULL,
  same_site           VARCHAR(20)  NULL,
  partitioned         BOOL         NULL,
  observed_origin     VARCHAR(255) NOT NULL,
  observed_via_url    TEXT         NULL,
  callstack           MEDIUMTEXT   NULL,
  is_known            BOOL         NOT NULL DEFAULT 0,
  known_from          VARCHAR(64)  NULL,
  is_leg              BOOL         NOT NULL DEFAULT 0,
  occurrence_count    INT          NOT NULL DEFAULT 1,
  KEY ix_visit (visit_id),
  KEY ix_name  (cookie_name_norm),
  CONSTRAINT fk_obs_visit FOREIGN KEY (visit_id) REFERENCES site_visits(visit_id)
);

observations_classified — derived view

CREATE OR REPLACE VIEW observations_classified AS
SELECT
  o.*,
  v.consent_state,
  v.consent_accepted_at,
  CASE
    WHEN v.consent_state <> 'accepted' THEN 'no_consent'
    WHEN o.observed_at < v.consent_accepted_at THEN 'before_consent'
    ELSE 'after_consent'
  END AS consent_phase
FROM observations o
JOIN site_visits v ON v.visit_id = o.visit_id;

Per-run evidence pack — what gets shipped to storage

run-<run_id>.zip
├── manifest.json              run metadata + SHA-256 of every file
├── manifest.json.sig          detached signature (cosign / minisign / GPG)
├── runs.csv                   one row from runs
├── visits.csv                 rows from site_visits for this run
├── observations.csv           full observations rows for this run
├── csvoutput/
│   └── cookiesoutput-<site>.csv
├── screenshots/
│   └── <site>.jpeg
├── har/
│   └── <site>.har             Playwright HAR per visit (network trail)
└── README.md                  generated; scope, methodology, software version

Section 06

Presenting evidence to controllers.

Controllers want three things, in this order: a clean summary they can read in two minutes, the ability to drill down to a single cookie row, and confidence that the evidence has not been edited after the fact.

Tamper evidence — three options ranked by DPA defensibility

Recommendation: B2 with Object Lock, governance retention 6 years — matches accountability obligations under GDPR Art. 5(2). Tag run name YYYY-MM-DD-<run_id>. Include the manifest signature in a Sigstore Rekor entry so the signature itself is also append-only.

Section 07

Roadmap.

Order moves the system to defensible state in the cheapest sequence. After step 6 the headline DPA challenges are answered; steps 7–9 close the residual scope gaps.

1. 1

   Add runs and site_visits tables

   Behind a feature flag. Stop using a single pipeline_id as the only run key. F-016

2. 2

   Drop value truncation in exportToDb

   Add cookie_value_sha256. F-009

3. 3

   Snapshot afterConsent at event entry

   Write consent_accepted_at in site_visits; derive consent phase at read time. F-001

4. 4

   Persist raw cookie name

   Stop overwriting it during regex normalisation. F-010

5. 5

   Capture TCF TC-string

   After consent click, call window.__tcfapi('getTCData', 2, cb). F-025

6. 6

   Build the per-run ZIP + manifest + SHA-256 chain

   Ship to append-only storage. F-026

7. 7

   Add CMP probe

   Broaden detection beyond Didomi/CPEx. F-003

8. 8

   IndexedDB enumeration per visit

   Or document the gap explicitly. F-014

9. 9

   Tighten the dataset loader

   Explicit delimiter + header row + assertion on minimum rows loaded. F-007

Section 08

Methodology.

Every claim on this site is anchored to a verified codebase fact. The verification record lives in docs/audit/state.json in the repository: 32 facts, each with file path, line range, and excerpt. Each finding in docs/audit/findings.json references one or more facts. The full machine-readable evidence set is shipped alongside this report.

How a future verifier can re-walk the evidence

jq '.facts[] | {id, file: .evidence[0].file, claim}' docs/audit/state.json
jq '.findings[] | {id, severity: .regulatory_severity, claim}' docs/audit/findings.json

What this audit explicitly does not cover

• Service Worker / Cache Storage coverage (subset of F-014).
• Browser fingerprinting evidence — out of cookie/storage scope; arguable for ePrivacy Art. 5(3) but treated separately.
• Independence: this is a self-audit by the controller. A DPA may discount it relative to an independent third-party audit. The artifacts here lean heavily on reproducibility and signed evidence to compensate.

Document control