[Examples & benchmarks](https://cdpfleet.com/docs/cases) / #37 · 2026-10-12 · Benchmark · Medium · for [geo testing](https://cdpfleet.com/geo-testing), [testing](https://cdpfleet.com/cross-browser-testing)

# Cookie banners by country: the same five sites from Germany and from the US

The Guardian, BBC, AP News, The Verge and HuffPost through a German and a US exit: which show a consent banner, which consent API they expose, what loads before anyone clicks, and what "reject" and "accept" change.

[Chromium](https://cdpfleet.com/docs/browsers/chromium)

Run on production on 2026-10-21: ✓ Node.js ✓ Python ✓ Java ✓ C# ✓ Go

A benchmark: the results below are what this setup measured on the dates shown, on our own test runs. Sites, networks and browsers change, so rerun the code before you rely on the numbers.

## The problem

Your scraper's exit country decides which version of a site it gets. From the EU most news and media sites put a consent banner over the page, run IAB Europe's TCF consent API and are supposed to hold back ad and analytics tags until you choose; from the US the same sites often show nothing and load everything. That changes what you see (an overlay hiding the content), how much bandwidth a page costs and which cookies your session carries afterwards. How big is the difference, and what do "reject" and "accept" actually do?

## What we used, and why

| What | Why |
| --- | --- |
| `PROXY_URL_DE` and `PROXY_URL_US` | One headful Chromium per country; every site is loaded in a fresh context from each. |
| A probe in `page.evaluate` | Detects the consent platform (OneTrust, Sourcepoint, Didomi, Cookiebot, Usercentrics), whether a OneTrust or Sourcepoint banner is visible, and the consent APIs the page exposes: `__tcfapi` (IAB Europe's TCF) and `__gpp` (IAB Tech Lab's GPP, used for US state privacy laws). |
| `locator.waitFor({ state: "visible" })` | Waits up to 30 s for the banner (on 2026-10-21 HuffPost's took 28 s from Germany) and records how long after `domcontentloaded` it appeared. |
| `page.on("request")` | Counts the hosts on other domains the page contacted — ad, analytics and CDN hosts — up to each moment. |
| `context.cookies()` | All cookies in the context, including third-party ones a page script can't see. |
| `frameLocator('iframe[id^="sp_message_iframe"]')` | Sourcepoint draws its banner in an iframe; OneTrust's buttons (`#onetrust-reject-all-handler`, `#onetrust-accept-btn-handler`) are in the page. |

## How it works

1.  Launch Chromium through a US proxy and load each site in a fresh context; wait for a banner (up to 30 s, at least 6 s in all) and record the platform, banner, consent APIs, cookies and other-domain hosts. Then close that session and launch one through a German proxy (an idle session ends after 2 minutes by default, so the countries run one after the other).
2.  From Germany, load each site twice in fresh contexts: record the same before any choice, then click the first-layer "reject" in one and "accept" in the other — clicking once more if the banner is still up 5 s later — and measure again 6 s after that.
3.  A free "reject" counts only if the first layer offers one — "Reject all and subscribe" isn't clicked. A page that answers with an error status is loaded once more (the rotating proxy gives the retry another IP).

## The code

The same program in five languages (also [on GitHub](https://github.com/cdpfleet/cdpfleet/tree/main/cases/37-cookie-consent-by-country), with the raw output). Set these environment variables first:

-   `CDPFLEET_API_KEY` — your API key (dashboard → API keys)
-   `PROXY_URL` — your proxy, e.g. `http://user:pass@proxy.example.com:8000`
-   `PROXY_URL_DE` — a proxy exit in Germany
-   `PROXY_URL_US` — a proxy exit in the United States

 

```node
// npm install playwright@1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL_DE (a German proxy), PROXY_URL_US (a US proxy)
import { chromium } from 'playwright';

const KEY = process.env.CDPFLEET_API_KEY;
const SITES = ['https://www.theguardian.com/international', 'https://www.bbc.com/', 'https://apnews.com/', 'https://www.theverge.com/', 'https://www.huffpost.com/'];
// First-layer buttons of the two consent platforms these sites use. OneTrust renders in the
// page; Sourcepoint in an iframe. Only a free "reject" counts: "Reject all and subscribe" doesn't.
const REJECT = { OneTrust: '#onetrust-reject-all-handler', Sourcepoint: 'button[title="Reject all"], button[title="I do not agree"]' };
const ACCEPT = { OneTrust: '#onetrust-accept-btn-handler', Sourcepoint: 'button[title="Accept all"], button[title="I agree"]' };
const SP_FRAME = 'iframe[id^="sp_message_iframe"]';
const BANNER = '#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]';

// Which consent platform the page runs, whether its banner is showing, and which consent
// APIs it exposes (__tcfapi: IAB Europe's TCF; __gpp: IAB Tech Lab's GPP, used for US state
// privacy laws).
const PROBE = `() => {
  const shown = (s) => [...document.querySelectorAll(s)].some((e) => e.getClientRects().length && getComputedStyle(e).visibility !== 'hidden');
  const cmp = window.OneTrust || document.querySelector('#onetrust-consent-sdk') ? 'OneTrust'
    : window._sp_ || document.querySelector('iframe[id^="sp_message_iframe"]') ? 'Sourcepoint'
    : window.Didomi ? 'Didomi' : window.Cookiebot ? 'Cookiebot' : window.UC_UI ? 'Usercentrics' : null;
  return { cmp, banner: shown('#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'),
    tcf: typeof window.__tcfapi === 'function', gpp: typeof window.__gpp === 'function' };
}`;

async function launch(proxy) {
  const res = await fetch('https://starter.cdpfleet.com/chromium/session', {
    method: 'POST',
    headers: { 'x-api-key': KEY, 'content-type': 'application/json' },
    body: JSON.stringify({ proxy, headless: false }),
  });
  if (!res.ok) throw new Error(`launch ${res.status} ${await res.text()}`);
  const { wsUrl } = await res.json();
  return chromium.connect(wsUrl, { headers: { 'x-api-key': KEY } });
}

const onDomain = (host, domain) => host === domain || host.endsWith(`.${domain}`);

// Clicks a consent button; true if the click went through (one retry: a consent iframe that
// re-renders can swallow a click).
async function press(button) {
  for (let attempt = 1; attempt <= 2; attempt++) {
    try { await button.click({ timeout: 10000 }); return true; } catch { /* retry once */ }
  }
  return false;
}

// Load a site in a fresh context, then optionally click a consent button. Counts cookies and
// the hosts on other domains the page contacted (ads, analytics, CDNs) up to each moment.
async function visit(browser, url, choice) {
  const domain = new URL(url).hostname.split('.').slice(-2).join('.');
  const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } });
  const page = await ctx.newPage();
  const hosts = new Set();
  page.on('request', (r) => { const h = new URL(r.url()).hostname; if (h && !onDomain(h, domain)) hosts.add(h); });
  const measure = async () => {
    const cookies = await ctx.cookies();
    return { cookies: cookies.length, third_party_cookies: cookies.filter((c) => !onDomain(c.domain.replace(/^\./, ''), domain)).length, other_hosts: hosts.size };
  };
  try {
    // Proxies drop a connection now and then: one retry.
    const open = () => page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
    const res = await open().catch(open);
    // Consent scripts load late on busy pages: wait up to 30 s for the banner, and give
    // every page at least 6 s before measuring.
    const t0 = Date.now();
    const seen = await page.locator(BANNER).first().waitFor({ state: 'visible', timeout: 30000 }).then(() => true, () => false);
    const bannerAfterMs = seen ? Date.now() - t0 : null;
    if (Date.now() - t0 < 6000) await page.waitForTimeout(6000 - (Date.now() - t0));
    const probe = await page.evaluate(`(${PROBE})()`);
    const before = { status: res?.status() ?? null, ...probe, banner_after_ms: bannerAfterMs, ...(await measure()) };
    if (!choice) return { before };
    const selector = (choice === 'reject' ? REJECT : ACCEPT)[probe.cmp];
    const scope = probe.cmp === 'Sourcepoint' ? page.frameLocator(SP_FRAME) : page;
    const button = selector && scope.locator(selector).first();
    if (!button || !(await button.waitFor({ state: 'visible', timeout: 10000 }).then(() => true, () => false))) {
      return { before, after: null, missing: probe.banner ? `no ${choice} button on the first layer` : 'no banner' };
    }
    const label = (await button.getAttribute('title')) || (await button.innerText()).trim();
    await page.waitForTimeout(2000); // let the banner finish animating in, or the click can be lost
    if (!(await press(button))) return { before, after: null, missing: 'click failed' };
    // If the banner is still up, the click was lost: click once more.
    let clicks = 1;
    if (!(await page.locator(BANNER).first().waitFor({ state: 'hidden', timeout: 5000 }).then(() => true, () => false))) {
      clicks = 2;
      await press(button);
    }
    await page.waitForTimeout(6000);
    return { before, after: { button: label, clicks, ...(await measure()) } };
  } finally {
    await ctx.close();
  }
}

// A site may refuse one exit IP (an error status): visit again in a fresh context, whose new
// proxy connection gets another IP from the rotating proxy.
async function visitOnce(browser, url, choice) {
  const r = await visit(browser, url, choice);
  return r.before.status >= 200 && r.before.status < 300 ? r : visit(browser, url, choice);
}

// Sessions on shared threads end after 2 minutes without traffic by default, so the countries run one after the other.
const fromUs = [];
const us = await launch(process.env.PROXY_URL_US);
try {
  for (const url of SITES) fromUs.push(await visitOnce(us, url));
} finally {
  await us.close();
}
const sites = [];
const de = await launch(process.env.PROXY_URL_DE);
try {
  for (const [i, url] of SITES.entries()) {
    const rejected = await visitOnce(de, url, 'reject');
    const accepted = await visitOnce(de, url, 'accept');
    sites.push({
      site: new URL(url).hostname.replace(/^www\./, ''), us: fromUs[i].before, de: rejected.before,
      de_reject: rejected.after, de_reject_missing: rejected.missing ?? null, de_accept: accepted.after, de_accept_missing: accepted.missing ?? null,
    });
  }
} finally {
  await de.close();
}
console.log(JSON.stringify({ sites }, null, 2));
```

```python
# pip install playwright==1.60.0 requests
# env: CDPFLEET_API_KEY, PROXY_URL_DE (a German proxy), PROXY_URL_US (a US proxy)
import json
import os
import time
from urllib.parse import urlparse

import requests
from playwright.sync_api import sync_playwright

KEY = os.environ["CDPFLEET_API_KEY"]
SITES = ["https://www.theguardian.com/international", "https://www.bbc.com/", "https://apnews.com/", "https://www.theverge.com/", "https://www.huffpost.com/"]
# First-layer buttons of the two consent platforms these sites use. OneTrust renders in the
# page; Sourcepoint in an iframe. Only a free "reject" counts: "Reject all and subscribe" doesn't.
REJECT = {"OneTrust": "#onetrust-reject-all-handler", "Sourcepoint": 'button[title="Reject all"], button[title="I do not agree"]'}
ACCEPT = {"OneTrust": "#onetrust-accept-btn-handler", "Sourcepoint": 'button[title="Accept all"], button[title="I agree"]'}
SP_FRAME = 'iframe[id^="sp_message_iframe"]'
BANNER = '#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'

# Which consent platform the page runs, whether its banner is showing, and which consent
# APIs it exposes (__tcfapi: IAB Europe's TCF; __gpp: IAB Tech Lab's GPP, used for US state
# privacy laws).
PROBE = """() => {
  const shown = (s) => [...document.querySelectorAll(s)].some((e) => e.getClientRects().length && getComputedStyle(e).visibility !== 'hidden');
  const cmp = window.OneTrust || document.querySelector('#onetrust-consent-sdk') ? 'OneTrust'
    : window._sp_ || document.querySelector('iframe[id^="sp_message_iframe"]') ? 'Sourcepoint'
    : window.Didomi ? 'Didomi' : window.Cookiebot ? 'Cookiebot' : window.UC_UI ? 'Usercentrics' : null;
  return { cmp, banner: shown('#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'),
    tcf: typeof window.__tcfapi === 'function', gpp: typeof window.__gpp === 'function' };
}"""

def launch(p, proxy):
    res = requests.post("https://starter.cdpfleet.com/chromium/session",
        headers={"x-api-key": KEY}, json={"proxy": proxy, "headless": False}, timeout=60)
    if not res.ok:
        raise RuntimeError(f"launch {res.status_code} {res.text}")
    return p.chromium.connect(res.json()["wsUrl"], headers={"x-api-key": KEY})

def on_domain(host, domain):
    return host == domain or host.endswith("." + domain)

def press(button):
    """Clicks a consent button; True if the click went through (one retry: a consent iframe that
    re-renders can swallow a click)."""
    for _ in range(2):
        try:
            button.click(timeout=10000)
            return True
        except Exception:
            pass
    return False

def visit(browser, url, choice=None):
    """Load a site in a fresh context, then optionally click a consent button. Counts cookies and
    the hosts on other domains the page contacted (ads, analytics, CDNs) up to each moment."""
    domain = ".".join(urlparse(url).hostname.split(".")[-2:])
    ctx = browser.new_context(viewport={"width": 1280, "height": 900})
    page = ctx.new_page()
    hosts = set()

    def on_request(r):
        h = urlparse(r.url).hostname or ""
        if h and not on_domain(h, domain):
            hosts.add(h)
    page.on("request", on_request)

    def measure():
        cookies = ctx.cookies()
        return {"cookies": len(cookies), "third_party_cookies": len([c for c in cookies if not on_domain(c["domain"].lstrip("."), domain)]), "other_hosts": len(hosts)}

    def open_page():
        return page.goto(url, wait_until="domcontentloaded", timeout=60000)

    try:
        try:
            res = open_page()
        except Exception:  # proxies drop a connection now and then: one retry
            res = open_page()
        # Consent scripts load late on busy pages: wait up to 30 s for the banner, and give
        # every page at least 6 s before measuring.
        t0 = time.monotonic()
        try:
            page.locator(BANNER).first.wait_for(state="visible", timeout=30000)
            banner_after_ms = round((time.monotonic() - t0) * 1000)
        except Exception:
            banner_after_ms = None
        elapsed = (time.monotonic() - t0) * 1000
        if elapsed < 6000:
            page.wait_for_timeout(6000 - elapsed)
        probe = page.evaluate(f"({PROBE})()")
        before = {"status": res.status if res else None, **probe, "banner_after_ms": banner_after_ms, **measure()}
        if not choice:
            return {"before": before}
        missing = f"no {choice} button on the first layer" if probe["banner"] else "no banner"
        selector = (REJECT if choice == "reject" else ACCEPT).get(probe["cmp"])
        if not selector:
            return {"before": before, "after": None, "missing": missing}
        scope = page.frame_locator(SP_FRAME) if probe["cmp"] == "Sourcepoint" else page
        button = scope.locator(selector).first
        try:
            button.wait_for(state="visible", timeout=10000)
        except Exception:
            return {"before": before, "after": None, "missing": missing}
        label = button.get_attribute("title") or button.inner_text().strip()
        page.wait_for_timeout(2000)  # let the banner finish animating in, or the click can be lost
        if not press(button):
            return {"before": before, "after": None, "missing": "click failed"}
        clicks = 1
        try:  # if the banner is still up, the click was lost: click once more
            page.locator(BANNER).first.wait_for(state="hidden", timeout=5000)
        except Exception:
            clicks = 2
            press(button)
        page.wait_for_timeout(6000)
        return {"before": before, "after": {"button": label, "clicks": clicks, **measure()}}
    finally:
        ctx.close()

def visit_once(browser, url, choice=None):
    """A site may refuse one exit IP (an error status): visit again in a fresh context, whose new
    proxy connection gets another IP from the rotating proxy."""
    r = visit(browser, url, choice)
    status = r["before"]["status"]
    return r if status and 200 <= status < 300 else visit(browser, url, choice)

with sync_playwright() as p:
    # Sessions on shared threads end after 2 minutes without traffic by default, so the countries run one after the other.
    us = launch(p, os.environ["PROXY_URL_US"])
    try:
        from_us = [visit_once(us, url) for url in SITES]
    finally:
        us.close()
    sites = []
    de = launch(p, os.environ["PROXY_URL_DE"])
    try:
        for i, url in enumerate(SITES):
            rejected = visit_once(de, url, "reject")
            accepted = visit_once(de, url, "accept")
            sites.append({"site": urlparse(url).hostname.removeprefix("www."), "us": from_us[i]["before"], "de": rejected["before"],
                          "de_reject": rejected["after"], "de_reject_missing": rejected.get("missing"),
                          "de_accept": accepted["after"], "de_accept_missing": accepted.get("missing")})
    finally:
        de.close()
    print(json.dumps({"sites": sites}, indent=2))
```

```java
// Maven: com.microsoft.playwright:playwright:1.60.0, com.google.code.gson:gson:2.11.0
// Run with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1. env: CDPFLEET_API_KEY, PROXY_URL_DE (a German proxy), PROXY_URL_US (a US proxy)
import com.google.gson.*;
import com.microsoft.playwright.*;
import com.microsoft.playwright.options.WaitForSelectorState;
import com.microsoft.playwright.options.WaitUntilState;
import java.net.URI;
import java.net.http.*;
import java.util.*;

public class Main {
  static final String KEY = System.getenv("CDPFLEET_API_KEY");
  static final Gson GSON = new GsonBuilder().serializeNulls().create();
  static final List<String> SITES = List.of("https://www.theguardian.com/international", "https://www.bbc.com/", "https://apnews.com/", "https://www.theverge.com/", "https://www.huffpost.com/");
  // First-layer buttons of the two consent platforms these sites use. OneTrust renders in the
  // page; Sourcepoint in an iframe. Only a free "reject" counts: "Reject all and subscribe" doesn't.
  static final Map<String, String> REJECT = Map.of("OneTrust", "#onetrust-reject-all-handler", "Sourcepoint", "button[title=\"Reject all\"], button[title=\"I do not agree\"]");
  static final Map<String, String> ACCEPT = Map.of("OneTrust", "#onetrust-accept-btn-handler", "Sourcepoint", "button[title=\"Accept all\"], button[title=\"I agree\"]");
  static final String SP_FRAME = "iframe[id^=\"sp_message_iframe\"]";
  static final String BANNER = "#onetrust-banner-sdk, iframe[id^=\"sp_message_iframe\"]";

  // Which consent platform the page runs, whether its banner is showing, and which consent
  // APIs it exposes (__tcfapi: IAB Europe's TCF; __gpp: IAB Tech Lab's GPP, used for US state
  // privacy laws).
  static final String PROBE = """
      () => {
        const shown = (s) => [...document.querySelectorAll(s)].some((e) => e.getClientRects().length && getComputedStyle(e).visibility !== 'hidden');
        const cmp = window.OneTrust || document.querySelector('#onetrust-consent-sdk') ? 'OneTrust'
          : window._sp_ || document.querySelector('iframe[id^="sp_message_iframe"]') ? 'Sourcepoint'
          : window.Didomi ? 'Didomi' : window.Cookiebot ? 'Cookiebot' : window.UC_UI ? 'Usercentrics' : null;
        return { cmp, banner: shown('#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'),
          tcf: typeof window.__tcfapi === 'function', gpp: typeof window.__gpp === 'function' };
      }""";

  static Browser launch(Playwright playwright, String proxy) throws Exception {
    String body = "{\"proxy\": " + GSON.toJson(proxy) + ", \"headless\": false}";
    HttpResponse<String> res = HttpClient.newHttpClient().send(HttpRequest.newBuilder(URI.create("https://starter.cdpfleet.com/chromium/session"))
        .header("x-api-key", KEY).header("content-type", "application/json")
        .POST(HttpRequest.BodyPublishers.ofString(body)).build(), HttpResponse.BodyHandlers.ofString());
    if (res.statusCode() / 100 != 2) throw new RuntimeException("launch: " + res.statusCode() + " " + res.body());
    String wsUrl = JsonParser.parseString(res.body()).getAsJsonObject().get("wsUrl").getAsString();
    return playwright.chromium().connect(wsUrl, new BrowserType.ConnectOptions().setHeaders(Map.of("x-api-key", KEY)));
  }

  // Request URLs aren't always valid java.net.URI input (unescaped {} or "), so cut the host out.
  static String hostOf(String url) {
    return url.replaceFirst("^[a-zA-Z][a-zA-Z0-9+.-]*://", "").split("[/?#:]", 2)[0];
  }

  static String domainOf(String url) {
    String[] parts = hostOf(url).split("\\.");
    return parts[parts.length - 2] + "." + parts[parts.length - 1];
  }

  static boolean onDomain(String host, String domain) {
    return host.equals(domain) || host.endsWith("." + domain);
  }

  // Clicks a consent button; true if the click went through (one retry: a consent iframe that
  // re-renders can swallow a click).
  static boolean press(Locator button) {
    for (int attempt = 1; attempt <= 2; attempt++) {
      try {
        button.click(new Locator.ClickOptions().setTimeout(10000));
        return true;
      } catch (PlaywrightException e) {
        // retry once
      }
    }
    return false;
  }

  static JsonObject measure(BrowserContext ctx, Set<String> hosts, String domain) {
    List<com.microsoft.playwright.options.Cookie> cookies = ctx.cookies();
    JsonObject m = new JsonObject();
    m.addProperty("cookies", cookies.size());
    m.addProperty("third_party_cookies", cookies.stream().filter(c -> !onDomain(c.domain.replaceFirst("^\\.", ""), domain)).count());
    m.addProperty("other_hosts", hosts.size());
    return m;
  }

  // Load a site in a fresh context, then optionally click a consent button. Counts cookies and
  // the hosts on other domains the page contacted (ads, analytics, CDNs) up to each moment.
  // Returns [before, after, missing]; after is null (and missing says why) when there was
  // nothing to click.
  static JsonElement[] visit(Browser browser, String url, String choice) {
    String domain = domainOf(url);
    BrowserContext ctx = browser.newContext(new Browser.NewContextOptions().setViewportSize(1280, 900));
    Page page = ctx.newPage();
    Set<String> hosts = new HashSet<>();
    page.onRequest(r -> {
      String h = r.url().startsWith("http") ? hostOf(r.url()) : "";
      if (!h.isEmpty() && !onDomain(h, domain)) hosts.add(h);
    });
    try {
      Page.NavigateOptions load = new Page.NavigateOptions().setWaitUntil(WaitUntilState.DOMCONTENTLOADED).setTimeout(60000);
      Response res;
      try {
        res = page.navigate(url, load);
      } catch (PlaywrightException e) { // proxies drop a connection now and then: one retry
        res = page.navigate(url, load);
      }
      // Consent scripts load late on busy pages: wait up to 30 s for the banner, and give
      // every page at least 6 s before measuring.
      long t0 = System.nanoTime();
      Long bannerAfterMs = null;
      try {
        page.locator(BANNER).first().waitFor(new Locator.WaitForOptions().setState(WaitForSelectorState.VISIBLE).setTimeout(30000));
        bannerAfterMs = (System.nanoTime() - t0) / 1_000_000;
      } catch (PlaywrightException e) {
        // no banner
      }
      long elapsed = (System.nanoTime() - t0) / 1_000_000;
      if (elapsed < 6000) page.waitForTimeout(6000 - elapsed);
      JsonObject probe = GSON.toJsonTree(page.evaluate("(" + PROBE + ")()")).getAsJsonObject();
      JsonObject before = new JsonObject();
      if (res == null) before.add("status", JsonNull.INSTANCE); else before.addProperty("status", res.status());
      for (Map.Entry<String, JsonElement> e : probe.entrySet()) before.add(e.getKey(), e.getValue());
      before.addProperty("banner_after_ms", bannerAfterMs);
      for (Map.Entry<String, JsonElement> e : measure(ctx, hosts, domain).entrySet()) before.add(e.getKey(), e.getValue());
      if (choice == null) return new JsonElement[] { before, JsonNull.INSTANCE, JsonNull.INSTANCE };
      JsonPrimitive missing = new JsonPrimitive(probe.get("banner").getAsBoolean() ? "no " + choice + " button on the first layer" : "no banner");
      String cmp = probe.get("cmp").isJsonNull() ? null : probe.get("cmp").getAsString();
      String selector = cmp == null ? null : (choice.equals("reject") ? REJECT : ACCEPT).get(cmp);
      if (selector == null) return new JsonElement[] { before, JsonNull.INSTANCE, missing };
      Locator button = "Sourcepoint".equals(cmp) ? page.frameLocator(SP_FRAME).locator(selector).first() : page.locator(selector).first();
      try {
        button.waitFor(new Locator.WaitForOptions().setState(WaitForSelectorState.VISIBLE).setTimeout(10000));
      } catch (PlaywrightException e) {
        return new JsonElement[] { before, JsonNull.INSTANCE, missing };
      }
      String title = button.getAttribute("title");
      String label = title != null ? title : button.innerText().trim();
      page.waitForTimeout(2000); // let the banner finish animating in, or the click can be lost
      if (!press(button)) return new JsonElement[] { before, JsonNull.INSTANCE, new JsonPrimitive("click failed") };
      int clicks = 1;
      try { // if the banner is still up, the click was lost: click once more
        page.locator(BANNER).first().waitFor(new Locator.WaitForOptions().setState(WaitForSelectorState.HIDDEN).setTimeout(5000));
      } catch (PlaywrightException e) {
        clicks = 2;
        press(button);
      }
      page.waitForTimeout(6000);
      JsonObject after = new JsonObject();
      after.addProperty("button", label);
      after.addProperty("clicks", clicks);
      for (Map.Entry<String, JsonElement> e : measure(ctx, hosts, domain).entrySet()) after.add(e.getKey(), e.getValue());
      return new JsonElement[] { before, after, JsonNull.INSTANCE };
    } finally {
      ctx.close();
    }
  }

  // A site may refuse one exit IP (an error status): visit again in a fresh context, whose new
  // proxy connection gets another IP from the rotating proxy.
  static JsonElement[] visitOnce(Browser browser, String url, String choice) {
    JsonElement[] r = visit(browser, url, choice);
    JsonElement status = r[0].getAsJsonObject().get("status");
    return !status.isJsonNull() && status.getAsInt() / 100 == 2 ? r : visit(browser, url, choice);
  }

  public static void main(String[] args) throws Exception {
    try (Playwright playwright = Playwright.create()) {
      // Sessions on shared threads end after 2 minutes without traffic by default, so the countries run one after the other.
      List<JsonElement[]> fromUs = new ArrayList<>();
      Browser us = launch(playwright, System.getenv("PROXY_URL_US"));
      try {
        for (String url : SITES) fromUs.add(visitOnce(us, url, null));
      } finally {
        us.close();
      }
      Browser de = launch(playwright, System.getenv("PROXY_URL_DE"));
      try {
        JsonArray sites = new JsonArray();
        for (int i = 0; i < SITES.size(); i++) {
          String url = SITES.get(i);
          JsonElement[] rejected = visitOnce(de, url, "reject");
          JsonElement[] accepted = visitOnce(de, url, "accept");
          JsonObject site = new JsonObject();
          site.addProperty("site", hostOf(url).replaceFirst("^www\\.", ""));
          site.add("us", fromUs.get(i)[0]);
          site.add("de", rejected[0]);
          site.add("de_reject", rejected[1]);
          site.add("de_reject_missing", rejected[2]);
          site.add("de_accept", accepted[1]);
          site.add("de_accept_missing", accepted[2]);
          sites.add(site);
        }
        JsonObject out = new JsonObject();
        out.add("sites", sites);
        System.out.println(new GsonBuilder().serializeNulls().setPrettyPrinting().disableHtmlEscaping().create().toJson(out));
      } finally {
        de.close();
      }
    }
  }
}
```

```csharp
// dotnet add package Microsoft.Playwright --version 1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL_DE (a German proxy), PROXY_URL_US (a US proxy)
using System.Net.Http.Json;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Playwright;

string[] sites = ["https://www.theguardian.com/international", "https://www.bbc.com/", "https://apnews.com/", "https://www.theverge.com/", "https://www.huffpost.com/"];
// First-layer buttons of the two consent platforms these sites use. OneTrust renders in the
// page; Sourcepoint in an iframe. Only a free "reject" counts: "Reject all and subscribe" doesn't.
var reject = new Dictionary<string, string> { ["OneTrust"] = "#onetrust-reject-all-handler", ["Sourcepoint"] = "button[title=\"Reject all\"], button[title=\"I do not agree\"]" };
var accept = new Dictionary<string, string> { ["OneTrust"] = "#onetrust-accept-btn-handler", ["Sourcepoint"] = "button[title=\"Accept all\"], button[title=\"I agree\"]" };
const string SpFrame = "iframe[id^=\"sp_message_iframe\"]";
const string Banner = "#onetrust-banner-sdk, iframe[id^=\"sp_message_iframe\"]";

// Which consent platform the page runs, whether its banner is showing, and which consent
// APIs it exposes (__tcfapi: IAB Europe's TCF; __gpp: IAB Tech Lab's GPP, used for US state
// privacy laws).
const string Probe = """
    () => {
      const shown = (s) => [...document.querySelectorAll(s)].some((e) => e.getClientRects().length && getComputedStyle(e).visibility !== 'hidden');
      const cmp = window.OneTrust || document.querySelector('#onetrust-consent-sdk') ? 'OneTrust'
        : window._sp_ || document.querySelector('iframe[id^="sp_message_iframe"]') ? 'Sourcepoint'
        : window.Didomi ? 'Didomi' : window.Cookiebot ? 'Cookiebot' : window.UC_UI ? 'Usercentrics' : null;
      return { cmp, banner: shown('#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'),
        tcf: typeof window.__tcfapi === 'function', gpp: typeof window.__gpp === 'function' };
    }
    """;

var key = Environment.GetEnvironmentVariable("CDPFLEET_API_KEY")!;
using var http = new HttpClient();
http.DefaultRequestHeaders.Add("x-api-key", key);
using var playwright = await Playwright.CreateAsync();

async Task<IBrowser> Launch(string proxy)
{
    var res = await http.PostAsJsonAsync("https://starter.cdpfleet.com/chromium/session", new { proxy, headless = false });
    if (!res.IsSuccessStatusCode) throw new Exception($"launch: {(int)res.StatusCode} {await res.Content.ReadAsStringAsync()}");
    var wsUrl = (await res.Content.ReadFromJsonAsync<JsonElement>()).GetProperty("wsUrl").GetString()!;
    return await playwright.Chromium.ConnectAsync(wsUrl, new() { Headers = new Dictionary<string, string> { ["x-api-key"] = key } });
}

static bool OnDomain(string host, string domain) => host == domain || host.EndsWith("." + domain, StringComparison.Ordinal);

// Clicks a consent button; true if the click went through (one retry: a consent iframe that
// re-renders can swallow a click).
static async Task<bool> Press(ILocator button)
{
    for (var attempt = 1; attempt <= 2; attempt++)
    {
        try { await button.ClickAsync(new() { Timeout = 10000 }); return true; }
        catch (Exception) { /* retry once */ }
    }
    return false;
}

// Load a site in a fresh context, then optionally click a consent button. Counts cookies and
// the hosts on other domains the page contacted (ads, analytics, CDNs) up to each moment.
// After is null (and Missing says why) when there was nothing to click.
async Task<(JsonObject Before, JsonObject? After, string? Missing)> Visit(IBrowser browser, string url, string? choice)
{
    var domain = string.Join('.', new Uri(url).Host.Split('.')[^2..]);
    var ctx = await browser.NewContextAsync(new() { ViewportSize = new() { Width = 1280, Height = 900 } });
    var page = await ctx.NewPageAsync();
    var hosts = new HashSet<string>();
    page.Request += (_, r) =>
    {
        var h = Uri.TryCreate(r.Url, UriKind.Absolute, out var u) ? u.Host : "";
        if (h != "" && !OnDomain(h, domain)) lock (hosts) hosts.Add(h);
    };
    async Task<JsonObject> Measure()
    {
        var cookies = await ctx.CookiesAsync();
        int count;
        lock (hosts) count = hosts.Count;
        return new JsonObject
        {
            ["cookies"] = cookies.Count,
            ["third_party_cookies"] = cookies.Count(c => !OnDomain(c.Domain.TrimStart('.'), domain)),
            ["other_hosts"] = count,
        };
    }
    var load = new PageGotoOptions { WaitUntil = WaitUntilState.DOMContentLoaded, Timeout = 60000 };
    try
    {
        IResponse? res;
        try { res = await page.GotoAsync(url, load); }
        catch (PlaywrightException) { res = await page.GotoAsync(url, load); } // proxies drop a connection now and then: one retry
        // Consent scripts load late on busy pages: wait up to 30 s for the banner, and give
        // every page at least 6 s before measuring.
        var clock = System.Diagnostics.Stopwatch.StartNew();
        long? bannerAfterMs = null;
        try
        {
            await page.Locator(Banner).First.WaitForAsync(new() { State = WaitForSelectorState.Visible, Timeout = 30000 });
            bannerAfterMs = clock.ElapsedMilliseconds;
        }
        catch (Exception) { /* no banner */ }
        if (clock.ElapsedMilliseconds < 6000) await page.WaitForTimeoutAsync(6000 - clock.ElapsedMilliseconds);
        var probe = (JsonObject)Plain(await page.EvaluateAsync<JsonElement>($"({Probe})()"))!;
        var before = new JsonObject { ["status"] = res?.Status };
        foreach (var prop in probe.ToList()) before[prop.Key] = prop.Value?.DeepClone();
        before["banner_after_ms"] = bannerAfterMs;
        foreach (var prop in (await Measure()).ToList()) before[prop.Key] = prop.Value?.DeepClone();
        if (choice is null) return (before, null, null);
        var missing = probe["banner"]!.GetValue<bool>() ? $"no {choice} button on the first layer" : "no banner";
        var cmp = probe["cmp"]?.GetValue<string>();
        if (cmp is null || !(choice == "reject" ? reject : accept).TryGetValue(cmp, out var selector)) return (before, null, missing);
        var button = cmp == "Sourcepoint" ? page.FrameLocator(SpFrame).Locator(selector).First : page.Locator(selector).First;
        try { await button.WaitForAsync(new() { State = WaitForSelectorState.Visible, Timeout = 10000 }); }
        catch (Exception) { return (before, null, missing); }
        var label = await button.GetAttributeAsync("title") ?? (await button.InnerTextAsync()).Trim();
        await page.WaitForTimeoutAsync(2000); // let the banner finish animating in, or the click can be lost
        if (!await Press(button)) return (before, null, "click failed");
        // If the banner is still up, the click was lost: click once more.
        var clicks = 1;
        try { await page.Locator(Banner).First.WaitForAsync(new() { State = WaitForSelectorState.Hidden, Timeout = 5000 }); }
        catch (Exception)
        {
            clicks = 2;
            await Press(button);
        }
        await page.WaitForTimeoutAsync(6000);
        var after = new JsonObject { ["button"] = label, ["clicks"] = clicks };
        foreach (var prop in (await Measure()).ToList()) after[prop.Key] = prop.Value?.DeepClone();
        return (before, after, null);
    }
    finally
    {
        await ctx.CloseAsync();
    }
}

// A site may refuse one exit IP (an error status): visit again in a fresh context, whose new
// proxy connection gets another IP from the rotating proxy.
async Task<(JsonObject Before, JsonObject? After, string? Missing)> VisitOnce(IBrowser browser, string url, string? choice)
{
    var r = await Visit(browser, url, choice);
    var status = r.Before["status"]?.GetValue<int>();
    return status is >= 200 and < 300 ? r : await Visit(browser, url, choice);
}

// Sessions on shared threads end after 2 minutes without traffic by default, so the countries run one after the other.
var fromUs = new List<(JsonObject Before, JsonObject? After, string? Missing)>();
var us = await Launch(Environment.GetEnvironmentVariable("PROXY_URL_US")!);
try
{
    foreach (var url in sites) fromUs.Add(await VisitOnce(us, url, null));
}
finally
{
    await us.CloseAsync();
}
var de = await Launch(Environment.GetEnvironmentVariable("PROXY_URL_DE")!);
try
{
    var results = new JsonArray();
    for (var i = 0; i < sites.Length; i++)
    {
        var url = sites[i];
        var rejected = await VisitOnce(de, url, "reject");
        var accepted = await VisitOnce(de, url, "accept");
        var host = new Uri(url).Host;
        results.Add(new JsonObject
        {
            ["site"] = host.StartsWith("www.") ? host[4..] : host,
            ["us"] = fromUs[i].Before,
            ["de"] = rejected.Before,
            ["de_reject"] = rejected.After,
            ["de_reject_missing"] = rejected.Missing,
            ["de_accept"] = accepted.After,
            ["de_accept_missing"] = accepted.Missing,
        });
    }
    var result = new JsonObject { ["sites"] = results };
    Console.WriteLine(result.ToJsonString(new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
}
finally
{
    await de.CloseAsync();
}

// EvaluateAsync<JsonElement> tags every object with a "$id" reference key; drop those.
static JsonNode? Plain(JsonElement e) => e.ValueKind switch
{
    JsonValueKind.Object => new JsonObject(e.EnumerateObject().Where(p => p.Name != "$id")
        .Select(p => KeyValuePair.Create(p.Name, Plain(p.Value)))),
    JsonValueKind.Array => new JsonArray(e.EnumerateArray().Select(Plain).ToArray()),
    _ => JsonNode.Parse(e.GetRawText()),
};
```

```go
// go get github.com/playwright-community/playwright-go@v0.6000.0
// Driver: build playwright-core 1.60.0 from npm and set PLAYWRIGHT_DRIVER_PATH (see /docs/quickstart).
// env: CDPFLEET_API_KEY, PROXY_URL_DE (a German proxy), PROXY_URL_US (a US proxy)
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
	"net/url"
	"os"
	"strings"
	"sync"
	"time"

	"github.com/playwright-community/playwright-go"
)

var key = os.Getenv("CDPFLEET_API_KEY")

var sites = []string{"https://www.theguardian.com/international", "https://www.bbc.com/", "https://apnews.com/", "https://www.theverge.com/", "https://www.huffpost.com/"}

// First-layer buttons of the two consent platforms these sites use. OneTrust renders in the
// page; Sourcepoint in an iframe. Only a free "reject" counts: "Reject all and subscribe" doesn't.
var reject = map[string]string{"OneTrust": "#onetrust-reject-all-handler", "Sourcepoint": `button[title="Reject all"], button[title="I do not agree"]`}
var accept = map[string]string{"OneTrust": "#onetrust-accept-btn-handler", "Sourcepoint": `button[title="Accept all"], button[title="I agree"]`}

const spFrame = `iframe[id^="sp_message_iframe"]`
const banner = `#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]`

// Which consent platform the page runs, whether its banner is showing, and which consent
// APIs it exposes (__tcfapi: IAB Europe's TCF; __gpp: IAB Tech Lab's GPP, used for US state
// privacy laws).
const probe = `() => {
  const shown = (s) => [...document.querySelectorAll(s)].some((e) => e.getClientRects().length && getComputedStyle(e).visibility !== 'hidden');
  const cmp = window.OneTrust || document.querySelector('#onetrust-consent-sdk') ? 'OneTrust'
    : window._sp_ || document.querySelector('iframe[id^="sp_message_iframe"]') ? 'Sourcepoint'
    : window.Didomi ? 'Didomi' : window.Cookiebot ? 'Cookiebot' : window.UC_UI ? 'Usercentrics' : null;
  return { cmp, banner: shown('#onetrust-banner-sdk, iframe[id^="sp_message_iframe"]'),
    tcf: typeof window.__tcfapi === 'function', gpp: typeof window.__gpp === 'function' };
}`

type before struct {
	Status            *int   `json:"status"`
	CMP               any    `json:"cmp"`
	Banner            bool   `json:"banner"`
	TCF               bool   `json:"tcf"`
	GPP               bool   `json:"gpp"`
	BannerAfterMs     *int64 `json:"banner_after_ms"`
	Cookies           int    `json:"cookies"`
	ThirdPartyCookies int    `json:"third_party_cookies"`
	OtherHosts        int    `json:"other_hosts"`
}

type after struct {
	Button            string `json:"button"`
	Clicks            int    `json:"clicks"`
	Cookies           int    `json:"cookies"`
	ThirdPartyCookies int    `json:"third_party_cookies"`
	OtherHosts        int    `json:"other_hosts"`
}

type siteResult struct {
	Site            string  `json:"site"`
	US              *before `json:"us"`
	DE              *before `json:"de"`
	DEReject        *after  `json:"de_reject"`
	DERejectMissing *string `json:"de_reject_missing"`
	DEAccept        *after  `json:"de_accept"`
	DEAcceptMissing *string `json:"de_accept_missing"`
}

func launch(proxy string) (string, error) {
	body, _ := json.Marshal(map[string]any{"proxy": proxy, "headless": false})
	req, _ := http.NewRequest("POST", "https://starter.cdpfleet.com/chromium/session", bytes.NewReader(body))
	req.Header.Set("x-api-key", key)
	req.Header.Set("content-type", "application/json")
	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return "", err
	}
	defer res.Body.Close()
	if res.StatusCode/100 != 2 {
		msg, _ := io.ReadAll(res.Body)
		return "", fmt.Errorf("launch: %s %s", res.Status, msg)
	}
	var session struct {
		WsURL string `json:"wsUrl"`
	}
	return session.WsURL, json.NewDecoder(res.Body).Decode(&session)
}

func domainOf(u string) string {
	parsed, _ := url.Parse(u)
	parts := strings.Split(parsed.Hostname(), ".")
	return strings.Join(parts[len(parts)-2:], ".")
}

func onDomain(host, domain string) bool {
	return host == domain || strings.HasSuffix(host, "."+domain)
}

// press clicks a consent button and reports whether the click went through (one retry: a
// consent iframe that re-renders can swallow a click).
func press(button playwright.Locator) bool {
	for attempt := 1; attempt <= 2; attempt++ {
		if button.Click(playwright.LocatorClickOptions{Timeout: playwright.Float(10000)}) == nil {
			return true
		}
	}
	return false
}

// visit loads a site in a fresh context, then optionally clicks a consent button. It counts
// cookies and the hosts on other domains the page contacted (ads, analytics, CDNs) up to each
// moment. after is nil (and missing says why) when there was nothing to click.
func visit(browser playwright.Browser, site, choice string) (*before, *after, *string, error) {
	domain := domainOf(site)
	ctx, err := browser.NewContext(playwright.BrowserNewContextOptions{Viewport: &playwright.Size{Width: 1280, Height: 900}})
	if err != nil {
		return nil, nil, nil, err
	}
	defer ctx.Close()
	page, err := ctx.NewPage()
	if err != nil {
		return nil, nil, nil, err
	}
	var mu sync.Mutex
	hosts := map[string]bool{}
	page.On("request", func(r playwright.Request) {
		u, err := url.Parse(r.URL())
		if err != nil || u.Hostname() == "" || onDomain(u.Hostname(), domain) {
			return
		}
		mu.Lock()
		hosts[u.Hostname()] = true
		mu.Unlock()
	})
	measure := func() (int, int, int, error) {
		cookies, err := ctx.Cookies()
		if err != nil {
			return 0, 0, 0, err
		}
		third := 0
		for _, c := range cookies {
			if !onDomain(strings.TrimPrefix(c.Domain, "."), domain) {
				third++
			}
		}
		mu.Lock()
		defer mu.Unlock()
		return len(cookies), third, len(hosts), nil
	}

	load := playwright.PageGotoOptions{WaitUntil: playwright.WaitUntilStateDomcontentloaded, Timeout: playwright.Float(60000)}
	res, err := page.Goto(site, load)
	if err != nil { // proxies drop a connection now and then: one retry
		if res, err = page.Goto(site, load); err != nil {
			return nil, nil, nil, err
		}
	}
	// Consent scripts load late on busy pages: wait up to 30 s for the banner, and give every
	// page at least 6 s before measuring.
	t0 := time.Now()
	var bannerAfterMs *int64
	if page.Locator(banner).First().WaitFor(playwright.LocatorWaitForOptions{State: playwright.WaitForSelectorStateVisible, Timeout: playwright.Float(30000)}) == nil {
		ms := time.Since(t0).Milliseconds()
		bannerAfterMs = &ms
	}
	if elapsed := time.Since(t0).Milliseconds(); elapsed < 6000 {
		page.WaitForTimeout(float64(6000 - elapsed))
	}
	v, err := page.Evaluate("(" + probe + ")()")
	if err != nil {
		return nil, nil, nil, err
	}
	m := v.(map[string]any)
	b := &before{CMP: m["cmp"], Banner: m["banner"] == true, TCF: m["tcf"] == true, GPP: m["gpp"] == true, BannerAfterMs: bannerAfterMs}
	if res != nil {
		status := res.Status()
		b.Status = &status
	}
	if b.Cookies, b.ThirdPartyCookies, b.OtherHosts, err = measure(); err != nil {
		return nil, nil, nil, err
	}
	if choice == "" {
		return b, nil, nil, nil
	}
	missing := "no banner"
	if b.Banner {
		missing = "no " + choice + " button on the first layer"
	}
	cmp, _ := m["cmp"].(string)
	selectors := accept
	if choice == "reject" {
		selectors = reject
	}
	selector, ok := selectors[cmp]
	if !ok {
		return b, nil, &missing, nil
	}
	var button playwright.Locator
	if cmp == "Sourcepoint" {
		button = page.FrameLocator(spFrame).Locator(selector).First()
	} else {
		button = page.Locator(selector).First()
	}
	if err := button.WaitFor(playwright.LocatorWaitForOptions{State: playwright.WaitForSelectorStateVisible, Timeout: playwright.Float(10000)}); err != nil {
		return b, nil, &missing, nil
	}
	label, err := button.GetAttribute("title")
	if err != nil || label == "" {
		text, err := button.InnerText()
		if err != nil {
			return nil, nil, nil, err
		}
		label = strings.TrimSpace(text)
	}
	page.WaitForTimeout(2000) // let the banner finish animating in, or the click can be lost
	if !press(button) {
		failed := "click failed"
		return b, nil, &failed, nil
	}
	// If the banner is still up, the click was lost: click once more.
	clicks := 1
	if err := page.Locator(banner).First().WaitFor(playwright.LocatorWaitForOptions{State: playwright.WaitForSelectorStateHidden, Timeout: playwright.Float(5000)}); err != nil {
		clicks = 2
		press(button)
	}
	page.WaitForTimeout(6000)
	a := &after{Button: label, Clicks: clicks}
	if a.Cookies, a.ThirdPartyCookies, a.OtherHosts, err = measure(); err != nil {
		return nil, nil, nil, err
	}
	return b, a, nil, nil
}

// visitOnce: a site may refuse one exit IP (an error status), so visit again in a fresh
// context, whose new proxy connection gets another IP from the rotating proxy.
func visitOnce(browser playwright.Browser, site, choice string) (*before, *after, *string, error) {
	b, a, missing, err := visit(browser, site, choice)
	if err != nil || (b.Status != nil && *b.Status/100 == 2) {
		return b, a, missing, err
	}
	return visit(browser, site, choice)
}

func runUS(us playwright.Browser) ([]*before, error) {
	results := []*before{}
	for _, site := range sites {
		b, _, _, err := visitOnce(us, site, "")
		if err != nil {
			return nil, err
		}
		results = append(results, b)
	}
	return results, nil
}

func runDE(de playwright.Browser, fromUs []*before) ([]siteResult, error) {
	results := []siteResult{}
	for i, site := range sites {
		deBefore, rejected, rejectMissing, err := visitOnce(de, site, "reject")
		if err != nil {
			return nil, err
		}
		_, accepted, acceptMissing, err := visitOnce(de, site, "accept")
		if err != nil {
			return nil, err
		}
		u, _ := url.Parse(site)
		results = append(results, siteResult{Site: strings.TrimPrefix(u.Hostname(), "www."), US: fromUs[i], DE: deBefore,
			DEReject: rejected, DERejectMissing: rejectMissing, DEAccept: accepted, DEAcceptMissing: acceptMissing})
	}
	return results, nil
}

func main() {
	pw, err := playwright.Run(&playwright.RunOptions{SkipInstallBrowsers: true})
	if err != nil {
		log.Fatal(err)
	}
	defer pw.Stop()
	fatal := func(err error) { // log.Fatal would skip the deferred pw.Stop()
		pw.Stop()
		log.Fatal(err)
	}
	connect := func(proxy string) playwright.Browser {
		wsURL, err := launch(proxy)
		if err != nil {
			fatal(err)
		}
		browser, err := pw.Chromium.Connect(wsURL, playwright.BrowserTypeConnectOptions{Headers: map[string]string{"x-api-key": key}})
		if err != nil {
			fatal(err)
		}
		return browser
	}
	// Sessions on shared threads end after 2 minutes without traffic by default, so the countries run one after the other.
	us := connect(os.Getenv("PROXY_URL_US"))
	fromUs, err := runUS(us)
	us.Close()
	if err != nil {
		fatal(err)
	}
	de := connect(os.Getenv("PROXY_URL_DE"))
	results, err := runDE(de, fromUs)
	de.Close()
	if err != nil {
		fatal(err)
	}
	var buf bytes.Buffer
	enc := json.NewEncoder(&buf)
	enc.SetEscapeHTML(false)
	enc.SetIndent("", "  ")
	enc.Encode(map[string]any{"sites": results})
	fmt.Print(buf.String())
}
```

## What we got

| Site | Exit / choice | Banner | Cookies | Third-party cookies | Other-domain hosts |
| --- | --- | --- | --- | --- | --- |
| theguardian.com | US · Sourcepoint, GPP | yes, after 3.5 s | 17 | 5 | 46 |
| theguardian.com | DE · before a choice · Sourcepoint, TCF | yes, after 2.0 s | 7 | 0 | 5 |
| theguardian.com | DE · reject: no reject button on the first layer |  | — | — | — |
| theguardian.com | DE · "Accept all" |  | 102 | 80 | 146 |
| bbc.com | US · Sourcepoint, GPP, TCF | yes, after 4.3 s | 18 | 4 | 14 |
| bbc.com | DE · before a choice · Sourcepoint, TCF, GPP | yes, after 7.3 s | 18 | 2 | 15 |
| bbc.com | DE · "I do not agree" |  | 30 | 4 | 38 |
| bbc.com | DE · "I agree" |  | 31 | 6 | 31 |
| apnews.com | US · no platform | page refused (403) | 1 | 0 | 2 |
| apnews.com | DE · before a choice · OneTrust, TCF | yes, after 4.9 s | 36 | 15 | 112 |
| apnews.com | DE · "I Reject All (except Strictly Necessary)" |  | 42 | 15 | 121 |
| apnews.com | DE · "I Accept All" |  | 192 | 165 | 282 |
| theverge.com | US · OneTrust, GPP | no | 328 | 291 | 285 |
| theverge.com | DE · before a choice · OneTrust, TCF | yes, after 1.8 s | 7 | 0 | 33 |
| theverge.com | DE · "Reject Non-Essential" |  | 10 | 0 | 38 |
| theverge.com | DE · "Accept" |  | 41 | 14 | 74 |
| huffpost.com | US · OneTrust, GPP | yes, after 3.8 s | 76 | 67 | 170 |
| huffpost.com | DE · before a choice · OneTrust, TCF | yes, after 6.7 s | 9 | 0 | 55 |
| huffpost.com | DE · "Reject All" |  | 12 | 0 | 66 |
| huffpost.com | DE · "I Accept" |  | 149 | 118 | 261 |

From the Node.js run on 2026-10-21. IP addresses are replaced with placeholders (203.0.113.x); equal addresses stay equal. The other languages produced the same findings.

Raw output (Node.js)

```
{
  "sites": [
    {
      "site": "theguardian.com",
      "us": {
        "status": 200,
        "cmp": "Sourcepoint",
        "banner": true,
        "tcf": false,
        "gpp": true,
        "banner_after_ms": 3538,
        "cookies": 17,
        "third_party_cookies": 5,
        "other_hosts": 46
      },
      "de": {
        "status": 200,
        "cmp": "Sourcepoint",
        "banner": true,
        "tcf": true,
        "gpp": false,
        "banner_after_ms": 1974,
        "cookies": 7,
        "third_party_cookies": 0,
        "other_hosts": 5
      },
      "de_reject": null,
      "de_reject_missing": "no reject button on the first layer",
      "de_accept": {
        "button": "Accept all",
        "clicks": 1,
        "cookies": 102,
        "third_party_cookies": 80,
        "other_hosts": 146
      },
      "de_accept_missing": null
    },
    {
      "site": "bbc.com",
      "us": {
        "status": 200,
        "cmp": "Sourcepoint",
        "banner": true,
        "tcf": true,
        "gpp": true,
        "banner_after_ms": 4330,
        "cookies": 18,
        "third_party_cookies": 4,
        "other_hosts": 14
      },
      "de": {
        "status": 200,
        "cmp": "Sourcepoint",
        "banner": true,
        "tcf": true,
        "gpp": true,
        "banner_after_ms": 7347,
        "cookies": 18,
        "third_party_cookies": 2,
        "other_hosts": 15
      },
      "de_reject": {
        "button": "I do not agree",
        "clicks": 1,
        "cookies": 30,
        "third_party_cookies": 4,
        "other_hosts": 38
      },
      "de_reject_missing": null,
      "de_accept": {
        "button": "I agree",
        "clicks": 1,
        "cookies": 31,
        "third_party_cookies": 6,
        "other_hosts": 31
      },
      "de_accept_missing": null
    },
    {
      "site": "apnews.com",
      "us": {
        "status": 403,
        "cmp": null,
        "banner": false,
        "tcf": false,
        "gpp": false,
        "banner_after_ms": null,
        "cookies": 1,
        "third_party_cookies": 0,
        "other_hosts": 2
      },
      "de": {
        "status": 200,
        "cmp": "OneTrust",
        "banner": true,
        "tcf": true,
        "gpp": false,
        "banner_after_ms": 4918,
        "cookies": 36,
        "third_party_cookies": 15,
        "other_hosts": 112
      },
      "de_reject": {
        "button": "I Reject All (except Strictly Necessary)",
        "clicks": 1,
        "cookies": 42,
        "third_party_cookies": 15,
        "other_hosts": 121
      },
      "de_reject_missing": null,
      "de_accept": {
        "button": "I Accept All",
        "clicks": 1,
        "cookies": 192,
        "third_party_cookies": 165,
        "other_hosts": 282
      },
      "de_accept_missing": null
    },
    {
      "site": "theverge.com",
      "us": {
        "status": 200,
        "cmp": "OneTrust",
        "banner": false,
        "tcf": false,
        "gpp": true,
        "banner_after_ms": null,
        "cookies": 328,
        "third_party_cookies": 291,
        "other_hosts": 285
      },
      "de": {
        "status": 200,
        "cmp": "OneTrust",
        "banner": true,
        "tcf": true,
        "gpp": false,
        "banner_after_ms": 1814,
        "cookies": 7,
        "third_party_cookies": 0,
        "other_hosts": 33
      },
      "de_reject": {
        "button": "Reject Non-Essential",
        "clicks": 1,
        "cookies": 10,
        "third_party_cookies": 0,
        "other_hosts": 38
      },
      "de_reject_missing": null,
      "de_accept": {
        "button": "Accept",
        "clicks": 1,
        "cookies": 41,
        "third_party_cookies": 14,
        "other_hosts": 74
      },
      "de_accept_missing": null
    },
    {
      "site": "huffpost.com",
      "us": {
        "status": 200,
        "cmp": "OneTrust",
        "banner": true,
        "tcf": false,
        "gpp": true,
        "banner_after_ms": 3766,
        "cookies": 76,
        "third_party_cookies": 67,
        "other_hosts": 170
      },
      "de": {
        "status": 200,
        "cmp": "OneTrust",
        "banner": true,
        "tcf": true,
        "gpp": false,
        "banner_after_ms": 6663,
        "cookies": 9,
        "third_party_cookies": 0,
        "other_hosts": 55
      },
      "de_reject": {
        "button": "Reject All",
        "clicks": 1,
        "cookies": 12,
        "third_party_cookies": 0,
        "other_hosts": 66
      },
      "de_reject_missing": null,
      "de_accept": {
        "button": "I Accept",
        "clicks": 1,
        "cookies": 149,
        "third_party_cookies": 118,
        "other_hosts": 261
      },
      "de_accept_missing": null
    }
  ]
}
```

## Takeaways

-   **The exit country picks the site you get.** Measured again on 2026-10-21 (five runs, one per language): from Germany all five sites showed a consent banner and exposed IAB Europe's TCF API in four runs; in the fifth The Verge showed neither. The banner appeared 0.2–28.1 s after `domcontentloaded`. From the US, only the BBC exposed TCF (it runs both APIs everywhere); every site that loaded exposed IAB Tech Lab's GPP instead. US banners: the Guardian's privacy message in four runs, the BBC's and HuffPost's in three each, none on AP News (in the two runs where it loaded) or The Verge.
-   **Some sites refuse some exits.** AP News answered 403 to the US exit in three of five runs, even after a second try from a fresh context with a new IP; from Germany it loaded every time. Pick the exit for the site, and check the status before you parse.
-   **A banner doesn't mean nothing loads.** Before any choice, the German visitor's page had already contacted 5–6 other-domain hosts on the Guardian, 13–15 on the BBC, 29–48 on The Verge, 45–71 on HuffPost and 75–137 on AP News — with 0–2 third-party cookies, except AP News at 4–19.
-   **"Reject" changes little; "accept" changes a lot on three of the five.** After a reject, cookies rose by 0–15 and third-party cookies stayed at 0–2 on The Verge and HuffPost, 2–4 on the BBC and 4–22 on AP News. After "accept", the Guardian carried 78–99 third-party cookies, AP News 131–165 and HuffPost 87–166, with 133–282 other-domain hosts. On the BBC (6–14 against 2–4) and The Verge (14–17 against 0–2) accepting added far less.
-   **From the US nothing is held back.** With no click at all, the US visitor carried up to 291 third-party cookies on The Verge, 249 on HuffPost and 248 on AP News in some runs — far more than accepting everything from Germany gave on The Verge (14–17).
-   **Not every banner has a free "no".** The Guardian's first layer had an "Accept all" but no free reject in any run — its alternative is pay-or-consent.
-   **For scraping:** pick the exit country on purpose ([a country-pinned residential token](https://cdpfleet.com/docs/proxies#residential) or your own proxy), expect an overlay from the EU, and click the first-layer reject — `#onetrust-reject-all-handler` in the page, Sourcepoint's buttons inside `iframe[id^="sp_message_iframe"]`. Wait for the banner rather than sleeping a fixed time, and wait long: it took up to 20.0 s on HuffPost from the US and 28.1 s from Germany, and 21.9 s on AP News from Germany. Check it's gone after the click and click again if not (no second click was needed in these runs, but one "accept" click on The Verge changed nothing — consent iframes that re-render can swallow one).

---
Source: https://cdpfleet.com/docs/cases/cookie-consent-by-country · Markdown: https://cdpfleet.com/docs/cases/cookie-consent-by-country.md · All docs for AI tools: https://cdpfleet.com/llms.txt