Cases / #21 · 2026-10-05 · Medium
Four ways to make a request, four different visitors: goto, in-page fetch, page.request and your own client
The same URL fetched from the browser, from a script inside the page, from Playwright's request API and from your machine — and what the server saw each time: exit IP, user agent, HTTP version and TLS fingerprint.
Run on production on 2026-10-05: ✓ Node.js ✓ Python ✓ Java ✓ C# ✓ Go
The problem
Playwright gives you several ways to fetch a URL once a browser is running, and they look interchangeable: page.goto, a fetch() inside page.evaluate, page.request.get, or plain HTTP from your script with the browser's cookies copied over. They are not interchangeable. Each runs on a different machine or in a different network stack, so the server sees a different IP, a different TLS handshake and sometimes a different HTTP version — and an anti-bot system compares exactly those things with the user agent. Which paths go through your proxy, which look like the browser on the wire, and which only pretend to?
What we used, and why
| What | Why |
|---|---|
chromium, headless: "new" | One session; the question is about request paths, not the browser. |
https://tls.peet.ws/api/all | Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4) — the facts a server-side bot check keys on. |
page.goto and fetch() in page.evaluate | Both run in the browser: its proxy, cookies, headers and TLS stack. |
page.request.get | Playwright's HTTP client (Node.js), run by the Playwright server next to the browser; it copies the browser's user agent and cookies. |
fetch() in the script itself | Your machine, your runtime's TLS, no proxy — the reference point. |
http://ip-api.com/json/<ip>?fields=hosting | Whether each exit is a residential ISP (the proxy) or a hosting provider (a server). |
How it works
- Launch one headless Chromium session through the proxy.
- Fetch the fingerprint endpoint four ways: a navigation, a same-origin
fetchfrom inside the page,page.request.get, and a plainfetchfrom the script. - For each: record exit IP and its type, user agent, HTTP version, JA4, number of TLS extensions and whether the handshake was resumed (
pre_shared_key). - Compare the cipher-suite part of JA4 with the navigation's to tell "same TLS stack" from "same connection".
The code
The same program in five languages (also on GitHub, with the raw output). Set these environment variables first:
CDPFLEET_API_KEY— your API key (dashboard → API keys)PROXY_URL— your proxy, e.g.http://user:[email protected]:8000
// npm install [email protected]
// env: CDPFLEET_API_KEY, PROXY_URL
import { chromium } from 'playwright';
const KEY = process.env.CDPFLEET_API_KEY;
// Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4).
const PEET = 'https://tls.peet.ws/api/all';
const res = await fetch('https://starter.cdpfleet.com/chromium/session', {
method: 'POST',
headers: { 'x-api-key': KEY, 'content-type': 'application/json' },
body: JSON.stringify({ proxy: process.env.PROXY_URL, headless: 'new' }),
});
if (!res.ok) throw new Error(`launch ${res.status} ${await res.text()}`);
const { wsUrl } = await res.json();
const browser = await chromium.connect(wsUrl, { headers: { 'x-api-key': KEY } });
async function row(path, seen, runs_on) {
const ip = seen.ip.split(':')[0];
// Who owns the exit: a residential ISP (your proxy) or a hosting provider (a server)?
const who = await (await fetch(`http://ip-api.com/json/${ip}?fields=hosting`)).json();
return {
path, runs_on, exit_ip: ip, exit_type: who.hosting ? 'datacenter' : 'residential',
user_agent: seen.user_agent, http_version: seen.http_version, ja4: seen.tls.ja4,
tls_extensions: seen.tls.extensions.length,
resumed_tls: seen.tls.extensions.some((e) => /pre_shared_key/.test(e.name)),
};
}
try {
const page = await browser.newPage();
// 1. A navigation: the browser itself makes the request.
const nav = await (await page.goto(PEET, { timeout: 60000 })).json();
// 2. fetch() inside the page (same origin): the browser's network stack, cookies and headers.
const inPage = await page.evaluate(() => fetch('/api/all').then((r) => r.json()));
// 3. page.request: Playwright's own HTTP client, run by the Playwright server next to the browser.
const viaRequest = await (await page.request.get(PEET, { timeout: 60000 })).json();
// 4. Your own HTTP client on your machine, for reference.
const local = await (await fetch(PEET)).json();
const out = [
await row('page.goto', nav, 'the browser'),
await row('fetch() in page.evaluate', inPage, 'the browser'),
await row('page.request.get', viaRequest, 'Playwright server'),
await row('fetch() in your script', local, 'your machine'),
];
// JA4's middle part hashes the cipher suites: the same TLS stack keeps it across connections.
for (const r of out) r.same_tls_stack_as_browser = r.ja4.split('_')[1] === out[0].ja4.split('_')[1];
console.log(JSON.stringify(out, null, 2));
} finally {
await browser.close();
}
# pip install playwright==1.60.0 requests
# env: CDPFLEET_API_KEY, PROXY_URL
import json
import os
import requests
from playwright.sync_api import sync_playwright
KEY = os.environ["CDPFLEET_API_KEY"]
# Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4).
PEET = "https://tls.peet.ws/api/all"
res = requests.post("https://starter.cdpfleet.com/chromium/session", headers={"x-api-key": KEY}, timeout=60,
json={"proxy": os.environ["PROXY_URL"], "headless": "new"})
if not res.ok:
raise SystemExit(f"launch {res.status_code} {res.text}")
session = res.json()
def row(path, seen, runs_on):
ip = seen["ip"].split(":")[0]
# Who owns the exit: a residential ISP (your proxy) or a hosting provider (a server)?
who = requests.get(f"http://ip-api.com/json/{ip}?fields=hosting", timeout=30).json()
return {
"path": path, "runs_on": runs_on, "exit_ip": ip, "exit_type": "datacenter" if who.get("hosting") else "residential",
"user_agent": seen.get("user_agent"), "http_version": seen.get("http_version"), "ja4": seen["tls"]["ja4"],
"tls_extensions": len(seen["tls"]["extensions"]),
"resumed_tls": any("pre_shared_key" in (e.get("name") or "") for e in seen["tls"]["extensions"]),
}
with sync_playwright() as p:
browser = p.chromium.connect(session["wsUrl"], headers={"x-api-key": KEY})
try:
page = browser.new_page()
# 1. A navigation: the browser itself makes the request.
nav = page.goto(PEET, timeout=60000).json()
# 2. fetch() inside the page (same origin): the browser's network stack, cookies and headers.
in_page = page.evaluate("() => fetch('/api/all').then((r) => r.json())")
# 3. page.request: Playwright's own HTTP client, run by the Playwright server next to the browser.
via_request = page.request.get(PEET, timeout=60000).json()
# 4. Your own HTTP client on your machine, for reference.
local = requests.get(PEET, timeout=60).json()
out = [
row("page.goto", nav, "the browser"),
row("fetch() in page.evaluate", in_page, "the browser"),
row("page.request.get", via_request, "Playwright server"),
row("fetch() in your script", local, "your machine"),
]
# JA4's middle part hashes the cipher suites: the same TLS stack keeps it across connections.
for r in out:
r["same_tls_stack_as_browser"] = r["ja4"].split("_")[1] == out[0]["ja4"].split("_")[1]
print(json.dumps(out, indent=2))
finally:
browser.close()
// Maven: com.microsoft.playwright:playwright:1.60.0, com.google.code.gson:gson:2.11.0
// Run with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1. env: CDPFLEET_API_KEY, PROXY_URL
import com.google.gson.*;
import com.microsoft.playwright.*;
import com.microsoft.playwright.options.RequestOptions;
import java.net.URI;
import java.net.http.*;
import java.util.Map;
public class Main {
static final String KEY = System.getenv("CDPFLEET_API_KEY");
// Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4).
static final String PEET = "https://tls.peet.ws/api/all";
static final Gson GSON = new GsonBuilder().setPrettyPrinting().disableHtmlEscaping().serializeNulls().create();
static final HttpClient HTTP = HttpClient.newHttpClient();
static JsonObject get(String url) throws Exception {
HttpResponse<String> res = HTTP.send(HttpRequest.newBuilder(URI.create(url)).GET().build(), HttpResponse.BodyHandlers.ofString());
return JsonParser.parseString(res.body()).getAsJsonObject();
}
static JsonObject row(String path, JsonObject seen, String runsOn) throws Exception {
String ip = seen.get("ip").getAsString().split(":")[0];
// Who owns the exit: a residential ISP (your proxy) or a hosting provider (a server)?
JsonObject who = get("http://ip-api.com/json/" + ip + "?fields=hosting");
JsonObject tls = seen.getAsJsonObject("tls");
boolean resumed = false;
for (JsonElement e : tls.getAsJsonArray("extensions")) {
JsonElement name = e.getAsJsonObject().get("name");
if (name != null && !name.isJsonNull() && name.getAsString().contains("pre_shared_key")) resumed = true;
}
JsonObject row = new JsonObject();
row.addProperty("path", path);
row.addProperty("runs_on", runsOn);
row.addProperty("exit_ip", ip);
row.addProperty("exit_type", who.has("hosting") && who.get("hosting").getAsBoolean() ? "datacenter" : "residential");
row.add("user_agent", seen.get("user_agent"));
row.add("http_version", seen.get("http_version"));
row.add("ja4", tls.get("ja4"));
row.addProperty("tls_extensions", tls.getAsJsonArray("extensions").size());
row.addProperty("resumed_tls", resumed);
return row;
}
public static void main(String[] args) throws Exception {
String body = "{\"proxy\": " + GSON.toJson(System.getenv("PROXY_URL")) + ", \"headless\": \"new\"}";
HttpResponse<String> res = HTTP.send(HttpRequest.newBuilder(URI.create("https://starter.cdpfleet.com/chromium/session"))
.header("x-api-key", KEY).header("content-type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(body)).build(), HttpResponse.BodyHandlers.ofString());
if (res.statusCode() != 200) throw new RuntimeException("launch " + res.statusCode() + " " + res.body());
JsonObject session = JsonParser.parseString(res.body()).getAsJsonObject();
try (Playwright playwright = Playwright.create()) {
Browser browser = playwright.chromium().connect(session.get("wsUrl").getAsString(),
new BrowserType.ConnectOptions().setHeaders(Map.of("x-api-key", KEY)));
try {
Page page = browser.newPage();
// 1. A navigation: the browser itself makes the request.
JsonObject nav = JsonParser.parseString(page.navigate(PEET, new Page.NavigateOptions().setTimeout(60000)).text()).getAsJsonObject();
// 2. fetch() inside the page (same origin): the browser's network stack, cookies and headers.
JsonObject inPage = GSON.toJsonTree(page.evaluate("() => fetch('/api/all').then((r) => r.json())")).getAsJsonObject();
// 3. page.request(): Playwright's own HTTP client, run by the Playwright server next to the browser.
JsonObject viaRequest = JsonParser.parseString(page.request().get(PEET, RequestOptions.create().setTimeout(60000)).text()).getAsJsonObject();
// 4. Your own HTTP client on your machine, for reference.
JsonObject local = get(PEET);
JsonArray out = new JsonArray();
out.add(row("page.goto", nav, "the browser"));
out.add(row("fetch() in page.evaluate", inPage, "the browser"));
out.add(row("page.request.get", viaRequest, "Playwright server"));
out.add(row("fetch() in your script", local, "your machine"));
// JA4's middle part hashes the cipher suites: the same TLS stack keeps it across connections.
String browserCiphers = out.get(0).getAsJsonObject().get("ja4").getAsString().split("_")[1];
for (JsonElement e : out) {
JsonObject r = e.getAsJsonObject();
r.addProperty("same_tls_stack_as_browser", r.get("ja4").getAsString().split("_")[1].equals(browserCiphers));
}
System.out.println(GSON.toJson(out));
} finally {
browser.close();
}
}
}
}
// dotnet add package Microsoft.Playwright --version 1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL
using System.Net.Http.Json;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Playwright;
var key = Environment.GetEnvironmentVariable("CDPFLEET_API_KEY")!;
// Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4).
const string Peet = "https://tls.peet.ws/api/all";
using var http = new HttpClient();
http.DefaultRequestHeaders.Add("x-api-key", key);
var res = await http.PostAsJsonAsync("https://starter.cdpfleet.com/chromium/session",
new { proxy = Environment.GetEnvironmentVariable("PROXY_URL"), headless = "new" });
if (!res.IsSuccessStatusCode) throw new Exception($"launch {(int)res.StatusCode} {await res.Content.ReadAsStringAsync()}");
var session = await res.Content.ReadFromJsonAsync<JsonElement>();
// A plain client for third-party URLs: no API key, no proxy, the default (empty) user agent.
using var plain = new HttpClient();
async Task<JsonObject> Row(string path, JsonNode seen, string runsOn)
{
var ip = seen["ip"]!.GetValue<string>().Split(':')[0];
// Who owns the exit: a residential ISP (your proxy) or a hosting provider (a server)?
var who = JsonNode.Parse(await plain.GetStringAsync($"http://ip-api.com/json/{ip}?fields=hosting"))!;
var tls = seen["tls"]!;
var extensions = tls["extensions"]!.AsArray();
return new JsonObject
{
["path"] = path,
["runs_on"] = runsOn,
["exit_ip"] = ip,
["exit_type"] = who["hosting"]?.GetValue<bool>() == true ? "datacenter" : "residential",
["user_agent"] = seen["user_agent"]?.DeepClone(),
["http_version"] = seen["http_version"]?.DeepClone(),
["ja4"] = tls["ja4"]?.DeepClone(),
["tls_extensions"] = extensions.Count,
["resumed_tls"] = extensions.Any(e => (e?["name"]?.GetValue<string>() ?? "").Contains("pre_shared_key")),
};
}
using var playwright = await Playwright.CreateAsync();
var browser = await playwright.Chromium.ConnectAsync(session.GetProperty("wsUrl").GetString()!,
new() { Headers = new Dictionary<string, string> { ["x-api-key"] = key } });
try
{
var page = await browser.NewPageAsync();
// 1. A navigation: the browser itself makes the request.
var nav = JsonNode.Parse(await (await page.GotoAsync(Peet, new() { Timeout = 60000 }))!.TextAsync())!;
// 2. fetch() inside the page (same origin): the browser's network stack, cookies and headers.
var inPage = JsonNode.Parse((await page.EvaluateAsync<JsonElement>("() => fetch('/api/all').then((r) => r.json())")).GetRawText())!;
// 3. page.APIRequest: Playwright's own HTTP client, run by the Playwright server next to the browser.
var viaRequest = JsonNode.Parse(await (await page.APIRequest.GetAsync(Peet, new() { Timeout = 60000 })).TextAsync())!;
// 4. Your own HTTP client on your machine, for reference.
var local = JsonNode.Parse(await plain.GetStringAsync(Peet))!;
var output = new JsonArray
{
await Row("page.goto", nav, "the browser"),
await Row("fetch() in page.evaluate", inPage, "the browser"),
await Row("page.request.get", viaRequest, "Playwright server"),
await Row("fetch() in your script", local, "your machine"),
};
// JA4's middle part hashes the cipher suites: the same TLS stack keeps it across connections.
var browserCiphers = output[0]!["ja4"]!.GetValue<string>().Split('_')[1];
foreach (var r in output)
r!["same_tls_stack_as_browser"] = r["ja4"]!.GetValue<string>().Split('_')[1] == browserCiphers;
Console.WriteLine(output.ToJsonString(new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
}
finally
{
await browser.CloseAsync();
}
// go get github.com/playwright-community/[email protected]
// Driver: build playwright-core 1.60.0 from npm and set PLAYWRIGHT_DRIVER_PATH (see /docs/quickstart).
// env: CDPFLEET_API_KEY, PROXY_URL
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"strings"
"github.com/playwright-community/playwright-go"
)
var key = os.Getenv("CDPFLEET_API_KEY")
// Reports the caller's IP, user agent, HTTP version and TLS fingerprint (JA4).
const peet = "https://tls.peet.ws/api/all"
type row struct {
Path string `json:"path"`
RunsOn string `json:"runs_on"`
ExitIP string `json:"exit_ip"`
ExitType string `json:"exit_type"`
UserAgent any `json:"user_agent"`
HTTPVersion any `json:"http_version"`
JA4 string `json:"ja4"`
TLSExtensions int `json:"tls_extensions"`
ResumedTLS bool `json:"resumed_tls"`
SameTLSAsBrowser bool `json:"same_tls_stack_as_browser"`
}
func must[T any](v T, err error) T {
if err != nil {
log.Fatal(err)
}
return v
}
// GET a JSON document with Go's own HTTP client: no proxy, the default user agent.
func getJSON(url string) map[string]any {
res := must(http.Get(url))
defer res.Body.Close()
var v map[string]any
must(0, json.NewDecoder(res.Body).Decode(&v))
return v
}
func makeRow(path string, seen map[string]any, runsOn string) row {
ip := strings.Split(seen["ip"].(string), ":")[0]
// Who owns the exit: a residential ISP (your proxy) or a hosting provider (a server)?
who := getJSON("http://ip-api.com/json/" + ip + "?fields=hosting")
exitType := "residential"
if hosting, _ := who["hosting"].(bool); hosting {
exitType = "datacenter"
}
tls := seen["tls"].(map[string]any)
extensions, _ := tls["extensions"].([]any)
resumed := false
for _, e := range extensions {
if name, _ := e.(map[string]any)["name"].(string); strings.Contains(name, "pre_shared_key") {
resumed = true
}
}
return row{Path: path, RunsOn: runsOn, ExitIP: ip, ExitType: exitType, UserAgent: seen["user_agent"], HTTPVersion: seen["http_version"],
JA4: tls["ja4"].(string), TLSExtensions: len(extensions), ResumedTLS: resumed}
}
func main() {
body, _ := json.Marshal(map[string]any{"proxy": os.Getenv("PROXY_URL"), "headless": "new"})
req, _ := http.NewRequest("POST", "https://starter.cdpfleet.com/chromium/session", bytes.NewReader(body))
req.Header.Set("x-api-key", key)
req.Header.Set("content-type", "application/json")
res := must(http.DefaultClient.Do(req))
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
msg, _ := io.ReadAll(res.Body)
log.Fatalf("launch %s %s", res.Status, msg)
}
var session map[string]any
must(0, json.NewDecoder(res.Body).Decode(&session))
pw := must(playwright.Run(&playwright.RunOptions{SkipInstallBrowsers: true}))
defer pw.Stop()
browser := must(pw.Chromium.Connect(session["wsUrl"].(string), playwright.BrowserTypeConnectOptions{Headers: map[string]string{"x-api-key": key}}))
defer browser.Close()
page := must(browser.NewPage())
// 1. A navigation: the browser itself makes the request.
var nav map[string]any
must(0, must(page.Goto(peet, playwright.PageGotoOptions{Timeout: playwright.Float(60000)})).JSON(&nav))
// 2. fetch() inside the page (same origin): the browser's network stack, cookies and headers.
inPage := must(page.Evaluate("() => fetch('/api/all').then((r) => r.json())")).(map[string]any)
// 3. page.Request(): Playwright's own HTTP client, run by the Playwright server next to the browser.
var viaRequest map[string]any
must(0, must(page.Request().Get(peet, playwright.APIRequestContextGetOptions{Timeout: playwright.Float(60000)})).JSON(&viaRequest))
// 4. Your own HTTP client on your machine, for reference.
local := getJSON(peet)
out := []row{
makeRow("page.goto", nav, "the browser"),
makeRow("fetch() in page.evaluate", inPage, "the browser"),
makeRow("page.request.get", viaRequest, "Playwright server"),
makeRow("fetch() in your script", local, "your machine"),
}
// JA4's middle part hashes the cipher suites: the same TLS stack keeps it across connections.
browserCiphers := strings.Split(out[0].JA4, "_")[1]
for i := range out {
out[i].SameTLSAsBrowser = strings.Split(out[i].JA4, "_")[1] == browserCiphers
}
text, _ := json.MarshalIndent(out, "", " ")
fmt.Println(string(text))
}
What we got
| Path | Runs on | Exit | HTTP | JA4 | TLS ext. | Resumed TLS | Browser's TLS stack | User agent |
|---|---|---|---|---|---|---|---|---|
| page.goto | the browser | residential | h2 | t13d1516h2_8daaf6152771_d8a2da3f94cd | 18 | no | yes | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36 |
| fetch() in page.evaluate | the browser | residential | h2 | t13d1517h2_8daaf6152771_b6f405a00624 | 19 | yes | yes | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36 |
| page.request.get | Playwright server | residential | HTTP/1.1 | t13d5211_b262b3658495_8e6e362c5eac | 11 | no | no | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36 |
| fetch() in your script | your machine | datacenter | HTTP/1.1 | t13d5911h1_a33745022dd6_1f22a2ca17c4 | 11 | no | no | node |
From the Node.js run on 2026-10-05. IP addresses are replaced with placeholders (203.0.113.x); equal addresses stay equal. The other languages produced the same findings.
Raw output (Node.js)
[
{
"path": "page.goto",
"runs_on": "the browser",
"exit_ip": "203.0.113.1",
"exit_type": "residential",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36",
"http_version": "h2",
"ja4": "t13d1516h2_8daaf6152771_d8a2da3f94cd",
"tls_extensions": 18,
"resumed_tls": false,
"same_tls_stack_as_browser": true
},
{
"path": "fetch() in page.evaluate",
"runs_on": "the browser",
"exit_ip": "203.0.113.2",
"exit_type": "residential",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36",
"http_version": "h2",
"ja4": "t13d1517h2_8daaf6152771_b6f405a00624",
"tls_extensions": 19,
"resumed_tls": true,
"same_tls_stack_as_browser": true
},
{
"path": "page.request.get",
"runs_on": "Playwright server",
"exit_ip": "203.0.113.3",
"exit_type": "residential",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/148.0.7778.96 Safari/537.36",
"http_version": "HTTP/1.1",
"ja4": "t13d5211_b262b3658495_8e6e362c5eac",
"tls_extensions": 11,
"resumed_tls": false,
"same_tls_stack_as_browser": false
},
{
"path": "fetch() in your script",
"runs_on": "your machine",
"exit_ip": "203.0.113.4",
"exit_type": "datacenter",
"user_agent": "node",
"http_version": "HTTP/1.1",
"ja4": "t13d5911h1_a33745022dd6_1f22a2ca17c4",
"tls_extensions": 11,
"resumed_tls": false,
"same_tls_stack_as_browser": false
}
]Takeaways
- Only the browser paths are the browser:
page.gotoand an in-pagefetchshare Chrome's cipher suites, HTTP/2 and user agent. Their JA4 differs by one extension — the fetch reused the TLS session (pre_shared_key), a normal browser behaviour — so compare the cipher hash, not the whole string, when checking "is this the same stack". page.requestgoes through your proxy but is not the browser: the exit was residential (the fleet runs the request through the session's proxy), the user agent is Chrome's — and the handshake is Node.js's: HTTP/1.1, 11 extensions, a JA4 that no Chrome ever sends. A Chrome user agent on a Node TLS stack is the textbook bot signature.- Your own client is a third identity: a datacenter exit, your runtime's TLS and user agent (
node,python-requests/…,Java-http-client/…,Go-http-client/2.0, none at all from .NET), HTTP/1.1 from Node, Python and C#, HTTP/2 from Java and Go. Copying the browser's cookies into it ties that identity to the browser's session. - Exit IPs rotate per connection on a rotating residential proxy: four requests, four exits, all residential except your own. Use sticky ports when a session must keep one IP; see three visitors, one thread.
- Rule of thumb: data the site should see you fetch as a browser — fetch it in the page (
page.evaluate(() => fetch(url)), as in infinite scroll, two ways). Usepage.requestfor things the site never fingerprints, or not at all.