Cases / #30 · 2026-10-09 · Easy

Five pages at once: parallel tabs in one browser session

Open five unrelated pages concurrently in one session and compare with loading them one by one — same thread, same proxy, a fraction of the wall time.

Chromium

Run on production on 2026-10-09: ✓ Node.js ✓ Python ✓ Java ✓ C# ✓ Go

The problem

You need something from several unrelated pages. The obvious loop opens one, extracts, closes, and moves on — and most of each wait is latency through the proxy while the browser sits idle. A session can hold many tabs, and each language has its own way to wait on several at once. How much does that save, and what does it cost?

What we used, and why

What	Why
`chromium`, `headless: "new"`	One session (1 thread) for all five pages.
Five independent URLs	books.toscrape.com, quotes.toscrape.com, example.com, httpbin.org/html and scrapethissite.com: public, no login, safe to load at once.
`browser.newPage()` per URL	Each tab has its own navigation; all share the session's proxy and connection.
`Promise.all` · `asyncio.gather` · `Task.WhenAll` · goroutines	Each language's own way to wait on five loads at once. Java starts all five navigations and then waits for each — no threads.
`waitUntil: "domcontentloaded"`	Wait for the HTML, not every image and script: one slow asset would otherwise set the pace.
`page.title()`	The smallest proof a page loaded; put your extraction here.

How it works

Launch one Chromium session.
Sequential: open each URL in a new tab, read its title, close it — timed.
Parallel: open all five at once, read the titles, close the tabs — timed.
Print both timings, the speedup and every title.

The code

The same program in five languages (also on GitHub, with the raw output). Set these environment variables first:

CDPFLEET_API_KEY — your API key (dashboard → API keys)
PROXY_URL — your proxy, e.g. http://user:[email protected]:8000

// npm install [email protected]
// env: CDPFLEET_API_KEY, PROXY_URL
import { chromium } from 'playwright';

const KEY = process.env.CDPFLEET_API_KEY;
const URLS = [
  'https://books.toscrape.com/',
  'https://quotes.toscrape.com/',
  'https://example.com',
  'https://en.wikipedia.org/wiki/Web_scraping',
  'https://news.ycombinator.com/',
];

const res = await fetch('https://starter.cdpfleet.com/chromium/session', {
  method: 'POST',
  headers: { 'x-api-key': KEY, 'content-type': 'application/json' },
  body: JSON.stringify({ proxy: process.env.PROXY_URL, headless: 'new' }),
});
if (!res.ok) throw new Error(`launch ${res.status} ${await res.text()}`);
const { wsUrl } = await res.json();
const browser = await chromium.connect(wsUrl, { headers: { 'x-api-key': KEY } });

async function extract(browser, url) {
  const page = await browser.newPage();
  try {
    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
    return { url, title: await page.title() };
  } finally {
    await page.close();
  }
}

try {
  const t1 = Date.now();
  const sequential = [];
  for (const url of URLS) {
    sequential.push({ ...await extract(browser, url), method: 'sequential' });
  }
  const seqSeconds = Math.round((Date.now() - t1) / 10) / 100;

  const t2 = Date.now();
  const parallel = await Promise.all(URLS.map((url) => extract(browser, url).then((r) => ({ ...r, method: 'parallel' }))));
  const parSeconds = Math.round((Date.now() - t2) / 10) / 100;

  console.log(JSON.stringify({
    sequential_seconds: seqSeconds,
    parallel_seconds: parSeconds,
    speedup: `${Math.round(seqSeconds / parSeconds * 10) / 10}x`,
    results: [...sequential, ...parallel],
  }, null, 2));
} finally {
  await browser.close();
}

# pip install playwright==1.60.0 requests
# env: CDPFLEET_API_KEY, PROXY_URL
# Playwright's sync API isn't thread-safe, so the parallel loads use the async API.
import asyncio
import json
import os
import time

import requests
from playwright.async_api import async_playwright

KEY = os.environ["CDPFLEET_API_KEY"]
URLS = [
    "https://books.toscrape.com/",
    "https://quotes.toscrape.com/",
    "https://example.com",
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://news.ycombinator.com/",
]


async def main():
    res = requests.post("https://starter.cdpfleet.com/chromium/session",
        headers={"x-api-key": KEY},
        json={"proxy": os.environ["PROXY_URL"], "headless": "new"}, timeout=60)
    if not res.ok:
        raise SystemExit(f"launch {res.status_code} {res.text}")

    async with async_playwright() as p:
        browser = await p.chromium.connect(res.json()["wsUrl"], headers={"x-api-key": KEY})
        try:
            async def extract(url):
                page = await browser.new_page()
                try:
                    await page.goto(url, wait_until="domcontentloaded", timeout=60000)
                    return {"url": url, "title": await page.title()}
                finally:
                    await page.close()

            t1 = time.time()
            sequential = [{**(await extract(url)), "method": "sequential"} for url in URLS]
            seq_seconds = round(time.time() - t1, 2)

            t2 = time.time()
            parallel = [{**r, "method": "parallel"} for r in await asyncio.gather(*(extract(url) for url in URLS))]
            par_seconds = round(time.time() - t2, 2)

            print(json.dumps({
                "sequential_seconds": seq_seconds,
                "parallel_seconds": par_seconds,
                "speedup": f"{round(seq_seconds / par_seconds, 1)}x",
                "results": sequential + parallel,
            }, indent=2, ensure_ascii=False))
        finally:
            await browser.close()


asyncio.run(main())

// Maven: com.microsoft.playwright:playwright:1.60.0, com.google.code.gson:gson:2.11.0
// Run with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1. env: CDPFLEET_API_KEY, PROXY_URL
import com.google.gson.*;
import com.microsoft.playwright.*;
import com.microsoft.playwright.options.WaitUntilState;
import java.net.URI;
import java.net.http.*;
import java.util.*;
import java.util.concurrent.*;

public class Main {
  static final String KEY = System.getenv("CDPFLEET_API_KEY");
  static final Gson GSON = new GsonBuilder().serializeNulls().create();
  static final String[] URLS = {
      "https://books.toscrape.com/",
      "https://quotes.toscrape.com/",
      "https://example.com",
      "https://en.wikipedia.org/wiki/Web_scraping",
      "https://news.ycombinator.com/",
  };

  record Result(String url, String title, String method) {}

  static Result extract(Browser browser, String url) {
    Page page = browser.newPage();
    try {
      page.navigate(url, new Page.NavigateOptions().setWaitUntil(WaitUntilState.DOMCONTENTLOADED).setTimeout(60000));
      return new Result(url, page.title(), null);
    } finally {
      page.close();
    }
  }

  public static void main(String[] args) throws Exception {
    String body = "{\"proxy\": " + GSON.toJson(System.getenv("PROXY_URL")) + ", \"headless\": \"new\"}";
    HttpResponse<String> res = HttpClient.newHttpClient().send(HttpRequest.newBuilder(URI.create("https://starter.cdpfleet.com/chromium/session"))
        .header("x-api-key", KEY).header("content-type", "application/json")
        .POST(HttpRequest.BodyPublishers.ofString(body)).build(), HttpResponse.BodyHandlers.ofString());
    if (res.statusCode() != 200) throw new RuntimeException("launch: " + res.statusCode() + " " + res.body());
    String wsUrl = JsonParser.parseString(res.body()).getAsJsonObject().get("wsUrl").getAsString();

    try (Playwright playwright = Playwright.create()) {
      Browser browser = playwright.chromium().connect(wsUrl, new BrowserType.ConnectOptions().setHeaders(Map.of("x-api-key", KEY)));
      try {
        long t1 = System.currentTimeMillis();
        List<Result> sequential = new ArrayList<>();
        for (String url : URLS) {
          Result r = extract(browser, url);
          sequential.add(new Result(r.url, r.title, "sequential"));
        }
        double seqSeconds = Math.round((System.currentTimeMillis() - t1) / 10.0) / 100.0;

        // Playwright for Java isn't thread-safe: start all five navigations, then wait for each.
        long t2 = System.currentTimeMillis();
        List<Page> pages = new ArrayList<>();
        for (String url : URLS) {
          Page page = browser.newPage();
          page.evaluate("u => { location.href = u; }", url);
          pages.add(page);
        }
        List<Result> parallel = new ArrayList<>();
        for (int i = 0; i < URLS.length; i++) {
          Page page = pages.get(i);
          page.waitForURL(u -> !u.startsWith("about:"), new Page.WaitForURLOptions().setWaitUntil(WaitUntilState.DOMCONTENTLOADED).setTimeout(60000));
          parallel.add(new Result(URLS[i], page.title(), "parallel"));
          page.close();
        }
        double parSeconds = Math.round((System.currentTimeMillis() - t2) / 10.0) / 100.0;

        JsonObject out = new JsonObject();
        out.addProperty("sequential_seconds", seqSeconds);
        out.addProperty("parallel_seconds", parSeconds);
        out.addProperty("speedup", Math.round(seqSeconds / parSeconds * 10) / 10.0 + "x");
        JsonArray results = new JsonArray();
        for (Result r : sequential) results.add(GSON.toJsonTree(r));
        for (Result r : parallel) results.add(GSON.toJsonTree(r));
        out.add("results", results);

        System.out.println(new GsonBuilder().serializeNulls().setPrettyPrinting().disableHtmlEscaping().create().toJson(out));
      } finally {
        browser.close();
      }
    }
  }
}

// dotnet add package Microsoft.Playwright --version 1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL
using System.Diagnostics;
using System.Net.Http.Json;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Playwright;

string[] urls = [
    "https://books.toscrape.com/",
    "https://quotes.toscrape.com/",
    "https://example.com",
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://news.ycombinator.com/",
];

var key = Environment.GetEnvironmentVariable("CDPFLEET_API_KEY")!;
using var http = new HttpClient();
http.DefaultRequestHeaders.Add("x-api-key", key);
var res = await http.PostAsJsonAsync("https://starter.cdpfleet.com/chromium/session",
    new { proxy = Environment.GetEnvironmentVariable("PROXY_URL"), headless = "new" });
if (!res.IsSuccessStatusCode) throw new Exception($"launch: {(int)res.StatusCode} {await res.Content.ReadAsStringAsync()}");
var wsUrl = (await res.Content.ReadFromJsonAsync<JsonElement>()).GetProperty("wsUrl").GetString()!;

using var playwright = await Playwright.CreateAsync();
var browser = await playwright.Chromium.ConnectAsync(wsUrl, new() { Headers = new Dictionary<string, string> { ["x-api-key"] = key } });
try
{
    async Task<JsonObject> Extract(string url)
    {
        var page = await browser.NewPageAsync();
        try
        {
            await page.GotoAsync(url, new() { WaitUntil = WaitUntilState.DOMContentLoaded, Timeout = 60000 });
            return new JsonObject { ["url"] = url, ["title"] = await page.TitleAsync() };
        }
        finally
        {
            await page.CloseAsync();
        }
    }

    var sw1 = Stopwatch.StartNew();
    var sequential = new List<JsonObject>();
    foreach (var url in urls)
    {
        var r = await Extract(url);
        r["method"] = "sequential";
        sequential.Add(r);
    }
    var seqSeconds = Math.Round(sw1.Elapsed.TotalSeconds, 2);

    var sw2 = Stopwatch.StartNew();
    var tasks = urls.Select(async url =>
    {
        var r = await Extract(url);
        r["method"] = "parallel";
        return r;
    }).ToArray();
    var parallelResults = await Task.WhenAll(tasks);
    var parSeconds = Math.Round(sw2.Elapsed.TotalSeconds, 2);

    var results = new JsonArray();
    foreach (var r in sequential) results.Add(r);
    foreach (var r in parallelResults) results.Add(r);

    var output = new JsonObject
    {
        ["sequential_seconds"] = seqSeconds,
        ["parallel_seconds"] = parSeconds,
        ["speedup"] = $"{Math.Round(seqSeconds / parSeconds, 1)}x",
        ["results"] = results,
    };
    Console.WriteLine(output.ToJsonString(new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
}
finally
{
    await browser.CloseAsync();
}

// go get github.com/playwright-community/[email protected]
// Driver: build playwright-core 1.60.0 from npm and set PLAYWRIGHT_DRIVER_PATH (see /docs/quickstart).
// env: CDPFLEET_API_KEY, PROXY_URL
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"math"
	"net/http"
	"os"
	"sync"
	"time"

	"github.com/playwright-community/playwright-go"
)

var key = os.Getenv("CDPFLEET_API_KEY")

var urls = []string{
	"https://books.toscrape.com/",
	"https://quotes.toscrape.com/",
	"https://example.com",
	"https://en.wikipedia.org/wiki/Web_scraping",
	"https://news.ycombinator.com/",
}

func launch(name string, options map[string]any) (map[string]any, error) {
	body, _ := json.Marshal(options)
	req, _ := http.NewRequest("POST", "https://starter.cdpfleet.com/"+name+"/session", bytes.NewReader(body))
	req.Header.Set("x-api-key", key)
	req.Header.Set("content-type", "application/json")
	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != http.StatusOK {
		msg, _ := io.ReadAll(res.Body)
		return nil, fmt.Errorf("launch %s: %s %s", name, res.Status, msg)
	}
	var session map[string]any
	return session, json.NewDecoder(res.Body).Decode(&session)
}

type result struct {
	URL    string `json:"url"`
	Title  string `json:"title"`
	Method string `json:"method"`
}

func extract(browser playwright.Browser, url string) (string, string) {
	page, err := browser.NewPage()
	if err != nil {
		log.Fatal(err)
	}
	defer page.Close()
	if _, err := page.Goto(url, playwright.PageGotoOptions{WaitUntil: playwright.WaitUntilStateDomcontentloaded, Timeout: playwright.Float(60000)}); err != nil {
		log.Fatal(err)
	}
	title, err := page.Title()
	if err != nil {
		log.Fatal(err)
	}
	return url, title
}

func main() {
	session, err := launch("chromium", map[string]any{"proxy": os.Getenv("PROXY_URL"), "headless": "new"})
	if err != nil {
		log.Fatal(err)
	}
	pw, err := playwright.Run(&playwright.RunOptions{SkipInstallBrowsers: true})
	if err != nil {
		log.Fatal(err)
	}
	defer pw.Stop()
	browser, err := pw.Chromium.Connect(session["wsUrl"].(string), playwright.BrowserTypeConnectOptions{Headers: map[string]string{"x-api-key": key}})
	if err != nil {
		log.Fatal(err)
	}
	defer browser.Close()

	t1 := time.Now()
	var sequential []result
	for _, u := range urls {
		url, title := extract(browser, u)
		sequential = append(sequential, result{URL: url, Title: title, Method: "sequential"})
	}
	seqSeconds := math.Round(time.Since(t1).Seconds()*100) / 100

	t2 := time.Now()
	parallel := make([]result, len(urls))
	var wg sync.WaitGroup
	for i, u := range urls {
		wg.Add(1)
		go func(i int, u string) {
			defer wg.Done()
			url, title := extract(browser, u)
			parallel[i] = result{URL: url, Title: title, Method: "parallel"}
		}(i, u)
	}
	wg.Wait()
	parSeconds := math.Round(time.Since(t2).Seconds()*100) / 100

	all := append(sequential, parallel...)
	out := struct {
		SequentialSeconds float64  `json:"sequential_seconds"`
		ParallelSeconds   float64  `json:"parallel_seconds"`
		Speedup           string   `json:"speedup"`
		Results           []result `json:"results"`
	}{
		SequentialSeconds: seqSeconds,
		ParallelSeconds:   parSeconds,
		Speedup:           fmt.Sprintf("%.1fx", math.Round(seqSeconds/parSeconds*10)/10),
		Results:           all,
	}
	var buf bytes.Buffer
	enc := json.NewEncoder(&buf)
	enc.SetEscapeHTML(false)
	enc.SetIndent("", "  ")
	enc.Encode(out)
	fmt.Print(buf.String())
}

What we got

Method	Pages	Seconds
Sequential	5	28.86
Parallel	5	4.26
Speedup		6.8x

From the Node.js run on 2026-10-09. IP addresses are replaced with placeholders (203.0.113.x); equal addresses stay equal. The other languages produced the same findings.

Raw output (Node.js)

{
  "sequential_seconds": 28.86,
  "parallel_seconds": 4.26,
  "speedup": "6.8x",
  "results": [
    {
      "url": "https://books.toscrape.com/",
      "title": "All products | Books to Scrape - Sandbox",
      "method": "sequential"
    },
    {
      "url": "https://quotes.toscrape.com/",
      "title": "Quotes to Scrape",
      "method": "sequential"
    },
    {
      "url": "https://example.com",
      "title": "Example Domain",
      "method": "sequential"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "title": "Web scraping - Wikipedia",
      "method": "sequential"
    },
    {
      "url": "https://news.ycombinator.com/",
      "title": "Hacker News",
      "method": "sequential"
    },
    {
      "url": "https://books.toscrape.com/",
      "title": "All products | Books to Scrape - Sandbox",
      "method": "parallel"
    },
    {
      "url": "https://quotes.toscrape.com/",
      "title": "Quotes to Scrape",
      "method": "parallel"
    },
    {
      "url": "https://example.com",
      "title": "Example Domain",
      "method": "parallel"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "title": "Web scraping - Wikipedia",
      "method": "parallel"
    },
    {
      "url": "https://news.ycombinator.com/",
      "title": "Hacker News",
      "method": "parallel"
    }
  ]
}

Takeaways

Tabs overlap the waiting: 2–7× faster in our runs (11–29 s one by one, 2.4–6.6 s in parallel) — the wall time approaches the slowest page.
No extra threads or proxy slots: five tabs are one session — the plan meter counts one thread.
The gain depends on page time, not page count: fast pages gain little; slow ones through a distant proxy gain most.
Don't share Playwright across threads: its Python sync API and its Java API aren't thread-safe (a thread pool fails with Target closed / NoSuchElementException). Use Python's async API, and in Java start every navigation before waiting on any.
Wait for the HTML, not the whole page: with load, one page with a slow asset (18 s through our proxy) made the parallel run as slow as the sequential one.
Close tabs when done — every open page holds memory in the remote browser.