[Examples & benchmarks](https://cdpfleet.com/docs/cases) / #30 · 2026-10-09 · Example · Easy · for [web scraping](https://cdpfleet.com/web-scraping)

# Five pages at once: parallel tabs in one browser session

Open five unrelated pages concurrently in one session and compare with loading them one by one — same thread, same proxy, a fraction of the wall time.

[Chromium](https://cdpfleet.com/docs/browsers/chromium)

Run on production on 2026-10-21: ✓ Node.js ✓ Python ✓ Java ✓ C# ✓ Go

## The problem

You need something from several unrelated pages. The obvious loop opens one, extracts, closes, and moves on — and most of each wait is latency through the proxy while the browser sits idle. A session can hold many tabs, and each language has its own way to wait on several at once. How much does that save, and what does it cost?

## What we used, and why

| What | Why |
| --- | --- |
| `chromium`, `headless: "new"` | One session (1 thread) for all five pages. |
| Five independent URLs | books.toscrape.com, quotes.toscrape.com, example.com, a Wikipedia article and the Hacker News front page: public, no login, safe to load at once. |
| `browser.newPage()` per URL | Each tab has its own navigation; all share the session's proxy and connection. |
| `Promise.all` · `asyncio.gather` · `Task.WhenAll` · goroutines | Each language's own way to wait on five loads at once. Java starts all five navigations and then waits for each — no threads. |
| `waitUntil: "domcontentloaded"` | Wait for the HTML, not every image and script: one slow asset would otherwise set the pace. |
| `page.title()` | The smallest proof a page loaded; put your extraction here. |

## How it works

1.  Launch one Chromium session.
2.  Sequential: open each URL in a new tab, read its title, close it — timed.
3.  Parallel: open all five at once, read the titles, close the tabs — timed.
4.  Do both three times, alternating, and keep each method's best time: a single page that stalls in the proxy (we saw 14–28 s now and then) would otherwise decide one comparison on its own.
5.  Print both timings, the speedup, every round and every title.

## The code

The same program in five languages (also [on GitHub](https://github.com/cdpfleet/cdpfleet/tree/main/cases/30-parallel-pages), with the raw output). Set these environment variables first:

-   `CDPFLEET_API_KEY` — your API key (dashboard → API keys)
-   `PROXY_URL` — your proxy, e.g. `http://user:pass@proxy.example.com:8000`

 

```node
// npm install playwright@1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL
import { chromium } from 'playwright';

const KEY = process.env.CDPFLEET_API_KEY;
const URLS = [
  'https://books.toscrape.com/',
  'https://quotes.toscrape.com/',
  'https://example.com',
  'https://en.wikipedia.org/wiki/Web_scraping',
  'https://news.ycombinator.com/',
];

const res = await fetch('https://starter.cdpfleet.com/chromium/session', {
  method: 'POST',
  headers: { 'x-api-key': KEY, 'content-type': 'application/json' },
  body: JSON.stringify({ proxy: process.env.PROXY_URL, headless: 'new' }),
});
if (!res.ok) throw new Error(`launch ${res.status} ${await res.text()}`);
const { wsUrl } = await res.json();
const browser = await chromium.connect(wsUrl, { headers: { 'x-api-key': KEY } });

async function extract(browser, url) {
  // One retry, in a fresh tab: over 30 loads through a rotating proxy, one occasionally fails.
  for (let attempt = 1; ; attempt++) {
    const page = await browser.newPage();
    try {
      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
      return { url, title: await page.title() };
    } catch (err) {
      if (attempt === 2) throw err;
    } finally {
      await page.close();
    }
  }
}

try {
  // Three rounds of each, alternating; the best time of each method counts. One page that
  // stalls for 20 s in the proxy would otherwise decide a single comparison on its own.
  const rounds = [];
  let sequential = [];
  let parallel = [];
  for (let round = 0; round < 3; round++) {
    const t1 = Date.now();
    sequential = [];
    for (const url of URLS) {
      sequential.push({ ...await extract(browser, url), method: 'sequential' });
    }
    const seqSeconds = Math.round((Date.now() - t1) / 10) / 100;

    const t2 = Date.now();
    parallel = await Promise.all(URLS.map((url) => extract(browser, url).then((r) => ({ ...r, method: 'parallel' }))));
    const parSeconds = Math.round((Date.now() - t2) / 10) / 100;
    rounds.push({ sequential_seconds: seqSeconds, parallel_seconds: parSeconds });
  }
  const seqSeconds = Math.min(...rounds.map((r) => r.sequential_seconds));
  const parSeconds = Math.min(...rounds.map((r) => r.parallel_seconds));

  console.log(JSON.stringify({
    sequential_seconds: seqSeconds,
    parallel_seconds: parSeconds,
    speedup: `${Math.round(seqSeconds / parSeconds * 10) / 10}x`,
    rounds,
    results: [...sequential, ...parallel],
  }, null, 2));
} finally {
  await browser.close();
}
```

```python
# pip install playwright==1.60.0 requests
# env: CDPFLEET_API_KEY, PROXY_URL
# Playwright's sync API isn't thread-safe, so the parallel loads use the async API.
import asyncio
import json
import os
import time

import requests
from playwright.async_api import async_playwright

KEY = os.environ["CDPFLEET_API_KEY"]
URLS = [
    "https://books.toscrape.com/",
    "https://quotes.toscrape.com/",
    "https://example.com",
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://news.ycombinator.com/",
]

async def main():
    res = requests.post("https://starter.cdpfleet.com/chromium/session",
        headers={"x-api-key": KEY},
        json={"proxy": os.environ["PROXY_URL"], "headless": "new"}, timeout=60)
    if not res.ok:
        raise SystemExit(f"launch {res.status_code} {res.text}")

    async with async_playwright() as p:
        browser = await p.chromium.connect(res.json()["wsUrl"], headers={"x-api-key": KEY})
        try:
            async def extract(url):
                # One retry, in a fresh tab: over 30 loads through a rotating proxy, one occasionally fails.
                for attempt in (1, 2):
                    page = await browser.new_page()
                    try:
                        await page.goto(url, wait_until="domcontentloaded", timeout=60000)
                        return {"url": url, "title": await page.title()}
                    except Exception:
                        if attempt == 2:
                            raise
                    finally:
                        await page.close()

            # Three rounds of each, alternating; the best time of each method counts. One page that
            # stalls for 20 s in the proxy would otherwise decide a single comparison on its own.
            rounds = []
            for _ in range(3):
                t1 = time.time()
                sequential = [{**(await extract(url)), "method": "sequential"} for url in URLS]
                seq = round(time.time() - t1, 2)

                t2 = time.time()
                parallel = [{**r, "method": "parallel"} for r in await asyncio.gather(*(extract(url) for url in URLS))]
                par = round(time.time() - t2, 2)
                rounds.append({"sequential_seconds": seq, "parallel_seconds": par})
            seq_seconds = min(r["sequential_seconds"] for r in rounds)
            par_seconds = min(r["parallel_seconds"] for r in rounds)

            print(json.dumps({
                "sequential_seconds": seq_seconds,
                "parallel_seconds": par_seconds,
                "speedup": f"{round(seq_seconds / par_seconds, 1)}x",
                "rounds": rounds,
                "results": sequential + parallel,
            }, indent=2, ensure_ascii=False))
        finally:
            await browser.close()

asyncio.run(main())
```

```java
// Maven: com.microsoft.playwright:playwright:1.60.0, com.google.code.gson:gson:2.11.0
// Run with PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1. env: CDPFLEET_API_KEY, PROXY_URL
import com.google.gson.*;
import com.microsoft.playwright.*;
import com.microsoft.playwright.options.WaitUntilState;
import java.net.URI;
import java.net.http.*;
import java.util.*;
import java.util.concurrent.*;

public class Main {
  static final String KEY = System.getenv("CDPFLEET_API_KEY");
  static final Gson GSON = new GsonBuilder().serializeNulls().create();
  static final String[] URLS = {
      "https://books.toscrape.com/",
      "https://quotes.toscrape.com/",
      "https://example.com",
      "https://en.wikipedia.org/wiki/Web_scraping",
      "https://news.ycombinator.com/",
  };

  record Result(String url, String title, String method) {}

  static Result extract(Browser browser, String url) {
    // One retry, in a fresh tab: over 30 loads through a rotating proxy, one occasionally fails.
    for (int attempt = 1; ; attempt++) {
      Page page = browser.newPage();
      try {
        page.navigate(url, new Page.NavigateOptions().setWaitUntil(WaitUntilState.DOMCONTENTLOADED).setTimeout(60000));
        return new Result(url, page.title(), null);
      } catch (PlaywrightException e) {
        if (attempt == 2) throw e;
      } finally {
        page.close();
      }
    }
  }

  public static void main(String[] args) throws Exception {
    String body = "{\"proxy\": " + GSON.toJson(System.getenv("PROXY_URL")) + ", \"headless\": \"new\"}";
    HttpResponse<String> res = HttpClient.newHttpClient().send(HttpRequest.newBuilder(URI.create("https://starter.cdpfleet.com/chromium/session"))
        .header("x-api-key", KEY).header("content-type", "application/json")
        .POST(HttpRequest.BodyPublishers.ofString(body)).build(), HttpResponse.BodyHandlers.ofString());
    if (res.statusCode() != 200) throw new RuntimeException("launch: " + res.statusCode() + " " + res.body());
    String wsUrl = JsonParser.parseString(res.body()).getAsJsonObject().get("wsUrl").getAsString();

    try (Playwright playwright = Playwright.create()) {
      Browser browser = playwright.chromium().connect(wsUrl, new BrowserType.ConnectOptions().setHeaders(Map.of("x-api-key", KEY)));
      try {
        // Three rounds of each, alternating; the best time of each method counts. One page that
        // stalls for 20 s in the proxy would otherwise decide a single comparison on its own.
        JsonArray rounds = new JsonArray();
        List<Result> sequential = new ArrayList<>();
        List<Result> parallel = new ArrayList<>();
        double seqSeconds = Double.MAX_VALUE;
        double parSeconds = Double.MAX_VALUE;
        for (int round = 0; round < 3; round++) {
          long t1 = System.currentTimeMillis();
          sequential = new ArrayList<>();
          for (String url : URLS) {
            Result r = extract(browser, url);
            sequential.add(new Result(r.url, r.title, "sequential"));
          }
          double seq = Math.round((System.currentTimeMillis() - t1) / 10.0) / 100.0;

          // Playwright for Java isn't thread-safe: start all five navigations, then wait for each.
          long t2 = System.currentTimeMillis();
          List<Page> pages = new ArrayList<>();
          for (String url : URLS) {
            Page page = browser.newPage();
            page.evaluate("u => { location.href = u; }", url);
            pages.add(page);
          }
          parallel = new ArrayList<>();
          for (int i = 0; i < URLS.length; i++) {
            Page page = pages.get(i);
            page.waitForURL(u -> !u.startsWith("about:"), new Page.WaitForURLOptions().setWaitUntil(WaitUntilState.DOMCONTENTLOADED).setTimeout(60000));
            parallel.add(new Result(URLS[i], page.title(), "parallel"));
            page.close();
          }
          double par = Math.round((System.currentTimeMillis() - t2) / 10.0) / 100.0;
          JsonObject r = new JsonObject();
          r.addProperty("sequential_seconds", seq);
          r.addProperty("parallel_seconds", par);
          rounds.add(r);
          seqSeconds = Math.min(seqSeconds, seq);
          parSeconds = Math.min(parSeconds, par);
        }

        JsonObject out = new JsonObject();
        out.addProperty("sequential_seconds", seqSeconds);
        out.addProperty("parallel_seconds", parSeconds);
        out.addProperty("speedup", Math.round(seqSeconds / parSeconds * 10) / 10.0 + "x");
        out.add("rounds", rounds);
        JsonArray results = new JsonArray();
        for (Result r : sequential) results.add(GSON.toJsonTree(r));
        for (Result r : parallel) results.add(GSON.toJsonTree(r));
        out.add("results", results);

        System.out.println(new GsonBuilder().serializeNulls().setPrettyPrinting().disableHtmlEscaping().create().toJson(out));
      } finally {
        browser.close();
      }
    }
  }
}
```

```csharp
// dotnet add package Microsoft.Playwright --version 1.60.0
// env: CDPFLEET_API_KEY, PROXY_URL
using System.Diagnostics;
using System.Net.Http.Json;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Playwright;

string[] urls = [
    "https://books.toscrape.com/",
    "https://quotes.toscrape.com/",
    "https://example.com",
    "https://en.wikipedia.org/wiki/Web_scraping",
    "https://news.ycombinator.com/",
];

var key = Environment.GetEnvironmentVariable("CDPFLEET_API_KEY")!;
using var http = new HttpClient();
http.DefaultRequestHeaders.Add("x-api-key", key);
var res = await http.PostAsJsonAsync("https://starter.cdpfleet.com/chromium/session",
    new { proxy = Environment.GetEnvironmentVariable("PROXY_URL"), headless = "new" });
if (!res.IsSuccessStatusCode) throw new Exception($"launch: {(int)res.StatusCode} {await res.Content.ReadAsStringAsync()}");
var wsUrl = (await res.Content.ReadFromJsonAsync<JsonElement>()).GetProperty("wsUrl").GetString()!;

using var playwright = await Playwright.CreateAsync();
var browser = await playwright.Chromium.ConnectAsync(wsUrl, new() { Headers = new Dictionary<string, string> { ["x-api-key"] = key } });
try
{
    async Task<JsonObject> Extract(string url)
    {
        // One retry, in a fresh tab: over 30 loads through a rotating proxy, one occasionally fails.
        for (var attempt = 1; ; attempt++)
        {
            var page = await browser.NewPageAsync();
            try
            {
                await page.GotoAsync(url, new() { WaitUntil = WaitUntilState.DOMContentLoaded, Timeout = 60000 });
                return new JsonObject { ["url"] = url, ["title"] = await page.TitleAsync() };
            }
            catch (PlaywrightException) when (attempt == 1)
            {
            }
            finally
            {
                await page.CloseAsync();
            }
        }
    }

    // Three rounds of each, alternating; the best time of each method counts. One page that
    // stalls for 20 s in the proxy would otherwise decide a single comparison on its own.
    var rounds = new JsonArray();
    var sequential = new List<JsonObject>();
    JsonObject[] parallelResults = [];
    double seqSeconds = double.MaxValue, parSeconds = double.MaxValue;
    for (var round = 0; round < 3; round++)
    {
        var sw1 = Stopwatch.StartNew();
        sequential = new List<JsonObject>();
        foreach (var url in urls)
        {
            var r = await Extract(url);
            r["method"] = "sequential";
            sequential.Add(r);
        }
        var seq = Math.Round(sw1.Elapsed.TotalSeconds, 2);

        var sw2 = Stopwatch.StartNew();
        var tasks = urls.Select(async url =>
        {
            var r = await Extract(url);
            r["method"] = "parallel";
            return r;
        }).ToArray();
        parallelResults = await Task.WhenAll(tasks);
        var par = Math.Round(sw2.Elapsed.TotalSeconds, 2);
        rounds.Add(new JsonObject { ["sequential_seconds"] = seq, ["parallel_seconds"] = par });
        seqSeconds = Math.Min(seqSeconds, seq);
        parSeconds = Math.Min(parSeconds, par);
    }

    var results = new JsonArray();
    foreach (var r in sequential) results.Add(r);
    foreach (var r in parallelResults) results.Add(r);

    var output = new JsonObject
    {
        ["sequential_seconds"] = seqSeconds,
        ["parallel_seconds"] = parSeconds,
        ["speedup"] = $"{Math.Round(seqSeconds / parSeconds, 1)}x",
        ["rounds"] = rounds,
        ["results"] = results,
    };
    Console.WriteLine(output.ToJsonString(new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
}
finally
{
    await browser.CloseAsync();
}
```

```go
// go get github.com/playwright-community/playwright-go@v0.6000.0
// Driver: build playwright-core 1.60.0 from npm and set PLAYWRIGHT_DRIVER_PATH (see /docs/quickstart).
// env: CDPFLEET_API_KEY, PROXY_URL
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"math"
	"net/http"
	"os"
	"sync"
	"time"

	"github.com/playwright-community/playwright-go"
)

var key = os.Getenv("CDPFLEET_API_KEY")

var urls = []string{
	"https://books.toscrape.com/",
	"https://quotes.toscrape.com/",
	"https://example.com",
	"https://en.wikipedia.org/wiki/Web_scraping",
	"https://news.ycombinator.com/",
}

func launch(name string, options map[string]any) (map[string]any, error) {
	body, _ := json.Marshal(options)
	req, _ := http.NewRequest("POST", "https://starter.cdpfleet.com/"+name+"/session", bytes.NewReader(body))
	req.Header.Set("x-api-key", key)
	req.Header.Set("content-type", "application/json")
	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != http.StatusOK {
		msg, _ := io.ReadAll(res.Body)
		return nil, fmt.Errorf("launch %s: %s %s", name, res.Status, msg)
	}
	var session map[string]any
	return session, json.NewDecoder(res.Body).Decode(&session)
}

type result struct {
	URL    string `json:"url"`
	Title  string `json:"title"`
	Method string `json:"method"`
}

func extract(browser playwright.Browser, url string) (string, string) {
	// One retry, in a fresh tab: over 30 loads through a rotating proxy, one occasionally fails.
	for attempt := 1; ; attempt++ {
		title, err := load(browser, url)
		if err == nil {
			return url, title
		}
		if attempt == 2 {
			log.Fatal(err)
		}
	}
}

func load(browser playwright.Browser, url string) (string, error) {
	page, err := browser.NewPage()
	if err != nil {
		return "", err
	}
	defer page.Close()
	if _, err := page.Goto(url, playwright.PageGotoOptions{WaitUntil: playwright.WaitUntilStateDomcontentloaded, Timeout: playwright.Float(60000)}); err != nil {
		return "", err
	}
	return page.Title()
}

func main() {
	session, err := launch("chromium", map[string]any{"proxy": os.Getenv("PROXY_URL"), "headless": "new"})
	if err != nil {
		log.Fatal(err)
	}
	pw, err := playwright.Run(&playwright.RunOptions{SkipInstallBrowsers: true})
	if err != nil {
		log.Fatal(err)
	}
	defer pw.Stop()
	browser, err := pw.Chromium.Connect(session["wsUrl"].(string), playwright.BrowserTypeConnectOptions{Headers: map[string]string{"x-api-key": key}})
	if err != nil {
		log.Fatal(err)
	}
	defer browser.Close()

	// Three rounds of each, alternating; the best time of each method counts. One page that
	// stalls for 20 s in the proxy would otherwise decide a single comparison on its own.
	type timing struct {
		SequentialSeconds float64 `json:"sequential_seconds"`
		ParallelSeconds   float64 `json:"parallel_seconds"`
	}
	var rounds []timing
	var sequential, parallel []result
	seqSeconds, parSeconds := math.MaxFloat64, math.MaxFloat64
	for round := 0; round < 3; round++ {
		t1 := time.Now()
		sequential = nil
		for _, u := range urls {
			url, title := extract(browser, u)
			sequential = append(sequential, result{URL: url, Title: title, Method: "sequential"})
		}
		seq := math.Round(time.Since(t1).Seconds()*100) / 100

		t2 := time.Now()
		parallel = make([]result, len(urls))
		var wg sync.WaitGroup
		for i, u := range urls {
			wg.Add(1)
			go func(i int, u string) {
				defer wg.Done()
				url, title := extract(browser, u)
				parallel[i] = result{URL: url, Title: title, Method: "parallel"}
			}(i, u)
		}
		wg.Wait()
		par := math.Round(time.Since(t2).Seconds()*100) / 100
		rounds = append(rounds, timing{seq, par})
		seqSeconds = math.Min(seqSeconds, seq)
		parSeconds = math.Min(parSeconds, par)
	}

	all := append(sequential, parallel...)
	out := struct {
		SequentialSeconds float64  `json:"sequential_seconds"`
		ParallelSeconds   float64  `json:"parallel_seconds"`
		Speedup           string   `json:"speedup"`
		Rounds            []timing `json:"rounds"`
		Results           []result `json:"results"`
	}{
		SequentialSeconds: seqSeconds,
		ParallelSeconds:   parSeconds,
		Speedup:           fmt.Sprintf("%.1fx", math.Round(seqSeconds/parSeconds*10)/10),
		Rounds:            rounds,
		Results:           all,
	}
	var buf bytes.Buffer
	enc := json.NewEncoder(&buf)
	enc.SetEscapeHTML(false)
	enc.SetIndent("", "  ")
	enc.Encode(out)
	fmt.Print(buf.String())
}
```

## What we got

| Method | Pages | Seconds |
| --- | --- | --- |
| Sequential | 5 | 12.77 |
| Parallel | 5 | 9.39 |
| Speedup |  | 1.4x |

From the Node.js run on 2026-10-21. IP addresses are replaced with placeholders (203.0.113.x); equal addresses stay equal. The other languages produced the same findings.

Raw output (Node.js)

```
{
  "sequential_seconds": 12.77,
  "parallel_seconds": 9.39,
  "speedup": "1.4x",
  "rounds": [
    {
      "sequential_seconds": 16.61,
      "parallel_seconds": 30.7
    },
    {
      "sequential_seconds": 15.14,
      "parallel_seconds": 9.39
    },
    {
      "sequential_seconds": 12.77,
      "parallel_seconds": 13.3
    }
  ],
  "results": [
    {
      "url": "https://books.toscrape.com/",
      "title": "All products | Books to Scrape - Sandbox",
      "method": "sequential"
    },
    {
      "url": "https://quotes.toscrape.com/",
      "title": "Quotes to Scrape",
      "method": "sequential"
    },
    {
      "url": "https://example.com",
      "title": "Example Domain",
      "method": "sequential"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "title": "Web scraping - Wikipedia",
      "method": "sequential"
    },
    {
      "url": "https://news.ycombinator.com/",
      "title": "Hacker News",
      "method": "sequential"
    },
    {
      "url": "https://books.toscrape.com/",
      "title": "All products | Books to Scrape - Sandbox",
      "method": "parallel"
    },
    {
      "url": "https://quotes.toscrape.com/",
      "title": "Quotes to Scrape",
      "method": "parallel"
    },
    {
      "url": "https://example.com",
      "title": "Example Domain",
      "method": "parallel"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "title": "Web scraping - Wikipedia",
      "method": "parallel"
    },
    {
      "url": "https://news.ycombinator.com/",
      "title": "Hacker News",
      "method": "parallel"
    }
  ]
}
```

## Takeaways

-   **Tabs overlap the waiting:** 1.4–4.4× faster in the 2026-10-21 runs, taking each method's best of three rounds (6.9–12.8 s one by one, 1.7–9.4 s in parallel) — the wall time approaches the slowest page.
-   **…which is also the catch:** in 3 of the 15 rounds one page stalled in the proxy (up to 30 s) and the parallel batch came out slower than loading one by one. Give each load a timeout and a retry, and compare methods over several rounds, not one.
-   **No extra threads or proxy slots:** five tabs are one session — the plan meter counts one thread.
-   **The gain depends on page time, not page count:** fast pages gain little; slow ones through a distant proxy gain most.
-   **Don't share Playwright across threads:** its Python sync API and its Java API aren't thread-safe (a thread pool fails with `Target closed` / `NoSuchElementException`). Use Python's async API, and in Java start every navigation before waiting on any.
-   **Wait for the HTML, not the whole page:** with `load`, one page with a slow asset (18 s through our proxy) made the parallel run as slow as the sequential one.
-   **Close tabs when done** — every open page holds memory in the remote browser.

---
Source: https://cdpfleet.com/docs/cases/parallel-pages · Markdown: https://cdpfleet.com/docs/cases/parallel-pages.md · All docs for AI tools: https://cdpfleet.com/llms.txt