Cookies & Headers

You can pass custom HTTP headers and browser cookies to scrape content that requires authentication or specific request context.

Custom headers

Pass a headers object to set HTTP headers on the request:

{
  "url": "https://example.com",
  "headers": {
    "Accept-Language": "en-US",
    "Referer": "https://google.com"
  }
}

Common use cases:

• Accept-Language — request content in a specific language
• Referer — some sites serve different content based on the referrer
• User-Agent — PagePry uses a realistic browser User-Agent by default, but you can override it

Custom cookies

Pass a cookies array to set browser cookies before the page loads:

{
  "url": "https://example.com/dashboard",
  "cookies": [
    {
      "name": "session_id",
      "value": "abc123def456",
      "domain": ".example.com"
    },
    {
      "name": "preferences",
      "value": "lang=en",
      "domain": ".example.com"
    }
  ]
}

Each cookie requires three fields:

Full example

curl -X POST https://api.pagepry.com/v1/scrape \
  -H "x-api-key: pp_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/account",
    "cookies": [
      {"name": "session", "value": "tok_abc123", "domain": ".example.com"}
    ],
    "headers": {
      "Accept-Language": "en-US"
    },
    "cache": false
  }'

Tips

• Disable caching when scraping with cookies — personalized content shouldn't be served from cache. Set cache: false.
• Use the correct domain — cookies are only sent to pages on the matching domain. Use a leading . (e.g., .example.com) to match all subdomains.
• Session cookies expire — if your session cookie has a short TTL, grab a fresh one before scraping.