PagePry
Web Scraping

Caching

How result caching works and when to disable it.

PagePry caches scrape results by URL to deliver faster responses and reduce credit costs. Cached responses cost 0 credits.

How it works

  1. When you scrape a URL with cache: true (the default), PagePry checks if a cached result exists for that URL
  2. If a cached result is found and still valid, it's returned immediately — no browser is launched
  3. If no cached result exists, the page is scraped normally and the result is cached for future requests

You can tell whether a response came from cache by checking the metadata.fromCache field:

{
  "success": true,
  "html": "...",
  "metadata": {
    "fromCache": true,
    "loadTimeMs": 2
  }
}

Cached responses have extremely low loadTimeMs values (typically under 10ms) since they skip browser rendering entirely.

Disabling cache

Set cache: false to always get a fresh result:

{
  "url": "https://example.com",
  "cache": false
}

Use this when:

  • The page content changes frequently and you need the latest version
  • You're scraping personalized content (with cookies/headers) that shouldn't be shared across requests
  • You're debugging and want to see a fresh render

Cache behavior

  • Cache is keyed by the full URL (including query parameters)
  • Custom headers and cookies create separate cache entries
  • Cache entries expire automatically
  • There is no way to manually invalidate cache entries — set cache: false to bypass