Web Scraping
Caching
How result caching works and when to disable it.
PagePry caches scrape results by URL to deliver faster responses and reduce credit costs. Cached responses cost 0 credits.
How it works
- When you scrape a URL with
cache: true(the default), PagePry checks if a cached result exists for that URL - If a cached result is found and still valid, it's returned immediately — no browser is launched
- If no cached result exists, the page is scraped normally and the result is cached for future requests
You can tell whether a response came from cache by checking the metadata.fromCache field:
{
"success": true,
"html": "...",
"metadata": {
"fromCache": true,
"loadTimeMs": 2
}
}Cached responses have extremely low loadTimeMs values (typically under 10ms) since they skip browser rendering entirely.
Disabling cache
Set cache: false to always get a fresh result:
{
"url": "https://example.com",
"cache": false
}Use this when:
- The page content changes frequently and you need the latest version
- You're scraping personalized content (with cookies/headers) that shouldn't be shared across requests
- You're debugging and want to see a fresh render
Cache behavior
- Cache is keyed by the full URL (including query parameters)
- Custom headers and cookies create separate cache entries
- Cache entries expire automatically
- There is no way to manually invalidate cache entries — set
cache: falseto bypass

