Skip to content
ScrapeField

Caching and freshness

We keep each answer for a while, so asking twice is quick. Every answer says how old it is, and you can always ask for a fresh one.

How old an answer is

Every response carries meta.cached, true when the answer came from our cache, and meta.fetched_at, when the data was actually read from the platform. A cached answer is never presented as new: its fetched_at is the original time.

A cached answer costs the same credits as a fetched one. You pay for the answer, not for how we produced it, so the price of a call never depends on who asked before you.

How long each answer is kept

The same call with the same parameters is answered from the cache until its window ends. After that the answer is deleted from the cache, not just refreshed, and the next call fetches it again. What else we keep, and why, is in Privacy.

An answer that the thing asked for does not exist, such as profile_not_found, is kept too: for 6 hours, or the endpoint’s window if that is shorter. Asked again in that time, it answers at once. Like the first time, it costs nothing.

Forcing a fresh fetch

Add fresh=true to skip the cache and read the platform now. It costs the endpoint’s usual credits, and the new answer replaces the cached one for everyone after you.

const params = new URLSearchParams({
  "username": "nasa",
  "fresh": "true"
});

const res = await fetch(`https://api.scrapefield.com/v1/instagram/profile?${params}`, {
  headers: { Authorization: `Bearer ${process.env.SCRAPEFIELD_KEY}` },
});

const { data, meta } = await res.json();
console.log(data, meta.credits_charged);
Error codes
CodeHTTPWhen, and what to do
fresh_not_available403`fresh=true` on an account that has never bought credits, or in the demo. Trial credits are cache-first. Buy any amount of credits to bypass the cache.