This site tests how crawlers handle content freshness (changing pages with proper HTTP caching headers) and whether they can discover API endpoints hidden in various locations.
Content changes every hour. Uses Last-Modified and
supports If-Modified-Since conditional requests.
Content changes every day. Uses ETag and supports
If-None-Match conditional requests.
Content changes on every request. Uses
Cache-Control: no-store.
An API endpoint is referenced in app.js. Crawlers that parse JavaScript should find it.
An API endpoint is hidden in an HTML comment on this page. Crawlers that inspect comments should find it.
An API endpoint is listed in sitemap.xml. Crawlers that process sitemaps should find it.
An API endpoint is referenced via a Link HTTP header
on this page. Crawlers that inspect headers should find it.