How to find broken pages and redirects in your XML sitemap

Updated 2026-10-10 · Tool: Sitemap Scraper - All URLs from XML Sitemaps + Status Check · $0.20 per 1,000 URLs ($0.50 per 1,000 status checks); sites without a sitemap are free

Search engines expect a sitemap to list only live, canonical URLs. After migrations or product deletions, sitemaps often contain 404s and redirected URLs, which waste crawl budget and show up as errors in Search Console.

This tutorial checks every URL in a sitemap and gives you a CSV of the problems.

Open the tool on Apify →

Step by step

Open the Sitemap Scraper and enter your domain (or the sitemap URL).
Turn on checkStatus. URLs disallowed by robots.txt are skipped by default.
Run it, then filter the dataset for finalStatusCode 400 and above, or rows with a non-empty redirectChain.
To get the rows into Google Sheets: download the run's dataset as CSV and import it, or, for automatic updates, add an integration on the actor's Integrations tab (Apify's Google Sheets Import & Export actor, Make, Zapier or n8n) so every finished run appends its rows to your sheet.
Alternatively, in Google Sheets use =IMPORTDATA("https://api.apify.com/v2/acts/tinyrex~sitemap-scraper/runs/last/dataset/items?format=csv&status=SUCCEEDED&token=YOUR_TOKEN"). Anyone with edit access to the sheet can see the token, so only do this in private sheets (or use a scoped token).
Add a Schedule in Apify (daily or weekly) so the sheet stays fresh.

Example input

{
  "startUrls": [
    "https://crawlee.dev"
  ],
  "checkStatus": true,
  "maxUrlsPerSite": 5000
}

Same thing from the command line (returns CSV directly):

curl -X POST 'https://api.apify.com/v2/acts/tinyrex~sitemap-scraper/run-sync-get-dataset-items?format=csv' -H 'Authorization: Bearer $APIFY_TOKEN' -H 'Content-Type: application/json' -d '{"startUrls": ["https://crawlee.dev"], "checkStatus": true, "maxUrlsPerSite": 5000}' > results.csv

More code examples (Python, JavaScript, MCP for AI agents) are in the full guide.

FAQ

Will it overload my server?

No. It sends one request per URL with low concurrency per site (3 by default, adjustable).

Does it crawl links on the pages?

No, it checks only the URLs listed in the sitemap.

Try it on Apify →