CrawlwiseData
Free Tech stack lookup — no key needed

Website change alerts, as signed webhooks.

Tell Crawlwise which sites to watch. We check them on a schedule and POST your app (or AI agent) when something changes — new pages, new posts, or a tech-stack shift. On-demand lookups still work. You pay for changes, not empty checks.

Try the free demo Create free account Get a free trial key

New here? See how it works in 5 steps →

We watch sites on a scheduleYou pay when something changesHMAC-signed webhooks to your app

Billing mistakes get fixed — duplicate charges or credits you paid for but never received. Details.

# Free demo: no key needed
curl -G https://crawlwisedata.com/v1/techstack \
  -d url=wordpress.org

# With your key: only feed items you haven't seen yet
curl -G https://crawlwisedata.com/v1/feed \
  -H "Authorization: Bearer $CRAWLWISE_KEY" \
  -d url=https://github.blog -d monitor=gh-blog
const res = await fetch(
  'https://crawlwisedata.com/v1/techstack?url=wordpress.org',
  { headers: { Authorization: `Bearer ${process.env.CRAWLWISE_KEY}` } },
);
const { ok, data, charged } = await res.json();
console.log(data.technologyNames, charged.credits);
import os, requests

r = requests.get(
    'https://crawlwisedata.com/v1/techstack',
    params={'url': 'wordpress.org'},
    headers={'Authorization': f"Bearer {os.environ['CRAWLWISE_KEY']}"},
    timeout=60,
)
print(r.json()['data']['technologyNames'])
200 OK · real demo response (trimmed)
{
  "ok": true,
  "endpoint": "techstack",
  "mode": "demo",
  "charged": { "credits": 0, "usd": 0 },
  "balance": null,
  "data": {
    "url": "https://wordpress.org/",
    "domain": "wordpress.org",
    "statusCode": 200,
    "technologyCount": 13,
    "technologyNames": [
      "WordPress 7.2", "Gutenberg 24.0.0", "Nginx", "PHP",
      "MySQL", "Google Tag Manager", "HSTS", "RSS", ...
    ],
    "technologies": [
      {
        "name": "Nginx",
        "version": null,
        "confidence": 100,
        "categories": ["Web servers", "Reverse proxies"],
        "implied": false,
        "detectedBy": ["headers"]
      },
      ...
    ],
    "categories": {
      "CMS": ["WordPress"], "Web servers": ["Nginx"], ...
    }
  }
}
What you get
Signed change webhooks
Empty checks
2 credits (not full price)
Failed calls
Always free
Try free
1,000-credit trial key
How it works

Website → Crawlwise → signed webhook → your app

You are not buying “another scrape.” You are buying scheduling, change detection, and reliable delivery so software and AI agents stay informed.

  1. 1. Website

    Point us at a public site, feed, or sitemap you care about.

  2. 2. Crawlwise

    We schedule checks, keep state, detect new/updated/removed pages or tech changes, and skip empty noise.

  3. 3. Signed webhook

    When something changes we POST JSON with an HMAC signature you can verify.

  4. 4. Your app

    Update a dashboard, notify a team, or wake an AI agent — without polling yourself.

You want outcomes

  • Know when a competitor ships a page
  • Catch new posts with scheduled checks, usually within an hour, plus check-now on demand
  • Detect CMS / analytics / CDN shifts
  • Keep agents current without babysitting cron

We handle the hard parts

  • Scheduling & polling
  • State, dedupe & change detection
  • Retries & signed delivery
  • robots.txt & honest crawling

You pay for changes

  • Noop checks: 2 credits (5 for tech)
  • Change found: normal endpoint price
  • Failed / blocked / robots: $0
  • Same prepaid credit balance for lookups
Live demo

Try the real API right here

These requests go to the production endpoints in demo mode: no key, 20 calls per day per IP (also capped per network), smaller result sizes and no monitors. Demo attempts count toward the limit even when they fail. On 429, respect Retry-After (seconds until 00:00 UTC).

1. Pick an endpoint

Detects CMS, frameworks, analytics, CDN, server and DNS providers.

2. Enter a public URL

Quickstart →

Demo calls are free. With a key you get full result sizes, monitors (new items only) and higher rate limits.

GET /v1/techstack?url=wordpress.org
Press “Run request” to call the API. The JSON response appears here.
Free trial

Try it with a real key, free

Get 1,000 free credits on a real API key: full result sizes, named monitors and signed webhooks, no card. One key per email every 30 days; per network up to 5 requests an hour and 10 a day.

Get a free trial key

1,000 free credits, one time (about 33 tech-stack lookups or 100 feed reads). No card. One key per email every 30 days; per network up to 5 requests an hour and 10 a day. We never email you.

Trial limits: 20 requests/min, 5 monitors. If a call costs more than your balance, it's refused and you're charged nothing. Disposable email addresses are refused. We keep only a salted hash of your email (Privacy). Agents: POST /v1/trial with {"email": "…", "acceptTerms": true} (or accept later with POST /v1/terms/accept).

Lookups when you need them · monitors when you want alerts

Three signals. One prepaid balance.

On-demand tech, feed, and sitemap reads — or hand us a webhook and we watch for you. Deterministic code, no AI guessing: every detection shows which signal matched.

Tech-stack detection

Find the CMS, e-commerce platform, frameworks, analytics, CDN, web server and email/DNS providers behind any public page, using 7,600+ open-source fingerprints.

  • Versions and confidence scores
  • Shows which signal matched (headers, HTML, DNS…)
  • Filter by confidence, hide implied tech
GET /v1/techstack$0.003 / page

Feeds, new items only

RSS 2.0, Atom, RDF and JSON Feed, normalized into one schema. Pass a site URL and we discover the feed. Name a monitor and each call returns only items you haven't seen.

  • Keyword include/exclude filters
  • Relative dates like “7 days”
  • Optional full item HTML
GET /v1/feed$0.001 / read

Sitemap & tech change alerts

Every URL a site lists in its XML, gzipped or text sitemaps (found through robots.txt), with lastmod, images and hreflang. Monitors report new, updated and removed pages.

  • Up to 10,000 URLs per call
  • Regex include/exclude filters
  • No false “removed” on truncated results
GET /v1/sitemap$0.001 + $0.10/1k URLs

Read the API reference Free tech stack lookup

Also: free sitemap & feed checker · same demo API as the box above.

For AI agents & automation

OpenAPI, llms.txt, and MCP

Wire Crawlwise into agent stacks without scraping our HTML. Deterministic tools — Crawlwise never runs an LLM on your request.

OpenAPI 3

Import /openapi.json (or YAML) into your client generator. Includes the signed monitor.change webhook event schema.

MCP server

Local stdio package crawlwise-mcp (techstack, feed, sitemap, monitors, account). Install it from npm with npx crawlwise-mcp and add it to Claude Desktop or Cursor. Set CRAWLWISE_API_KEY. See docs.

Get started

From free trial to first change alert

No sales calls. Start with a free key, create a monitor, and receive a signed webhook when something moves. See the 5-step walkthrough.

  1. Get a free trial key

    Enter your email in the trial form for 1,000 credits (or buy a pack/plan). The key is shown once. Lost it? Recover.

  2. Create a monitor

    POST /v1/monitors with a target URL, schedule: true, and your https webhook. We run scheduled checks, usually within an hour, plus check-now on demand.

  3. Verify the webhook

    We POST a signed monitor.change event. Check X-Crawlwise-Signature with your secret from GET /v1/webhooks/secret. Don't want to wait for a change? POST /v1/monitors/test sends a free signed sample now, and our demo target changes every 5 minutes. Copyable examples: docs.

Fair billing

You only pay for results

  • Unreachable sites, blocked pages (HTTP ≥ 400), timeouts, pages disallowed by robots.txt, invalid input and our own errors cost nothing.
  • Credits are prepaid, and calls stop when your balance runs out.
  • Pack credits never expire while your account is active. Starter credits carry over for one billing period, then expire.
  • No surprise charges. Monthly plans renew only while you keep them, and you can cancel in two clicks and keep your plan until the end of the period.
  • Purchases are non-refundable except where law requires (unused credits / change of mind / mid-period cancel ≠ refund). We still fix our mistakes: duplicate charges and credits never delivered. See refund terms.
Responsible crawling

A polite, honest crawler

  • robots.txt is always checked first (RFC 9309). If a site disallows us, the call fails and costs you nothing.
  • Every request identifies itself as CrawlwiseBot with a link to this site.
  • Public data only: we never log in, submit forms, run JavaScript, solve CAPTCHAs or get around bot protection.
  • Site owners can opt out entirely, and we then refuse all requests for their domain.
  • Private networks and internal addresses are blocked, including after DNS resolution and redirects.
Pricing

Prepaid credits. Clear outcomes.

Credits power both lookups and monitors. Empty scheduled checks are cheap (2 credits). Buy once, or subscribe for more watching capacity. 1 credit = $0.0001.

Credit packs

One-time, no renewal

from $10

100,000 credits

≈ 10,000 feed reads, or about 520 days of one monitor at the minimum interval with no changes. Also $25 (262,500 credits, +5%). Never expire while active.

See packs
One plan

Starter

Side projects & small watch lists

$19/mo

250,000 credits / month, about 43 monitors

500 monitors · scheduled checks, usually within an hour, plus check-now on demand

Choose Starter

Need more than Starter? Email Crawlwise Data support — crawlwise.support@gmail.com for high-volume / custom limits — same product, higher caps.

Purchases are non-refundable except where law requires. Cancel stops future renewals — it is not a refund. Duplicate charges and credits never delivered get fixed. Refund terms.

FAQ

Frequently asked questions

Can I try it without paying?

Yes, three ways. Free account: create an account (0 credits) after accepting the Terms, then buy when ready. Free trial key: enter your email in the trial form (or POST /v1/trial) and get a real key with 1,000 one-time credits: full result sizes, monitors and webhooks, no card. That's enough to watch a real change arrive as a signed webhook: monitor our demo target, which changes every 5 minutes (about 50 credits). One per email every 30 days (soft IP/network caps for shared egress); disposable addresses are refused. No key at all: call any endpoint without a key, 20 calls per day per IP, with smaller results (feeds up to 15 items, sitemaps up to 75 URLs) and no monitors, use the demo box above, or the dedicated tech stack lookup / sitemap checker pages.

What exactly counts as a failed (free) call?

Unreachable sites, blocked pages (HTTP ≥ 400), timeouts, pages disallowed by robots.txt, invalid input and our own errors. Every error has the same JSON shape with "billable": false and charged.credits = 0.

What's the difference between packs and plans?

Packs are one-time purchases. Nothing renews, and the credits never expire while your account is active. Plans are monthly subscriptions that renew until you cancel. Each paid month adds credits, and plans give more credits per dollar and higher limits.

How do monitors work?

We run them. POST /v1/monitors with a webhook URL and Crawlwise checks on a schedule, then POSTs a signed event when something changes. Noop checks cost 2 credits (5 for tech). You can still poll with ?monitor=NAME if you prefer. See docs.

How do I cancel a plan?

Dashboard → Manage / cancel subscription opens the Stripe Customer Portal. Cancellation takes effect at the end of the paid period. You keep your plan and credits until then and aren't charged again.

Can I get a refund?

Purchases are non-refundable except where law requires. Canceling stops future renewals — it is not a refund. We still fix duplicate charges and credits you paid for but never received. See the Terms.

I lost my API key. What now?

Use key recovery. You prove you own your checkout email through Stripe's billing portal and we issue a new key (old keys are revoked). We can never show an existing key again because we only store a hash.

Do you support AI agents / MCP?

Yes. Use OpenAPI, llms.txt, or the local crawlwise-mcp MCP server (stdio). Install it with npx crawlwise-mcp. Crawlwise stays deterministic — no AI at runtime. Details: docs.

Can I call the API from browser JavaScript?

Only the free demo and public endpoints. Keyed endpoints intentionally don't send CORS headers, so your key stays on your server. Call the API from your backend.

More answers in Support & FAQ.

Start free today

Get a free trial key (1,000 credits) and create your first monitor, or try the free demo with no key. When you need continuous watching, pick a pack or plan.

Get a free trial key See pricing