Where Sockt data comes from
Every technology on a Sockt profile comes with the capture it was found in, how it was found and the dates we first and last saw it. This page explains the sources, how detection works, how accurate it is and what it misses.
- Websites analysed
- 26,318,494
- Technologies detected
- 5,850
- Detections stored
- 286,179,438
- Last updated
- 13 October 2026
Detector sockt-2026.10.1
| Source | Captured | Websites |
|---|---|---|
| Common Crawl CC-MAIN-2026-39 | 7 Aug 2026 to 17 Sep 2026 | 23,196,740 |
| SocktBot visit | 12 Oct 2026 to 12 Oct 2026 | 38 |
Sources
Common Crawl
Most profiles come from Common Crawl, a public archive of web pages. The current profiles use crawl CC-MAIN-2026-39, captured between 7 August 2026 and 17 September 2026. For each domain we read the stored homepage, or the page its homepage redirects to on the same site, and run our detector over the saved headers and HTML. Use of the archive follows the Common Crawl terms of use.
Archive captures are usually one to two months old by the time they are served, and each profile shows its capture date. Sites that block Common Crawl's crawler have no archive capture.
SocktBot
SocktBot visits sites directly to fill gaps in the archive and to recheck domains on customers' watchlists. It identifies itself and follows robots.txt. Profiles built from a SocktBot visit name it as the source. How SocktBot works.
DNS
Where a profile shows an email or DNS provider, it comes from the domain's public MX and nameserver records, with the date they were checked.
AI estimates
Company name, description, industry, business type and country are estimated by an AI model from the site's own text. They are labelled as estimates with the date they were made, and they are never used as evidence that a site runs a technology.
Site rank
Site rank comes from the Majestic Million, published by Majestic under CC BY 3.0. For sites outside it, Sockt uses the host rank Common Crawl publishes from its web graph (harmonic centrality). Each profile says which one it shows. Sockt uses rank to choose which domains to analyse first and to order lists.
What dates mean
First observed is the first capture where we saw a technology, not the day the site installed it. Last seen is the latest capture where it was still there.
A failed or blocked check never removes a technology. Only a newer successful analysis of the same site can.
How detection works
Detection rules come from the open WebAppAnalyzer fingerprints, licensed under GPL-3.0, with a small set of Sockt rules added. The detector checks HTTP headers, cookies, meta tags, page HTML, script addresses and page structure, and records which of these matched.
It reads the saved page and does not run its JavaScript. Tags that only appear after a script loads them, such as some analytics and consent tools, can be missed. Each detection carries a confidence from the fingerprint, and a technology implied by another, such as PHP behind WordPress, is marked as implied.
Accuracy
Measured 11 October 2026 to 12 October 2026 on a sample of 10,000 domains. We checked 230 detections by hand against the archived page and headers they came from.
- Detections that were right
- 94.7%
- 95% interval
- 90.9% to 96.9%
- Widely used technologies
- 96.1%
- Versions that were right
- 32 of 34
Widely used technologies means 153 judged detections of platforms such as Shopify, WordPress, Cloudflare and Stripe; 147 were right.
Misses are harder to count. Where a page clearly loaded the tag, Sockt found Google Tag Manager 89% of the time and Google Analytics 82% of the time; every confirmed miss was a tag added by another script.
Coverage: 44.5% of the sample had a usable homepage capture in the crawl, and 53.4% of the domains that appear in the Majestic Million. A domain without a usable capture has no profile until SocktBot visits it.
Attribution
- Web page captures from Common Crawl, used under the Common Crawl terms of use.
- Site rank from the Majestic Million by Majestic, licensed under CC BY 3.0, and from the Common Crawl web graph host ranks.
- Technology fingerprints from WebAppAnalyzer, licensed under GPL-3.0. Technology descriptions on sockt.com come from the same fingerprints.
- Technology logos from Simple Icons (CC0). Names and logos belong to their owners.