Search engines, AI answer engines, RSS readers, shopping catalogs, and link preview systems can discover the same public catalog facts through HTML, structured data, feeds, sitemap, robots, and llms.txt.
Crawler access does not receive special ranking content: public HTML, structured data, feeds, sitemap, robots, and llms.txt describe the same products, policies, and educational resources shown to users.
Discovery files
- Sitemap index: /sitemap-index.xml
- Robots policy: /robots.txt
- AI-readable map: /llms.txt
- Product feeds: /feeds/products.json, /feeds/products.xml, /feeds/products.csv