Hacker News guide
How to scrape data from Hacker News in 2026
Hacker News combines submitted links, self-posts, scores, authors, and discussion metadata across several public page types. Normalizing those pages gives agents a predictable post, profile, or bounded feed while retaining both the discussion URL and submitted source.
Quick answer
Use the public page URL you already have.
Send the ordinary public Hacker News URL to webstractor.com. Choose JSON for stable fields or Markdown when an AI model will read the result directly. The API uses GET, so an identical successful request can be served from Cloudflare’s edge cache.
curl --get 'https://api.webstractor.com/v1/extract' \
--data-urlencode 'url=https://news.ycombinator.com/item?id=1' \
--data-urlencode 'format=json'Available data
What you can extract
- Story titles, authors, dates, scores, and comment counts
- Discussion URLs and submitted-page URLs when available
- Public profiles with karma and up to 20 recent story submissions
- Up to 30 stories from supported listings or story-search pages
AI workflows
Where normalized data helps
- Technical trend research
- Startup and developer-news monitoring
- Source discovery for RAG pipelines
- Public story and author analysis
AI-ready output
Markdown for models. JSON for systems.
Raw HTML consumes tokens on navigation, scripts, styling, and interface labels. Clean Markdown keeps the readable hierarchy for LLM prompts and RAG chunks. Normalized JSON is better when your application needs an explicit semantic type, source, author, publication date, media, attributes, and collection items.
Always retain the canonical URL from the response. AI-generated summaries should remain traceable to the public source, especially when the underlying page can change.
Boundaries
Public data only
- Comment bodies, comment search, and complete discussion trees are not included.
- Story feeds are capped at 30 items and profiles contain a bounded recent-submission view.
- Voting, posting, accounts, crawling, complete user history, and media downloads are unavailable.
webstractor.com does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.
Ready to try it?