ParseForge
Hacker News Stories Scraper
parseforge/hackernews-stories-scraper
News & mediaSocial mediaOther
Scrapes Hacker News stories by category and returns each story as a flat row with title, URL, score, author, and comment count.
- Total users
- 1
- Monthly active
- 0
- Total runs
- 98
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 9 days ago
Overview
Hacker News Stories Scraper
Scrape Hacker News stories from any category, up to a million per run. Each story comes with its title, URL, score, author, and comment count. No login or API key. Export to CSV, JSON, Excel, or XML.
Hacker News has no official API for its story lists, and scraping it yourself means handling rate limits and HTML parsing. This reads the public story feeds directly, filtered by category, and returns each match in one fixed schema.
| Who uses it | What they scrape Hacker News for |
|---|---|
| Market researchers | Which topics the tech community is discussing this week |
| Product managers | What features or pain points developers mention most |
| Content marketers | Which headlines and domains get the most traction |
| Data analysts | A clean dataset of stories for trend analysis |
| Job seekers | The latest job postings from Y Combinator startups |
What it does
This Actor collects Hacker News stories by category and returns each one as a flat row.
- 📊 Computed signals: story score, comment count, and rank position.
- 🔍 Category filters: top, new, best, ask, show, and job stories.
- ⚡ Fast collection: up to a million stories per run, no API key needed.
- 📁 Multiple exports: CSV, JSON, Excel, and XML.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with Hacker News data
📈 Track topics and trends.
A market researcher runs the Actor daily on the top stories category to see which technologies and companies are gaining attention.
🧠 Understand developer pain points.
A product manager scrapes Ask HN stories to identify common problems developers face and prioritize feature development.
📰 Curate content for a newsletter.
A content marketer collects the best stories each morning and selects the most relevant ones for a daily tech newsletter.
💼 Find startup jobs.
A job seeker scrapes job stories to get a list of current openings at Y Combinator startups.
Why choose this scraper
| What you get | |
|---|---|
| No API key | Scrape public story feeds without registration or OAuth. |
| Fixed schema | Every story returns the same fields, ready for analysis. |
| High volume | Collect up to a million stories in a single run. |
| Flexible categories | Top, new, best, ask, show, and job stories. |
How it compares
This Actor focuses on story lists with a simple category selector and high volume, while the competitors below offer additional features like comments or multi-source aggregation.
| Feature | ParseForge | Hacker News Data | Trends Aggregator |
|---|---|---|---|
| Story categories (top, new, best, ask, show, job) | Yes | Yes | Not listed |
| Comments scraping | Not listed | Yes | Not listed |
| Date range scanning | Not listed | Yes | Not listed |
| Multi-source aggregation (Reddit, GitHub, etc.) | Not listed | Not listed | Yes |
| No API key required | Yes | Not listed | Yes |
| Up to 1,000,000 stories per run | Yes | Not listed | Not listed |
Configure the run
Drive the Actor from story categories, and set a maximum number of stories to collect per run. The Input tab lists every parameter.
A first run with the defaults:
{
"maxItems": 10
}
A larger pull:
{
"maxItems": 200
}
Pricing
Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the Hacker News Stories Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to Hacker News through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/hackernews-stories-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check that the story category you selected has stories. For example, job stories may have fewer items than top stories. Also ensure your maximum stories is set to at least 1.
Why is the run taking so long?
Collecting a very large number of stories, such as 1,000,000, will take time. Try reducing the maximum stories or running more frequently with smaller batches.
Why are some stories missing fields?
Occasionally Hacker News may not provide all fields for a story. The Actor returns whatever is available, so some fields may be empty.
Can I get stories from a specific date range?
This Actor does not support date filtering. It returns the current stories in the selected category. For historical data, consider the Hacker News API or another Actor.
FAQ
| Question | Answer |
|---|---|
| Do I need an API key to use this Actor? | No. It reads the public Hacker News story feeds directly, so no registration or API key is required. |
| What story categories can I scrape? | You can choose from top, new, best, ask, show, and job stories. |
| How many stories can I collect in one run? | You can set the maximum number of stories, up to 1,000,000 per run. |
| What data does each story include? | Each story includes its title, URL, score, author, comment count, and rank position. |
| Can I export the data to Excel? | Yes. You can export to CSV, JSON, Excel, or XML. |
| Is this Actor legal to use? | Yes. It only accesses publicly available data from Hacker News. You should still review Hacker News' terms of service for your use case. |
| How often should I run this Actor? | It depends on your needs. Many users run it daily or hourly to track changes in story rankings. |
| Can I scrape comments as well? | This Actor focuses on stories only. For comments, consider a different Actor or the official Hacker News API. |
| What is the difference between top, new, and best stories? | Top stories are the current front page, new stories are the most recent submissions, and best stories are the highest voted in recent days. |
| Does this Actor handle pagination automatically? | Yes. It will keep fetching stories until it reaches your maximum number or the category is exhausted. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Y Combinator. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| maxItems | integer | Maximum number of stories to collect per run. | 10 |
| storyType | string (6 options) | Story Type | top |
Pricing
from $19.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| result | Single result in the default dataset. | $0.019 to $0.021 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~hackernews-stories-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.