ParseForge
News Across the Web Scraper
parseforge/news-across-the-web-scraper
News & mediaAutomationLead generation
Scrapes news articles from Semafor, Axios, Medium, The Intercept, The Verge, TechCrunch, and Wired. Returns each article as a flat row with title, summary, source, URL, and publication date. Filter by search term and set a maximum number of articles.
- Total users
- 2
- Monthly active
- 1
- Total runs
- 94
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 9 days ago
Overview
News Across the Web Scraper
Scrape news articles from Semafor, Axios, Medium, The Intercept, The Verge, TechCrunch, and Wired in one run. Filter by search term and collect up to a million articles. Export to CSV, JSON, Excel, or XML.
News sites each have their own layout, feeds, and rate limits. This actor reads the public article feeds from seven major sources at once, filters them by a search term you provide, and returns every match in one fixed schema. No API keys, no per-site setup, no pagination code.
| Who uses it | What they scrape News Across the Web for |
|---|---|
| Market researchers | Which topics a niche is discussing this week |
| Journalists | What competing outlets are covering on a story |
| PR teams | Where a brand or executive is mentioned across tech media |
| Data analysts | Building a clean dataset of recent articles for trend analysis |
What it does
This Actor collects news articles from the selected sources, filters them by an optional search term, and returns each article as a flat row.
- 🔎 Search term filter: case-insensitive substring match on title and summary, so only relevant articles reach your dataset.
- 📰 Seven sources in one run: Semafor, Axios, Medium (Towards Data Science), The Intercept, The Verge, TechCrunch, and Wired.
- 📦 Flat row output: every article is one record with title, summary, source, URL, and publication date.
- ⚙️ Max articles control: set a hard cap from 1 to 1,000,000 per run to stay within your storage budget.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with News Across the Web data
📈 Track topics and trends.
A market researcher runs the actor weekly with a search term like 'generative AI' across all seven sources to see which angles are gaining traction.
🗞️ Monitor competitor coverage.
A journalist sets the search term to a breaking story and collects articles from The Verge, TechCrunch, and Wired to compare framing and sources.
📊 Build a media mention dataset.
A PR team runs the actor with their company name and exports the results to CSV for a monthly coverage report.
🧪 Feed a research pipeline.
A data analyst collects all articles from Medium (Towards Data Science) and filters by 'machine learning' to create a clean dataset for a literature review.
Why choose this scraper
| What you get | |
|---|---|
| One schema for seven sites | No per-site parsing or cleanup, every article arrives in the same flat format |
| No API keys or logins | Reads public feeds directly, so you can start a run immediately |
| Search term filter built in | Only articles matching your term are returned, saving storage and processing time |
| Scales to a million articles | Set maxItems as high as you need for large-scale collection |
How it compares
No other Store actor targets News Across the Web the same way, so the honest comparison is with the alternatives teams actually weigh.
| News Across the Web Scraper | Build it in-house | By hand | |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When News Across the Web changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |
Configure the run
Drive the Actor from a search term and a list of sources, and the filter runs as each article is read so only matches reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"maxItems": 10
}
A larger pull:
{
"maxItems": 200
}
Pricing
Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the News Across the Web Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to News Across the Web through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/news-across-the-web-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check that your search term is spelled correctly and is not too specific. Also verify that at least one source is selected in the sources array. If the term is very niche, try broadening it or removing it entirely.
Why are some sources returning fewer articles than others?
Each source has a different feed size and update frequency. Some sources may only expose a limited number of recent articles in their public feeds.
Why did my run stop before reaching maxItems?
The actor stops when it has read all available articles from the selected sources. If the total number of articles across those sources is less than maxItems, the run ends early.
Can I get the full text of an article?
No, this actor only returns the title and summary from the source feeds. To get full text, you would need a different actor that fetches individual article pages.
Why is my search term not matching articles I can see on the site?
The filter is applied to the title and summary fields only. If the term appears only in the article body or in metadata not included in the feed, it will not match.
FAQ
| Question | Answer |
|---|---|
| Which news sources does this actor scrape? | It scrapes Semafor, Axios, Medium (Towards Data Science), The Intercept, The Verge, TechCrunch, and Wired. You can select any subset of these sources in the input. |
| Do I need API keys or logins for these sites? | No. The actor reads the public article feeds directly, so no API keys, OAuth, or account registration is required. |
| How does the search term filter work? | The query input is a case-insensitive substring filter applied to the article title and summary. Only articles containing the term are returned. |
| Can I scrape all articles from a source without a search term? | Yes. Leave the query field empty and the actor will return all articles it can find from the selected sources, up to the maxItems limit. |
| What is the maximum number of articles I can collect? | You can set maxItems up to 1,000,000 articles per run. The default is 10. |
| What format is the output data in? | Each article is returned as a flat row with fields like title, summary, source, URL, and publication date. You can export the dataset to CSV, JSON, Excel, or XML. |
| Does this actor scrape full article text? | No, it returns the title and summary as shown in the source feeds. It does not fetch the full body text of each article. |
| Can I schedule this actor to run automatically? | Yes, you can schedule runs on Apify with any interval, such as hourly or daily, to keep your dataset up to date. |
| Is this actor free to use? | The actor itself is free to run on the Apify platform. You only pay for the platform compute units you consume, and the free tier includes a monthly allowance. |
| Can I filter by publication date? | No, the current input schema does not include a date filter. The actor returns the most recent articles available from each source feed. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by News Across the Web. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| query | string | Optional case-insensitive substring filter on title/summary. | not set |
| sources | array | Which news sources to query. | ["semafor","axios","medium","theintercep |
| maxItems | integer | How many articles to collect per run. | 10 |
Pricing
from $19.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| result | Single result in the default dataset. | $0.019 to $0.021 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~news-across-the-web-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.