ParseForge Scrapers

Tech Jobs Substack Newsletter Aggregator

parseforge/tech-jobs-newsletter-substack-aggregator-scraper

JobsBusinessNews & media

Aggregates tech job posts from a list of Substack newsletter RSS feeds. Returns each post with its title, body, link, date, and source newsletter, filtered by keyword and lookback days.

Run this scraper See the API call
Total users
2
Monthly active
1
Total runs
42
Bookmarked
0
Rating
Not rated yet
Last modified
12 days ago

Overview

ParseForge

Tech Jobs Substack Newsletter Aggregator

Aggregate tech job posts from multiple Substack newsletters into one feed. Each post comes with its full text, publication date, and source newsletter. Filter by keyword and date range, then export to CSV, JSON, Excel, or XML.

Scouring individual Substack newsletters for tech job leads wastes hours. This actor reads the public RSS feeds of multiple tech-focused newsletters at once, extracts every post, and lets you filter them by a job title or company keyword. You get a single, deduplicated dataset of opportunities without reading a dozen inboxes.

Who uses it What they scrape Substack for
Job seekers in tech Monitor multiple niche job boards and newsletters for new openings in one place.
Recruiters Track which companies are hiring and what roles are being advertised across the tech newsletter ecosystem.
Market analysts Identify hiring trends by aggregating job posts from influential tech publications.
Newsletter curators Source and repackage the latest job listings from across the tech community.

What it does

This Actor collects posts from a list of Substack newsletter RSS feeds and returns each matching post as a flat row with its title, body, link, publication date, and source newsletter.

  • ๐Ÿ“ฐ Multi-source aggregation: Feed in any number of Substack newsletter RSS URLs and scrape them all in a single run.
  • ๐Ÿ” Keyword filtering: Supply a job title like 'backend engineer' or a company name like 'OpenAI' to return only matching posts.
  • ๐Ÿ“… Date windowing: Set a lookback period in days to ignore old posts and see only recent opportunities.
  • โš™๏ธ Flexible output: Cap the total number of posts returned and download the results as CSV, JSON, Excel, or XML.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Substack data

๐Ÿ’ผ Build a personal job feed.

A software engineer adds a dozen tech-job newsletters and sets a keyword for 'remote' to get a daily CSV of relevant openings.

๐Ÿ“Š Track hiring velocity.

A VC analyst aggregates posts from founder newsletters and filters by portfolio company names to gauge team growth.

๐Ÿ“ฌ Curate a weekly job roundup.

A newsletter author scrapes multiple sources, filters for senior roles, and republishes the best finds for their own subscribers.

๐ŸŽฏ Source passive candidates.

A recruiter monitors niche engineering newsletters for 'hiring' posts to identify companies with active reqs.

Why choose this scraper

What you get
One feed for all newsletters Combine job posts from The Pragmatic Engineer, Lenny's Newsletter, and any other Substack into a single dataset.
Keyword match on title and body Filter for specific roles, skills, or company names so you only see relevant posts.
Time-boxed results Limit posts to the last 7, 14, or 30 days to keep your dataset fresh and manageable.

How it compares

No other Store actor targets Substack the same way, so the honest comparison is with the alternatives teams actually weigh.

Tech Jobs Substack Newsletter Aggregator Build it in-house By hand
Setup Run it now, zero config Days of engineering None, but hours per pull
When Substack changes Maintained for you You fix it You re-learn the page
Proxies, retries, anti-bot Built in Your problem Browser only
Output Fixed JSON schema, CSV/Excel export Whatever you build Copy-paste
Cost Pay per result Engineering time Analyst hours

Configure the run

Drive the Actor from a list of Substack newsletter RSS feed URLs, then narrow results with an optional keyword filter and a lookback window in days. The Input tab lists every parameter.

A first run with the defaults:

{
 "newsletters": [
 "https://www.jobhuntingsux.com/feed",
 "https://nonlinearnews.com/feed",
 "https://beyondbayst.substack.com/feed",
 "https://aitidbits.substack.com/feed",
 "https://thepragmaticengineer.substack.com/feed",
 "https://lennysnewsletter.substack.com/feed"
 ],
 "lookbackDays": 30,
 "maxItems": 10
}

A larger pull:

{
 "newsletters": [
 "https://www.jobhuntingsux.com/feed",
 "https://nonlinearnews.com/feed",
 "https://beyondbayst.substack.com/feed",
 "https://aitidbits.substack.com/feed",
 "https://thepragmaticengineer.substack.com/feed",
 "https://lennysnewsletter.substack.com/feed"
 ],
 "lookbackDays": 30,
 "maxItems": 200
}

Pricing

Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.

Results collected Approximate cost
100 results $2.10
1,000 results $21.00
10,000 results $210.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

  1. Create a free Apify account with $5 in credit.
  2. Open the Tech Jobs Substack Newsletter Aggregator.
  3. Set your inputs and any filters, then click Start.
  4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Substack through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/tech-jobs-newsletter-substack-aggregator-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your newsletter URLs end in /feed and are publicly accessible. Also verify that your keyword filter is not too restrictive and that the lookback days window covers the publication dates of the posts you expect.

Why are some posts missing the full body text?

Some Substack authors configure their RSS feeds to show only a summary or excerpt. The actor returns whatever the feed provides. There is no workaround for feeds that truncate content.

The run timed out. What should I do?

Reduce the number of newsletter URLs or lower the max items limit. If you are scraping many feeds with long histories, consider splitting them across multiple runs.

I get an error about an invalid feed URL.

Confirm the URL loads in a browser with /feed at the end. Some newsletters use a custom domain; make sure you are using the correct Substack subdomain.

Why do I see duplicate posts?

If the same job is cross-posted to multiple newsletters in your list, it will appear once per feed. Deduplicate in your spreadsheet or database after export.

FAQ

Question Answer
What is a Substack newsletter RSS feed URL? It is the web address that ends in /feed for a Substack publication. For example, https://thepragmaticengineer.substack.com/feed. You can find it by adding /feed to the newsletter's homepage URL.
Can I add newsletters that are not on Substack? This actor is built for Substack RSS feeds. It may work with other RSS feeds that share the same structure, but non-Substack feeds are not officially supported.
How does the keyword filter work? It performs a case-insensitive substring search across the post title and body text. A keyword like 'iOS' will match 'iOS Engineer', 'Senior iOS Developer', and any post mentioning iOS in the body.
What happens if I set lookback days to 0? Setting lookback days to 0 disables the date filter entirely. The actor will return all available posts from the feeds, subject to the max items limit.
Is there a limit on how many newsletters I can add? There is no hard limit on the number of newsletter URLs. The run time will increase with each additional feed, but you can add as many as you need.
What does the free preview include? Free users can scrape up to 10 newsletter posts per run to test the output. Upgrade to a paid plan to increase the limit up to 1,000,000 posts.
Does this actor require a Substack login or API key? No. It reads the public RSS feeds that every Substack newsletter provides. No login, API key, or authentication is needed.
Can I get the full body text of each post? Yes. The actor extracts the full HTML or text body of each post as it appears in the RSS feed.
How often should I run this actor? You can schedule it daily or weekly. A daily run with a 1-day lookback captures every new post without duplicates.
What export formats are supported? You can export your results to CSV, JSON, Excel, or XML from the dataset tab after the run completes.

Related actors

Browse the full ParseForge collection for more scrapers.

๐Ÿ†˜ Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

โš ๏ธ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Substack, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

Input

FieldTypeWhat it doesDefault
newsletters array Substack-powered newsletter feed URLs to aggregate. Each URL should be the RSS endpoint (typically /feed). ["https://www.jobhuntingsux.com/feed","h
lookbackDays integer Only include posts published within the last N days. Set 0 to skip date filtering. 30
keyword string Filter by job title or company keyword (case-insensitive substring match). Leave empty for all. not set
maxItems integer Free users: Limited to 10 newsletter posts (preview). Paid users: Optional, max 1,000,000 10

Pricing

from $19.00 per 1,000 results

Charged forWhat it isPrice each
result Single result in the default dataset. $0.019 to $0.021

Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.

API

One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.

POST ยท run and get results
curl -X POST "https://api.apify.com/v2/acts/parseforge~tech-jobs-newsletter-substack-aggregator-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "helloWorld": 123
  }'

Examples

Input that runs as-is.

input.json
{
  "helloWorld": 123
}

Reviews

No reviews yet. Be the first.

Issues

We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.

Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.

Related scrapers

Run Tech Jobs Substack Newsletter Aggregator on Apify All scrapers