Substack Publication Scraper

Substack Publication Scraper

Scrape Substack publications for posts, authors, and engagement data, up to a million per run. Each post comes with its title, subtitle, publication date, likes, comments, and full text. No login or API key. Export to CSV, JSON, Excel, or XML.

Substack's official API requires an account and limits access to your own publication. This actor reads public publication pages directly, so you can collect posts from any Substack you specify, filtered by post type and sorted by recency, popularity, or community picks. It returns each post as a flat row with consistent fields.

Who uses it	What they scrape Substack for
Content marketers	Monitor competitor newsletters for topic ideas and publishing frequency.
Journalists	Track what writers on a beat are publishing and how readers respond.
Researchers	Build a dataset of Substack posts for analysis of trends or discourse.
Newsletter operators	Benchmark their own engagement against similar publications.

What it does

This actor collects Substack posts from specified publications and returns each post as a flat row with metadata and content.

• 📰 Publication targeting: Provide one or more Substack subdomains or full URLs, like lex or https://www.platformer.news.
• 🔀 Post type filter: Choose all, newsletter, podcast, thread, or discussion to narrow results.
• 📊 Sorting options: Order posts by new (newest first), top (most liked), or community (community picks).
• 📦 Bulk collection: Set a maximum number of publications to scrape per run, up to 1,000,000.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Substack data

📈 Track competitor newsletters.

A content marketer enters a list of rival Substack subdomains, scrapes all newsletter posts sorted by new, and reviews titles and engagement to spot trending topics.

📰 Monitor a beat.

A journalist scrapes posts from several writers covering a niche, filters to newsletter type, and checks recent posts daily to stay current.

🔍 Research public discourse.

An academic researcher collects posts from a set of publications, exports the dataset, and analyzes text and engagement metrics for a study.

📊 Benchmark engagement.

A newsletter operator scrapes their own publication and a few peers, compares likes and comments per post, and adjusts content strategy.

Why choose this scraper

	What you get
No API key needed	Scrape public Substack pages without registration or authentication.
Flexible input	Use subdomains or full URLs, and mix multiple publications in one run.
Structured output	Get consistent fields for every post, ready for analysis or export.
Scalable	Collect up to a million publications per run, limited only by your plan.

How it compares

No other Store actor targets Substack the same way, so the honest comparison is with the alternatives teams actually weigh.

	Substack Publication Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When Substack changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the actor from a list of Substack publications, then filter by post type and sort order to control which posts are returned. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10,
  "publications": [
    "lex",
    "stratechery"
  ],
  "sortBy": "new",
  "postType": "all"
}

A larger pull:

{
  "maxItems": 200,
  "publications": [
    "lex",
    "stratechery"
  ],
  "sortBy": "new",
  "postType": "all"
}

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Substack Publication Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Substack through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/substack-publication-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that the publication subdomain or URL is correct and publicly accessible. Also ensure your postType filter is not too restrictive; try all first.

The actor is slow or times out.

Reduce maxItems or split large lists into multiple runs. Substack pages can be slow to load, so consider increasing the run timeout in Apify settings.

Some posts are missing from the output.

The actor only scrapes public posts. If a post is behind a paywall or deleted, it will not appear. Also verify the sort order; new may not include older posts if pagination is limited.

I get an error about invalid input.

Ensure publications are provided as subdomains (e.g., lex) or full URLs (e.g., https://www.platformer.news). Remove any trailing slashes or extra characters.

FAQ

Question	Answer
Do I need a Substack account or API key?	No. This actor scrapes public Substack pages directly, so no login or key is required.
Can I scrape multiple publications in one run?	Yes. Provide a list of subdomains or URLs in the input, and the actor will process each one.
What post types can I filter by?	You can choose all, newsletter, podcast, thread, or discussion using the postType input.
How do I sort the posts?	Use the sortBy input: new for newest first, top for most liked, or community for community picks.
What data does each post include?	Each post returns fields like title, subtitle, publication date, likes, comments, and full text. See the sample output for the complete list.
Can I export the results?	Yes. You can export to CSV, JSON, Excel, or XML from the Apify dataset.
Is there a limit on how many posts I can scrape?	You can set maxItems up to 1,000,000 publications per run, subject to your Apify plan limits.
Does this actor handle pagination?	Yes, it automatically paginates through the publication's post list to collect all available posts up to your maxItems.
Can I scrape a publication that requires a paid subscription?	Only public posts are accessible. Paid or subscriber-only content will not be returned.
What if a publication URL is invalid?	The actor will skip invalid entries and continue with the rest. Check the run log for details.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Substack, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.