Puppet Forge Modules Scraper

Puppet Forge Modules Scraper

Scrape Puppet Forge modules by keyword, up to a million per run. Each module comes with its name, summary, tags, and release metadata. No login or API key. Export to CSV, JSON, Excel, or XML.

Puppet Forge's official API needs an account and rate-limits you. This reads the public module list directly, filtered by keyword, and returns each match in one fixed schema.

Who uses it	What they scrape Puppet Forge for
DevOps engineers	Which Puppet modules exist for a tool they need to manage
IT automation consultants	A list of modules to recommend for a client's infrastructure
Security researchers	Which modules are published for a specific software package
Market researchers	How many modules exist for a technology category

What it does

This Actor collects Puppet Forge modules by keyword and returns each one as a flat row.

• 🔍 Keyword search: find modules by name, summary, or tags, for example nginx, apache, mysql.
• 📦 Full catalog: leave the keyword empty to browse the complete module list.
• ⚙️ Max items control: set a hard cap from 1 to 1,000,000 modules per run.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Puppet Forge data

🧰 Build a module inventory.

A DevOps engineer searches for 'nginx' and exports every matching module to CSV to compare options.

📊 Track module popularity.

A market researcher collects all modules for a category and sorts by release date to spot trends.

🔎 Audit available automation.

A security researcher searches for a specific software name to see which modules exist before choosing one.

📋 Recommend modules to clients.

An IT consultant runs a keyword search and shares the JSON export with a client as a shortlist.

Why choose this scraper

	What you get
No API key	Reads the public module list directly, no registration or OAuth.
One fixed schema	Every module returns the same flat row, ready for analysis.
Any volume	Collect one module or a million, the cap is yours.

How it compares

This Actor focuses on keyword search and a simple flat schema, while the competitors below offer different metadata or export options.

Feature	ParseForge	Puppet Forge Modules Scraper	Puppet Forge Scraper
Keyword search	Yes	Not listed	Not listed
Full catalog browsing	Yes	Not listed	Not listed
Max items control	Yes	Not listed	Not listed
Module name and summary	Yes	Yes	Not listed
Release metadata	Yes	Yes	Not listed
Versions and downloads	Not listed	Not listed	Yes
OS compatibility and dependencies	Not listed	Not listed	Yes

Configure the run

Drive the Actor from a keyword search, and the max items cap runs as modules are read so only the first N matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10
}

A larger pull:

{
 "maxItems": 200
}

Pricing

Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$2.10
1,000 results	$21.00
10,000 results	$210.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Puppet Forge Modules Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Puppet Forge through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/puppet-forge-modules-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Your keyword may be too specific or misspelled. Try a broader term or leave the field empty to browse the full list.

Why did the run stop before reaching my max items?

The Actor stops when there are no more matching modules. If you expected more, try a different keyword or remove the keyword to get the full catalog.

Why are some module fields empty?

Puppet Forge does not always provide every field for every module. Empty values mean the data was not available on the public page.

Can I get more metadata like downloads or dependencies?

This Actor collects the public module list fields. For deeper metadata, you would need a scraper that calls the Puppet Forge API endpoints directly.

FAQ

Question	Answer
Do I need a Puppet Forge account or API key?	No. This Actor reads the public module list directly, so no registration or authentication is required.
What does each module record include?	Each record includes the module name, summary, tags, and release metadata as returned by Puppet Forge. The exact fields are shown in the sample output.
Can I scrape the entire Puppet Forge catalog?	Yes. Leave the keyword field empty and set a high max items value to collect the full list, up to one million modules per run.
How do I search for a specific module?	Enter a keyword in the search query field. The Actor matches against module name, summary, and tags.
What export formats are supported?	You can export the results as CSV, JSON, Excel, or XML from the Apify dataset.
Is there a limit on how many modules I can collect?	You set the limit yourself with the max items field, from 1 to 1,000,000 modules per run.
Does this Actor get module download counts or dependencies?	The Actor returns the fields available on the public module list. For deeper metadata like downloads or dependencies, consider a dedicated Puppet Forge API scraper.
Can I schedule this Actor to run regularly?	Yes, you can schedule it in Apify to run daily, weekly, or at any interval you need.
What if my keyword returns no results?	Check the spelling and try a broader term. The search matches name, summary, and tags, so a more general keyword may help.
Is this Actor free to use?	You can run it on the Apify free plan with a limited number of results. For larger runs, you need a paid plan.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Puppet, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.