ParseForge
MetaCPAN Perl Modules Scraper
parseforge/metacpan-perl-scraper
Developer toolsAutomationOther
Scrapes MetaCPAN for Perl module metadata by keyword or Lucene query. Returns version, author, release date, and distribution fields as flat rows.
- Total users
- 2
- Monthly active
- 1
- Total runs
- 56
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 9 days ago
Overview
MetaCPAN Perl Modules Scraper
Scrape MetaCPAN Perl modules by keyword or query, up to a million per run. Each module comes with its version, author, release date, and distribution metadata. No API key or login. Export to JSON, CSV, Excel, or XML.
MetaCPAN's search API is capable but returns nested JSON that needs cleaning before you can use it. This Actor queries the public MetaCPAN index directly, flattens each module into one row, and lets you filter to the latest releases only. Feed it a module name like JSON or Moose, a Lucene query, or leave it empty to list the newest uploads.
| Who uses it | What they scrape MetaCPAN for |
|---|---|
| Perl developers | Finding the current version and author of a module before adding it to a project |
| DevOps engineers | Auditing which Perl distributions are installed across an environment |
| Security researchers | Checking module release dates and authors for supply chain risk |
| Data analysts | Building a dataset of Perl module metadata for trend analysis |
What it does
This Actor collects Perl module metadata from MetaCPAN by keyword or Lucene query, and returns each module as a flat row with its version, author, release date, and distribution fields.
- 🔍 Keyword or Lucene query: search by module name, author, or any indexed field.
- 🆕 Latest releases only: toggle to return only the current version of each distribution.
- 📦 Flat rows: every module is one record, ready for CSV or a database.
- ⚡ Up to 1,000,000 modules per run: collect the whole index or a focused subset.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with MetaCPAN data
📦 Audit Perl dependencies.
A DevOps engineer runs the Actor with a list of module names to confirm the installed versions and authors before a production release.
🔎 Discover new modules.
A Perl developer searches for a keyword like 'JSON' to see the most recently released modules and their metadata.
📊 Track release trends.
A data analyst collects all latest releases weekly to chart which distributions are updated most often.
🛡️ Check supply chain risk.
A security researcher pulls release dates and authors for a set of modules to spot abandoned or suspicious packages.
Why choose this scraper
| What you get | |
|---|---|
| No API key | Queries the public MetaCPAN endpoint directly, no registration or token. |
| Flat output | Nested API responses are flattened into one row per module. |
| Latest only | Filter to the current release of each distribution with one checkbox. |
| Large scale | Collect up to a million modules in a single run. |
How it compares
This Actor focuses on flat, query-driven module metadata with a simple latest-only filter, while the competitors below add dependency, documentation, or exact-name lookup features.
| Feature | ParseForge | MetaCPAN Perl Module Scraper - CPAN Distributions & Deps | MetaCPAN Modules Scraper - Perl Package Data | CPAN Module Scraper |
|---|---|---|---|---|
| Keyword or Lucene query search | Yes | Not listed | Yes | Yes |
| Latest releases only filter | Yes | Not listed | Not listed | Not listed |
| Dependency information | Not listed | Yes | Not listed | Not listed |
| Documentation fields | Not listed | Not listed | Yes | Not listed |
| Fetch by exact module name | Not listed | Not listed | Not listed | Yes |
| Export to JSON, CSV, Excel | Yes | Yes | Not listed | Not listed |
Configure the run
Drive the Actor with a module name, keyword, or raw Lucene query, and set latestOnly to true to skip older releases. maxItems caps how many modules are returned. The Input tab lists every parameter.
A first run with the defaults:
{
"query": "JSON",
"maxItems": 10
}
A larger pull:
{
"query": "JSON",
"maxItems": 200
}
Pricing
Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the MetaCPAN Perl Modules Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to MetaCPAN through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/metacpan-perl-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check your query syntax. If you are using a Lucene field query, make sure the field name and value are correct. Also try a simpler keyword like 'JSON' or leave the query empty to list latest releases.
Why are some modules missing?
If latestOnly is true, only the current release of each distribution is returned. Set it to false to include older releases. Also check maxItems; if it is too low, the run stops early.
Why does the run take so long?
Large maxItems values mean more API requests. MetaCPAN rate limits may slow down the Actor. Reduce maxItems or narrow your query.
Why do I get an error about the query?
MetaCPAN uses Lucene syntax. If your query has special characters, escape them or use a simple keyword. The Actor passes the query string directly to the API.
FAQ
| Question | Answer |
|---|---|
| Do I need a MetaCPAN API key? | No. This Actor uses the public MetaCPAN API endpoint, which does not require authentication. |
| What is a Lucene query? | MetaCPAN uses Lucene query syntax. You can pass a simple keyword like 'Moose' or a field query like 'author:OALDERS AND status:latest'. |
| What does 'latestOnly' do? | When set to true, the Actor only returns releases whose status is 'latest', meaning the current version of each distribution. Older releases are skipped. |
| How many modules can I scrape in one run? | You can set maxItems up to 1,000,000. The actual number returned depends on how many modules match your query. |
| Can I scrape all modules on MetaCPAN? | Yes. Leave the query empty and set maxItems high enough. The Actor will list the latest releases from the index. |
| What fields are returned for each module? | Each row includes the module name, distribution, version, author, release date, and other metadata available from the MetaCPAN API. |
| Can I export the results to CSV? | Yes. Apify supports exporting your dataset to JSON, CSV, Excel, XML, and other formats. |
| Is this Actor free to use? | The Actor itself is free to run on the Apify platform. You only pay for the compute resources your runs consume, based on your Apify plan. |
| How do I search by author? | Use a Lucene field query in the query input, for example 'author:OALDERS'. |
| Can I get only the latest version of each distribution? | Yes, set latestOnly to true. This is the default behavior. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by MetaCPAN. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| query | string | Lucene query passed to MetaCPAN. Use a keyword (for example JSON, Moose, HTTP) or a raw field query. Leave empty to list the latest releases. | JSON |
| latestOnly | boolean | Only include releases whose status is 'latest' (the current version of each distribution). | true |
| maxItems | integer | How many modules to collect per run. | 10 |
Pricing
from $19.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| result | Single result in the default dataset. | $0.019 to $0.021 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~metacpan-perl-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.