RFC Editor Index Scraper

RFC Editor Index Scraper

Scrape RFC documents from the RFC Editor index by number, keyword, status, or stream, up to a million per run. Each RFC comes with its number, title, authors, publication date, status, stream, page count, and abstract. No API key required. Export to CSV, JSON, Excel, or XML.

The RFC Editor publishes every numbered Internet standards document, but browsing the index manually or hitting the site for bulk analysis is slow. This Actor reads the public RFC Editor index directly, filters by publication status, stream, or keyword, and returns each matching RFC in one fixed schema. You can also fetch specific RFCs by number for targeted extraction.

Who uses it	What they scrape RFC Editor for
Network engineers	Pull the latest PROPOSED STANDARD RFCs to review protocol changes.
Technical writers	Gather all INFORMATIONAL RFCs on a topic for a reference document.
Compliance auditors	Extract every INTERNET STANDARD RFC to verify current obligations.
Computer science researchers	Build a dataset of RFC abstracts and statuses for citation analysis.

What it does

This Actor collects RFC metadata from the RFC Editor index and returns each document as a flat row with its number, title, authors, status, stream, and abstract.

• 🔢 Direct lookup: fetch specific RFCs by number, e.g. 791, 2616, 9110, skipping the index browse.
• 🔍 Keyword search: filter the index by RFC number, title, or keyword to find documents on a topic.
• 📋 Status and stream filters: narrow results to INTERNET STANDARD, PROPOSED STANDARD, INFORMATIONAL, or any other publication status, and by IETF, IAB, IRTF, or Independent stream.
• 📄 Full metadata per RFC: number, title, authors, publication date, status, stream, page count, and abstract.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with RFC Editor data

📡 Monitor new protocol standards.

A network engineer runs the Actor weekly with status set to PROPOSED STANDARD and stream IETF to catch every new protocol document and update internal specs.

📚 Build an RFC knowledge base.

A technical writer fetches all INFORMATIONAL RFCs containing 'HTTP' in the title to compile a historical reference of web protocol evolution.

🔎 Audit compliance against Internet Standards.

A compliance auditor extracts every INTERNET STANDARD RFC and cross-references the list against the organization's implemented protocols.

📊 Analyze RFC publication trends.

A researcher scrapes the full index with no filters, then charts RFC counts by year, stream, and status to study standardization patterns.

Why choose this scraper

	What you get
No API key or registration	Reads the public RFC Editor index with no authentication.
Up to a million RFCs per run	Collect the entire index or a filtered subset in one execution.
Fixed flat schema	Every RFC returns the same fields: number, title, authors, date, status, stream, pages, abstract.
Multiple export formats	Save results as CSV, JSON, Excel, or XML for offline analysis.

How it compares

No other Store actor targets RFC Editor the same way, so the honest comparison is with the alternatives teams actually weigh.

	RFC Editor Index Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When RFC Editor changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor from specific RFC numbers, a keyword search, or the full index, and filters run as each RFC is read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10
}

A larger pull:

{
 "maxItems": 200
}

Pricing

Pay-per-result: $0.01467 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.47
1,000 results	$14.67
10,000 results	$146.70

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the RFC Editor Index Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to RFC Editor through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/rfc-editor-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your filters are not too restrictive. Try clearing the search query, status, and stream, and set maxItems to a small number to test. If you provided RFC numbers, verify they are valid integers.

The run finished but the dataset is smaller than maxItems.

The Actor stops when it has read all RFCs matching your filters. If the index has fewer matching documents than maxItems, you will get all of them. Increase maxItems only if you need a larger cap.

I entered RFC numbers but the Actor still browsed the index.

Make sure the RFC Numbers field is a valid JSON array of integers, e.g. [791, 2616]. If the field is empty or invalid, the Actor falls back to the index browse with your other filters.

The abstract field is empty for some RFCs.

Some older or very short RFCs may not have an abstract in the index. This is expected. The Actor returns an empty string for those documents.

FAQ

Question	Answer
Can I scrape all RFCs ever published?	Yes. Set maxItems to a high number and leave all filters empty to collect the entire index. The Actor will return every RFC listed on the RFC Editor site.
How do I get only the latest Internet Standards?	Set the status filter to INTERNET STANDARD and leave the search query and stream filters empty. The Actor returns every RFC with that status in the index.
Does this Actor download the full RFC text?	No. It collects metadata from the index: number, title, authors, publication date, status, stream, page count, and abstract. The full RFC text is not included.
Can I fetch a specific set of RFCs by number?	Yes. Use the RFC Numbers field to list the exact RFCs you want, e.g. [791, 2616, 9110]. When you provide numbers, the Actor skips the index browse and fetches those directly.
What is the difference between status and stream?	Status is the publication maturity level like PROPOSED STANDARD or INFORMATIONAL. Stream is the originating body like IETF, IAB, or IRTF. You can filter by one, both, or neither.
Does the keyword search look inside the RFC text?	No. The search query filters RFCs whose title or number contains your keyword. It does not search the full document body or abstract.
How many RFCs can I collect in one run?	You can set maxItems up to 1,000,000. The actual number depends on how many RFCs match your filters in the index.
Do I need an API key or login?	No. The Actor reads the public RFC Editor index pages directly. No registration, API key, or authentication is required.
What output formats are supported?	You can export the dataset as CSV, JSON, Excel, or XML from the Apify platform after the run completes.
Can I filter by author name?	The input schema does not include an author filter. You can fetch the full index and filter authors in your own post-processing, or use the keyword search if the author name appears in the title.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by RFC Editor. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.