ParseForge
Energy.gov Scraper
parseforge/energy-gov-scraper
OtherAutomationNews & media
Scrapes Energy.gov articles and content by search term, article type, office, and language. Returns a flat dataset with title, URL, date, and content fields.
- Total users
- 1
- Monthly active
- 0
- Total runs
- 238
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 12 days ago
Overview
Energy.gov Scraper
Scrape Energy.gov articles, press releases, and documents by keyword, office, or type, up to a million per run. Each result comes with its title, URL, date, and content fields. No login or API key. Export to CSV, JSON, Excel, or XML.
Energy.gov publishes thousands of articles, press releases, and technical documents across dozens of DOE offices, but there is no public API for bulk access. This Actor reads the public search results directly, filtered by keyword, article type, office, and language, and returns each match in one fixed schema.
| Who uses it | What they scrape Energy.gov for |
|---|---|
| Energy policy analysts | Track DOE announcements and funding opportunities across offices |
| Market researchers | Monitor clean energy technology trends and program updates |
| Journalists | Find press releases and remarks for energy sector reporting |
| Grant writers | Discover new funding opportunities and success stories |
| Academic researchers | Collect documents and reports for literature reviews |
What it does
This Actor collects Energy.gov articles and content by search term, article type, office, and language, and returns each one as a flat row.
- 🔍 Keyword search: Filter articles by any search term, from 'solar' to 'hydrogen'.
- 🏢 Office filter: Narrow results to a specific DOE office, like the Solar Energy Technologies Office.
- 📄 Article type: Choose from press releases, blogs, documents, success stories, and more.
- 🌐 Language: Get results in English or Spanish.
- 📅 Sort order: Order by date or relevance.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with Energy.gov data
📈 Track funding opportunities.
A grant consultant runs the Actor with the 'Funding Opportunities' office filter and keyword 'solar' to get a weekly list of new DOE solicitations.
📰 Monitor press releases.
A journalist sets the article type to 'Press Releases' and office to 'Office of Nuclear Energy' to stay current on reactor announcements.
🔬 Collect technical documents.
A researcher filters by article type 'Document' and keyword 'battery storage' to gather reports for a literature review.
🌍 Analyze clean energy trends.
A market analyst scrapes all articles from the 'Solar Energy Technologies Office' over the past year to identify investment patterns.
📚 Build an energy news archive.
A data engineer schedules daily runs with no filters to maintain a complete archive of Energy.gov content.
Why choose this scraper
| What you get | |
|---|---|
| No API key | Access public Energy.gov content without registration or rate limits |
| Bulk export | Save up to a million articles per run to CSV, JSON, Excel, or XML |
| Fixed schema | Every row has the same fields for easy analysis |
| Office targeting | Filter by over 100 DOE offices and programs |
How it compares
No other Store actor targets Energy.gov the same way, so the honest comparison is with the alternatives teams actually weigh.
| Energy.gov Scraper | Build it in-house | By hand | |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Energy.gov changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |
Configure the run
Drive the Actor with a search keyword, article type, office, and language, alone or together, and filters run as each article is read so only matches reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"maxItems": 10
}
A larger pull:
{
"maxItems": 200
}
Pricing
Pay-per-result: $0.0075 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $0.75 |
| 1,000 results | $7.50 |
| 10,000 results | $75.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the Energy.gov Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to Energy.gov through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/energy-gov-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check your filters. If you set a keyword and an office, both must match. Try removing filters one by one to see which is too restrictive.
Why did the run stop before reaching maxItems?
The Actor stops when there are no more matching results. Try broadening your search or removing some filters.
Can I get the full text of each article?
The Actor returns the content fields available in the search results. For full text, you may need to visit the article URL separately.
Why are some articles missing?
Energy.gov updates its content regularly. If an article was recently removed or changed, it may not appear. Try running again later.
How do I filter by multiple offices?
The office input accepts one office per run. To get multiple offices, run the Actor multiple times or use a keyword search without an office filter.
FAQ
| Question | Answer |
|---|---|
| Does this Actor require an API key? | No. It reads the public Energy.gov search pages directly, so no registration or key is needed. |
| Can I filter by a specific DOE office? | Yes. The office input accepts over 100 offices, from the Solar Energy Technologies Office to the National Nuclear Security Administration. |
| What article types can I scrape? | You can filter by press releases, blogs, documents, success stories, media advisories, multimedia, remarks/testimony, energy profiles, and more. |
| How many articles can I get in one run? | You can set maxItems up to 1,000,000. The Actor will collect until it reaches that number or no more results match. |
| Can I search by keyword? | Yes. The keywords input accepts any search term, like 'hydrogen' or 'grid modernization'. |
| What languages are supported? | You can filter results to English (en) or Spanish (es). |
| Can I sort results by date or relevance? | Yes. The sort input lets you choose 'Date' or 'Relevance'. |
| What format is the output? | The Actor returns a flat dataset with fields like title, URL, date, and content. You can export to CSV, JSON, Excel, or XML. |
| Is this legal to scrape? | Energy.gov is a public government website. Scraping public data is generally allowed, but you should review the site's terms of use and robots.txt. |
| Can I schedule this Actor to run automatically? | Yes. On Apify, you can set up a schedule to run the Actor daily, weekly, or at any interval. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by U.S. Department of Energy. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| maxItems | integer | Maximum number of articles to collect per run. | 10 |
| keywords | string | Search keywords to filter articles. | not set |
| articleType | string (13 options) | Filter by article type | not set |
| language | string (2 options) | Filter by language | not set |
| office | string (157 options) | Filter by DOE office or organization | not set |
| sort | string (2 options) | Sort order for results | date |
Pricing
from $1.80 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Actor Start | Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). | $0.005 to $0.0075 |
| result | Single result in the default dataset. | $0.0018 to $0.0027 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~energy-gov-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.