KAPSARC Energy Data Scraper

KAPSARC Energy Data Scraper

Scrape KAPSARC energy datasets by ID, search query, or refine filter, up to a million rows per run. Each record comes with its full metadata and values, ready for CSV, JSON, Excel, or XML export. No API key or registration needed.

KAPSARC's official data portal requires manual browsing and downloads, one dataset at a time. This Actor reads the public catalog directly, so you can pull any dataset by its ID, search across the catalog, or refine by fields like country and year. It returns every matching record in one flat, consistent schema.

Who uses it	What they scrape KAPSARC for
Energy analysts	Pull historical crude oil indicators for a specific country to model supply trends.
Market researchers	Compare energy consumption across regions using refined dataset filters.
Data engineers	Automate ingestion of KAPSARC datasets into a warehouse without manual downloads.
Academic researchers	Collect large samples of energy data for econometric studies.

What it does

This Actor collects records from KAPSARC energy datasets by dataset ID, search query, or refine filter, and returns each record as a flat row.

• 🔍 Search query: full-text search across the KAPSARC catalog to find relevant datasets.
• 🎯 Refine filters: narrow results by any field, such as country or year, before collection.
• 📊 Sort control: order results by any field, ascending or descending, for consistent exports.
• ⚙️ Flexible limits: set a maximum number of datasets per run, from 1 to 1,000,000.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with KAPSARC data

📈 Track crude oil indicators.

An energy analyst pulls the India crude oil indicators dataset to monitor production and consumption trends for a monthly report.

🌍 Compare regional energy data.

A market researcher uses refine filters to collect energy consumption records for multiple countries and compares them in a single spreadsheet.

🔁 Automate data pipelines.

A data engineer schedules the Actor to fetch updated KAPSARC datasets daily and load them into a data warehouse.

🎓 Support academic research.

A PhD student collects a large sample of energy pricing data for a regression analysis on market dynamics.

Why choose this scraper

	What you get
No API key	Access public KAPSARC data without registration or authentication.
Flat schema	Every record is returned as a flat row, ready for analysis or export.
Bulk collection	Collect up to a million datasets in a single run.
Flexible filtering	Refine by any field and sort results to match your analysis needs.

How it compares

No other Store actor targets KAPSARC the same way, so the honest comparison is with the alternatives teams actually weigh.

	KAPSARC Energy Data Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When KAPSARC changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor with a dataset ID, an optional search query, and refine filters that run as records are read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
  "dataset": "india-crude-oil-indicators",
  "maxItems": 10
}

A larger pull:

{
  "dataset": "india-crude-oil-indicators",
  "maxItems": 200
}

Pricing

Pay-per-result: $0.0085 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$0.85
1,000 results	$8.50
10,000 results	$85.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the KAPSARC Energy Data Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to KAPSARC through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/kapsarc-energy-data-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that the dataset ID is correct and exists in the KAPSARC catalog. If using search or refine filters, try removing them to see if the dataset itself returns records.

The refine filter is not working.

Ensure the refine field name matches exactly, including case. You can check available fields by viewing the dataset on the KAPSARC portal.

Sorting returns an error.

Verify the sort field exists in the dataset. Some fields may not be sortable; try a different field or remove sorting.

The run takes too long.

Reduce the maximum datasets value or add refine filters to narrow the result set before collection.

FAQ

Question	Answer
How do I find the dataset ID?	Browse the KAPSARC catalog at datasource.kapsarc.org/explore. Each dataset page shows its ID, which you can copy into the dataset field.
Can I search for datasets by keyword?	Yes, use the search query field to perform a full-text search across the catalog. The Actor will collect all matching datasets.
What refine fields are available?	Any field in the dataset can be used, such as country, year, or indicator type. Check the dataset's metadata to see available fields.
How do I sort results?	Enter a field name in the sort field. Prefix with a minus sign for descending order, for example -year.
Is there a limit on how many datasets I can collect?	You can set a maximum from 1 to 1,000,000 datasets per run. The default is 10.
Do I need an API key?	No, this Actor accesses public KAPSARC data without authentication.
What formats can I export to?	You can export results to CSV, JSON, Excel, or XML from the Apify platform.
Can I schedule this Actor to run automatically?	Yes, use Apify's scheduler to run it at regular intervals and keep your data up to date.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by King Abdullah Petroleum Studies and Research Center. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.