ParseForge
data.go.kr Dataset Scraper
parseforge/data-go-kr-scraper
AutomationOther
Scrapes dataset listings from Korea's official open data portal (data.go.kr) by keyword, dataset type, or recommendation status. Returns each dataset as a flat row with title, description, provider, and file links.
- Total users
- 2
- Monthly active
- 1
- Total runs
- 249
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 9 days ago
Overview
data.go.kr Dataset Scraper
Scrape public datasets from Korea's data.go.kr portal by keyword, type, or recommendation status. Each dataset record includes its title, description, provider, format, and download links. No API key required. Export to CSV, JSON, Excel, or XML.
Korea's official open data portal, data.go.kr, hosts tens of thousands of public datasets but offers no bulk export or programmatic access without navigating its web interface. This Actor searches the portal by keyword, dataset type, or recommendation status, and returns each matching dataset as a structured row. It reads the public listing pages directly, so you can collect dataset metadata without writing a custom crawler.
| Who uses it | What they scrape Korea's Open Government Data Portal (data.go.kr) for |
|---|---|
| Data journalists | Finding government datasets on a specific topic for a story. |
| Researchers and academics | Building a catalog of available public data in a domain like health or transportation. |
| Open data advocates | Monitoring newly published datasets from Korean government agencies. |
| Data engineers | Discovering dataset endpoints and file URLs for automated ingestion pipelines. |
What it does
This Actor collects dataset listings from data.go.kr by search term or direct URL, and returns each dataset as a flat row with its title, description, provider, type, and available file formats.
- π Keyword search: Find datasets by title, description, or keyword using a search term like 'weather' or 'population'.
- π Dataset type filter: Narrow results to file-based datasets (CSV, Excel), APIs, standard data, or linked data.
- β Recommendation filter: Limit results to datasets recommended by the portal for quality and frequent use.
- π Direct URL input: Paste a pre-configured search results URL from data.go.kr to scrape an exact page.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with Korea's Open Government Data Portal (data.go.kr) data
π Build a research data catalog.
A university researcher scrapes all datasets matching 'air quality' to compile a list of available environmental data sources for a grant proposal.
π° Find data for investigative reporting.
A data journalist searches for 'crime statistics' and filters by file type to locate downloadable CSV datasets for analysis.
π Monitor new open data releases.
An open data advocate runs the Actor weekly with no keyword to collect all newly listed datasets and publishes a digest for the community.
π Discover API endpoints for a data pipeline.
A data engineer filters by API type and keyword 'traffic' to find real-time transportation data feeds for a city dashboard.
Why choose this scraper
| What you get | |
|---|---|
| No API key needed | Reads the public listing pages directly, no registration or authentication required. |
| Structured output | Every dataset comes back as a flat row with consistent columns for title, description, provider, and formats. |
| Flexible input | Use a keyword search with filters or paste a URL from the portal's own search interface. |
| Scalable collection | Paid users can scrape up to a million datasets in a single run. |
How it compares
No other Store actor targets Korea's Open Government Data Portal (data.go.kr) the same way, so the honest comparison is with the alternatives teams actually weigh.
| data.go.kr Dataset Scraper | Build it in-house | By hand | |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Korea's Open Government Data Portal (data.go.kr) changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |
Configure the run
Drive the Actor with a search keyword, dataset type, recommendation flag, or a direct data.go.kr URL. Filters are applied as each page is read so only matching datasets reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"maxItems": 10,
"keyword": "weather"
}
A larger pull:
{
"maxItems": 200,
"keyword": "weather"
}
Pricing
Pay-per-result: $0.0075 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $0.75 |
| 1,000 results | $7.50 |
| 10,000 results | $75.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the data.go.kr Dataset Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to Korea's Open Government Data Portal (data.go.kr) through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/data-go-kr-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check that your keyword or filters match existing datasets. Try a broad term like 'weather' or leave the keyword empty to see all datasets. Also ensure you are not using both a start URL and search filters simultaneously.
The Actor stops after only a few datasets.
Check the maximum items setting. Free users are capped at 10 datasets. If you need more, upgrade your plan and increase the limit or leave it empty.
My start URL is not working.
Make sure the URL begins with 'https://www.data.go.kr/' and was copied from a dataset search results page, not an individual dataset detail page. The URL should contain query parameters from your filter selections.
Korean characters appear as gibberish in the output.
Ensure your output format (CSV, JSON, Excel) is opened with UTF-8 encoding. Most spreadsheet applications let you specify encoding when importing a CSV file.
FAQ
| Question | Answer |
|---|---|
| Do I need a data.go.kr account or API key? | No. This Actor reads the public dataset listing pages that anyone can view in a browser. No login, registration, or API key is required. |
| What information does each scraped dataset include? | Each row contains the dataset title, description, provider organization, dataset type, available file formats, and download or API endpoint links as shown on the portal. |
| Can I scrape all datasets from the portal? | Yes. Leave the keyword field empty and set the maximum items high enough. The Actor will paginate through all available dataset listings. |
| How do I get a direct URL to scrape? | Visit data.go.kr, apply your desired filters on the dataset list page, and copy the full URL from your browser's address bar. Paste it into the startUrl field. |
| What is the difference between FILE, API, STD, and LINKED dataset types? | FILE datasets are downloadable files like CSV or Excel. API datasets provide REST endpoints returning JSON or XML. STD datasets follow specific standard formats. LINKED datasets use semantic web standards. |
| Does this Actor download the actual dataset files? | No. It scrapes the metadata and links from the listing pages. You get the URLs to the files or API endpoints, not the file contents themselves. |
| Can I search in Korean? | Yes. The keyword field accepts Korean text. Use terms like 'λ μ¨' or 'μΈκ΅¬' to search for datasets with Korean metadata. |
| What does the recommendation filter do? | Setting it to 'Y' returns only datasets marked as recommended by the portal, which are typically high-quality and frequently used. 'N' returns all datasets including non-recommended ones. |
| Is there a limit on how many datasets I can scrape? | Free users are limited to 10 datasets per run. Paid users can set a higher limit, up to 1,000,000, or leave it empty for unlimited scraping. |
| Can I use both a start URL and search filters? | No. They are mutually exclusive. Use either a direct URL from the portal or the keyword and filter fields, not both at the same time. |
Related actors
Browse the full ParseForge collection for more scrapers.
π Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
β οΈ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Ministry of the Interior and Safety. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| startUrl | string | Direct URL to a search results page from data.go.kr. Use this OR search filters below, not both. To get a URL: 1) Go to https://www.data.go.kr/en/tcs/dss/selectDataSetList.do, 2) Apply your desired filters, 3) Copy the URL from your browser address bar. This is useful when you want to scrape a specific search result page you've already configured on the website. | not set |
| maxItems | integer | Free users: Limited to 100. Paid users: Optional, max 1,000,000. Leave empty for unlimited (paid users only). | 10 |
| keyword | string | Search term to find datasets. This will search across dataset titles, descriptions, and keywords. Only works when using search filters (not with startUrl). Example: 'weather', 'population', 'economic' | weather |
| svcType | string (4 options) | Filter by dataset type. FILE: Downloadable file-based datasets (CSV, XML, Excel, etc.). API: REST API endpoints that return JSON/XML data. STD: Standard datasets following specific data formats. LINKED: Linked data using semantic web standards. Only works when using search filters (not with startUrl). | not set |
| recmSe | string (2 options) | Filter by recommendation status. Y: Only show datasets recommended by the portal (typically high-quality, frequently used datasets). N: Show all datasets including non-recommended ones. Leave empty to show all datasets. Only works when using search filters (not with startUrl). | not set |
| conditionType | string (2 options) | Search condition type. 'init': Initial search state (default, shows all results). 'search': Active search mode (typically used after applying filters). Usually leave as 'init' unless you need specific search behavior. Only works when using search filters (not with startUrl). | not set |
| kwrdArray | string | Comma-separated keywords for advanced search. Example: 'temperature,precipitation,humidity'. Only works when using search filters (not with startUrl). | not set |
Pricing
from $1.40 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Actor Start | Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). | $0.005 to $0.0075 |
| result | Single result in the default dataset. | $0.0014 to $0.0021 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~data-go-kr-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.