ParseForge
DOAJ Open Access Journals Scraper
parseforge/doaj-scraper
OtherEducationAutomation
Scrapes journal metadata from the DOAJ API. Returns journal title, ISSN, publisher, subject codes, and APC information as flat rows. Filter by subject, language, or country.
- Total users
- 9
- Monthly active
- 0
- Total runs
- 261
- Bookmarked
- 1
- Rating
- 5.0 (1)
- Last modified
- 12 days ago
Overview
DOAJ Open Access Journals Scraper
Scrape open access journal metadata from the Directory of Open Access Journals, filtered by subject, language, or country. Every journal record includes its ISSN, publisher, article count, and APC information. No API key required. Export to CSV, JSON, Excel, or XML.
The DOAJ indexes over 20,000 peer-reviewed open access journals, but browsing its web interface makes it hard to compile lists for analysis. This Actor queries the public DOAJ API directly, letting you pull journal records by subject code, language, publisher country, or a free-text Lucene search. You get a clean, flat dataset of journal metadata without writing a single line of code.
| Who uses it | What they scrape DOAJ for |
|---|---|
| Librarians and collection managers | Building a list of open access journals in a specific subject for library guides. |
| Researchers and PhD students | Identifying reputable open access journals to publish in or review for a literature survey. |
| Publishers and journal editors | Benchmarking their own journals against the DOAJ index by subject and country. |
| Scientometric analysts | Gathering journal-level metadata for studies on open access publishing trends. |
What it does
This Actor collects journal metadata from the DOAJ API and returns each journal as a flat row with its title, ISSN, publisher, subject codes, and other indexed fields.
- 🔍 Free-text search: Use Lucene query syntax to search across journal titles, keywords, and publisher names.
- 🏷️ Subject filtering: Narrow results to a single LCC subject code like Medicine (R) or Science (Q).
- 🌐 Language and country filters: Restrict journals by the language of publication or the publisher's country using ISO codes.
- 📊 Sorted output: Order results by journal title or ISSN, ascending or descending.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with DOAJ data
📚 Build a subject-specific journal list.
A librarian filters DOAJ journals by the Medicine subject code and exports a CSV to update the university's open access guide.
🌍 Map open access publishing by country.
An analyst scrapes all journals with a publisher country of Brazil to study the growth of Latin American OA publishing.
🔎 Find journals by keyword.
A researcher searches for 'bibjson.keywords:machine learning' to discover open access venues in their field.
📈 Monitor APC trends.
A publisher scrapes journals in their discipline and compares article processing charge information across competitors.
Why choose this scraper
| What you get | |
|---|---|
| No API key or registration | Queries the public DOAJ API endpoint with no authentication needed. |
| Structured journal records | Each row is a flat object with consistent fields for ISSN, publisher, and subject. |
| Flexible search syntax | Lucene queries support fielded search, wildcards, and boolean operators. |
| Respectful defaults | Built-in concurrency control and rate limiting to avoid overloading the DOAJ API. |
How it compares
This Actor focuses on journal-level metadata from the DOAJ API with Lucene search and ISO code filters. Here is how it compares to other DOAJ scrapers on Apify.
| Feature | ParseForge | DOAJ Open Access Journal & Article Scraper | DOAJ Scraper · Open Access Journals, Articles & Publishers | DOAJ Scraper |
|---|---|---|---|---|
| Journal-level metadata | Yes | Yes | Yes | Yes |
| Article-level data | Not listed | Yes | Yes | Yes |
| Lucene search syntax | Yes | Not listed | Not listed | Not listed |
| Subject code filter | Yes | Yes | Not listed | Yes |
| Language filter | Yes | Yes | Not listed | Not listed |
| Publisher country filter | Yes | Not listed | Not listed | Yes |
| No API key required | Yes | Not listed | Not listed | Not listed |
Configure the run
Drive the Actor with a Lucene search query and optional filters for subject, language, and country. Filters are applied at the API level so only matching journals count toward your max items limit. The Input tab lists every parameter.
A first run with the defaults:
{
"searchQuery": "*",
"maxItems": 10
}
A larger pull:
{
"searchQuery": "*",
"maxItems": 200
}
Pricing
Pay-per-result: $0.16 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $16.00 |
| 1,000 results | $160.00 |
| 10,000 results | $1600.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the DOAJ Open Access Journals Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to DOAJ through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/doaj-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check your search query syntax. Lucene queries are case-sensitive for field names. Try using a simple wildcard '*' first to confirm the API is reachable, then add filters one at a time.
The Actor is running very slowly.
The DOAJ API may be rate-limiting your requests. Reduce the Max Concurrency setting to 1 or 2 and try again. Large result sets will also take longer to paginate through.
My subject filter is not working.
Subject codes must be valid LCC codes. Use single-letter codes like 'R' for Medicine or 'Q' for Science. You can also use the searchQuery field with 'bibjson.subject.code:R' for more control.
I got fewer results than my maxItems setting.
The DOAJ API returned fewer matching journals than your limit. Try broadening your search query or removing some filters to capture more results.
The Actor returns an error about the API.
The DOAJ public API may experience occasional downtime. Wait a few minutes and retry. If the problem persists, check the DOAJ website to confirm the service is operational.
FAQ
| Question | Answer |
|---|---|
| What is the DOAJ? | The Directory of Open Access Journals is a community-curated index of peer-reviewed open access journals. It lists over 20,000 journals and is a standard reference for quality OA publishing. |
| Do I need an API key to use this Actor? | No. This Actor uses the public DOAJ API endpoint, which does not require authentication or an API key. |
| Can I scrape article-level data with this Actor? | No, this Actor is designed for journal-level metadata only. It returns journal records with fields like title, ISSN, and publisher, not individual articles. |
| How do I search for journals in a specific subject? | Use the Subject Code input with an LCC code like 'R' for Medicine or 'Q' for Science. You can also use the searchQuery field with Lucene syntax, for example 'bibjson.subject.code:R'. |
| What is Lucene query syntax? | Lucene is a search syntax that supports fielded queries, wildcards, and boolean operators. For example, 'bibjson.title:ecology' searches for journals with 'ecology' in the title, and 'publisher:elsevier' filters by publisher name. |
| How many journals can I scrape in one run? | You set the limit with the maxItems input. The DOAJ API returns results in pages, and the Actor will stop once it reaches your specified maximum. |
| What export formats are supported? | You can export your dataset in CSV, JSON, Excel, or XML from the Apify platform. |
| Can I filter journals by language? | Yes, use the Language Code input with an ISO 639-1 code like 'EN' for English, 'FR' for French, or 'ES' for Spanish. |
| How do I filter by publisher country? | Use the Publisher Country Code input with an ISO 3166-1 alpha-2 code like 'US' for the United States, 'GB' for the United Kingdom, or 'DE' for Germany. |
| What if I get rate-limited? | Lower the Max Concurrency setting to reduce the number of parallel requests. The default is 5, but you can set it as low as 1. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by DOAJ. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| searchQuery | string | Lucene query string to filter journals. Use * for all journals. Examples: bibjson.keywords:medicine, publisher:elsevier, bibjson.title:ecology | * |
| maxItems | integer | How many articles to collect per run. | 10 |
| subject | string | Filter by LCC subject code. Examples: R (Medicine), Q (Science), H (Social Sciences). | not set |
| language | string | Filter by language code (ISO 639-1). Examples: EN, FR, ES, DE, ZH. | not set |
| country | string | Filter by publisher country code (ISO 3166-1 alpha-2). Examples: US, GB, DE, IN, BR. | not set |
| sort | string (4 options) | Sort results. Options: title:asc, title:desc, issn:asc, issn:desc. | title:asc |
| maxConcurrency | integer | Maximum number of parallel requests. Lower this if you get rate-limited. | 5 |
Pricing
from $8.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Actor Start | Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). | $0.05 to $0.16 |
| Result | Single result in the default dataset. | $0.008 to $0.012 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~doaj-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
Rated 5.0 out of 5 across 1 review. Read them on Apify.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.