ParseForge
DOAB Open Access Books Scraper
parseforge/doabooks-directory-open-access-scraper
EducationOtherFOR_CREATORS
Scrapes open access book records from the Directory of Open Access Books by search query, language, subject, or publisher. Returns each book as a flat row with metadata and full-text links.
- Total users
- 2
- Monthly active
- 1
- Total runs
- 104
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 9 days ago
Overview
DOAB Open Access Books Scraper
Scrape open access academic books from the Directory of Open Access Books by search query, subject, language, or publisher. Each record includes the full metadata, abstract, license, and direct links to the free PDF or EPUB. Export to CSV, JSON, Excel, or XML.
Finding peer-reviewed open access books across hundreds of publishers is slow when you browse the DOAB website one page at a time. This Actor queries the full DOAB catalog directly, letting you filter by language, subject, or publisher, and returns a structured dataset of matching titles with their abstracts, authors, and download links.
No API key, no registration. a search term and the fields you need.
| Who uses it | What they scrape Directory of Open Access Books for |
|---|---|
| Academic librarians | Building a topical collection of free eBooks for a university reading list |
| Researchers and PhD students | Gathering the latest open access monographs in a narrow field for a literature review |
| Data analysts in scholarly publishing | Tracking the growth of open access book output by language, publisher, or subject over time |
| Repository managers | Ingesting metadata for batch import into an institutional repository |
What it does
This Actor collects open access book records from the Directory of Open Access Books and returns each one as a flat row with its title, authors, abstract, license, and direct links to the full text.
- 🔍 Full-text search: Query across titles, authors, and abstracts with support for field-qualified terms like title:democracy.
- 🌐 Language filter: Restrict results to a single language using its ISO code or name, such as en, de, es, or fr.
- 📚 Subject filter: Narrow the catalog to a discipline like philosophy, mathematics, or history.
- 🏢 Publisher filter: Limit results to books from a specific publisher or university press.
- 🎯 Direct handle lookup: Fetch a single book instantly by its DOAB handle ID, bypassing search entirely.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with Directory of Open Access Books data
📖 Build a subject bibliography.
A librarian scrapes all philosophy books in Spanish to create a curated open access reading list for a new undergraduate course.
🌍 Map global OA book output.
A publishing analyst runs the Actor quarterly with no filters to collect the entire catalog and chart growth by language and publisher.
🔬 Feed a literature review.
A doctoral candidate searches for 'digital humanities' and exports the abstracts to screen titles for a systematic review.
🏛️ Populate a repository.
A repository manager pulls all books from a specific university press and imports the metadata batch into their DSpace instance.
Why choose this scraper
| What you get | |
|---|---|
| Full metadata per book | Title, authors, abstract, publisher, publication year, ISBN, and subject classifications |
| Direct download links | URLs to the free PDF, EPUB, or HTML versions hosted by the publisher |
| License information | The Creative Commons or other open license governing each title |
| Scalable collection | Collect up to a million books per run, controlled by a single maxItems setting |
How it compares
No other Store actor targets Directory of Open Access Books the same way, so the honest comparison is with the alternatives teams actually weigh.
| DOAB Open Access Books Scraper | Build it in-house | By hand | |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Directory of Open Access Books changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |
Configure the run
Drive the Actor with a search query, and optionally narrow results by language, subject, or publisher. Filters are applied as the catalog is read so only matching books reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"maxItems": 10,
"searchQuery": "*"
}
A larger pull:
{
"maxItems": 200,
"searchQuery": "*"
}
Pricing
Pay-per-result: $0.01467 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $1.47 |
| 1,000 results | $14.67 |
| 10,000 results | $146.70 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the DOAB Open Access Books Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to Directory of Open Access Books through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/doabooks-directory-open-access-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check that your search query is not too restrictive. Try using '*' to return everything, then add filters one at a time. Also verify that the language code or subject name matches the terms used on doabooks.org.
The Actor returns fewer books than I expected.
The maxItems setting controls the upper limit. If you set it to 100 but expect 500, increase the number. Also confirm that your subject or publisher filter is not narrowing the results more than intended.
My handle ID lookup returns an error or empty result.
Ensure the handle is complete and copied exactly from the DOAB URL, including the prefix (e.g., 20.500.12854/...). A missing digit or a trailing slash can cause the lookup to fail.
The search query with a colon is not working.
Field-qualified queries like 'title:democracy' must have no space around the colon. Write it exactly as 'title:democracy'. If the problem persists, try a plain keyword search without the field prefix.
The run times out before collecting all books.
Increase the run timeout in the Actor's settings, or reduce the maxItems value and run the Actor in batches using different filters to partition the catalog.
FAQ
| Question | Answer |
|---|---|
| What is the Directory of Open Access Books? | DOAB is a community-driven discovery service that indexes peer-reviewed open access books from hundreds of academic publishers. It is maintained by the DOAB Foundation and provides a central search point for free scholarly monographs. |
| Do I need an API key or login to scrape DOAB? | No. This Actor reads the public catalog feeds directly. You do not need to register an application, obtain a token, or log in. |
| What data does each book record contain? | Every row includes the title, author list, abstract, publisher, publication year, ISBN, subject headings, license type, and direct links to the full-text PDF or EPUB. |
| Can I search by keyword or only by exact title? | You can search by keyword across titles, authors, and abstracts. The search also supports field-qualified queries, so you can write 'title:democracy' to match only in the title field. |
| How do I filter books by language? | Set the optional 'Language' input to an ISO 639-1 code like 'en' for English, 'de' for German, or 'fr' for French. You can also use the full language name. |
| Can I get one specific book instead of a list? | Yes. Paste a DOAB handle ID like '20.500.12854/12345' into the Handle ID field. The Actor will fetch that single book and return its full record. |
| What is a DOAB handle ID? | It is a persistent identifier assigned to every book in the directory, visible in the book's URL on doabooks.org. It looks like '20.500.12854/...' and uniquely identifies one title. |
| How many books can I collect in one run? | You can set the maximum up to 1,000,000 books. The Actor will stop when it reaches that number or when the catalog has no more matching results. |
| Are the books themselves downloaded? | No. The Actor collects the metadata and the URLs that point to the full text on the publisher's site. You get links, not the PDF files. |
| What export formats are supported? | Your dataset can be exported to CSV, JSON, Excel, or XML from the Apify platform. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by DOAB Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| maxItems | integer | Maximum number of books to collect per run. | 10 |
| searchQuery | string | Search across titles, authors, abstracts. Use * for everything. Supports field-qualified queries (e.g. title:democracy). | * |
| language | string | Optional ISO language code or name (e.g. en, de, es, fr). | not set |
| subject | string | Optional subject filter (e.g. philosophy, mathematics, history). | not set |
| publisher | string | Optional publisher name filter. | not set |
| handleId | string | Optional. Look up a single book by its DOAB handle (e.g. 20.500.12854/12345). Overrides search. | not set |
Pricing
from $11.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Result Item | Charged once per result collected. | $0.011 to $0.01467 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~doabooks-directory-open-access-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.