Academic Research Aggregator Scraper

Academic Research Aggregator Scraper

Aggregate research papers, datasets, and institutions from Crossref, OpenAlex, and Dryad in one run. Feed in a keyword or topic and get a unified stream of DOIs, titles, authors, and abstracts. No API keys or journal subscriptions required.

Finding papers across siloed academic databases means repeating the same keyword on Crossref, OpenAlex, and Dryad, then merging the results by hand. This Actor queries all three catalogues in parallel from a single research topic or keyword, deduplicates the hits, and returns one clean dataset. It works for literature reviews, meta-analyses, and institutional benchmarking without writing a line of code.

Who uses it	What they scrape Academic Research Aggregator for
PhD students and postdocs	Run a quick literature scan across multiple databases for a dissertation chapter.
Research librarians	Build a bibliography on a faculty member's topic from several catalogues at once.
Data journalists	Pull the latest datasets and papers on a public-health or climate keyword.
University strategy offices	Benchmark institutional research output by country and type from OpenAlex.

What it does

This Actor queries Crossref journals, OpenAlex institutions and topics, and Dryad datasets for a given keyword, and returns each matching paper, dataset, or institution as a flat row.

• 📚 Multi-source aggregation: Query Crossref, OpenAlex institutions, OpenAlex topics, and Dryad datasets in one run.
• 🏛️ Institution filtering: Narrow OpenAlex results to a country code and an institution type such as education, healthcare, or company.
• 🔢 Flexible volume: Grab up to 10 papers on the free tier for a preview, or scale to a million items on a paid plan.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Academic Research Aggregator data

📖 Systematic literature review.

A PhD candidate enters a dissertation keyword, selects all four sources, and exports a deduplicated CSV of papers and datasets for their PRISMA flowchart.

🏥 Public-health evidence scan.

A policy researcher runs a disease or intervention keyword across Crossref and Dryad to collect the latest papers and underlying datasets in one afternoon.

🏫 Institutional benchmarking.

A university strategy officer sets the OpenAlex country filter to 'gb' and type to 'education' to compare their institution's topic coverage against UK peers.

📊 Dataset discovery for replication.

A postdoc searches Dryad and Crossref for a specific analysis keyword to find both the original paper and the associated open data.

Why choose this scraper

	What you get
One keyword, four catalogues	Crossref, OpenAlex institutions, OpenAlex topics, and Dryad datasets queried in parallel.
Unified output schema	Every result lands in the same flat structure, ready for CSV, JSON, Excel, or XML export.
No API keys or subscriptions	Reads public APIs directly; no journal paywalls or developer registrations needed.
Country and type filters	Restrict OpenAlex institutions to a 2-letter country code and a specific institution type.

How it compares

No other Store actor targets Academic Research Aggregator the same way, so the honest comparison is with the alternatives teams actually weigh.

	Academic Research Aggregator Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When Academic Research Aggregator changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor with a research topic or keyword, pick which academic sources to query, and optionally filter OpenAlex institutions by country and type. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ]
}

A larger pull:

{
  "maxItems": 200,
  "sources": [
    "crossref",
    "openalex_institutions",
    "openalex_topics",
    "datadryad"
  ]
}

Pricing

Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$2.10
1,000 results	$21.00
10,000 results	$210.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Academic Research Aggregator Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Academic Research Aggregator through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/academic-research-aggregator-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that at least one source is enabled in the input. Try a broader keyword and remove the OpenAlex country or institution type filter. Some niche terms return few hits across all catalogues.

Why does the run stop at 10 items?

Free Apify accounts are capped at 10 items for this Actor. Upgrade to a paid plan and set maxItems to a higher number to collect more results.

Why are some fields empty in my dataset?

Different sources provide different metadata. Dryad datasets may not have an abstract, and Crossref records may lack institution types. Empty fields are expected when a source does not supply that attribute.

Why does my OpenAlex country filter not work?

Use an ISO 2-letter country code in lowercase, such as 'us' or 'gb'. The filter only applies to the OpenAlex institutions source. It is ignored for Crossref, OpenAlex topics, and Dryad.

Why do I see duplicate DOIs?

The Actor queries sources in parallel and does not deduplicate. If the same paper appears in Crossref and OpenAlex, you will see two rows. Deduplicate by DOI in your spreadsheet or analysis tool.

FAQ

Question	Answer
Which academic databases does this Actor search?	It queries Crossref journals, OpenAlex institutions, OpenAlex topics, and Dryad datasets. You can enable or disable each source in the input.
Do I need a university subscription or API key?	No. The Actor reads the public APIs of Crossref, OpenAlex, and Dryad. No login, no API key, and no institutional proxy are required.
How many results can I get in one run?	Free Apify users are limited to 10 items for a preview. Paid users can set maxItems up to 1,000,000 and pull a much larger corpus.
Can I filter by country or institution type?	Yes. Use the optional OpenAlex country code field with an ISO 2-letter code such as 'us' or 'de', and pick an institution type like 'education' or 'healthcare'.
What does the output look like?	Every result is a flat row with fields such as title, authors, DOI, abstract, publication date, and source catalogue. You can export to CSV, JSON, Excel, or XML.
Does it deduplicate papers that appear in more than one source?	The Actor queries each source independently and returns the raw hits. Deduplication by DOI or title can be done in a spreadsheet or with a simple post-processing script.
Can I search for a specific journal or dataset keyword?	Yes. The query field accepts journal names, topic keywords, or dataset keywords. The Actor passes your term to every source you have enabled.
Is this Actor suitable for a meta-analysis?	It is a good first-pass aggregation tool for a meta-analysis. It collects candidate papers and datasets across multiple databases, which you then screen and code offline.
What is the difference between OpenAlex institutions and OpenAlex topics?	OpenAlex institutions returns research organisations matching your keyword. OpenAlex topics returns concepts and fields of study. Enable one or both depending on your goal.
Does it scrape full-text PDFs?	No. The Actor collects metadata such as titles, abstracts, DOIs, and authors. It does not download or scrape full-text PDFs.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Crossref, OpenAlex, and Dryad. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.