ParseForge
PubMed Citation Scraper
parseforge/pubmed-citation-scraper
Developer toolsAutomationLead generation
Scrapes PubMed citations and abstracts by search query, author, journal, or direct URL. Returns each paper's title, authors, abstract, and publication metadata as a flat row.
- Total users
- 14
- Monthly active
- 1
- Total runs
- 278
- Bookmarked
- 0
- Rating
- 5.0 (1)
- Last modified
- 12 days ago
Overview
PubMed Citation Scraper
Scrape PubMed citations by search term, author, journal, or direct URL, up to a million per run. Every record returns the title, authors, abstract, publication date, and journal metadata. No API key required. Export to CSV, JSON, Excel, or XML.
PubMed's official E-utilities API requires an API key, rate-limits you, and returns XML that needs parsing. This actor reads the public search results directly, filtered by date range, publication type, journal, or author, and returns each citation in one flat schema. Build literature reviews, track research trends, or populate your own database without writing a single line of code.
| Who uses it | What they scrape PubMed for |
|---|---|
| Medical researchers | Gathering all recent clinical trials on a specific drug for a systematic review. |
| Pharma competitive intelligence teams | Monitoring new publications from competing labs or on a target molecule. |
| Academic librarians | Building a departmental publication database filtered by author and journal. |
| Data scientists in healthcare | Collecting a corpus of abstracts to train a biomedical NLP model. |
What it does
This Actor collects PubMed citations by search query, author, journal, or direct URL, and returns each one as a flat row with its title, abstract, authors, and metadata.
- 🔍 PubMed search syntax: use full Boolean queries like 'cancer AND therapy', field tags like 'Smith J[Author]', and MeSH terms.
- 📅 Date range filter: restrict results to a specific publication window with YYYY/MM/DD or YYYY precision.
- 📚 Publication type filter: narrow to Reviews, Clinical Trials, Meta-Analyses, Case Reports, and more.
- 👤 Author and journal filters: target a specific researcher or a high-impact journal directly.
- 🔗 Direct URL input: paste any PubMed search URL and the actor will paginate through all results.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with PubMed data
📖 Build a systematic literature review.
A research team enters a PubMed search term and a date range to collect every matching citation, then exports the dataset for screening in their review software.
📊 Monitor a competitor's research output.
A pharma analyst sets an author filter for a key opinion leader and a journal filter for top-tier publications, running the actor weekly to track new papers.
🧠 Create a training dataset for biomedical AI.
A data scientist scrapes 50,000 abstracts with the 'Clinical Trial' publication type filter, then exports the structured text to fine-tune a language model.
📋 Populate an institutional publication database.
A librarian runs the actor with a list of researcher names and their department's target journals, then loads the flat CSV into the university's repository.
Why choose this scraper
| What you get | |
|---|---|
| Full abstract text | The complete abstract as displayed on PubMed, not a truncated snippet. |
| Author affiliations | First author affiliation and all author names, parsed into a consistent list. |
| Publication metadata | Journal name, volume, issue, pages, DOI, and PMID for every record. |
| Flexible input | Start from a search query, an author name, a journal title, or a full URL. |
How it compares
This actor and the PubMed Search Scraper both extract citation data from PubMed, but they differ in input flexibility and filtering options.
| Feature | ParseForge | PubMed Search Scraper |
|---|---|---|
| Direct URL input | Yes | Not listed |
| Author filter | Yes | Not listed |
| Journal filter | Yes | Not listed |
| Publication type filter | Yes | Not listed |
| Date range filter | Yes | Not listed |
| Sort order selection | Yes | Not listed |
Configure the run
Drive the Actor from a search term, author, journal, or a direct PubMed URL, alone or together, and filters run as each citation is read so only matches reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"searchTerm": "cancer AND therapy",
"maxItems": 10
}
A larger pull:
{
"searchTerm": "cancer AND therapy",
"maxItems": 200
}
Pricing
Pay-per-result: $0.16 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $16.00 |
| 1,000 results | $160.00 |
| 10,000 results | $1600.00 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the PubMed Citation Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to PubMed through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/pubmed-citation-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results?
Check your search syntax. A typo in a field tag or an overly restrictive combination of filters (e.g., a rare author in a narrow date window) can return zero citations. Try the same query directly on the PubMed website to verify it returns results.
The actor is running very slowly.
PubMed's servers can be slow for large, complex queries. The actor includes polite delays to avoid being blocked. If you are scraping more than 100,000 citations, expect the run to take several hours. Consider narrowing your date range or adding more specific filters.
Some abstracts are missing or truncated.
PubMed does not always display the full abstract on the search results page for every citation. The actor extracts whatever text is publicly visible. For a small number of records, the abstract field may be empty if none is shown.
I get an error when using a start URL.
Make sure the URL is a valid PubMed search results page (it should contain '/?term='). A URL pointing to a single article or a different part of the site will not work. Copy the URL directly from your browser's address bar after running a search.
The actor stopped before reaching my maxItems limit.
PubMed searches sometimes return fewer total results than estimated. The actor stops when it has exhausted all available pages. Check the actual result count on the PubMed website for your query. If you need more data, try broadening your search terms.
FAQ
| Question | Answer |
|---|---|
| Can I scrape all results for a broad search like 'cancer'? | Yes. Set your maxItems high (up to 1,000,000 on a paid plan) and the actor will paginate through every page of results. Be aware that very broad queries can return millions of citations and take a long time to complete. |
| How do I use advanced PubMed search syntax? | Enter your query in the searchTerm field exactly as you would in the PubMed search box. You can use Boolean operators (AND, OR, NOT), field tags like [Author], [Journal], or [MeSH Terms], and parentheses for grouping. |
| Does this actor get the full text of articles? | No. This actor scrapes the citation metadata and abstract from PubMed search result pages. It does not access or download full-text PDFs, which are often behind publisher paywalls. |
| Can I filter by multiple authors or journals at once? | The input fields accept a single author or journal name. To combine multiple values, use the searchTerm field with Boolean operators, for example: '(Smith J[Author] OR Jones K[Author]) AND (Nature[Journal] OR Science[Journal])'. |
| What is the difference between using a search term and a start URL? | A search term lets you build a query using the actor's filter fields. A start URL bypasses those filters and scrapes exactly the results page you provide, which is useful if you have already crafted a complex search on the PubMed website. |
| How are authors formatted in the output? | Authors are returned as a list of full names as they appear on PubMed, typically in 'LastName FirstInitials' format. The first author and last author positions are preserved. |
| Can I get the DOI and PMID for each paper? | Yes. The actor extracts both the PMID (PubMed ID) and the DOI when available, making it easy to link back to the source or cross-reference with other databases. |
| Does this actor handle rate limiting or IP blocks? | The actor runs on Apify's distributed infrastructure with automatic retries and polite delays. If you are scraping a very large number of citations, it will pace itself to avoid overwhelming the PubMed servers. |
| Can I schedule this actor to run weekly? | Yes. Once you have configured your search, you can set up an Apify schedule to run it daily, weekly, or monthly. Each run will collect any new citations that match your query. |
| What export formats are supported? | Your dataset can be exported to CSV, JSON, Excel, XML, or RSS. You can also push the data directly to a webhook, Amazon S3, or Google Cloud Storage. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by National Library of Medicine. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| startUrl | string | Direct PubMed search URL to scrape. Example: https://pubmed.ncbi.nlm.nih.gov/?term=cancer+AND+therapy. If provided, all other search filters will be ignored. | not set |
| searchTerm | string | PubMed search query using PubMed search syntax. Examples: 'cancer AND therapy', 'Smith J[Author]', 'Nature[Journal]'. See PubMed search help for advanced syntax. | cancer AND therapy |
| dateFrom | string | Start date for publication date filter. Format: YYYY/MM/DD or YYYY. Example: 2020/01/01 or 2020 | not set |
| dateTo | string | End date for publication date filter. Format: YYYY/MM/DD or YYYY. Example: 2023/12/31 or 2023 | not set |
| publicationType | string | Filter by publication type. Examples: Review, Clinical Trial, Meta-Analysis, Case Reports, etc. | not set |
| journal | string | Filter by journal name. Example: Nature, Science, The Lancet | not set |
| author | string | Filter by author name. Use format: LastName FirstInitial. Example: Smith J | not set |
| sort | string (4 options) | Sort results by: relevance, pub_date (publication date), first_author, or journal | relevance |
| maxItems | integer | Free users: Limited to 100. Paid users: Optional, max 1,000,000 | 10 |
Pricing
from $8.00 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Actor Start | Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). | $0.05 to $0.16 |
| result | Single result in the default dataset. | $0.008 to $0.012 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~pubmed-citation-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
Rated 5.0 out of 5 across 1 review. Read them on Apify.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.