ParseForge
NerdWallet Personal Finance Scraper
parseforge/nerdwallet-scraper
BusinessLead generation
Scrapes NerdWallet articles by vertical or page type and returns each one as a flat row with full body text, author, and publish date.
- Total users
- 8
- Monthly active
- 2
- Total runs
- 139
- Bookmarked
- 0
- Rating
- Not rated yet
- Last modified
- 12 days ago
Overview
NerdWallet Personal Finance Scraper
Scrape NerdWallet personal finance pages for best-of roundups, product reviews, and guides, up to a million articles per run. Every article comes with its full body text, author, publish date, and structured data. No login or API key. Export to CSV, JSON, Excel, or XML.
NerdWallet publishes thousands of personal finance comparisons, reviews, and guides, but browsing them manually across verticals like mortgages, insurance, and banking is slow. This scraper reads the public sitemap and article pages directly, filtered by vertical and page type, and returns each match in one fixed schema.
| Who uses it | What they scrape NerdWallet for |
|---|---|
| Content strategists | Monitor which financial products NerdWallet is covering this month. |
| SEO analysts | Audit NerdWallet's best-of roundup structure and internal linking. |
| Financial product managers | Track competitor reviews and feature comparisons across lenders or insurers. |
| Data journalists | Gather personal finance guides and rate data for reporting. |
What it does
This Actor collects NerdWallet articles by vertical, page type, or direct URL, and returns each one as a flat row.
- 📄 Direct URL mode: Paste any single NerdWallet article or comparison page URL and scrape that page.
- 🏷️ Vertical filter: Restrict the crawl to one vertical, such as mortgages, banking, insurance, or investing.
- 📋 Page type filter: Emit only best-of roundups, single product reviews, guides, or hub pages.
- 📊 Structured output: Each article returns as a flat row with body text, author, publish date, and metadata.
Results export to CSV, JSON, Excel, or XML, or straight from the API.
What you can do with NerdWallet data
📈 Monitor competitor content.
A content strategist runs the scraper on the mortgages vertical weekly to see which lenders NerdWallet is featuring in its best-of lists.
🔍 Audit SEO page types.
An SEO analyst filters for best-of roundups and reviews to map NerdWallet's internal linking and heading structure.
📋 Track product reviews.
A product manager at an insurance carrier scrapes the insurance vertical for single-carrier reviews to benchmark feature mentions.
📰 Gather rate data for reporting.
A data journalist scrapes banking and mortgage guides to collect current rate tables and fee comparisons for a story.
Why choose this scraper
| What you get | |
|---|---|
| No API key | Scrapes public NerdWallet pages with no registration or token. |
| Full article text | Captures the complete body content of each guide or comparison. |
| Vertical targeting | Narrows the crawl to one financial vertical like loans or taxes. |
| Flexible export | Download results as CSV, JSON, Excel, or XML. |
How it compares
No other Store actor targets NerdWallet the same way, so the honest comparison is with the alternatives teams actually weigh.
| NerdWallet Personal Finance Scraper | Build it in-house | By hand | |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When NerdWallet changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |
Configure the run
Drive the Actor from a single NerdWallet URL or let it crawl the public sitemap filtered by vertical and page type, so only matching articles reach your dataset. The Input tab lists every parameter.
A first run with the defaults:
{
"startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders",
"maxItems": 10
}
A larger pull:
{
"startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders",
"maxItems": 200
}
Pricing
Pay-per-result: $0.02499 per result collected. You pay only for the results written to your dataset.
| Results collected | Approximate cost |
|---|---|
| 100 results | $2.50 |
| 1,000 results | $24.99 |
| 10,000 results | $249.90 |
New Apify accounts start with $5 in free credit.
Free users
Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.
Run it
- Create a free Apify account with $5 in credit.
- Open the NerdWallet Personal Finance Scraper.
- Set your inputs and any filters, then click Start.
- Export the results as CSV, Excel, JSON, or XML from the Dataset tab.
Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.
Use with AI agents (MCP)
Give an AI agent live access to NerdWallet through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/nerdwallet-scraper"
Then prompt it in plain language to run the scraper and read back the results.
Troubleshooting
Why am I getting no results for credit cards?
NerdWallet's credit card comparison pages use aggressive anti-bot protection. Switch to a different vertical like mortgages or banking, or use a direct article URL that you know is publicly accessible.
The actor returns fewer articles than my max items setting.
The Actor stops when it has read all matching pages in the sitemap. Try removing the vertical or page type filter to broaden the crawl, or increase the max items limit.
Some article body text is truncated.
NerdWallet sometimes lazy-loads content sections. The scraper captures the initial server-rendered body. If critical sections are missing, try scraping the same URL again or check if the content is behind a client-side paywall.
I pasted a URL but the actor ignored my filters.
When you set a start URL, the Actor scrapes only that single page and skips all vertical and page type filters. Remove the start URL to use the filtered crawl mode.
FAQ
| Question | Answer |
|---|---|
| Can I scrape credit card comparison pages? | Credit card comparison routes are gated by NerdWallet anti-bot protections and may not return data. Use mortgages, banking, insurance, or direct article URLs for reliable extraction. |
| How do I scrape one article? | Paste the full NerdWallet URL into the start URL field. The Actor will scrape only that page and ignore any vertical or page type filters. |
| What verticals are supported? | You can filter by credit cards, banking, mortgages, loans, insurance, investing, taxes, small business, or personal finance. Leave the vertical filter empty to crawl across all of them. |
| What page types can I filter? | Choose from best-of roundups, product or carrier reviews, articles and guides, or hub pages. Leave the filter empty to collect every page type. |
| Does this scraper need a NerdWallet login? | No. It reads only public pages that NerdWallet serves to anyone with a browser. No account, API key, or OAuth is required. |
| How many articles can I scrape in one run? | You set the maximum articles per run, up to one million. The Actor stops when it hits your limit or exhausts the matching pages. |
| What data fields does each row include? | Each row returns the article URL, title, author, publish date, full body text, and any structured metadata found on the page. |
| Can I export the data to Google Sheets? | You can export results as CSV or Excel from the Apify dataset tab and then import that file into Google Sheets. |
Related actors
Browse the full ParseForge collection for more scrapers.
🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.
⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by NerdWallet, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.
Input
| Field | Type | What it does | Default |
|---|---|---|---|
| startUrl | string | Optional. Any single NerdWallet URL to scrape (e.g. https://www.nerdwallet.com/mortgages/best/mortgage-lenders or https://www.nerdwallet.com/article/insurance/auto-insurance-rates). When set, only that single page is scraped and the filters below are ignored. | https://www.nerdwallet.com/mortgages/bes |
| maxItems | integer | Maximum number of articles to collect per run. | 10 |
| productVertical | string (9 options) | Restrict the crawl to a single NerdWallet vertical. Leave empty to scrape across every vertical. Note: credit-cards comparison routes are gated by NerdWallet anti-bot and may not return data; use mortgages, banking, insurance, or article URLs for reliable extraction. | not set |
| pageType | string (4 options) | Only emit pages of this template. 'roundup' = best-of comparison list (/best/...). 'review' = single product or carrier review. 'article' = guide / how-to. 'hub' = vertical landing page (e.g. /mortgages). | not set |
Pricing
from $18.74 per 1,000 results
| Charged for | What it is | Price each |
|---|---|---|
| Actor Start | Charged once when the run starts. | $0.005 |
| Result Item | Charged once per result collected. | $0.01874 to $0.02499 |
Tiered: the lower figure is the price on a higher Apify plan. Billing and the free credit live on Apify.
API
One POST returns the dataset directly. Same shape for every scraper in the library, so swapping the slug is the only change.
curl -X POST "https://api.apify.com/v2/acts/parseforge~nerdwallet-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"helloWorld": 123
}' Examples
Input that runs as-is.
{
"helloWorld": 123
} Reviews
No reviews yet. Be the first.
Issues
We build and maintain this scraper, so a problem with it comes to us. Report it on the Apify listing and the thread stays attached to the scraper where the next person can find it: open an issue.
Broken and urgent, or you would rather not post in public? Write to parseforge@protonmail.com and it reaches the people who wrote it.
Related scrapers
Run NerdWallet Personal Finance Scraper on Apify All scrapers