Firecrawl CLI is a command-line tool that can scrape pages, crawl sites, map URLs, run web search, and even drive a cloud browserâthen output results in LLM-friendly formats like clean Markdown or structured JSON. In Claude Code, you can use Firecrawl via an MCP connection, and (optionally) add a âSkillâ layer so you can trigger Firecrawl from slash commands while letting the agent choose the right endpoint and defaults for the job.
This guide breaks down Firecrawlâs six core capabilities, how to install it in Claude Code (MCP, CLI, and Skill-style workflows), how the escalation pattern works in practice, and what to watch for around credits, reliability, and security.
- How to choose between Firecrawlâs six core features: Scrape, Crawl, Map, Search, Extract, and Browser
- How to set up Firecrawl in Claude Code via MCP, the CLI, and Skill-based slash commands
- What a Firecrawl Skill isâand why it can be more productive than calling the CLI manually
- How to use an escalation pattern to start simple and âlevel upâ only when needed
- Credit usage basics and operational gotchas to avoid surprise spend
What is Firecrawl?
Firecrawl is a web data extraction platform focused on turning websites into LLM-ready Markdown or structured data. That âLLM-readyâ output is the main idea: instead of returning raw HTML and leaving you to clean it up, Firecrawl removes common noise (navigation, footers, repetitive layout blocks, ads) and produces a cleaner representation of the main content.
This differs from many traditional scrapers, where you typically need a post-processing pipeline (readability extraction, boilerplate removal, HTML-to-Markdown conversion, etc.) before the content is usable for retrieval, summarization, or structured extraction tasks.
Firecrawl offers six primary capabilities, and you can access them through multiple integration paths: API/SDK, CLI, MCP, and (in Claude Code) Skill-style slash commands.
Ways to use Firecrawl
- API / SDK: Embed Firecrawl in your application (Python and Node.js)
- CLI: Run commands from your terminal (great for local testing and CI automation)
- MCP: Connect Firecrawl as a tool in MCP-compatible clients like Claude Code
- Skill: Use slash commands in Claude Code (for example,
/firecrawl-scrape), with the agent choosing sensible defaults and options
Six core features
Firecrawl provides six capabilities you can mix and match depending on what youâre trying to achieve. Hereâs what each one doesâand when itâs the right tool.
Scrape: extract data from a single page
Scrape fetches the content of a given URL and returns it in formats such as Markdown, HTML, JSON, or even screenshots. Itâs the core âgrab this pageâ function, and itâs designed to work on dynamic sites that require JavaScript rendering. Proxy handling and caching are part of the platform.
# Basic scrape (remove navigation/footer and keep the main content)
firecrawl scrape "https://example.com/blog/post-1" --only-main-content -o .firecrawl/post.md
# Scrape multiple URLs in parallel
firecrawl scrape "https://example.com/page1" "https://example.com/page2" "https://example.com/page3"
# Output Markdown + link list as JSON
firecrawl scrape "https://example.com" --format markdown,links -o .firecrawl/page.jsonBest for: checking a specific page, preparing LLM inputs, analyzing competitor pages
Crawl: recursively crawl a whole site
Crawl starts from a URL and follows links to collect content across a site (or a section of it). You can control scope with sitemap discovery, depth limits, and include/exclude path filters. Crawls run as asynchronous jobs, so you can track progress and use webhooks for notifications.
# Crawl up to 50 pages under /docs
firecrawl crawl "https://docs.example.com" --include-paths /docs --limit 50 --wait -o .firecrawl/docs.json
# Crawl with progress output
firecrawl crawl "https://example.com" --progress --limit 100 --wait -o .firecrawl/site.jsonBest for: ingesting a documentation site, collecting RAG data, exporting a site before migration
Map: quickly list URLs on a site
Map returns a fast list of URLs by combining signals like sitemaps, search-engine discovery, and prior crawl knowledge. Itâs designed for speed rather than perfect completenessâideal for reconnaissance before you commit to a full crawl.
# Get a site-wide URL list
firecrawl map "https://example.com" -o .firecrawl/urls.txt
# Filter URLs related to a keyword
firecrawl map "https://example.com" --search "authentication" -o .firecrawl/auth-pages.txtBest for: finding specific pages in large sites, narrowing crawl scope, understanding site structure
Search: web search + content retrieval
Search runs a web search and (optionally) fetches full content for the resulting pages. This is useful when you donât have the URL yet: find candidates with Search, then switch to Scrape or Crawl when you know exactly what you want.
# Search the web and also scrape full content for results
firecrawl search "Firecrawl Claude Code how to" --scrape -o .firecrawl/results.json --json
# News search (past 24 hours)
firecrawl search "AI scraping tool" --sources news --tbs qdr:d -o .firecrawl/news.json --json
# Limit number of results
firecrawl search "web scraping best practices" --limit 5 -o .firecrawl/search.json --jsonBest for: technical research, competitive intelligence, collecting recent news
Extract: structured data extraction
Extract pulls structured data from multiple URLs (or an entire domain) aligned to a JSON schema. You can also describe what you want in natural language, and it supports wildcard URL patterns like /*.
Extract is currently in beta. Expect variability on large sites or complex extraction requests.
Best for: bulk product extraction, price comparison datasets, collecting forum comments into a consistent schema
Browser: cloud browser automation
Browser lets you control a cloud-hosted Chromium instance to handle sites that require interaction: clicks, form entry, login flows, pagination, and other dynamic behavior. You donât need to install Chromium locally. If you save session state as a profile, you can reconnect while preserving login state.
# Launch a browser session
firecrawl browser launch-session --ttl 600 --stream
# Example page interactions
firecrawl browser execute "open https://news.ycombinator.com"
firecrawl browser execute "snapshot" # inspect the element list
firecrawl browser execute "click @e5" # click a specific element
firecrawl browser execute "scrape" # extract content
# Save and reuse a logged-in session
firecrawl browser launch-session --profile my-app
firecrawl browser "open https://app.example.com/login"
firecrawl browser "fill @e3 'user@example.com'"
firecrawl browser "click @e7"Best for: login-required sites, pagination, JavaScript-heavy SPAs, workflows that need form submission
Feature comparison table
| Feature | Scope | Output | JS rendering | Page interaction | Speed |
|---|---|---|---|---|---|
| Scrape | Single page (batch supported) | Markdown, HTML, JSON, etc. | Supported | Actions supported | Fast |
| Crawl | Whole site (recursive) | Markdown, HTML, JSON, etc. | Supported | No | Slow (at scale) |
| Map | Whole site (URLs only) | URL list | â | No | Fastest |
| Search | Entire web (search results) | Results + content | â | No | Medium |
| Extract | Multiple URLs / whole domain | Structured JSON | â | No | Medium |
| Browser | Any website | Interaction outputs + content | Full support | Full support | Depends on workflow |
The escalation pattern
Firecrawlâs design encourages a simple workflow: start with the least complex approach, then âescalateâ only if you need more power. That keeps both complexity and cost under control.
| Step | Situation | Recommended feature |
|---|---|---|
| 1 | You donât know the URL yet | Use Search to find the right page |
| 2 | You already have the URL | Use Scrape to fetch the content directly |
| 3 | You need to locate specific pages in a large site | Use Map to list URLs, then Scrape what you need |
| 4 | You need a full section (or whole site) | Use Crawl for recursive bulk collection |
| 5 | The site requires clicks/login | Use Browser to interact, then extract |
Example: if you want to check a competitorâs pricing, you can run Search to find the pricing page, Scrape the result, and only move to Browser if the page is behind authentication. You usually donât need Browser as a first step.
How to set it up in Claude Code
You have three practical options for using Firecrawl with Claude Code: MCP integration, running the CLI directly, or using Skill-style slash commands (covered next).
Option 1: MCP integration (recommended)
This is the quickest way to get value. Firecrawl provides an MCP server you can add to Claude Code with a single command. Once itâs installed, Claude Code can call Firecrawl as a tool for scraping, crawling, mapping, and search.
# 1) Get a Firecrawl API key (create it in the Firecrawl dashboard)
# 2) Add the Firecrawl MCP server to Claude Code
claude mcp add firecrawl -e FIRECRAWL_API_KEY=your-api-key -- npx -y firecrawl-mcpAfter that, you can simply ask Claude Code things like âScrape this URLâ or âSearch for documentation about X,â and it will invoke Firecrawl as needed.
Firecrawl integrates powerful web scraping, crawling, and search capabilities directly into Claude Code.
Option 2: Install and use the CLI directly
If you prefer not to route through MCPâor you want tighter control over output files, logging, and debuggingârun the Firecrawl CLI in your terminal. This is also the most natural fit for CI/CD automation.
# Run the CLI via npx (no global install required)
npx -y firecrawl-cli@latest
# Authenticate (browser-based login is recommended)
firecrawl login --browser
# Or authenticate with an API key
firecrawl login --api-key "your-api-key"
Option 3: Use Skills (explained below)
Claude Code Skills let you trigger Firecrawl via slash commands like /firecrawl-scrape. The key advantage is that the agent can apply a consistent workflow (including output handling and safety defaults) instead of you manually picking commands and flags each time.
Common troubleshooting checks
- API key: Confirm the environment variable name is correct (
FIRECRAWL_API_KEY). - Network: Corporate proxies/firewalls can block
npxor outbound API callsâtest from a clean network if possible. - Output expectations: Dynamic sites often require waiting for the rendered state (use
--wait-forto set a wait time). - CLI version: Run
firecrawl --versionand confirm youâre on a current release.
What is a Firecrawl Skill?
In Claude Code, a Skill is an extension that streamlines a specific task. For Firecrawl, that typically means exposing the major functions as dedicated Skillsâso you can run them as predictable, repeatable slash commands.
If MCP is the mechanism that connects Firecrawl to Claude Code, Skills are the mechanism that teaches Claude Code how to use Firecrawl well: when to use Search vs Map, when to switch to Browser, where to store outputs, and how to treat scraped content safely.
Skill list
| Skill | How to call it | Purpose |
|---|---|---|
| firecrawl | /firecrawl |
Orchestrates the workflow and helps decide which feature to use |
| firecrawl-search | /firecrawl-search |
Web search plus full-content retrieval |
| firecrawl-scrape | /firecrawl-scrape |
Extract content from a specific URL |
| firecrawl-map | /firecrawl-map |
Discover and map URLs within a site |
| firecrawl-crawl | /firecrawl-crawl |
Bulk extraction for an entire site or section |
| firecrawl-browser | /firecrawl-browser |
Cloud browser automation (clicks, forms, login) |
| firecrawl-agent | /firecrawl-agent |
AI-driven autonomous extraction with structured JSON output |
| firecrawl-download | /firecrawl-download |
Save an entire site as local files |
What changes when you use a Skill?
If you call the CLI directly, you must decide (every time) which command to use, which options to pass, and where to write outputs. With Skills, Claude Code can automate those decisions and apply consistent operational defaults.
For example, if you say: âCheck this siteâs pricing page,â a Skill-driven flow can decide to:
- Use Map (or Search) to locate the pricing URL
- Scrape the page with
--only-main-contentto remove noise - Save results to
.firecrawl/to avoid bloating your context window
Skills can also include safety guidance by defaultâtreating scraped web content as untrusted third-party data to reduce prompt injection risk and keep outputs isolated as files.
CLI vs MCP vs Skill
| Dimension | CLI | MCP | Skill |
|---|---|---|---|
| Best for | Local testing, CI automation, deep debugging, browser workflows | Calling Firecrawl as a tool from Claude Code | Slash commands with opinionated, automated feature selection |
| How you run it | Terminal commands | Claude Code invokes tools automatically | Start with /firecrawl-scrape etc. |
| Choosing the right feature | Manual | Agent-driven | Skill provides a guided workflow |
| Output management | You choose | Often shown in the chat context | Typically saved to .firecrawl/ by default |
| Setup effort | Node environment + authentication | One command + API key | Skill file placement + CLI auth (varies by setup) |
| Strengths | Traceability and logs, fine-grained control | Fastest path to âit worksâ | Automation + safety defaults + consistent workflows |
In practice, teams often combine them: use Skills for everyday research tasks, then fall back to the CLI when something fails and you need to inspect flags, output formats, or network behavior.
Whatâs actually impressive about Firecrawl?
Firecrawlâs value isnât just âfetch a web page.â Itâs that Firecrawl produces content in formats that are easy to feed into LLM workflowsâand itâs designed to slot into automation pipelines.
LLM-optimized outputs
Firecrawl can return Markdown, HTML, structured JSON, and screenshots. For Markdown output, it automatically strips boilerplate like nav bars and footers and keeps the main content. It also adds useful metadata (for example title, description, status codes, and source URL), which makes provenance and debugging much easier.
Works on dynamic pages
Many scraping workflows break on JavaScript-rendered SPAs or client-side-rendered pages. Firecrawl is built to handle those cases, and it also acknowledges real-world operational issues like bot detection and proxy routing. When a page still canât be fetched reliably, Browser lets you interact with it directly (click flows, login, pagination) and extract the rendered state.
Structured extraction without hand-rolled parsers
Scrape can output JSON in addition to Markdown, and features like Extract and Agent are aimed at pulling consistent structured data across multiple pages. That reduces the amount of âparse HTML â normalize fields â validate schemaâ glue code you typically need to maintain.
Decision automation via Skills
The strongest argument for the Skill approach is workflow automation. Instead of you remembering when to use Search vs Map vs Crawlâand which flags prevent noisy outputsâSkills can turn a vague request (âresearch this siteâ) into concrete, repeatable steps like Search â Map â Scrape, with sensible options and output handling.
A smooth loop inside Claude Code
When Firecrawl is integrated into Claude Code (via MCP and/or Skills), you can run a complete loop inside your editor: collect sources, summarize key sections, and then implement changes or write documentation against those sources without constantly switching tools.
Credit usage: a practical baseline
Firecrawl uses credit-based usage billing. Different features and options consume different amounts of credits, so it helps to understand the âshapeâ of costs before you automate large jobs.
| Action | Credit usage |
|---|---|
| Scrape (per page) | 1 |
| Scrape + JSON mode (per page) | 5 (1 + 4) |
| Scrape + Enhanced Proxy (per page) | 5 (1 + 4) |
| Crawl (per page) | 1 |
| Map (per request) | 1 (fixed, regardless of URL count) |
| Search (10 results) | 2 |
| Browser (per minute) | 2 |
Cost optimization tips
- Start with Map: Map costs 1 credit per request, so you can list URLs and narrow scope before spending credits on many Scrape calls.
- Use JSON mode only when you need it: If Markdown is enough, avoid the +4 credits per page for JSON mode.
- Leverage caching: Re-fetching the same pages can be cheaper operationally if caching is enabled (and you can disable caching for sensitive workflows if needed).
Where it helps in real work
Requirements and technical research
Firecrawl is useful when you need to read multiple pages quickly: competitor feature lists, pricing comparisons, or tracking differences between SDK/API versions. A practical workflow is: Map to enumerate candidate URLs, then Scrape only the pages that matter.
# List competitor URLs and find pricing pages
firecrawl map "https://competitor.com" --search "pricing" -o .firecrawl/pricing-urls.txt
# Scrape the pricing page you found
firecrawl scrape "https://competitor.com/pricing" --only-main-content -o .firecrawl/pricing.mdKeeping up with documentation changes
You can crawl an SDK/API documentation site, extract key changes, and sync them into internal docs (for example a team wiki). Because Crawl runs as a job, itâs easier to manage completion and page-count limits in operations.
# Bulk-fetch the /api section of a docs site
firecrawl crawl "https://docs.example.com" --include-paths /api --limit 100 --wait -o .firecrawl/api-docs.jsonDebugging difficult dynamic sites
If headless scraping fails (missing elements, post-login pages, heavy client-side rendering), Browser can make troubleshooting easier. Capture snapshots to inspect the DOM and identify the minimum set of interactions needed before you extract content.
Collecting structured datasets
If you need structured dataâjob listings, ecommerce product catalogs, SaaS pricing tiersâExtract and Agent can help standardize output across many pages. With a JSON schema, you can enforce consistent fields and reduce downstream normalization work.
Important cautions and operating guidelines
Always verify legal and policy requirements.
- Follow the target siteâs terms of service, robots.txt, and privacy policy.
- Avoid collecting or storing personal or sensitive data (mask/minimize if you must handle it).
- Prevent undue load (rate limits, caching, and controlled concurrency).
Firecrawlâs public materials also emphasize that itâs your responsibility to respect site policies and run scraping ethically. Treat that as a non-negotiable constraint when you design production pipelines.
Security considerations
When you bring web content into an LLM workflow, treat it as untrusted third-party input. A practical set of guardrails looks like this:
- Prefer file output: Save results with
-o .firecrawl/to reduce prompt injection risk and avoid bloating your context window. - Read large files in chunks: Donât load entire dumps into contextâextract only what you need.
- Quote URLs: Wrap URLs in quotes to avoid shell interpretation of characters like
?and&.
Need Help with Web Scraping?
Whether you're exploring automation with Firecrawl, struggling with complex site structures, or need reliable data extraction at scale â our scraping specialists can help. From requirements analysis to implementation and maintenance, we've got you covered.
Summary
- Firecrawl organizes web data collection into six core features: Scrape, Crawl, Map, Search, Extract, and Browser.
- The escalation pattern (Search â Scrape â Map â Crawl â Browser) helps you start simple and switch to advanced tools only when necessary.
- In Claude Code, MCP is the fastest way to integrate Firecrawl, while Skills can automate feature selection, flags, and output management.
- Use Map and caching strategically to reduce credit spend, and avoid JSON mode unless you truly need structured output.
- Treat scraped content as untrusted input and default to file-based workflows to reduce security risk.
Review the official documentation and choose the right integration path for your workflow: CLI (debugging/automation), MCP (quick integration), or Skills (workflow automation).
