API Endpoints

ActiCrawl provides a RESTful API for web scraping and data extraction. All API requests should be made to the base URL with your API key for authentication.

Base URL

https://www.acticrawl.com/api/v1

Available Endpoints

1. Scrape Single URL

Extract content from a single web page with multiple output format options.

POST /scrape

Request Headers

Authorization: Bearer YOUR_API_KEY
Content-Type: application/json

Request Body

{
  "url": "https://example.com",
  "output_format": "markdown",
  "execute_js": true,
  "wait_for": 3000,
  "timeout": 30000,
  "extract_main_content": true,
  "use_premium_proxy": false,
  "exclude_tags": "script,style,nav",
  "include_only_tags": "article,main",
  "scraping_mode": "real"
}

Parameters

Output Formats

• html - Raw HTML content (default)
• markdown - Converted to Markdown format
• html_cleaned - Cleaned HTML without scripts/styles
• links - All links extracted from the page
• json - Structured JSON with title, content, and links
• screenshot - Page screenshot (when available)

Response

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "content": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "format": "markdown",
    "metadata": {
      "title": "Example Domain",
      "processing_time": 1.5,
      "plan": "Free",
      "scraping_mode": "real",
      "scraped_at": "2024-01-15T10:30:00Z",
      "response_time": 1500
    }
  }
}

Response Fields

Error Response

{
  "success": false,
  "error": "Rate limit exceeded",
  "data": {
    "url": "https://example.com",
    "error": "Rate limit exceeded",
    "metadata": {
      "processing_time": 0.1,
      "plan": "Free",
      "scraping_mode": "simulation"
    }
  }
}

Rate Limiting

API rate limits are based on your subscription plan:

• Free Plan: 100 requests/month, 1 concurrent request
• Basic Plan: 10,000 requests/month, 5 concurrent requests
• Pro Plan: 100,000 requests/month, 20 concurrent requests
• Enterprise Plan: Unlimited requests, 100 concurrent requests

Rate limit information is included in response headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1704844800

Examples

Basic HTML Scraping

curl -X POST https://www.acticrawl.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Markdown Extraction

curl -X POST https://www.acticrawl.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "output_format": "markdown",
    "extract_main_content": true
  }'

JavaScript-Rendered Content

curl -X POST https://www.acticrawl.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "output_format": "html_cleaned",
    "execute_js": true,
    "wait_for": 5000
  }'

Link Extraction

curl -X POST https://www.acticrawl.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "output_format": "links"
  }'

Structured JSON Data

curl -X POST https://www.acticrawl.com/api/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "output_format": "json",
    "extract_main_content": true
  }'

Best Practices

1. Use appropriate output formats - Choose the format that best suits your needs to minimize processing time
2. Set reasonable timeouts - Adjust timeout based on the complexity of the target site
3. Use JavaScript rendering sparingly - Only enable when necessary as it increases processing time
4. Implement retry logic - Handle transient failures with exponential backoff
5. Cache results - Store scraped data to avoid unnecessary repeated requests
6. Monitor rate limits - Check response headers to track your usage

Error Codes