Docs

Documentation

Learn how to automate your web scraping workflows with ActiCrawl

Authentication

ActiCrawl API uses API keys to authenticate requests. You can view and manage your API keys in your dashboard.

Getting Your API Key

  1. Sign up for an ActiCrawl account
  2. Navigate to your dashboard
  3. Click on API Keys in the sidebar
  4. Click Generate New API Key
  5. Copy your API key and store it securely

Important: Your API key is sensitive information. Never share it publicly or commit it to version control.

Using Your API Key

There are two ways to authenticate with the ActiCrawl API:

1. HTTP Header (Recommended)

Include your API key in the X-API-Key header:

bash
curl -X POST https://api.acticrawl.com/v1/scrape \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

2. Query Parameter

Include your API key as a query parameter:

bash
curl -X POST "https://api.acticrawl.com/v1/scrape?api_key=YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Note: Using the header method is more secure as query parameters may be logged in server access logs.

Authentication Examples

JavaScript (Node.js)

javascript
const axios = require('axios');

const apiKey = 'YOUR_API_KEY';
const apiUrl = 'https://api.acticrawl.com/v1/scrape';

async function scrapeWebsite(url) {
  try {
    const response = await axios.post(apiUrl, {
      url: url,
      formats: ['markdown', 'screenshot']
    }, {
      headers: {
        'X-API-Key': apiKey,
        'Content-Type': 'application/json'
      }
    });

    return response.data;
  } catch (error) {
    console.error('Error:', error.response.data);
  }
}

scrapeWebsite('https://example.com').then(console.log);

Python

python
import requests

api_key = 'YOUR_API_KEY'
api_url = 'https://api.acticrawl.com/v1/scrape'

def scrape_website(url):
    headers = {
        'X-API-Key': api_key,
        'Content-Type': 'application/json'
    }

    payload = {
        'url': url,
        'formats': ['markdown', 'screenshot']
    }

    response = requests.post(api_url, json=payload, headers=headers)

    if response.status_code == 200:
        return response.json()
    else:
        print(f"Error: {response.status_code} - {response.text}")
        return None

result = scrape_website('https://example.com')
print(result)

PHP

php
<?php
$apiKey = 'YOUR_API_KEY';
$apiUrl = 'https://api.acticrawl.com/v1/scrape';

$data = [
    'url' => 'https://example.com',
    'formats' => ['markdown', 'screenshot']
];

$ch = curl_init($apiUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'X-API-Key: ' . $apiKey,
    'Content-Type: application/json'
]);

$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

if ($httpCode === 200) {
    $result = json_decode($response, true);
    print_r($result);
} else {
    echo "Error: HTTP $httpCode - $response\n";
}
?>

Ruby

ruby
require 'net/http'
require 'json'
require 'uri'

api_key = 'YOUR_API_KEY'
api_url = 'https://api.acticrawl.com/v1/scrape'

uri = URI(api_url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

request = Net::HTTP::Post.new(uri)
request['X-API-Key'] = api_key
request['Content-Type'] = 'application/json'
request.body = {
  url: 'https://example.com',
  formats: ['markdown', 'screenshot']
}.to_json

response = http.request(request)

if response.code == '200'
  result = JSON.parse(response.body)
  puts result
else
  puts "Error: #{response.code} - #{response.body}"
end

API Key Security Best Practices

1. Environment Variables

Store your API key in environment variables instead of hardcoding:

bash
# .env file
ACTICRAWL_API_KEY=your_api_key_here
javascript
// Node.js
const apiKey = process.env.ACTICRAWL_API_KEY;
python
# Python
import os
api_key = os.environ.get('ACTICRAWL_API_KEY')

2. Server-Side Usage

Always use your API key on the server-side. Never expose it in client-side code:

javascript
// ❌ Bad: Client-side code
fetch('https://api.acticrawl.com/v1/scrape', {
  headers: {
    'X-API-Key': 'YOUR_API_KEY' // Exposed to users!
  }
});

// ✅ Good: Server-side proxy
// Client calls your server
fetch('/api/scrape', {
  method: 'POST',
  body: JSON.stringify({ url: 'https://example.com' })
});

// Your server calls ActiCrawl API with the key

3. Key Rotation

Regularly rotate your API keys:

  1. Generate a new API key
  2. Update your application to use the new key
  3. Test to ensure everything works
  4. Delete the old API key

4. Restrict Key Permissions

If you have multiple applications, create separate API keys for each with appropriate permissions.

Rate Limiting

API keys are subject to rate limits based on your plan:

Plan Rate Limit Burst Limit
Free 10 requests/minute 20 requests
Starter 30 requests/minute 60 requests
Professional 60 requests/minute 120 requests
Business 120 requests/minute 240 requests
Enterprise Custom Custom

If you exceed the rate limit, you'll receive a 429 Too Many Requests response:

json
{
  "success": false,
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Rate limit exceeded. Please retry after 60 seconds.",
    "retry_after": 60
  }
}

Error Handling

Authentication errors return a 401 Unauthorized response:

json
{
  "success": false,
  "error": {
    "code": "INVALID_API_KEY",
    "message": "Invalid or missing API key"
  }
}

Common authentication errors:

Error Code Description Solution
INVALID_API_KEY API key is invalid or has been deleted Check your API key is correct
MISSING_API_KEY No API key provided Include your API key in the request
EXPIRED_API_KEY API key has expired Generate a new API key
SUSPENDED_API_KEY API key has been suspended Contact support

Testing Authentication

You can test your API key with a simple request:

bash
curl -X GET https://api.acticrawl.com/v1/account \
  -H "X-API-Key: YOUR_API_KEY"

A successful response confirms your API key is valid:

json
{
  "success": true,
  "data": {
    "email": "user@example.com",
    "plan": "Professional",
    "credits": {
      "used": 1250,
      "limit": 10000
    }
  }
}