Bulk API Overview
The Bulk API allows you to process multiple URLs efficiently in a single request. This endpoint is available to users on paid plans and provides a queue-based system for handling large batches of URLs.
How It Works
1. Submit your URLs using the POST/v1.1/bulk endpoint
2. Receive a bulk_id and queue confirmation
3. Poll for results using the GET/v1.1/bulk endpoint with your bulk_id
Response Examples
When you submit URLs, you'll receive a response like this:
{
"count": 921,
"bulk_id": "4a7d9ca68177713593c713cc584a7edf9ee9017c"
}
When retrieving results, you'll get data in this format:
{
"data": {
"http://example.com/": {
"Facebook": {
"total_count": 1234,
"comment_count": 56,
"reaction_count": 789,
"share_count": 1234
},
"Pinterest": 567
}
},
"_meta": {
"urls_completed": 100,
"bulk_id": "abc123...",
"completed": true,
"urls_queued": 100
}
}
Batch Size Limits
The maximum number of URLs per batch depends on your plan:
Plan | URL Limit |
---|---|
Personal | 500 URLs |
Professional | 1,000 URLs |
Enterprise | 10,000 URLs |
Note: Each URL in a batch counts against your daily API quota. Requests exceeding your quota will be rejected.
Best Practices
Follow these guidelines for optimal bulk API usage:
Practice | Details |
---|---|
Polling Frequency | Poll every 30-60 seconds to avoid overwhelming the API |
Force Parameter | Use &force=1 for long-running jobs to get partial results |
Progress Monitoring | Check the _meta.completed field to know when all URLs are processed |
Debugging | Keep track of your bulk_id for troubleshooting and support |
Real-time Progress | Monitor urls_completed vs urls_queued fields to track processing progress in real-time |
For detailed implementation examples and complete API reference, see the POST/v1.1/bulk and GET/v1.1/bulk endpoint documentation.