Quick Start

Get up and running with Crawly in under 5 minutes.

This guide walks you through creating an API key, scraping your first webpage, extracting a YouTube transcript, and handling errors. By the end you will have working code in cURL, JavaScript, or Python.

1. Get Your API Key

Sign up at crawly.bikal.co and create an API key from the API Keys page. You get 100 free credits on sign up, no credit card required.

Copy the key immediately. It starts with cr_ and is only shown once.

2. Scrape a Webpage

Send any URL to the scrape endpoint and get back clean Markdown with metadata:

curl -X POST https://api.crawly.bikal.co/v1/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

3. Read the Response

A successful response includes the page content as Markdown, along with metadata like the title, description, language, and status code:

json
{
  "success": true,
  "url": "https://example.com",
  "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
  "metadata": {
    "title": "Example Domain",
    "description": "Example Domain for documentation",
    "language": "en",
    "status_code": 200
  }
}

4. Get a YouTube Transcript

Extract transcripts from any YouTube video. Pass a YouTube URL and choose your preferred format:

bash
curl -X POST https://api.crawly.bikal.co/v1/transcript \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://youtube.com/watch?v=dQw4w9WgXcQ", "format": "text"}'

Supported formats: text (plain transcript), segments (timed text), srt (SubRip subtitles), vtt (WebVTT subtitles).

5. Handle Errors

If something goes wrong, the response will have success: false with an error message and type:

json
{
  "success": false,
  "error": "Missing API key. Use Authorization: Bearer <key>",
  "error_type": "AUTH_ERROR"
}

See the Error Handling guide for a full list of error types and how to handle them.

Next Steps