Provider Architecture & Mechanics

How Vyla SDK sources are structured, scheduled, verified, proxied, and cached across the ecosystem.

Overview

Rather than relying on a single upstream host, the Vyla SDK aggregates over 60 independent provider scrapers. Each scraper handles its own domain logic, extraction mechanics, and header requirements.

Provider Configuration

Scrapers are configured in src/config.js:

export interface SourceConfig {
    key: string;
    label: string;
    sourceFile: string;
    proxyParam: string;
    timeout: number;
    sourcesTimeout?: number;
    jitter: number;
    retries: number;
    multiUrl: boolean;
    skipProxy?: boolean;
    skipVerify?: boolean;
    verifyHeaders?: boolean;
    disabled?: boolean;
    cdnHeaders?: {
        pattern: RegExp;
        headers: { Referer: string; Origin: string; };
    }[];
}

Query Pipeline

Source Filtering: Excludes disabled scrapers and loads active configurations.
Jitter Delay: Sleeps for a randomized duration between 0 and cfg.jitter ms to avoid burst requests.
Concurrent Scraping: Runs scrapers in parallel with provider-specific User-Agent, Referer, and Origin headers.
Two-Phase Verification: Validates candidate URLs via verifyStream() (HTTP status < 400) and verifyPlayable() (HLS playlist and segment checks).
Proxy URL Rewriting: Encodes verified streams into /api?url=... with required proxy flags.
Progressive Emission: Pushes candidate streams over Server-Sent Events (SSE) immediately.

Jitter Staggering

To prevent triggering upstream rate limits when querying dozens of scrapers simultaneously, each scraper applies random jitter:

const jitterDelay = Math.floor(Math.random() * (cfg.jitter || 400));
await new Promise(resolve => setTimeout(resolve, jitterDelay));

This spreads outbound requests across a 100ms–800ms window, reducing HTTP 429 errors.

Two-Phase Verification

Phase 1 (verifyStream): Sends a lightweight HTTP HEAD request with source-specific headers. Discards any response >= 400.
Phase 2 (verifyPlayable): Fetches the initial playlist bytes, verifies the #EXTM3U header, confirms playable segment paths, and tests the first chunk or key URI.

Only streams that pass both checks are returned to the client.

Proxy & Header Stripping

M3U8 Rewriting: Rewrites AES-128 encryption key URIs and .ts segment paths to route through the proxy with CORS headers (Access-Control-Allow-Origin: *).
120-Byte Mask Stripping: Slices off fake 120-byte image headers prepended by certain CDNs:

const needsStrip = searchParams.has('tt') || STRIP_REGEX.test(cleanUrl);
if (needsStrip && (chunk[0] === 0x89 || chunk[0] === 0xFF || chunk[0] === 0x00)) {
    return chunk.subarray(120);
}

In-Memory Caching

Verified stream candidates are cached in-memory for 5 minutes (300,000 ms):

Cache keys format: source_key + tmdb_id + season + episode.
In cluster mode, workers coordinate entries via IPC messages (cache:get, cache:hit, cache:set).
Expired entries are pruned every 30 seconds, maintaining a 1,500-entry ceiling.

3.2 KiB Raw Blame History Unescape Escape