Provider Architecture & Mechanics

How Vyla SDK sources are structured, scheduled, verified, proxied, and cached across the ecosystem.

Overview

At the heart of the Vyla ecosystem is the Provider Layer. Rather than relying on a single upstream video host, Vyla SDK aggregates over 60 independent, community-maintained scraping resolvers.

Each provider operates with distinct upstream mechanics, domain rotations, rate-limiting policies, and security defenses. This document explains how Vyla orchestrates these disparate sources into a unified, high-reliability streaming pipeline.

Provider Configuration Interface

In @vyla-entertainment/sdk, each provider is declared in src/config.js via the SourceConfig interface:

export interface SourceConfig {
    key: string;              // Unique source identifier (e.g. "vidlink", "vixsrc")
    label: string;            // Human-readable display label
    sourceFile: string;       // Scraper module filename in src/sources/
    proxyParam: string;       // Query parameter used for HLS proxy routing
    timeout: number;          // Total milliseconds allowed before source abort
    sourcesTimeout?: number;  // Secondary timeout for multi-server extraction
    jitter: number;           // Maximum randomized startup delay in ms
    retries: number;          // Number of automatic retry attempts on error
    multiUrl: boolean;        // Returns multiple stream candidate URLs if true
    skipProxy?: boolean;      // If true, returns direct upstream URLs without proxy
    skipVerify?: boolean;     // If true, bypasses verification checks
    verifyHeaders?: boolean;  // Injects custom headers into verification probes
    disabled?: boolean;       // If true, provider is completely excluded
    cdnHeaders?: {            // Custom headers injected for specific CDN patterns
        pattern: RegExp;
        headers: { Referer: string; Origin: string; };
    }[];
}

The Query Pipeline

When a client requests a title (getStream() in SDK or /movie / /tv in Stream API), the engine executes the following pipeline:

Pipeline Stage	Operation	Purpose & Verification
1. Source Filtering	`SOURCES.filter(!disabled)`	Excludes disabled scrapers and selects active providers based on configuration.
2. Jitter Staggering	Sleep between `0` and `cfg.jitter` ms	Disperses outbound connection spikes across time to prevent DDoS protection triggers.
3. Concurrent Scraping	Parallel async scraper execution	Scrapers parse HTML/JSON endpoints using customized `User-Agent`, `Referer`, and `Origin` headers.
4. Two-Phase Verification	`verifyStream()` then `verifyPlayable()`	Phase 1 validates HTTP status (`< 400` via HEAD). Phase 2 parses HLS manifests and verifies segment availability. Dead streams are discarded.
5. Proxy URL Rewriting	Encodes target URL into `/api?url=...`	Adds proxy flags, sets required playback headers, and prepares segment de-obfuscation.
6. Progressive Emission	Pushes SSE `source` event	Delivers playable stream candidates to the client player immediately without waiting for slower providers.

Jitter Staggering & Rate-Limit Defense

If a client triggers 60 scrapers instantaneously, your client IP address could send hundreds of simultaneous HTTP requests in the same millisecond, triggering upstream Cloudflare or DDoS rate limits.

To mitigate this, Vyla introduces per-source jitter:

const jitterDelay = Math.floor(Math.random() * (cfg.jitter || 400));
await new Promise(resolve => setTimeout(resolve, jitterDelay));

By staggering outbound requests across a 100ms–800ms window, upstream hosts perceive natural traffic patterns, dramatically reducing HTTP 429 (Too Many Requests) errors.

Two-Phase Playback Verification

Many streaming extractors successfully return a URL, only for that URL to return an HTTP 403 Forbidden, an empty manifest, or a dead video chunk. To protect players from stalling, Vyla validates every candidate through two independent verification stages before emitting it:

Phase 1: `verifyStream()`

Dispatches a lightweight HTTP HEAD request to the upstream target URL.
Enforces custom User-Agent, Referer, and Origin headers.
Requires HTTP status code < 400 within 4,000 milliseconds.

Phase 2: `verifyPlayable()`

Fetches the initial playlist bytes through the local proxy.
Validates the presence of the #EXTM3U header tag.
Checks for stream definitions:
- If a Master Playlist: ensures valid #EXT-X-STREAM-INF entries exist.
- If a Media Playlist: ensures valid #EXTINF duration tags and playable segment URIs exist.
Attempts a test read on the first media segment or key URI.
Only candidates that pass both phases are deemed verified.

The Universal HLS Proxy & Byte-Mask Stripper

Modern streaming hosts often protect their video segments by adding CORS restrictions, token signatures, or binary header masking. Stream API handles this transparently:

1. M3U8 Playlist Rewriting

When an HLS stream is proxied, Stream API fetches the upstream playlist and rewrites every internal URI:

AES-128 Key URIs: #EXT-X-KEY:METHOD=AES-128,URI="/api?url=<key>&<param>=1"
Video Chunks: Segment URIs are rewritten to route back through the /api proxy endpoint.
CORS Headers: Returned with Access-Control-Allow-Origin: * and Access-Control-Allow-Headers: *.

2. Segment Byte-Mask Stripping

Certain major CDNs mask MPEG-TS chunks by prepending a 120-byte fake image header (image/png, image/jpeg, TikTok CDN, or letsgocdn signatures) so firewalls treat them as images.

When Stream API proxies these segments, it inspects the leading bytes and automatically strips the 120-byte image wrapper:

const needsStrip = searchParams.has('tt') || STRIP_REGEX.test(cleanUrl);
if (needsStrip && (chunk[0] === 0x89 || chunk[0] === 0xFF || chunk[0] === 0x00)) {
    return chunk.subarray(120); // Strips fake header, leaving clean MPEG-TS
}

Inter-Process Caching

Verified stream results are cached in-memory with a 5-minute Time-To-Live (TTL):

Cache Key: source_key + tmdb_id + season + episode
In multi-core cluster deployments, cache entries are coordinated across workers via inter-process communication (IPC) messages (cache:get, cache:hit, cache:set).
Automatic prune routines run every 30 seconds to evict expired items and enforce maximum cache limits (default: 1,500 entries).

6.5 KiB Raw Blame History Unescape Escape