huge rewrite

This commit is contained in:
2026-11-14 18:21:12 -08:00
parent f465d1ec0d
commit 2cec85a8f3
69 changed files with 11479 additions and 1576 deletions

View File

@@ -1,44 +1,143 @@
# Our Architecture
# Architecture & System Design
<p class="page-description">How Vyla Home, SDK, and Stream API relate, and where each responsibility belongs.</p>
<p class="page-description">Deep architectural breakdown of the Vyla Ecosystem: component boundaries, communication protocols, video delivery pipelines, and failover topologies.</p>
## The Vyla stack
## High-Level Architecture
Vyla separates local application management, provider resolution, and client-facing delivery. That separation lets each layer be used on its own while keeping the integration points clear.
The Vyla architecture cleanly separates **presentation**, **gateway mediation**, and **content resolution**. This decoupling ensures that any individual layer can be replaced, scaled, or run independently.
### Vyla Home
| Architecture Tier | Components | Protocols & Interfaces | Core Responsibilities |
| :--- | :--- | :--- | :--- |
| **Presentation Tier** | Vyla Home, Vyla Go, Vyla TV, Third-Party Web Players | SSE (HTTP/1.1), REST (JSON), HLS / DASH | Render user interface, manage local state, execute video playback via HLS.js or ExoPlayer Media3. |
| **Gateway Tier** | Stream API (`server.js`) | Node.js Cluster IPC, HTTP/REST, SSE, Reverse Proxy | Expose public endpoints, manage 1,500-entry in-memory cache, rewrite M3U8 playlists & AES keys, strip 120-byte obfuscation masks. |
| **Resolver Tier** | Vyla SDK (`@vyla-entertainment/sdk`) | Direct ESM function calls, HTTP/HTTPS scrapers | Scrape 60+ upstream providers concurrently, enforce random jitter delays, run two-phase stream validation. |
| **Upstream Hosts** | Public Third-Party CDNs & Video Hosts | Encrypted TLS/HTTPS, M3U8, MP4 | Deliver raw media segments and video assets. |
Home is responsible for a desktop session: local configuration, encrypted local credentials, update coordination, managed processes, and the embedded interface. It is the right layer for launching and supervising a local experience.
---
### Vyla SDK
## Component Boundaries & Responsibilities
The SDK contains the provider-facing work: source configuration, stream resolution, subtitle retrieval, manga discovery and page resolution, download lookup, and health probes. It is the right layer when your Node.js application needs to decide how requests are authorized, cached, selected, and returned.
### 1. Vyla Home (Desktop)
- **Runtime Environment**: Built on [Tauri v2](https://v2.tauri.app/) using Rust and an embedded Node.js sidecar service.
- **IPC Protocol**: Communicates with the sidecar process via standard input/output (`stdin`/`stdout`) passing JSON-RPC messages.
- **Responsibilities**:
- Manages application lifecycle, window management, and native system tray.
- Automatically checks, verifies cryptographic signatures (`update-signing.pem`), and extracts delta updates from GitLab releases.
- Houses the local `ServiceManager` and `Patcher` for offline and local service fallback.
- Dispatches Discord Rich Presence activity updates (`discord-rpc.js`).
- Persists configuration and cache to platform-specific AppData locations.
### Stream API
### 2. Vyla Go (Mobile)
- **Runtime Environment**: Tauri v2 for Android (`tauri-android`) targeting ARM64 (`aarch64`) and x86_64 architectures.
- **Responsibilities**:
- Touch-optimized interface utilizing WebGL hardware shaders and glassmorphism styling.
- Automated resource management: fetches, unpacks, and hot-patches runtime assets directly from remote releases without requiring full APK reinstallations for minor script updates.
- Background playback coordination and media notification integration.
Stream API wraps the SDK in a self-hosted HTTP service. Its movie and TV endpoints use Server-Sent Events so metadata can arrive first and usable sources can be delivered progressively. It also provides subtitle, download, health, test, and optional debug routes.
### 3. Vyla TV (Leanback)
- **Runtime Environment**: Native Android application written in Kotlin utilizing Jetpack Compose and Google's [ExoPlayer (Media3)](https://developer.android.com/media/media3/exoplayer).
- **Responsibilities**:
- Strictly optimized for D-pad remote controls, 10-foot viewing distances, and Android TV / Google TV operating systems.
- Native HLS stream parsing, hardware-accelerated 4K/HDR decoding, playback speed manipulation, audio track selection, and subtitle overlay.
- Local client storage for user favorites, watch progress resume, and TMDB catalog browsing.
## Common deployment patterns
### 4. Stream API (Gateway)
- **Runtime Environment**: Node.js microservice utilizing the native `cluster` module for multi-core process forking.
- **Responsibilities**:
- Acts as an HTTP/SSE bridge for clients that cannot run the Node.js SDK directly (e.g. mobile apps, TV apps, browser frontends).
- Emits real-time playback events (`meta`, `source`, `done`) over Server-Sent Events.
- Proxies HLS media streams (`/api?url=...`), automatically rewriting playlist chunk URIs and AES encryption key endpoints to bypass browser CORS and network restrictions.
- Strips obfuscation masks (e.g. 120-byte fake PNG/JPEG headers used by ByteDance/TikTok-based content delivery networks).
- Caches verified results for 5 minutes (TTL = 300,000 ms) in a shared inter-process memory map.
| Pattern | Best for | Key decision |
| --- | --- | --- |
| Home only | a managed local desktop experience | Home owns the lifecycle of bundled local services |
| App + SDK | a Node.js backend with a custom product API | your app owns authentication, policy, and response design |
| Player + Stream API | browser or mobile clients that need a stable HTTP contract | deploy Stream API and place it behind TLS when exposed remotely |
| Home + Stream API | a local player backed by a local service | keep the service bound and access-controlled for the local environment |
### 5. Vyla SDK (`@vyla-entertainment/sdk`)
- **Runtime Environment**: Pure ES Module Node.js library.
- **Responsibilities**:
- Houses the core provider scraping modules (60+ scrapers in `src/sources/`).
- Implements jitter scheduling, retry backoff, and custom header injection (`User-Agent`, `Referer`, `Origin`).
- Performs two-phase verification on resolved candidate URLs.
- Houses the Manga resolver engine leveraging `node-tls-client` to bypass TLS fingerprint blocking on `atsu.moe`.
## Data flow for playback
---
1. A client provides a TMDB media ID (plus season and episode for TV).
2. Stream API, or your own SDK-based backend, asks active providers for candidates.
3. Candidates are checked and successful sources are returned progressively.
4. The player begins with the first compatible candidate and keeps later candidates as fallbacks.
5. Subtitle and download lookups are separate operations; neither should block first playback.
## Playback Data Flow: Step-by-Step
Provider availability varies by source and network environment. Build the player experience around fallbacks and empty-result states, as described in [Error handling](/guides/error-handling).
Understanding how a media stream is discovered, verified, and played:
## Data flow for manga
1. **Client Establishes SSE Connection**
The viewer selects a title (e.g., TMDB ID `155`), and the client initiates an HTTP Server-Sent Events handshake:
```http
GET /movie?id=155 HTTP/1.1
Host: api.vyla.cc
Accept: text/event-stream
```
1. A client provides a search term, or direct manga identifier.
2. The SDK queries the manga provider for title details, metadata, and chapter listings.
3. When reading a chapter, the SDK resolves ordered page assets with dimensions and aspect ratios for responsive layout rendering.
2. **Stream API Emits Initial Metadata**
Stream API validates the identifier, queries TMDB for metadata and artwork, queries subtitle providers, and immediately delivers a `meta` event:
```json
data: {"type": "meta", "meta": {"title": "The Dark Knight", "year": 2008}, "subtitles": [{"lang": "English", "url": "..."}]}
```
3. **SDK Initiates Concurrent Provider Queries**
The SDK filters active provider scrapers from configuration, applies randomized per-source jitter (0–500 ms) to avoid burst-rate triggers, and initiates upstream HTTP scraping requests concurrently.
4. **Two-Phase Stream Verification**
- **Phase A (`verifyStream`)**: Executes an HTTP `HEAD` or range `GET` request against candidate stream URLs. Responses with status $\ge 400$ are discarded immediately.
- **Phase B (`verifyPlayable`)**: Downloads and parses the master HLS manifest (`#EXT-X-STREAM-INF`), validates chunk resolution, and tests segment availability.
5. **Progressive Source Delivery**
As each provider scraper completes two-phase verification, Stream API immediately pushes a `source` event over the active SSE stream:
```json
data: {"type": "source", "source": "vidlink", "url": "https://api.vyla.cc/api?url=...", "quality": "1080p", "type": "hls"}
```
6. **Instant Client Playback**
The client media player (ExoPlayer, HLS.js, or AVPlayer) immediately begins playback upon receiving the first valid source event (typically within 3–8 seconds). Subsequent incoming sources are enqueued into a client-side fallback list.
7. **Stream Completion**
Once all configured source scrapers have either returned verified streams or reached their timeout thresholds (typically 15–25 seconds), Stream API emits the completion event and closes the connection:
```json
data: {"type": "done", "total": 4}
```
---
## The HLS Proxy & De-Obfuscation Mechanics
Many upstream streaming hosts employ strict CORS restrictions, IP-bound tokens, or binary header obfuscation. Vyla resolves this transparently:
### Playlist & Key Rewriting
When a stream URL is proxied through `/api?url=<URL>&<param>=1`, the gateway:
1. Requests the upstream playlist with provider-mandated `Referer` and `Origin` headers.
2. Intercepts `#EXT-X-KEY:METHOD=AES-128,URI="..."` and rewrites the key URI to route through the proxy.
3. Rewrites all chunk lines (`.ts` / `.m4s`) so that browser audio/video fetch calls pass through the proxy with valid `Access-Control-Allow-Origin: *` headers.
### Byte-Mask Stripping
Certain modern providers mask `.ts` video chunks as fake image files (`image/png`, `image/jpeg`, `letsgocdn`, or TikTok CDN formats) by prepending a 120-byte image header before the MPEG-TS sync byte (`0x47`):
```
[ 120-byte Fake PNG Header (0x89 0x50 0x4E 0x47...) ] + [ MPEG-TS Payload (0x47...) ]
```
When Stream API detects these signatures via regular expressions (`TIKTOK_REGEX`, `STRIP_REGEX`), it slices off the first 120 bytes:
```javascript
const stripped = (full[0] === 0x89 || full[0] === 0xFF || full[0] === 0x00)
? full.subarray(120)
: full;
```
The resulting clean MPEG-TS stream is delivered to players with `Content-Type: video/MP2T`, enabling seamless playback across all platforms.
---
## Deployment Topologies
Choose the deployment architecture that fits your infrastructure:
| Topology | Components Used | Best For | Security & Networking |
| :--- | :--- | :--- | :--- |
| **All-in-One Desktop** | Vyla Home | Casual users wanting local playback on a single PC | Fully local, runs on localhost, zero open ports |
| **Living Room TV** | Vyla TV + Hosted Stream API | Smart TVs, Chromecast, NVIDIA Shield | TV connects over LAN/WAN to a secure Stream API gateway |
| **Private Family Server** | Stream API on VPS/NAS | Multi-device households streaming across phones, PCs, and TVs | Run behind Nginx/Caddy with TLS, enable API key protection |
| **Custom Platform** | SDK in Node.js backend | Developers building bespoke streaming web applications | Complete control over DB, auth, caching, and playback logic |