English

Français

English

Français

Appearance

Architecture ​

Introduction ​

Vocabulary ​

Fediverse: several servers following one another, several users following each other. Designates federated communities in general.

Vidiverse: same as Fediverse, but federating videos specifically.

Instance: a server which runs PeerTube in the fediverse.

Origin/local instance: the instance on which the video was uploaded and that serves the video behind a HTTP server.

Cache instance: an instance that decided to mirror a remote video. It sends a CacheFile activity to notify the origin instance.

Following: the action of a PeerTube instance which will follow another instance (subscribe to its videos). You can read more about Follows in the admin doc, under following servers.

PeerTube instance ​

  • An instance is like a website: it has an administrator, and people can create an account on the instance. Users have multiple channels in which they decide to upload videos.
  • An instance acts like a normal webserver: users can upload videos and the instance will serve files behind an HTTP server. It also has a websocket tracker which responds to any request on local videos. Then, a classic video player can play the video using the HTTP protocol, but the tracker can also share some segments of the video using P2P.
  • An instance has an administrator that can follow other instances using ActivityPub protocol so that other instance videos can be displayed on the local instance.

Global Overview ​

PeerTube is consisted of two parts: a client application that executes in the web browser and a server part that resides on the machine of the instance's system administrator.

As is common among modern applications, it is in reality made of several more components than just a "client" and "server", each of them fulfilling a specific mission:

  • a modern database engine (PostgreSQL) to store long-term metadata
  • a reverse proxy (NGINX) to handle certificates, and directly serve static assets
  • a key-value store (Redis) to help application caching and task queueing
  • a server application with:
    • controllers to server static and clients files
    • a REST API server providing the actual logic
    • an ActivitPub API to receive messages and present local objects
    • a BitTorrent tracker to allow clients to do P2P
    • a socker.io server to notify clients
    • a task scheduler to send ActivityPub requests, run transcoding jobs using ffmpeg etc.
  • a Single Page Application web client that consumes the REST API

architecture overview

The client web interface ​

This refers to PeerTube's official web interface, which is a Single Page application written in Angular. This application will interact with PeerTube's API to retrieve or send data. Of course any alternative client interface can be used so long as it is compatible with the API, and PeerTube can be started without the official web interface to let you serve your client instead.

The reverse proxy ​

PeerTube's API server should never be exposed directly to the internet, as we require a reverse proxy (we provide support for Nginx) for performance and security. The reverse proxy will receive client HTTP requests, and:

  • Proxy them to the API server
  • Serve requested static files (Video files, stylesheets, javascript, fonts...)

The PeerTube server ​

The API server is the central piece of PeerTube. This component is responsible for answering and processing user requests, manipulating data from the database, sending long-running tasks to the local worker, etc.

It's written in Typescript (NodeJS) and is an Express application.

The database ​

Most of the data such as user accounts, video metadata, comments or channels are stored in a PostgreSQL database.

The cache/job queue ​

Fetching data from the database is sometimes slow or resource hungry. To reduce the load, Redis is used as a cache for route data meant to be stored temporarily.

It is also a message queue that will deliver tasks to the local worker. Indeed, PeerTube uses the BullMQ queue.

Server ​

The server is a web application developed with TypeScript/Express.

Main technologies ​

Files structure ​

The server main file is server/server.ts. The server modules descriptions are in the package.json at the project root. All other server files are in the server/core directory:

server.ts -> app initialization, main routes configuration (static routes...)
config    -> server YAML configurations (for tests, production...)
scripts   -> Scripts files for npm run
server
|__ assets       -> static assets files (images...)
|__ controllers  -> API routes/controllers files
|__ helpers      -> functions used by different part of the project (logger, utils...)
|__ initializers -> functions used at the server startup (installer, database, constants...)
|__ lib          -> library function (WebTorrent, OAuth2, ActivityPub...)
|__ middlewares  -> middlewares for controllers (requests validators, requests pagination...)
|__ models       -> Sequelize models for each SQL tables (videos, users, accounts...)
|__ tests        -> API tests and real world simulations (to test the decentralized feature...)

Concepts ​

The server is composed of:

Video files are statically served so can be downloaded using HTTP.

Architecture scheme

When a user uploads a video, the REST API creates the torrent file and adds it to its database.

If a user wants to watch the video, it will send a request to the tracker to download/share video segments from other peers. The user also downloads segments using HTTP.

Transcoding ​

Please read the documentation regarding Web Video and HLS transcoding first: https://docs.joinpeertube.org/admin/configuration#vod-transcoding

After the video upload:

  • An optimize transcoding job is created by PeerTube. This job will transcode the video using x264/aac codecs with custom bitrates in a .mp4 container. The purpose of this job is to create a Web Video compatible file.
  • On optimization success, PeerTube will:
    • Create the HLS job of the optimized file resolution if HLS transcoding is enabled
    • Create lower Web Video resolution jobs
  • On Web Video resolution job success, PeerTube will:
    • Create the HLS job for this resolution if HLS transcoding is enabled
  • On HLS job success:
    • If Web Video is disabled and the transcoded file was the highest video resolution:
    • Create lower HLS resolution jobs and delete Web Video files

Live ​

When the Live feature is enabled on the instance, PeerTube will create a RTMP server listening on a specific TCP port.

Then, when a user creates a live container:

  • PeerTube generates a specific token for this live
  • When it receives a RTMP stream, it checks the token to retrieve the associated live container
  • It creates a ffmpeg task that will consume RTMP stream to create an HLS playlist
  • PeerTube serves the HLS playlist static files to web browsers

Remote VOD/Live transcoding ​

A PeerTube instance can offload transcoding process using remote runners. If enabled:

  • PeerTube stops to process transcoding jobs in its own job queue. Instead, it stores them in the database and wait for runners to request jobs
  • PeerTube runners register themselves to the PeerTube instance using a registration token. Then, they are allowed to request jobs
  • PeerTube runners periodically fetch available jobs using HTTP REST API
  • For VOD jobs:
    • Runner accepts the job
    • Downloads the input file to transcode
    • Transcodes it
    • Posts the output file to the PeerTube instance
    • The PeerTube instance adds/replaces this new file to the video
  • For Live jobs:
    • Runner accepts the job
    • Runs a ffmpeg process to transcode the input RTMP stream to HLS
    • Periodically uploads HLS .m3u8 and .ts segments to the PeerTube instance
    • The PeerTube instance serves these static live files to its users
    • When the stream ends, the runner posts a success result to the PeerTube instance

Plugins ​

Plugins are installed using NPM in a directory dedicated to plugins so NPM manages dependencies installation. Then, PeerTube tracks installed plugins (name, description, latest available version etc) in a specific table in database.

To enable a plugin, the server require the main module and calls the register function. Client does the same things with a scope concept (to only load plugin client files that are needed in a specific page). Then, the server and the client register plugin hooks and fire them on specific application events (video creation, page initialization etc).

Themes are plugins that are only used in the client. Theme CSS files are injected in HTML using <link> with alternate stylesheet rel attribute.

Communications between instances ​

Sharing videos metadata to build an index of more than just local videos

  • We use the server-server ActivityPub protocol. Activities and model documentation can be found on https://docs.joinpeertube.org/api/activitypub.
  • ActivityPub messages are signed with JSON Linked Data Signatures with the private key of the account that authored the action.
  • ActivityPub messages are also signed with HTTP Signatures with the private key of the account that sent the message.
  • All the requests are retried several times if they fail.
  • A PeerTube instance has always a peertube ActivityPub Actor that other instances can follow. This peertube actor Announce videos uploaded on the instance so other actors are notified by new videos.
  • A video is uploaded by an account, that is an Actor. This video is also Announce by the channel in which it has been uploaded. Then, remote ActivityPub actors can follow the account or the channel to be notified by new videos.

Redundancy between instances ​

A PeerTube instance can mirror other PeerTube videos to improve bandwidth use.

The instance administrator can choose between multiple redundancy strategies (cache trending videos or recently uploaded videos etc.), set their maximum size and the minimum duplication lifetime. Then, they choose the instances they want to cache in Manage follows -> Following admin table.

Videos are kept in the cache for at least min_lifetime, and then evicted when the cache is full.

When PeerTube chooses a video to duplicate, it imports all the resolution files (to avoid consistency issues) using their magnet URI/playlist URL and put them in the storage.redundancy directory. Then it sends a Create -> CacheFile ActivityPub message to other federated instances. This new instance is injected as WebSeed in the torrent file for the BitTorrent/WebTorrent compatible players and injected in the video API response for the HLS player.

Cache servers

Client ​

The client is a HTML/CSS/JavaScript web application (single page application -> SPA) developed with TypeScript/Angular.

Main technologies ​

  • TypeScript -> Language
  • Angular -> JavaScript framework
  • Webpack -> Source builder (compile TypeScript, SASS files, bundle them...)
  • SASS -> CSS framework
  • Bootstrap -> CSS framework
  • VideoJS -> JavaScript player framework
  • P2P Media Loader - JavaScript library to use P2P in the browser (via HLS and BitTorrent over WebRTC)
  • hls.js -> JavaScript library to handle HLS playlists

Files structure ​

The client files are in the client directory. The Webpack configurations files are in client/webpack and the source files in client/src. The client modules descriptions are in the client/package.json. There are many modules that are used to compile the web application in development or production mode. Here is the description of the useful client files directory:

tsconfig.*.json -> TypeScript configuration for the compilation
webpack         -> Webpack configuration files (to build the embed page)
e2e             -> E2E tests (using WebdriverIO)
src
|__ app          -> TypeScript files for Angular application
|__ assets       -> static files (images...)
|__ locale       -> translation files used by Angular and the player
|__ root-helpers -> helpers used by the Angular application and/or by the player
|__ sass         -> SASS files that are global for the application
|__ standalone   -> files outside the Angular application (embed HTML page...)
|__ types        -> specific types needed by the client
|__ index.html   -> root HTML file for our Angular application
|__ main.ts      -> Main TypeScript file that boostraps our Angular application
|__ polyfills.ts -> Polyfills imports (ES 2015...)

Details of the Angular application file structure. It tries to follow the official Angular styleguide.

app
|__ +about                       -> About components (about the instance, about peertube...)
|__ +accounts                    -> Account components (list account videos...)
|__ +admin                       -> Admin components (followers, users...)
|__ +home                        -> Homepage components
|__ +login                       -> Login components
|__ ...
|__ +my-account                  -> My account components (update my profile, notifications...)
|__ ...
|__ +my-library                  -> My library components (list my channels, list my videos...)
|__ ...
|__ +search                      -> Search components (search channels/videos...)
|__ ...
|__ +video-channels              -> Video channels components (list channel videos, channel playlists...)
|__ +videos                      -> Edit video/watch video/list videos components
|__ core                         -> Core components/services that should be injected only once
|__ header                       -> Header components (logo, search...)
|__ helpers                      -> Helpers used by the application
|__ menu                         -> Menu component (on the left)
|__ shared                       -> Shared components/services used by multiple modules (search component, REST services...)
|__ app.component.{html,scss,ts} -> Main application component
|__ app-routing.module.ts        -> Main Angular routes
|__ app.module.ts                -> Angular root module that imports all submodules we need

Concepts ​

The PeerTube player ​

When a user is watching a video:

  • If using the HLS player (depending on the admin transcoding configuration):
    • The player loads the HLS playlist using hls.js from the origin server
    • PeerTube provides a custom loader to hls.js that downloads segments from HTTP but also from P2P via WebRTC
    • Segments are loaded by HTTP from the origin server + servers that mirrored the video and by WebRTC from other web browsers that are watching the video. They are used by hls.js to stream them into the <video> HTML element
  • If using the Web Video player (depending on the admin transcoding configuration):
    • PeerTube injects the raw source video file in the src attribute of the <video> HTML element

Other libraries ​

Other libraries are also provided by PeerTube.

@peertube/embed-api ​

Published on NPM so third party websites can easily control the PeerTube embed player wrapping Window.postMessage() calls.

Located in client/src/standalone/player.

@peertube/peertube-types ​

Published on NPM so Typescript projects can consumes PeerTube types. Mainly used by PeerTube plugins.

Located in packages/types.