Hey Everyone,
We've had multiple questions about how our control Websockets (and datachannels, for WebRTC) work and we do understand the question as they're quite interesting to look at if you want to implement a better control layer within the playback from MistServer,or implement your own playback that can use the same functionality. Simply put, the stream control channel allows you to do bidirectional information sharing, which allows the player and server to coordinate the details of media delivery, such as exact playback position and speed.
Sadly, there's no standard widely recognized specification for everything we needed. Many protocols come with some of the things we wanted build-in, however even when already build-in to the protocol they don't necessarily do all the things we wanted. WHEP is a good example as its target was live content, which means that there's no control layer for seeking within a video. This makes sense for live content, however we wanted to use WHEP for VoD content as well and that meant we needed some kind of (optional) control layer that can be used for typical VoD playback operations.
The goals of the specification were:
- A simple format that is nearly trivial to implement
- Flexible enough to transport any codec, past, current or future (the WS/RAW protocol, targeting WebCodecs)
- Framing is provided by the transport protocol
- Able to extend to stream track data in either direction, or both at once
- Compatible with prepackaged / non-raw codec data as well (WebRTC, MSE)
You can find the specification here:
[Bidirectional websocket documentation](https://docs.mistserver.org/mistserver/introduction/architecture/bidirectionalwebsockets)