Synchronized Media Metadata

Media is Not Just Audio and Video

When people think of media, visuals and audio often come to mind. However, media extends beyond just audio and video. Metadata is a generic term for non-audio and non-video data associated with a media asset. This data can be either timed (synchronized with the media) or untimed (properties of the media).

Metadata provides additional information to replace or enhance media. Strictly speaking, metadata is not necessary, but it can significantly enhance the use or experience of media as a whole.

Examples of Metadata Tracks

An example of metadata is subtitles, providing additional information to enhance the experience for those unable to use the audio information. Beyond subtitles, chapters within a movie can be added to facilitate easier seeking, and a movie summary with a poster can be included for list display.

The fascinating aspect of metadata is its evolution alongside the devices used for media playback and recording. In the early 2000s, metadata took the form of properties, such as tags containing information like content language, the device used for capture/encoding, and the time of recording. While generic information is valuable, the true power of metadata lies in timed (synchronized) metadata.

For instance, in recording footage from cars or drones, synchronized metadata could include GPS location, sensor outputs, a travel plan, and control inputs. This synchronized metadata, combined with video/audio content, provides a comprehensive understanding of intended control and instructions for recreating the video.

Security or weather cameras can adopt a similar approach, adding sensor outputs or on-screen content information, enabling better explanations of events or facilitating easier seeking for specific types of images.

Even sport or entertainment streams can benefit, adding information such as who is visible or what is happening on screen.

Challenges with Metadata Tracks

Introducing new information like this presents its own challenges. To do this properly, you need to ensure:

• You have a method to add this information to your media.
• It is in sync with the other tracks of your media.
• Your end-user can use or benefit from the added metadata.

Collecting information is a step you likely already have a method for, but integration work will be required to ensure compatibility with other media. Synchronization is often overlooked but crucial; verifying correct matching with video/audio data is essential. Finally, the metadata is only beneficial if it can be used by the end-user.

How MistServer Can Help

MistServer offers significant flexibility in both input and output. Its capability to take any input and transmux it to the desired output extends beyond audio or video. MistServer can ingest metadata confidently, provided you have a method to retrieve it.

While synchronization can be a concern, MistServer leverages the experience gained from optimizing synchronization between audio and video tracks. The metaplayer not only plays back media but also facilitates communication between the server and player, allowing for easy integration of new information into your playback.

There is no one-size-fits-all solution due to the often custom or non-matching specifications. If you have something you want to add to your live or VoD streams or are unsure how to maximize the benefits of your current metadata, contact us, and let's explore the possibilities.