Preliminar note:  This feature is still under development and in beta stage. The following are some notes for betatesters, not the final documentation.  


INTRODUCTION


Starting with version 8.7, Mistika Ultima provides new streaming capabilities for sending the equivalent to the "Live video" images and audio as a network stream. Then you can send it to standard streaming players on Internet,  smartphones over wifi, or VR headsets with support for url playback.


The main purpose is to help on remote sessions, where the client or the operator is in a remote location or when the client is wearing an VR Headset, even in Stereo3D mode


Please note that what Mistika offer is only the server side of the streaming, which means that it provides a standard URL that the client can playback with any application supporting url streaming.  But the client is responsible to provide network resources and the playback application.  However we will mention some of those applications in this document, but please note that they are from third parties and need to be documented and supported by them, not by SGO. 


Several codecs are supported or streaming, including NVidia hardware encoding for h264 and h265. For realtime purposes, the recommended codec is NVidia h264. It currently supports any resolution up to 4K Stereo3D, but it also depends on the network bandwidth  and the decode processing capabilities of the client side.


STREAMING CONFIGURATION


The streaming is configured with these tools:


- mConfig->Streaming (select streaming type, select preferred codec, and get urls to be access from other systems). 


- Mute->S : Once in Mistika Ultima, the  button permits to mute the streaming temporally. For example, when you are organising a stack of effects if you don't want this process to be watched.  Additionally, using the Ctrl modifier you can restart the streaming service in case of problems.


Note: Streaming use different system resources constantly, so we recommend to not activate it when not in use.


Codec presets:  mConfig provides encoding presets for mpeg4 (cpu based) and NVidia h264 and h265/HEVC (GPU based).  Being NVidia h264 the recommended encoder to produce realtime streams for most streaming players.  Modern NVidia boards are not only much faster than software encoding, but they also have dedicated hardware for this that is not used otherwise, so it will not interfere with Mistika image processing.


For systems with very old NVdia boards not supporting hardware streaming  we recommend to select the mpeg4 format.


GOP: This parameter controls the number of delta frames between keyframes, and the optimal value is content dependent. 


- Reduce it for low latency. In this way, the time between making a change and seeing the results in the player can be minimised. Also, please note that compression artifacts will last the whole time between keyframes, so reducing GOP should produce more pleasant results, but logically at the cost of extra bitrate. 


- Increase GOP to reduce the bandwidth needs and the decoder processing needs. 


Buffering:  Number of frames used as buffer in the server side.


- Reduce it to improve latency


- Increase buffer if you experience stabilty problems, due to hardware limitations or irregular networks.  


Please note that mConfig->Streaming->Buffering only control buffering in the server side. Most players also have a similar setting to control their side. Also,  hardware encoders and decoders will  introduce additional latency, but that can not be controlled by Mistika.


STREAMING MODES


TEST_PLAYER:  Only valid for doing tests in the local system. Mistika will open a streaming player when doing the first playback. The only purpose is to check the quality and stability of the streaming in a local window, before changing to one of the other modes.  


ONE_CLIENT_UDP:  It provides the lowest latency, but it is only usable with one only client and on it requires very reliable networks. This is because it uses UDP protocol, which does not have any error correction protocol.  In this mode Mistika will send the stream directly to the remote player


LOCAL_SERVER: The recommended method for local networks or Wifi streams. It will launch a streaming server in the Mistika system, and Mistika will pass the stream to it. 


Note: In certain cases it could also be used for Internet streaming to thrusted locations by opening the corresponding router ports or in a VPN, but it should not be used on untrusted networks, as in general Mistika system are not secured systems. 



REMOTE_SERVER (Custom made): It is the method for doing  internet streaming when you have your own server on internet  (without using public streaming servers from third parties, which is explained in the Workflow Examples sections below).  In this mode Mistika will send the stream to a server located out of your firewall trough UDP, and then the server will send the stream to the end clients. 


It can also be used to reduce the system overhead in the Mistika system, as it only needs to send it but not to serve it. Also, it can be used on local networks when security is a concern.


The typical way to configure it for Internet streaming is this:


- Put a secured server out of the local firewall, or rent it from an internet solution providers.


- In the remote server: a typical way to launch the vlc server application is to execute this.


 vlc -I dummy udp://@:20000 --sout '#http{mux=ts, dst=:8004/mistika.ts'


- In the Mistika system:  Go to mConfig->Streaming and select the Remote Server mode, and fill the fields for the private ip of the server ( as seen from the Mistika  system ) and the server public ip for the end clients. Mistika will send the stream directly to the server by using UDP protocol and using the private ip. And the end clients will access trough the public ip.


- Once Mistika is opened, the server will receive the streaming and it will start serving it to any internet clients.


- Once you have tested it, tell the clients to open this ur in his player:


http://public_ip_of_server:8004/mistika.ts


This url is also provided in mConfig, so an easy way is to copy / paste  as a link in an email and send it to the clients. 



Most other parameters are autoexplicative or they include an explanation in the mConfig streaming panel.



WORKFLOW EXAMPLES


VR HEADSETS:


NOTE: At the date of this document the available streaming solutions for VR Headsets are more a "proof of concept" than really usable solutions. We will report advances in this document as we find additional progress. 


For VR Headsets. you will normally use the "LOCAL_SERVER" streaming type, which is simple to configure and it is the most reliable.


Regarding the wifi connection for the Mistika Ultima system (server side), the normal usage is to use a good wifi router, but it is also possible to make a direct wifi connection as explained in this article:


http://support.sgo.es/support/solutions/articles/1000244989-centos-7-2-how-to-setup-a-wired-or-wireless-network-on-centos-7-2-tricks-tips-



Examples:


- Google Daydream:  


It uses a smartphone based headset, connected with Mistika over wifi or USB tiering cable (USB can provide increased bandwidth with some smartphones, but not all will support this mode ). 


VRTV: There is an application in the Google shop named "VRTV" . It works with Google Daydream (and also Google Cardboard), it gets the Mistika stream very easily and the quality is not bad. The bitrate is not limited (it supports  4K), and it supports Stereo3D.   However, at the date of this document VRTV is a beta version and the interface is really buggy. And the latency is also pretty high. Once you manage to get a playback the quality is reasonably good, as long as you only watch it  and you do not try to interact with the application  (for example the scene recentering is a mess).  



- Oculus Rift:  Some clients have reported some success using the "OpenVR player" for Oculus, which also supports Stereo3D.  This application have significant bitrate limitations (20Mb), so it is recommend using a big GOP parameter. Even with that do not expect a high quality, as  20Mb is not enough. 



HD /2K / 4K REMOTE PLAYBACK TO A DISPLAY (TV, Computer monitor, or Tablet):


This is the case with best results and it is easy to configure.  For these purposes, probably the best general purpose player is the VLC application, which is available on internet free of cost.  Each remote client can open a  open a VLC player in any smartphone, tablet or computer (VLC is available to Mac, Windows, and Linux, and also for  tablets and smarpthones). Just send the stream url by email and if the client has VLC installed he can open it by just clicking on the link.



PUBLIC STREAMING SERVICES FROM THIRD PARTIES:


These are public streaming servers on Internet that offer live streaming capabilities, thus acting as intermediary servers. 


Youtube live: An interesting case is "Youtube Live",  for clients  who have a good internet bandwidth.  The main purpose is to use it  for remote clients, but it can also be used for local client to provide a generc front end interface that is compatibie with most devices.  


It supports many different formats and resolutions, stereo3D, and some headsets are also starting to support it (for example Google Daydream comes with a Youtube app that supports Youtube live in 360 VR formats.  Youtube live  also supports private sessions so you are only streaming for the people you want to watch it.  


Youtube also requires a supported  transcoder  for the stream management  Please note that what Mistika sends is just the transport stream ( the  ".ts" url ), so you need a specialised transcoder software to insert metadata, do the upload and provide the interface to interact with the Youtube back end.   


This  intermediate piece of software should run in other computer to avoid overloading the Mistika system with this functions. There are many options for this, there are  transcoders  that run on  Windows, Mac, Linux,  or even dedicated encoder boxes.    Basically, the transcoder  will need to get the Mistika stream and it will provide the remaining metadata (the Youtube "key" to connect to your Youtube account and other metadata ), and it will route the stream to youtube, which will serve the stream to the end clients.  Youtube says that there is a low latency mode available, so it could be a good solution for some interactive sesions.


The use of an intermediary transcoder also means that  you will normally use the LOCAL_SERVER or the ONE_CLIENT_UDP modes, because Mistika will only be sending the stream to a transcoder located in your local network.  You will simply give the Mistika stream URL to the transcoder, and it will be sending it  to Youtube with additional metadata and also providing interface options.


Note: If you plan  to try this please note that Youtube requires to create the live account 24 hours before using it, so it is not a fast setup.  



ADVANCED SETTINGS


The mConfig advanced mode provides additional capabilities. This point can be useful for experts on streaming and ffmpeg:


The codec presets are defined in .vid text files in this folder (you can create your custom .vid files and they will appear in mconfig):


MISTIKA-ENV/lib/linux64/streaminglib/


Please check the predefined .vid  files for reference examples.