Preliminar note:  This feature is still under development and in beta stage. The following are some notes for betatesters, not the final documentation.  


INTRODUCTION


Starting with version 8.7, Mistika Ultima provides new streaming capabilities for sending the equivalent to the "Live video" images and audio as a network stream. Then you can send it to standard streaming players on Internet,  smartphones over wifi, or VR headsets with support for url playback.


The main purpose is to help on remote sessions, where the client or the operator is in a remote location or when the client is wearing an VR Headset, even in Stereo3D mode


Please note that what Mistika offer is only the server side of the streaming, which means that it provides a standard URL that the client can playback with any application supporting url streaming.  But the client is responsable to provide network resources and the playback application.  We will mention some of those applications in this document, but they are from third parties and not documented or supported by SGO. 


Several codecs are supported, including NVidia hardware encoding for h264 and h265. For realtime purposes, the recommended codec is NVidia h264. It supports any resolution up to 8K, but in general,  the realistic realtime capabilities to expect with normal hardware are up to 2K Stereo3D and 4K 2D 


STREAMING CONFIGURATION


The streaming is configured with these tools:


- mConfig->Streaming (select streaming type, select preferred codec, and get urls to be access from other systems). 


- Mute->S : Once in Mistika Ultima, the  button permits to mute the streaming temporally. For example, when you are organising a stack of effects if you don't want this process to be watched.  Additionally, using the Ctrl modifier you can restart the streaming service in case of problems.


Note: Streaming use different system resources constantly, so we recommend to not activate it when not in use.


Codec presets:  mConfig provides encoding presets for mpeg4 (cpu based) and NVidia h264 and h265/HEVC (GPU based).  Being NVidia h264 the recommended encoder to produce realtime streams for most streaming players.  Modern NVidia boards are not only much faster than software enconding, but they also have dedicated hardware for this that is not used otherwise, so it will not interfere with Mistika image processing.


For systems with very old NVdia boards not supporting streaming you can select the mpeg4 streaming mode.


GOP: This parameter controls the number of delta frames between keyframes. 


- Reduce it for low latency (ideally down to 1 in high bandwidth environments). In this way, the time between making a change and seing the results in the player can be minimised. Also, please note that compression artifacts will last the whole time between keyframes, so reducing GOP should produce more pleasant results, but logically at the cost of extra bitrate. 


- Increase GOP to reduce the bandwitdh needs. 


Buffering:  Number of frames used as buffer in the server side.


- Reduce it to improve latency


- Increase buffer if you experience stabilty problems, due to hardware limitations or irregular networks.  


Please note that mConfig->Streaming->Buffering only control buffering in the server side. Most players also have a similar setting to control their side.


Most other parameters are autoexplicative or they include an explanation in the mConfig streaming panel.


Known Problems (Beta stage):


In current betas the fps must be an integer, using values like 29.97 will make the streaming to fail.




Application examples


VR HEADSETS:


NOTE: At the date of this document the available streaming solutions for VR Headsets are more a "proof of concept" than really usable solutions. We will report advances in this docuent as we find additional progress. 


For VR Headsets. you will normally use the "LOCAL_SERVER" streaming type, which is simple to confgure and it is the most reliable.


- Smartphone based VR headsets: It is the most basic setup. It uses a smartphone based headset over wifi. Some applications that may act as players and support url streams are VLC 360 and VRTV, which also supports Stereo3D. 


Regarding the wifi connection for the Mistika Ultima system (server side), please check the article:


http://support.sgo.es/support/solutions/articles/1000244989-centos-7-2-how-to-setup-a-wired-or-wireless-network-on-centos-7-2-tricks-tips-

  

- Oculus Rift:  Clients have reported some success using the "OpenVR player" for Oculus, which also supports Stereo3D.  This application have significant bitrate limitations, so we recommend using a big GOP parameter.



HD/2K/4K REMOTE PLAYBACK:


For these purposes, probably the best general purpose player is the VLC application, which is available on internet free of cost.  Each remote client can open a  open a VLC player in any smartphone, tablet or computer (VLC is available to Mac, Windows, and Linux, and also for  tablets and smarpthones). 


mConfig provides these streaming modes:


TEST_PLAYER:  Only valid for doing tests in the local system. Mistika will open a streaming player when doing the first playback. The only purpose is to check the quality and stability of the streaming in a local window, before changing to one of the other modes.  


ONE_CLIENT_UDP:  It provides the lowest latency, but it is only usable with one only client and on it requires very reliable networks. This is because it uses UDP protocol, which does not have any error correction protocol.  In this mode Mistika will send the stream directly to the remote player


LOCAL_SERVER: The recommended method for local networks or Wifi streams. It will launch a streaming server in the Mistika system, and Mistika will pass the stream to it. 


Note: In certain cases it could also be used for Internet streaming to thrusted locations by opening the corresponding router ports or in a VPN, but it should not be used on untrusted networks, as in general Mistika system are not secured systems. 


REMOTE_SERVER: It is the recommended method for internet streaming. In this mode Mistika will send the stream to a server located out of your firewall trough UDP, and then the server will send the stream to the end clients. 


It can also be used to reduce the system overhead in the Mistika system, as it only needs to send it but not to serve it. Also, it can be used on local networks when security is a concern.


The typical way to configure it for Internet streaming is this:


- Put a secured server out of the local firewall, or rent it from an internet solution providers.


- In the remote server: a typical way to launch the vlc server application is to execute this.


 vlc -I dummy udp://@:20000 --sout '#http{mux=ts, dst=:8004/mistika.ts'


- In the Mistika system:  Go to mConfig->Streaming and select the Remote Server mode, and fill the fields for the private ip of the server ( as seen from the Mistika  system ) and the server public ip for the end clients. Mistika will send the stream directly to the server by using UDP protocol and using the private ip. And the end clients will access trough the public ip.


- Once Mistika is opened, the server will receive the streaming and it will start serving it to any internet clients.


- Once you have tested it, tell the clients to open this ur in his player:


http://public_ip_of_server:8004/mistika.ts


This url is also provided in mConfig, so an easy way is to copy / paste  as a link in an email and send it to the clients. 


Advanced settings


The mConfig advanced mode provides additional capabilities. This section is only useful for experts on streaming and ffmpeg:


The codec presets are defined in .vid text files in this folder (you can create your custom .vid files and they will appear in mconfig):


MISTIKA-ENV/lib/linux64/streaminglib/


Example: The following .vid is an example of an nvidia h264 encoder with gop=150, and manual bitrate definiton from 22M to 30M (check ffmpeg documentation for parameter meaning)


VIDEO_CODEC h264_nvenc

VIDEO_PARAM preset llhp

VIDEO_PARAM b 22M

VIDEO_PARAM bufsize 22M

VIDEO_PARAM maxrate 30M

VIDEO_PARAM g 150


This other example uses qmin and qmax quantization rather than bitrate.


VIDEO_CODEC  h264_nvenc

VIDEO_PARAM  preset llhp

VIDEO_PARAM  bufsize 20M

VIDEO_PARAM  g 5

VIDEO_PARAM  qmin 30

VIDEO_PARAM  qmax 51