The "Add to render queue" button in the render panel permits to send a batch of Mistika render jobs ( .rnd files ) to external render farms, or to pass those files to user defined scripts.
Please note that this tool require some preparation and it is not really necessary for simple cases. If what you want is to just render a batch of shots then you can simply select all shots and it will render all of them in one go.
But if you need more powerful render management, then the Add to render queue button is the right tool to use, in combination with a external render manager
Add to render queue - Main Purposes:
- To integrate with external batch manager tools and external render managers. These are specialized applications that permit to organise your render jobs in different ways. For example, render jobs can be added to render queues for posterior execution (for example during the night), change their priorities on the fly, stop / resume jobs as needed, etc.
- To use a external render manager to render in parallel a single enumerated sequence (.dpx, .tiff, .jpg, etc) in multiple computers (with each node rendering different frames of the same clip)
- To use a external render manager to render a different movie file (.mov, .avi...) on each render node. Movie files can not be rendered by parts (only enumerated sequences can), but movie files can still be distributed by ordering one movie file to each render node.
- To execute user scripts to automate production workflows
Configuration in the Mistika side:
Currently two Mistika applications support this capability, Mistika VR and Mistika Boutique. (it also works for the linux version "Mistika Ultima". In this document everything that is said for Mistika Boutique also works for Mistika Ultima)
The Render -> Add to render queue button works both in Current shot and All shots modes. In the second case, a whole batch of Mistika render jobs ( .rnd files) is created first, and then a set of command lines are executed for each of the .rnd files in that batch.
Depending on the Render->Split By Segments options the Render -> Add to render queue button will create one .rnd per shot or just one .rnd for the whole selection , and then a set of command lines are executed for each of the .rnd files.
The command lines are stored in a configuration file, runBatch.cfg, which can be configured in the File->Options menu (for the case of Mistika VR) or in the MistikaConfig->BatchManager (for the case of Mistika Boutique).
The configuration menu provides a graphic interface to integrate with the most common batch managers: Mistika Ultima BatchManager, Mistika Workflows, Uberware Smedge, and ThinkBox Deadline.
For more advanced usage or to integrate with other applications, the user can put all the command lines that he wants to be executed in the runBatch.cfg, passing the path of each .rnd file to them and also any parameters requiring variable substitution (the full list of available variables is given in the configuration file as a reference).
Supported batch managers:
A batch manager (or render manager, the name depends on the purpose) is an application that provides batch processing for different jobs. In the case of "Render Managers" they provide the capability to distribute render jobs between multiple computers, monitoring their progress and managing all aspects of the render nodes.
Most of these applications are agnostic about the actual render software, which needs to be installed separately. In this context, the actual renderer (vr or mistika) can be connected to the render manager by providing a description of its command line interface (CLI), which requires a different configuration process for each of those applications. (all this is explained below).
Once the render manager is also configured, thanks to the Add to render queue button the user does not need to type any command line to render each job. Once the proper commands for job submission are configured the Add to render queue button will execute them for each render job automatically.
These are the render managers that are available by default:
- Mistika BatchManager ( send to Mistika Ultima ):
It is the usual render manager that is available in all Mistika Ultima environments (these render farms are linux only, but you can still submit rnd jobs to them from any other platform ).
If it is selected, the Add to render queue button will copy the rnd files to a Mistika Ultima folder (either to render them with Mistika BatchManager or to open them in Mistika Ultima).
The configuration file also offer parameters for "path translation", so it can replace local paths for the corresponding paths that are equivalent in the destination system before sending the .rnd job (the complete instructions are in the runBatch.cfg file). However, since v8.10 each Mistika installation also keeps its own configuration file with all the path translation pairs used in the past (for automatic relinking), so using the path translation of this tool is not mandatory. You can use one or the other method as preferred.
- Uberware Smedge:
It is a third party multi-platform render manager, developed by Uberware. It works on Windows, Mac, and Linux. It is probably the easiest render manager to setup, as it provides a lot of automated configuration. For that reason we have also tried to simplify our side of the integration as much as possible.
Please note that Mistika does not include licenses for any 3rd party render manager. At the date of this document Uberware Smedge is free of cost for the first 4 systems, but additional nodes require to purchase Smedge licenses from Uberware.
The recommended installation procedure is as follows:
1 - First thing, make sure that all the computers are sharing a common storage volume for the source files and rendered files. Check that Mistika can read the media files from those volumes and also render on the planed destinations, otherwise nothing is going to work properly on the next steps (except if you only plan to use Smedge in the local computer only , which also makes sense as a simple Batch processor ).
2 - In all the render nodes, install both Mistika and Smedge (you can find Smedge at http://www.uberware.net/ ).
Make sure that all nodes have the corresponding licenses as necessary, and test that they can render using Mistika as usual, without using the Add to render queue function yet (obviously, it does not make sense to try the integration functions with Smedge until Mistika is rendering correctly by itself).
3 - Copy the attached Smedge modules ( MistikaVR.psx and MistikaUltimaAndBoutique.psx) to the "Modules" folder of the Smedge installation. Default paths are:
Windows: C\Program FIles\Smedge\Modules
. Then start Smedge (or restart it if it was active, as this is necessary to refresh the modules list).
From that point, Smedge should already offer "MistikaVR" and "MistikaUltimaAndBoutique" as available job types. (for more details please read the Smedge documentation about creating jobs).
NOTE: The .psx module files are the same for Windows, Mac, and Linux, only the Modules folder location will change.
4 - In MistikaVR, press File->Options->BatchManager (or MistikaConfig->BatchManager for the case of Mistika Boutique). It will offer several configuration options, just choose the one to Send .rnd jobs to Smedge.
From that point, every time you execute the Add to render queue button it will submit the render jobs to Smedge (one per shot).
Later, if you want to use a different render manager or add user scripts you can change it in the same menu.
Note: In the case of Smedge, it provides its own system for path translation, in case that the file paths are different on each of your systems. For more details please search for "Path Translation" on the Smedge documentation. Mistika also provide path translation abilities. Use what is more convenient for you.
Note: The Smedge progress bar is based on counting the number of rendered files. Enumerated sequences will show a progressive bar, but movie files will jump from 1% to 100% in one step. This is the expected behaviour, as a movie file is just one file. However, you may still see the frame number being rendered in the metadata columns at the right of the progress bar.
- Thinkbox Deadline:
The instructions for the integration with Thinkbox Deadline are here:
Please note that the Mistika executables path has changed since the original document. You may need to adjust it to your particular installation paths.
Compatibility with other render managers and workflow automation tools:
In addtiion to Smedge and Deadline, the Add to render queue function should be compatible with any other render manager that permits to submit jobs by command line. This is because the runBatch.cfg file permits to define the exact command line to execute for each rnd job, also passing the path to the rnd file and other common parameters in the command line (start frame, end frame, media paths, etc).
However, in the other side of the connection many render managers also require a plugin or a configuration file to describe the command line interface (CLI) of each product that they support. If your render manager does not have your Mistika product on the list or if you are integrating with your own workflow tools then you will need to make that missing piece. For more details check the articles about Mistika CLI interface
Support services and trouble shooting:
A characteristic of most render farms is that they are based on a combination of products from different manufacturers. Then, in the case of problems the first thing is to isolate which product is creating the problem (logically, each of the providers can only help you with the configuration of their own products). We recommend the following procedure to find where the problem comes from and which support services need to be contacted:
Recommended support procedure:
- First thing check that all your network and shared storage volumes are working correctly on all computers before installing any render farm management software , as network and permission issues are the most common source of problems in a render farm.
Specifically, test that you can can access all source files and overwrite render folders from all the computers without any permission problems. If you detect problems on this preliminary tests then please contact your system administrator, and do not continue with the next points until this part works perfectly. It is highly recommended to start with a working network and fully operational network drives before installing any render manager software on it, otherwise they may fail to execute any automated configuration provided by their installer tools, and when you finally fix your network then you may end with a configuration mess that may need manual tweaking for everything.
- If the problem has appeared at a later stage, then a useful test is to configure your local system as the only render node, and test the connectivity between Mistika and the render manager locally on that single node. If the integration works well on each computer but not when you add them all together then the cause of the problem is probably a network issue. In particular, check that all your computers are discoverable in the network, as many render managers will use that function. (foe example in the case of Windows make sure to activate Network Settings->SharingOptions->Turn on Network Discovery). And make sure that the firewalls and antivirus software are not blocking any necessary connection (if possible, deactivate them for a while and reactivate them when it works)
- Once the above problems are discarded, check if Mistika can render normally on each render node using the shared network drives (without using Add to render queue or any render manager yet).
- Once that Mistika standalone render is working well in all computers using the shared volumes then the only piece that is left is the render manager configuration. That should be the last piece to be installed and configured, and their documentation and support services will be the ones to be used for this purpose.
MistikaVR.psx module for Smedge - Only for MistikaVR version 8.8.6 and later
- MistikaVR versions older than 8.8.6 need to use the .psx module that comes included with Smedge 2018.
- MistikaVR 8.8.6 and later versions will need to install the attached .psx module: Copy it to the Smedge "Modules" folder and restart Smedge.
(Current .psx version: 29th June 2018. Please note that this file may change over the time )
Mistika_Ultima_And_Boutique.psx : This is the Smedge module for Mistika Ultima and Mistika Boutique products. Requires Mistika v8.10 or later.
Note for users with installation paths different than default:
The .psx module file will tell Smedge how to build the path to find the renderer application. For example, If you are using MistikaVR and the vr executable is in this folder:
It will need a .psx file with this section:
[ FindLatestExecutable ]
WindowsRoots = C:\Program Files\SGO Apps
WindowsBases = Mistika VR
WindowsExe = bin\vr.exe