This article explains how to install the Deadline server and several Deadline workers in a single Windows 10 system, mainly for testing Mistika integration and reproducing problems from support cases.
This is an internal document for SGO support, not intended for clients. The methods below are different than what clients will need to do when installing Deadline in their production systems, in two key aspects:
- Security is fully ignored in the below methods, as the purpose of this article is to create a simple testing setup.
- Different Deadline components are designed to be installed in different computers (repository, database, render nodes / multiple workers ...), also using a Windows Server OS (if Windows is used), while this method will install everything in a single PC running Windows 10 rather than Windows server, so the Deadline performance will be much slower when additional render nodes are connected (Windows 10 only manages up to 10 TCP/IP simultaneous connections...).
Note: Once the integration is finished we plan to send all the new plugins to Thinkbox so that they can come preinstalled with Deadline, but by now the installation process requires quite a few manual steps (described below).
Most of the points below require Mistika products from 10.5.1 and later. Previous Mistika versions will fail to work properly on Deadline environments.
Deadline Installation
Download Deadline from AWS Thinkbox website: https://downloads.thinkboxsoftware.com/#!/
Note: To access the download you will be asked to identify using an Amazon account. That needs to be a standard Amazon shop account (same were you buy your regular stuff, this is not about AWS services). But don't worry your account is not used for anything, it is just the Amazon way for "I'm not a robot")
Uncompress the zip installer and install two of the packages that are inside, the Deadline-repository (which will also install the "Mongo" database in first pass) and also the Deadline-client. Do not install the AWSPortalLink (that is only needed for rendering on AWS cloud...). During the installation process you will be asked several questions offering security, like TLS certificate creation an so on. But to simplify our setup just say No to everything related to make your installation more secure, so just leave empty passwords and do not create any certificates, etc. That will create a simple setup only usable on a local system (in a secured local network) but it will be easier to administrate and to use it for simple integration tests, which is the purpose of this particular article. (Alternatively, if you need security then create the TLS certificates and copy them to the other systems as instructed....)
Create this environment variable (otherwise the MistikaWorkflows plugin will not work, as the plugin is based on Python 2.7 while latest versions of Deadline expects Python 3 by default):
DCONFIG_PYTHON_VERSION=2
By default Deadline-Launcher (the Deadline app that launches the others) will auto start on Windows boot, which is a pain in the arse because among other things it is constantly showing a console every few seconds. To avoid this, I recommend to disable the Deadline-Launcher service in the TaskManager->Startup. Later, just launch Deadline-Launcher manually when needed.
Mistika Deadline plugins installation
The Mistika plugins are changing often, so when you read this document they may be different than the attached versions.
Each Mistika plugin is made of several files, mainly these:
plugin folder:
- MistikaProductName.py plugin file ( python script executed at render time)
- MistikaProductName.icon (icon file)
- MistikaProductName.params configuration file indicating where are the product executables
submission folder:
- MistikaProductNameSubmission.py script (python script executed when a job is submitted to deadline)
At this date it works as follows:
MistikaVR:
Deadline comes with a very obsolete plugin for MistikaVR, you will need to overwrite the DeadlineRepository10/plugins/MistikaVR/MistkaVR.py it with the attached version.
Also edit the MistikaVR.params in the same folder and set the proper paths to your vr.exe executable, (which were also very obsolete at the date of this document).
Mistika Boutique and Mistika Ultima (theoretically, not fully tested)
If you need those products, copy the DeadlineRepository10/plugins/MistikaVR folder as DeadlineRepository10/plugins/MistikaUltimaAndBoutique and rename the plugin files inside using "MistikaUltimaAndBoutique" in their names rather than "MistikaVR". Do the same with the submission script, and also edit the .params file to point to the corresponding executables.
Mistika Workflows
Note that this case is a bit different, the MistikaVR plugin was coming with Deadline so it had a placeholder in the official DeadlineRepository10/Plugins folder, but Mistika Workflows is currently a custom plugin not coming with Deadline (or not a the date of this document) so it needs to be in the "custom" folder. Attached is a "MistikaWorkflows plugin for Deadline", the "plugin" folder that it contains needs to be copied here (with all its content):
DeadlineRepository10/custom/plugins/MistikaWorkflows
And the submission script custom/scripts/submission/MistikaWorkflowsSubmission.py needs to be copied here:
DeadlineRepository10/custom/scripts/submission/MistikaWorkflowsSubmission.py
Finally edit the DeadlineRepository10/custom/plugins/MistikaWorkflows/MistikaWorkflows.params file and make sure it points to the workflows executable in your system.
Using Mistika paths different than default:
Once Deadline has been installed a first time it will cache all settings on its database. For this reason, if Mistika is not in the default place (C:/Program Files/SGO Apps) changing paths in the .params files described above is normally not enough, you will also need to change them in the plugin configuration tool available in the Deadline Monitor:
DeadlineMonitor->Tools->SuperUser mode (activate)
DeadlineMonitor->Tools->ConfigurePlugin->Mistika_productName (then change the Mistika installation paths as needed)
Creating two Deadline workers in the same system (critical for Mistika Workflows)
Mistika Workflows requires at least two Deadline workers, one to execute the Workflow job that is submitted and at least another worker to render independent Workflow nodes that have the renderFarm checkmark applied. If there is only one worker the job will never start to render.
To create two deadline workers in a single computer:
- Go to DeadlineMonitor and activate Tools->SuperUser mode
- Tools/Super User Mode/Manage User Groups - > Everyone -> Items -> LauncherMenu -> Launch New Named Worker -> Enable.
- Exit and restart Deadline Launcher
- In deadline Deadline Launcher, go to contextual menu (right mouse button) -> Launch Worker By Name -> New Worker Instance. Create a new worker with your preferred name. Now you have two Deadline workers.
Configuration in the Mistika side for using Deadline Plugins
MistikaBoutique, Mistika Ultima, and MistikaVR:
- Use the usual BatchManager/AddToRenderQueue settigs wheel to select Deadline as the preferred render farm
- If not using default paths, edit the runBatch.cfg file and change the Deadline submission line accordingly
Mistika Workflows
- In the preferences->RenderFarm put the paths to Deadline and MistikaWorkflows submission script as found in your system. Default paths are:
Deadline Path: C:/Program Files/Thinkbox/Deadline10/bin/deadlinecommand.exe
Deadline Submission Path: C:/DeadlineRepository10/custom/scripts/Submission/MistikaWorkflowsSubmission.py
When creating a workflow, activate the renderfarm icons of the nodes and/or the queue renderfarm icon as needed. Note that they do different things, the renderfarm icon in the queue panel is meant to submit a whole workflow to each render node, while the renderfarm icon on a specific node means to render that particular node in parallel by using multiple render nodes (if possible).
Rendering Mistika jobs in AWS virtual machines with Deadline
The setup for this point is very different than the above examples, but it is included as a reference for the case of simple tests with virtual machines.
As a difference to the above examples, in this case the Deadline installation needs to be done with all security restrictions in place, not only Deadline policies but also taking care of all the security aspects of AWS or the virtual machine provider in question. Having an insecure render node accessible on Internet can be extremely dangerous, as it could easily execute malign scripts submitted by any attacker.
Apart from the security aspects the other key difference is that Mistika render jobs need OpenGL/openCL/CUDA graphic contexts that can not be taken for granted on virtual machines, specially in these aspects:
1 - The virtual machine needs access to the GPU via an Nvidia native driver (normally by using "Nvidia RTX software for virtual workstations" or similar), 3rd party virtualized drivers are not enough. However this part depends on each Virtual machine provider and is not documented here (as a simple setup, once in AWS search for an AMI containing "NVidia virtual workstation" or "RTX software for virtual workstations", or similar), as AWS provides some images that come with everything that is necessary. And apart from that the Deadline installation is well documented by AWS.
2 - Once the instance is started, the Deadline worker should not start until the graphics session of the user that Deadline will be using for rendering has already been started and not before. For example, for simple tests you can simply launch the deadline launcher and workers manually from a remote desktop connection.
3 - During a Mistika render, if the user who launched the graphics session did it via Microsoft RDC (free version) then he can not disconnect until all the renders are finished. If his user account is locked or logged out then Windows will inmediately destroy the necessary graphics context at disconnection time, and the render will stop. Also, if you keep the connection open but a different user connect to the same VM then your old graphic sessions will experience the same problem.
The paid version of Microsoft RDC have the optional feature to keep the graphic session alive after disconnection. Another solution is to simply use an VNC application capable to keep the graphics session alive after disconnection (some VNC apps support it, but some not). Install and start the VNC server (also opening the corresponding ports) and then connect via VNC to connect and launch the graphic applications (Deadline clients and/or Mistika directly). In the case of linux, x11vnc is a good option.