This page introduces and provides information on setting up Distributed Rendering.
Distributed rendering is a technique for distributing a single render job within a single frame across many computers in a network. There are different approaches to doing this, but the main concept is to reduce the render times by dividing different parts of the rendering pipeline and giving each participant different parts of the job. The most common way to do this is to divide the frame to be rendered into small regions (buckets) and give each machine to render a number of them; then get the results and combine them into the final image.
The V-Ray Standalone version included in the 3ds Max installer is intended for DR use when V-Ray GPU is set. It is not a replacement for the 3ds Max render slave, which is intended for use with the V-Ray renderer.
V-Ray supports DR. It divides the frame into regions and spreads them across the participants in the distributed rendering. This is done completely through TCP/IP protocol which is the standard protocol of the Internet and thus the most common protocol that is supported by the hardware. V-Ray itself does not need additional file or directory sharing (note that you may actually need some file/directory sharing for the bitmaps or other additional files used during rendering). The distribution management is divided into Render Clients and Render Servers.
The render client is the computer that the user is currently using and from which the rendering is started. It divides the frame into rendering regions and spreads it across the Render Servers. It distributes data to the render servers for processing and collects the results.
A render server is a computer that is part of the so-called render farm - it requests render data from the render client, processes it and sends the result back.
If you create a fresh install of 3ds Max on a machine, you will need run at least one Backburner job with this machine as a server, before you can use the machine for distributed rendering.
How to Test
First start with the testing of the render server:
Start the vrayspawnerXX.exe program from the Start menu (Start menu > Programs > Chaos Group > V-Ray for 3dsmax > Distributed rendering > Launch V-Ray DR spawner). It will automatically try to find the 3dsmax.exe file and start it in server mode. You should end up with 3dsmax minimized down in the task bar. It should have the title "vraydummy.max". If 3ds Max stays there alive without closing then you have the Render Server part working. If you see a 3ds Max window flashing on the taskbar and then disappearing, right-click on the V-Ray DR spawner icon in the taskbar tray, select Exit to close the DR spawner, and try running a regular Backburner job with this machine as the server. After that, try to start the V-Ray DR spawner again.
Now test the render client:
Open your 3ds Max as you normally would.
Open a scene you would like to render (preferably a simple one for this test).
Choose V-Ray as your current renderer and make sure you have checked Distributed Rendering ON in the V-Ray System section.
Press the Settings button in the Distributed Rendering section.
Add the machines used for rendering – either with their IP address or their network name and close the dialog.
Render the scene as you normally would. You should see the buckets rendered by the different servers.
Practically every action taken by V-Ray DR is logged. You can find all the log files in the C:\ directory and find out what has failed. If you do not understand the problem, you can compress and send the files to Chaos Group Support to analyze them and try to help.
If any of the servers fails, you should get a notification and the render client will try to reassign the buckets to another server.
Distributed Rendering Settings
The Distributed rendering settings dialog is accessible from the System rollout of the renderer settings.
The settings themselves are stored in a text file called vray_dr.cfg, which is located in the 3ds Max plugcfg folder (normally, this is located in the local application data folder for the current Windows user).
These settings can be adjusted while V-Ray is rendering. Note that progressive production mode and GPU(CPU and GPU) mode only allow adding and enabling servers on the fly. CPU Bucket mode supports both adding and removing servers on the fly.
Add server – Allows the user to manually add a server by entering its IP address or network name.
Edit server – Allows the user to change the data about a render server.
Remove server – This button deletes the currently selected server(s) from the list.
Resolve servers – This button resolves the IP addresses of all servers.
Restart servers on render end – Enabling this option will cause the 3ds Max copy on the render slaves to be restarted once a DR rendering is finished. This may help to make distributed rendering more stable.
Save servers in the scene – This option causes V-Ray to store the list of render servers into the 3ds Max scene. This is useful if the scene is later sent to Backburner for distributed rendering. In that case, instead of taking the DR servers list from the vray_dr.cfg file on the machine where the backburner server is running, the list will be taken from the 3ds Max scene.
Max servers – Allows the user to limit the maximum number of servers from the list to be included in distributed rendering. When set to 0, all listed servers will be used.
Use local host – When enabled, the client machine (the one from which the user initiates the DR render) will also take part in the rendering calculations. If disabled, the client machine only organizes the DR process, sends rendering tasks to the render servers and displays the final result.
Render region warning – When enabled, V-Ray will display a warning if the render region option is enabled before sending a job to the network.
Transfer missing assets – Allows render servers to automatically download missing rendering assets from the client machine. This includes textures, V-Ray proxy files, IES profiles, GI solution caches (irradiance map, light cache), X-Ref scenes etc.
Use cached assets – This option tells the render servers to keep the downloaded rendering assets between different DR sessions. If disabled, the render assets are automatically deleted once the render server disconnects from the client machine at the end of the rendering. When enabled, assets are kept in a folder on disk. By default, it is named vray_assets_cache located in the temporary folder for the user that runs the V-Ray DR spawner. The location of the assets cache can be modified with the environment variable VRAY_ASSETS_CACHE_PATH. Modified assets are generally detected and re-transferred to the render servers even though they may already exist in the render servers' caches.
Cache limit type – Allows the user to specify under what conditions the assets cache is cleaned up. The possible values are:
None – Assets are never deleted from the render servers' cache.
Age (hours) – Assets are kept for the specified amount of time. At the end of the rendering, the render servers will delete assets that have been transferred before the specified time limit.
Size (GB) – When the size of the assets on a render server exceeds the specified amount, least recently used assets are automatically deleted at the end of each render until the folder size is less than the limit.
Cache limit value – The specific value of the limit for the asset cache. This is either hours or GB, depending on the value of Cache limit type.
V-Ray DR Spawner command line options
You can add command line options to the VRaySpawner.
You can specify Numa command line options to VRaySpawner.exe to better utilize processor nodes with Numa architecture:
-numa[=N] - select the number of slave processes to start. If N is not given, N=0 or N>=M, one process per NUMA node will be created. Here "M" is the number of selected nodes using "-node", or the number of all available NUMA nodes if no "-node" is used.
-node=node1[,node2][,node3]... - select NUMA nodes that will be used for the slave process(es). Default - use all available NUMA nodes.
-port=port1[,port2][,port3]... - select listening ports for the slave process(es). If the number of processes is greater than the number of ports given, the last port is auto incremented. The default listening port is 20204 for 3ds max. The render client should be set to use the corresponding port range.
Using 3ds max 2012 and a system with 8 NUMA nodes:
vrayspawner2012.exe -numa - spawns eight 3dsmax.exe processes, every running on single NUMA node and using listening ports 20204-20211
vrayspawner2012.exe -numa=4 - spawns four 3dsmax.exe processes, every running on two NUMA nodes and using listening ports 20204-20207
vrayspawner2012.exe -node=3,5,6 -ports=30000,40000 - spawns three 3dsmax.exe processes, running on NUMA nodes 3,5 & 6 and using listening ports 30000, 40000 & 40001
vrayspawner2012.exe -node=2,3,4,5,6 -numa=3 - spawns three 3dsmax.exe processes, running on nodes (2,3) (4,5) & (6) and using listening ports 20204,20205 & 20206
Running the DR Spawner as a Windows Service
You can set the V-Ray DR spawner to automatically run as a Windows service whenever the machine boots up.
Registering the DR Spawner
To register the DR spawner as a Windows service, go to Start menu > All Programs > Chaos Group > V-Ray for 3ds Max NNNN > Distributed rendering > Administration > Register V-Ray DR spawner as a service.
Mapped Drives and UNC Paths for Texture Maps and Other Rendering Assets
If you are using mapped drives or UNC paths for your rendering assets, you should set up Log On account for the V-Ray spawner service, otherwise, the textures or any other external assets won't be accessible. To set up the Log On account, open the "Services" applet from the Control Panel (Control Panel > Administrative Tools > Services) and find "VRaySpawner 20XX" service, then with the right mouse click expand the drop-down menu and choose Properties. Go to “Log On” tab and instead the default "Log on as local system account" choose "This account" and put there the account that has adequate privileges on the machine and access to the shared resources over the network.
Alternatively, you can use the automatic asset transfer mechanism of V-Ray as described above to send the required assets from the client machine to the render servers.
TCP/IP Port Numbers
Distributed rendering works over TCP/IP and requires the following ports:
|Renderer type||Port number||Port type|
Port 20205 is only used by render servers to broadcast a message that they are ready to join an ongoing DR rendering.
- Every render server must have all the plugins and texture maps in their proper directories loaded so that the scene you are sending will not cause them to abort. For example having a PHOENIX plugin used in the scene will cause a server failure in any of the servers that do not have the PHOENIX plugin installed. If you have mapped your object with a file named JUNGLEMAP.JPG and you do not have that map in the BITMAPS directories of the render server installation - you will get bucket rendered at that machine as if the map was turned off, unless you also turned on the Check for missing files option in the V-Ray System rollout, in which case the render server will refuse to render the scene.
- When you cancel a DR rendering, it may take some time for the render servers to finish working and they may not be immediately available for another render.
- Default lights are not supported in DR mode and will not render. If you need any lighting in the scene, you should add it explicitly.
- The default system temp folder, usually C:\Users\<username>\AppData\Local\Temp on Windows (%TEMP%), is used for copying the assets if the VRAY_ASSETS_CACHE_PATH environment variable is not set.