This page provides a tutorial for using the V-Ray Ptex Map to render Mudbox PTex textures.
In this tutorial, we will render an object with several PTex textures painted in Mudbox using the Ptex Map | VRayPtex texture.
Open your object in Mudbox. In this case, we are just going to use one of the stock head models.
To prepare the object for PTex paining, choose Mesh > PTEX Setup... and adjust the desired resolution of the PTex textures. In this example, we just used the default settings.
Start painting the object as you would normally. When you start painting a new layer, the Create New Paint Layer dialog will appear. For diffuse/specular/bump textures, an 8-bit PTex format will usually be enough. For our example, we just painted one diffuse layer.
Sculpt the object as you would normally, adding subdivision levels as needed.
Here is an example of the final painted and sculpted object in Mudbox:
Exporting the Object
Go back to the 0-level resolution of the object, select it, and export it to an .obj file from File > Export Selection ...
Extracting the Paint Layers
Right-click on the paint layer you want to extract, select Export selected..., and choose a name for the resulting PTex file.
Go to Maps > Extract Texture Maps > New operation ... and in the following dialog select Displacement Map.
For Target Models select the 0-level resolution of the object; for Source Models select the resolution that you want to bake (typically the highest resolution).
Turn on the Smooth Target Models option.
Make sure the Method is set to Subdivision (the Ray Casting method may produce artifacts);
Select a PTex file for the Base File Name option.
Set the Data Format to 32-bit float. This ensures that the displacement values stored in the texture are absolute and do not need further adjustment.
This is what the final extraction options should look like:
Click on Extract button to create the vector displacement map.
Importing the Object
In 3ds Max, go to the File > Import... menu and import the .obj file with the 0-level resolution mesh for your object. Make sure that the Retriangulate polygons option in the OBJ importer is disabled.
Create a new V-Ray Material | VRayMtl, assign it to the object, and attach a VRayPtex texture to the diffuse slot. Choose the saved diffuse PTex file in the texture.
Note that the texture will not appear properly in the Material Editor as the preview sphere has a different topology from our object.
If you render the object now, it should be able to access the diffuse texture:
Add a Displacement modifier to the object and set its Type to Subdivision. Click onTexmap button and select a VRayPtex texture. Instance the newly created VRayPtex texture into the Material Editor and choose the displacement PTex file. Leave the displacement Amount in the modifier to the default value of 1 generic unit.
For the following rendering, we disabled the diffuse texture so that we can better see the displacement result:
The texture seems to be clipped in the lowest and highest parts. This is because V-Ray assumes by default that the texture values are between black and white, which is not correct in this case. To work around this, go to the VRayDisplacementMod modifier, and adjust the Texmap min and Texmap max parameters at the bottom of the rollout. In our case, through some trial and error, we found that using -3.0 for the minimum value and 4.0 for the maximum removes the clipping:
For smoother results, especially if you observe the displacement map from a close distance, it might be useful to set the Filter parameter of the VRayPTex texture to a smooth filter like the Bicubic one.
For comparison, the following images show close-ups of displacement with the Box and Bicubic filters.
Box filter. The pixels in the PTex file are clearly visible.
Bicubic filter. The result is much smoother.
Now we can enable the diffuse texture and get our final result: