Make sure xgenVRay.py is loaded as a plugin into maya (Plug-in Manager - near xgenToolkit.mll) before creating scenes or rendering:
When rendering XGen descriptions with V-Ray make sure the Output Settings in the Preview/Output tab of the XGen window are as follows:
If VRay is not in the Output Settings as shown above, try restarting Maya (with xgenVRay.py as "Auto load" in the Plug-in manager)
To ensure that V-Ray is there, always start Maya 2015+ normally, check in the Plug-In Manager that the module xgenVRay.py is loaded and then load/create XGen scenes. Opening scenes with double-clicking on them does not ensure V-Ray being registered in the XGen UI.
V-Ray can render two kinds of primitives XGen has to offer - Splines and Archives. Spheres and Cards primitives are not supported yet:
Here is how instanced primitives look like when the base mesh has low and high subdivision; note how XGen clumps primitives more to the center of each face. This is the same behaviour for all kinds of primitives.
Base meshes onto which XGen populates primitives should use Maya's subdivision settings. V-Ray's subdivision settings will not work with them.
Working with XGen and many primitives can be made faster by reducing the percentage of drawn primitives in the viewport - go to the Preview/Output tab of the XGen window and there you can find the Percent attribute.
Patches can be added/removed to/from descriptions
Normally shape nodes have visibility attributes in the Render Stats section, but the XGen patch/description placeholder shapes do not have them visible. To control such visibility flags use V-Ray Object Properties.
Materials and V-Ray Object Properties may be assigned to descriptions and all primitives for that description will share them; they can also be assigned on a per-patches basis for fine-grained control over primitives on different patches. This means 2 patches in the same description can use different materials and have separate V-Ray Object Properties.
For example if you have a description with 4 patches in it you can assign a master material to the description and give a specific material to one of the patches and the result will be like this
Note that you should assign the description material to the transform node, not the shape - such assignments are hierarchical - they affect the selected node and all its children and the patches are not children of the description shape but of the description transform
Controlling XGen Attributes
Click on the Sigma button of an attribute:
and edit the expression in the lower textbox of the popup. Click Accept to use it.
Through texture maps
Here we will show how to use a texture map for the Length attribute.
1. Create a plane and create an XGen description on it with splines randomly across the surface.
2. Crank up the Density from the Primitives tab of the XGen window to 50.
3. Click on the arrow pointing down next to the Sigma button of the Length attribute and select Create Map:
4. A pop-up window appears:
5. Increase the Map Resolution to around 200 (This is the resolution of the PTex map in texels per-face. Use larger values for high resolution textures).
6. Name the map in some way and click Create.
7. You are now in PTex painting mode. You can paint on the surface of the plane with the 3D Paint tool.
8. If you wish to use an image from the filesystem, open the Hypershade and Node Editor and click on this icon next to the Length attribute:
9. You should see a file texture connected to the plane:
10. Select the file texture and open the texture map that you want to use to drive the length of the primitives. Texture maps with higher contrast work out better for controlling attributes.
11. When you are done painting/selectiong a texture, click on the Save icon next to the attribute; always do this after changing a texture map. Otherwise XGen will not refresh and the changes will not be visible in the viewport/renders.
12. Using a bitmap like this should correspond to a view in the viewport like this:
13. If we edit the expression of the Length attribute through the Sigma button from this:
to this (which inverts the result of the map):
and render the scene with a dome light in it, we should see something like this: