Main Site Links Resources Tutorials
News VB Gaming Code Downloads DirectX 7
Contact Webmaster VB Programming Product Reviews DirectX 8
  General Multimedia Articles DirectX 9
      Miscellaneous

DirectXGraphics: Vertex Blending
Author: Jack Hoxley
Written: 31st July 2002
Contact: [EMail]
Download: Vertex Skinning.zip (58kb)


Contents of this lesson
1. What is Vertex Blending?
2. How to set up simple vertex blending
3. Extending this to use all 4 matrices
4. Taking it all the way: Indexed vertex blending


1. What is Vertex Blending?

I'm going to have to assume that by this point in the series you have a reasonable understanding of the Direct3D pipeline - if not, this tutorial won't make a huge amount of sense to you.

As you should be aware, when you send vertices to the device for rendering (Device.Draw* calls) they go through a number of stages before final rasterization. In particular, there is the transformation engine - this is near to the start of the process, and deals with multiplying every vertex in the data stream by the world matrix (view and projection can be done later). The world matrix will typically rotate scale and translate a given object.

As it currently stands, the world matrix transforms every vertex in exactly the same way - the whole mesh is rotated, or the whole mesh is translated across world space. Using current techniques you cannot apply different transformations to different parts of the same mesh without rendering each part separately. This works well in most cases, but it's lacking a more fluid, dynamic and artistic touch.

Take a look at the following screenshot (from the sample):

Nothing hugely exciting about this - a simple textured octagonal prism. Given current methods, and without altering the raw-geometry how would you make the above mesh render like this:

Interesting eh? Even if you were allowed to alter the actual geometry being rendered, the above shape would still be a little tricky to create (unless you used a software emulation of the D3D technique!).

To sum up, vertex blending allows us to specify up to 4 matrices per rendered mesh, and to specify how the transformations are distributed across the geometry. The above image was generated using 2 matrices, with the second matrix focusing mostly on the bottom of the prism (hence why most of the distortion is visible there).

Using vertex blending with bones is a very common technique in commercial games. Take a humanoid mesh, and set up a simple skeleton (you may need to read up further on skeletal animation), typically a set of vertices are attached to each node in the skeleton. As the skeleton is animated it moves the (rendered) mesh around. Using vertex blending you can let any vertex be influenced by more than one matrix (where each matrix represents a bone for example). This creates much more realistic character animation - much smoother (you can even make muscles flex) and fluid.


2. How to set up simple vertex blending

Vertex blending is not a hard effect to master, the most difficult aspect is deciding how you are going to use it. You will need to determine a pattern or algorithm to set blending values for each vertex, and then to decide what transformation to store in each slot.

The first step is to check for hardware support of vertex blending matrices. You can only ever apply a maximum of 4 matrices per vertex whatever the hardware.

Dim DevCaps As D3DCAPS8
D3D.GetDeviceCaps D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, DevCaps
If DevCaps.MaxVertexBlendMatrices < 2 Then
    Unload Me
    End
End If

The important part is the MaxVertexBlendMatrices value, this will either be 0,1,2,3 or 4. 0 indicating no support for vertex blending (the majority of recent hardware has at least minimal support). For the sample code presented with this tutorial, a simple 2 matrix blending method is used, hence the above code only checks for 2 or more.

The next step is to alter the vertex format - we must specify a blending 'weight' for each vertex you want to render. This is a floating point value indicating how much each matrix is going to contribute to the overall result. Typically this will be left in the range of 0.0 to 1.0, with 0.0 being disabled/no effect, and 1.0 being fully effective. 1.0 doesn't mean that it will be the only matrix that affects the current vertex, it just means that the current matrix is not scaled. The general formula being:

Pfinal = Pvertex * W1*M1 * W2*M2 * W3*M3 * (1.0-W1-W2-W3)*M4

where Pfinal being the final world-space coordinate, Pvertex being the original model-space vertex coordinate. W1, W2 and W3 are the three floating-point blending weights. M1, M2, M3 and M4 are the four matrix slots.

The vertex data type, for this simple sample, is altered in two ways: The actual data structure and the the FVF descriptor.

Const FVF_BLENDEDVERTEX2 = (D3DFVF_XYZB1 Or D3DFVF_NORMAL Or D3DFVF_TEX1)

Private Type BLENDEDVERTEX2
    P As D3DVECTOR
    B1 As Single
    N As D3DVECTOR
    T As D3DVECTOR2
End Type

The data structure has an additional single, this MUST come between the position data and the normal data - or Direct3D will get confused. The FVF has changed so that the first flag is now D3DFVF_XYZB1 - the B1 indicating that we have 1 weight defined in the vertex.

This sample only uses the first two matrix slots (of the four available), that is the D3DTS_WORLD and D3DTS_WORLD1 slots defined in the Direct3DDevice8.SetTransform(  ) function. If we plug this into the general equation, we can now see the following:

Pfinal = Pvertex * B1*D3DTS_WORLD * (1.0-B1)*D3DTS_WORLD1

that is, as you might recognize, the standard linear interpolation equation. As B1 tends towards 1.0 the vertex will be more affected by the D3DTS_WORLD matrix, as it tends towards 0.0 it will be affected by the D3DTS_WORLD1 matrix. Anything in the middle will be a mix of both!

One slight problem remains. When we load geometry from a .x file, it probably won't know that it's going to be used for vertex blending purposes - and as such, won't have the necessary blending weight parameter in the vertex structure. We must therefore add this member to the mesh's vertex format. When you've loaded the mesh as per normal, we use this following short piece of code to alter the vertex format:

If Not (TubeMesh.GetFVF = FVF_BLENDEDVERTEX2) Then
   
Set TubeMesh = TubeMesh.CloneMeshFVF(D3DXMESH_MANAGED, FVF_BLENDEDVERTEX2, D3DDevice)
End If

Obviously, it's not necessary to apply this process to "hand-made" geometry, as you should have programmed the algorithm/code to use blending weights from the beginning!

CloneMeshFVF( ) will have now allocated space for the blending weight, BUT it won't have put any data in - it will be left as 0. Therefore the next step is the all important - setting blending weights:

Dim MeshVerts() As BLENDEDVERTEX2
ReDim MeshVerts(TubeMesh.GetNumVertices) As BLENDEDVERTEX2
D3DXMeshVertexBuffer8GetData TubeMesh, 0, TubeMesh.GetNumVertices * Len(MeshVerts(0)), 0, MeshVerts(0)

'we can now process each vertex
'4a. I want to know the maximum Y value

For I = 0 To TubeMesh.GetNumVertices
   
If MeshVerts(I).P.Y > TubeMesh_YHeight Then TubeMesh_YHeight = MeshVerts(I).P.Y
Next I

'4b. we can now calculate the blending weights
For I = 0 To TubeMesh.GetNumVertices
    MeshVerts(I).B1 = Sin(((MeshVerts(I).P.Y - 0) / TubeMesh_YHeight) * (PI / 2))
Next I

D3DXMeshVertexBuffer8SetData TubeMesh, 0, TubeMesh.GetNumVertices * Len(MeshVerts(0)), 0, MeshVerts(0)

There isn't anything new to the above code, it's essentially the same code used by the keyframe animation tutorials (original, extended). As already mentioned, the method you use to determine blending weights can be as simple or as complicated as you like. Bare in mind that if you have to regularly alter the blending weights that calculating it for a 1000 vertex model (as in this sample) can slow the application down quite a bit.

By this point we've done all the initialization necessary to use vertex blending in our application. There is one final render state to observe before rendering the mesh:

'D3DVBF_DISABLE - render using the standard 1-matrix transformation pipeline
'D3DVBF_0WEIGHTS - use first matrix only, but no weights specified in each vertex
'D3DVBF_1WEIGHT   - one weight value per vertex, use first two matrices
'D3DVBF_2WEIGHTS - two weight values per vertex, use first three matrices
'D3DVBF_3WEIGHTS - three weight values per vertex, use all four matrices

D3DDevice.SetRenderState D3DRS_VERTEXBLEND, D3DVBF_1WEIGHT

'standard code: tell D3D what data is being rendered next...
D3DDevice.SetVertexShader FVF_BLENDEDVERTEX2

As a side note, D3DVBF_0WEIGHTS has the same outputting result as D3DVBF_DISABLE, but doesn't actually disable the extended transformation pipeline - so probably isn't as efficient. If you're not going to use blending weights for a while, it would be best to use D3DVBF_DISABLE. Also, in the SDK help files, D3DVBF_1WEIGHT is spelt wrong! _1WEIGHT is correct, unlike the _1WEIGHTS in the SDK help files.

That is all of the vertex blending samples at the simple level. As it currently stands, whatever type of matrices are defined in D3DTS_WORLD and D3DTS_WORLD1 will affect all rendered geometry. The sample code makes use of rotating through two axis (Y and Z) to get the results shown in original screenshots. The code for this is not very important, and you can find it in the downloadable archive.


3. Extending this to use all 4 matrices

The previous section dealt with using only the first two matrix slots. Just to cover all the bases I'm going to give a quick demonstration of how you could use all 4 matrices - with these two examples you should be able to work out how to do 3-matrix blending. The downloadable sample code does not include any of the source from this point onwards, but you should see how it will easily fit in...

For using all 4 matrices we must change the FVF and the vertex declarator again:

'increase _XYZB1 to _XYZB3
Const FVF_BLENDEDVERTEX4 = (D3DFVF_XYZB3 Or D3DFVF_NORMAL Or D3DFVF_TEX1)

'add B2 and B3
Private Type BLENDEDVERTEX4
    P
As D3DVECTOR
    B1
As Single
    B2 As Single
    B3 As Single
    N As D3DVECTOR
    T
As D3DVECTOR2
End Type

By using this data format, you need to put valid matrices in all four slots: D3DTS_WORLD, D3DTS_WORLD1, D3DTS_WORLD2, D3DTS_WORLD3. The weighting of all 4 matrices is calculated as follows:
D3DTS_WORLD = B1
D3DTS_WORLD1 = B2
D3DTS_WORLD2 = B3
D3DTS_WORLD3 = (1.0 - B1 - B2 - B3)
which follows the format of the general equation stated earlier.

When you are ready to render the geometry you must use the following two lines of code:

D3DDevice.SetRenderState D3DRS_VERTEXBLEND, D3DVBF_3WEIGHTS
D3DDevice.SetVertexShader FVF_BLENDEDVERTEX4

upon doing this you will be ready to go!

4. Taking it all the way: Indexed vertex blending

Vertex blending has been available through Direct3D for several versions now, the methods discussed thus far are nothing new. However, there is one new addition to vertex blending for Direct3D8 - indexed vertex blending.

This still limits you to a maximum of 4 matrices per vertex, BUT it allows you to specify up to 256 matrices for each call. How does this work? well, it's actually very clever...

If you're an experienced DirectX programmer, you may well remember using/learning how 8-bit palletized textures work. Each pixel stores an 8bit number - this indexes to to a palette of 256 different colors, but each color in the palette is defined as a 24 bit RGB triplet. A similar idea applies here.

For each vertex we still have the blending weights (1,2 or 3 of them), but we also add a 4-byte packed index value. This 32bit long is 4 indices packed into one variable. The idea being that you can store the 3 weights as appropriate, but then use the indices to select which 4 matrices they apply to - from a list of up to 256.

Say for example you had a list of 35 matrices setup, and for a triangle of 3 vertices you wanted 11 different matrices (based on each vertices distance from a bone node for example). This wouldn't be possible using the traditional method.

For example:
V0 = matrices 1, 3, 5, 7 weights of 0.2, 0.3, 0.4
V1 = matrices 1, 3, 6, 9 weights of 0.2, 0.6, 0.1
V2 = matrices 2, 4, 6, 8 weights of 0.1, 0.05, 0.3

to use indexed vertex blending we must (again) alter the vertex structure and the FVF:

Const FVF_INDEXBLENDEDVERTEX4 = (D3DFVF_XYZB3 Or D3DFVF_LASTBETA_UBYTE4 Or _
                                                         D3DFVF_NORMAL Or D3DFVF_TEX1)

Private Type INDEXBLENDEDVERTEX4
    P
As D3DVECTOR
    B1
As Single
    B2
As Single
    B3
As Single
    IdxList
As Long
    N
As D3DVECTOR
    T
As D3DVECTOR
End Type

A useful trick for setting the four indices is to use the D3DColorARGB( ) function. Because VB doesn't have an unsigned 32bit integer variable, when bit shifting/packing the 4 indices we have to mess around with the signed bit. Using D3DColorARGB( ) avoids this problem. For example:
'//based on the data from the example above.
V(0).IdxList = D3DColorARGB(1, 3, 5, 7)
V(1).IdxList = D3DColorARGB(1, 3, 6, 9)
V(2).IdxList = D3DColorARGB(2, 4, 6, 8)

If you do use the above technique, but choose not to use 4 matrices then the color components refer to:
A=D3DTS_WORLD3
R=D3DTS_WORLD2
G=D3DTS_WORLD1
B=D3DTS_WORLD

Before you actually use indexed vertex blending you need to check hardware support. Only the more recent hardware will support any significant number of matrices - my ATI Radeon8500 supports 58 matrices (slots 0 to 57), and not the full 256 possible.

Dim DevCaps As D3DCAPS8
D3D.GetDeviceCaps D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, DevCaps
DevCaps.MaxVertexBlendMatrixIndex
'this stores the number you mustn't exceed.

When you actually want to use indexed vertex blending there are 3 things you must do:

first: create and apply each matrix. You can't apply more matrices to the device than specified in the above enumeration. The following code will allow you to set a matrix in slot "I":

D3DDevice.SetTransform D3DTS_WORLD + I, matTheMatrix
'More specifically, D3DTS_WORLD = 256, therefore, slots 256 to 511 are reserved for
'the matrix palette.

secondly, you must enable indexed vertex blending:
D3DDevice.SetRenderState D3DRS_INDEXVERTEXBLENDENABLE, 1

lastly, you must set the FVF:
D3DDevice.SetVertexShader FVF_INDEXBLENDEDVERTEX4

Having completed all these steps, all geometry rendered from now on will be probably blended based on the matrix palette indices and the weighting values.

Configuring Direct3D to let you use indexed vertex blending is not a complicated task - however, writing an algorithm that selects the correct weights and matrix indices is going to be the far harder challenge. It may well be worth writing a plugin/tool for a 3D renderer that allows an artist to specify weighting values.

One solution I have thought of - assuming you were using this technique in conjunction with skeletal animation (I would!):
assume a reasonable skeleton with 58 bones (the same number of matrices my Radeon8500 supports). We can generate a matrix for each bone node (as a trivial example - shoulder->elbow->wrist nodes) quite easily. Then, we loop through every vertex in the mesh/skin, and calculate a list of distances from the vertex to each bone-node. Sort this list so we can pop-out the 4 closest bones (this isn't a very smart method, but it would work). We can then work out as a proportional value how close each bone is to the vertex and generate 3 blending weights. Algorithm complete.

In using a solution like that the artist need only specify a skeleton animation and a mesh-skin, and the resulting animation would be a very convincing and fluid system.


So, that is this tutorial completed. Uses for vertex blending can be huge - but I'll have to let you do your own research there.

You can (and I strongly advise you do) download the sample code from the top of the page, or from the DirectX8 Downloads page.

Enjoy!

DirectX 4 VB 2000 Jack Hoxley. All rights reserved.
Reproduction of this site and it's contents, in whole or in part, is prohibited,
except where explicitly stated otherwise.
Design by Mateo
Contact Webmaster