Implementing a wave-distortion shader with Nvidia FX Composer

I spent most of the day fighting with one of most difficult and arcane pieces of software I’ve ever used: Nvidia FX Composer. I think it was written five or so years ago by members of a suicide cult, who upon releasing it promptly all drank the poison cool aid. Nvidia makes a pretense of running a customer support forum for the product, but most of the posts are desperate questions without a single response. On the other hand, the included documentation is relatively massive… but years out of date and therefore occasionally completely inaccurate, which in many ways is worse than not having any at all. Of course, my own total lack of experience writing HLSL didn’t help, either.

In the end I persevered and got this effect implemented and merged into my game:

Once I finally figured out how to use it, FX Composer was a huge help in being able to rapidly experiment with different effect parameters and see the result. Here’s what the software looks like once it stops kicking you in the balls:

What I had so much trouble with was figuring out how to convince the software that my shader was a post-processing effect to be rendered onto the entire scene, not onto a 3D object in the scene. After a lot of research and experimentation, I started with a working post-processing shader provided as a sample and starting taking things out of it until it broke. That’s when I learned you need these special semantics to tell FX Composer that the shader is for post-processing:

float Script : STANDARDSGLOBAL <
    string UIWidget = "none";
    string ScriptClass = "scene";
    string ScriptOrder = "postprocess";
    string ScriptOutput = "color";
    string Script = "Technique=Main;";
> = 0.8;

If this looks like gibberish to you, it does to me too. But these are the magic words needed to get FX Composer to let you apply a shader to an entire scene. It’s also important that you have a vertex shader defined in your technique, even if you’re only using a pixel shader — nothing will work otherwise. Here’s a pass-through shader you can use:

struct VertexShaderInput
    float4 Position : POSITION0;
	float2 Txr1: TEXCOORD0;
struct VertexShaderOutput
    float4 Position : POSITION0;
	float2 Txr1: TEXCOORD0;

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
    VertexShaderOutput output;
    output.Position = input.Position;
	output.Txr1 = input.Txr1;
    return output;

The final piece of the puzzle was getting the updated shader back working in XNA. There are lots of articles on this on Google, so no need for me to repeat it here. But I did discover something interesting: if you have an entirely 2D game, not a single 3D triangle rendered anywhere, then using a vertex shader in your effect, even the pass-through shader, will cause no pixels to get through to your pixel shader. The solution is to just comment out the vertex shader in your pass. Since FX Composer requires this shader and XNA requires it be absent, toggling the comment is an easy way to make both systems happy.

technique Main <
	string Script =
> {
    pass Pass0 <
       	string Script= "RenderColorTarget0=;"
    //VertexShader = compile vs_2_0 VertexShaderFunction();
	PixelShader = compile ps_2_0 PS_wave();

This was a bit more of a rabbit hole than I really intended to climb into, but at least I learned a few somethings.

Also, I filmed the motion-capture sequences for the main character’s animations today in Roark’s back yard. I need to do a lot of video editing on the result, then the hard labor of cleaning up all the pixels and shoving it into the game, but that’s tomorrow. For now, enjoy this outtake from the shoot:

Leave a Reply