I have a question concerning the use of overlay nodes and custom vertex/fragment shaders. my fragment shader does nothing more than modify each pixel's color value (modeled after the faded layer node's shaders to take into account the 4 textures and 8 light sources). Prior to including the shaders I had an overlay node which projected a simple polygon over terrain. The node was included in the scenegraph by using the MapNode installOverlayNode method. this worked well, a blue polygon was projected onto the terrain.
However once I added the shader I got very odd results. It appeared to have several black polygons that resembled the shape of the original overlay but they scaled to different sizesall from the same geometry within the overlay node.
Q1: Has anyone had experience with using overlay nodes and shaders. did i miss a step?
Q2: A slightly unrelated note: Is there an easy way to incorporate multiple overlay nodes on the same MapNode?
After touring the source ( as one should before posting a question ) I see that the overlay node itself installs a set of shaders in order ( of course ) to modify the pixels colors where the overlay is supposed to exist. Does this means that I need to create a custom shader to accommodate for both the overlay and my custom requirements? if so, does this program get installed to the overlay node or to the MapNode? Id at any given time there are 0 to n geometries I need overlayed then i suppose this becomes a fairly involved shader.
Thoughts? comments? ... insults?
( p.s. Q2 is still in the air if anyone has experience with it)
Well, OSG does not yet have shader composition - this means that one shader will completely override another in the scene graph. I have not looked at the OverlayNode source in a while but I thought it just used a shader on the overlay graph itself. If you are trying to also shade the overlay graph, then yes, Is suspect you will need to combine you shader with the overlay shader manually.
I'm getting a bit confused about some of this, (and it may be how osg handles the shaders or when the rendering pipeline gets executed) :
I am installing my custom shader to the MapNode itself, but if that map node installs the overlay node, which inserts itself infront of the terrains group
overlay->addChild( this->getChild(CHILD_TERRAINS) );
this->replaceChild( this->getChild(CHILD_TERRAINS), overlay );
then the mapNode shaders should execute... after the overlay node has been processed... but in this traversal, the final shader should be the only one used? Then the map node should exclude the overlay as a whole. However, the multiple black overlay geometries are getting rendered as well. [ sorry, that is a regurgitation from my head ]
So the question I suppose I have is: how does osg (or perhaps opengl) use multiple shaders (via osg::Programs)? are they isolated to subtrees where only the last node closest to the root will apply the final shader programs?
AFAIK, OSG treats programs like any other state. It basically gets pushed on the top of the stack when encountered. So the last program traversed will be the active one.
Overlay nodes are different though because they embed a camera and do RTT. That RTT subgraph is a separate rendering pass than the main traversal b/c it renders a subgraph to a texture, then projects that texture in the main graph. If the OverlayNode shaders are on the overlay graph, those shaders should not affect rendering on the main graph (including the MapNode).
Of course this is just my theory. I have not studied the innards of OverlayNode. I try to avoid it b/c historically it has never quite been reliable :)