arma 3 normal space problems

[olli_ 13]

Im trying to get my normal maps to look good in in buldozer but the normal space is really weird. My normals look fine in 3ds max, but in bulldozer it doesnt render out properly.

http://i.imgur.com/AKTcLnH.png

(left is 3ds max and right is buldozer)

some info:

model is exported as fbx 2010

normal map is 32 bit tga (tried 24bit as well) with _NOHQ suffix

ive tried all output options in handplane with an object space normal but none of them worked in arma. source engine was closest i think but still noticably off)

have tried triangulate / and triangulate \ in O2. neither worked.

have triangulated my fbx before i exported but for some reason in the O2 fbx importer makes it back in to quads

edit: is there some way to import explicit vertex normals? that might help

Edited September 11, 2014 by Olli_

[olli_ 13]

alright i did some testing and ive outlined my results on polycount.. might as well copy the wall of text here if anyone finds any help in it:

alrighty. I decided to break out the "insane graph editing stuff" after all since nothing else was working. I created a control mesh which was basically a flattened cylinder but with the normals preserved (so a flat plane with a round normal surface)

I baked normals on to this control mesh from a flat plane, so in max the normals were working as intended and the control mesh was rendering as a flat plane with the normals.

Next I moved on to oxygen where i realized i cant export explicit vertex normals with the FBX importer (im pretty sure its oxygens fault since i double checked that the normals are indeed exported with the fbx) BUT i managed to import vertex normals with the .obj importer, although even then i had to manually set all the normals to explicit (edit normals>parameters>make explicit).

Now the only problem is that the .obj importer is a piece of shit compared to the FBX importer, but thats not relevant to the tangent space topic so ill leave that rant for another day.

now that i had a control mesh, and a normal map which should make the mesh flat, i moved on to photoshop and started a long process of trial and error with the curve editor.. Here's what i came up with:

With this curve, the normal map correctly flattened my control mesh normals. (at least within the available spectrum)

as you can see almost half of the spectrum is completely unused. If I hadnt spent the last month bashing my head against the wall with this engine I'd have discarded these results as absurd but quite frankly im not the least bit surprised.

(all values below are approximate)

To explain the curve a little more, basically what it does is instead of the usually -90 to 90 degree range of normal maps, the range is a lot less. Around -50 to 50 for 0-255. Additionally the "gradient" isnt linear, its slightly curved at the ends. The scale seems linear between around 95-160 and from there it starts to loosen, so one grayscale unit equals a smaller amount of degrees than it does within 95-160. My curve corrects that for the most part, but I made it completely by hand so its obviously not 100% accurate.

Now if you'll excuse me i have a date with the obj importer.

edit: I had a lot of trouble getting the obj importer to work properly (invertex faces, mirrored normals, all kinds of creepy stuff) so i might as well post solutions i found here. in order to import a model properly with the .obj importer, first i had to rotate my mesh 180 degrees around the X axis (so its upside down) then I reset xform and exported the .obj with all optimization options off, flip YZ on and all the normal stuff ticked. (also export as quads, its easier to triangulate in o2 since results may vary so you dont have to re-export)

In o2 you just import the obj with FLIP Y ticked. I also had to triangulate \ (NOT triangulate / as a lot of people in this thread claim to be the correct way)

the reason you have to rotate the mesh 180 degrees before you export is because if you dont have flipY ticked in the importer, your faces will be flipped, and if you dont have the mesh rotated 180 degrees, flipY will import your mesh upside down, and if you try to rotate it in o2 the mesh will rotate but the normals wont, so itll FUBAR.

i would give anything to meet whoever implemented this pipeline so i could give them a real good slap on the face.

[olli_ 13]

Im getting really tired of this engine's shenanigans. Saving completely wrecks the shading..

seriously what the hell am i supposed to do?

[p1nga 23]

Im getting really tired of this engine's shenanigans. Saving completely wrecks the shading..

http://i.imgur.com/bxoX2Iq.png

seriously what the hell am i supposed to do?

So it is not just me that has issues with getting some parts of surfaces to look as the should..........

[kiory 405]

Try triangulating the model in Max before hand, then import as whatever format you desire into O2, then re triangulate in O2, either / or \, whatever gives you the best results. Also, if you make any slight changes, rotating, changing size etc, don't forget to recalculate your normals in O2.

Oh and for the record, don't ever rely on 3DS Max viewport for accurate normals display, it's terrible.

One more thing, I've had best results using .3ds for O2, it seems to favour this format over all others for static objects.

Edited September 14, 2014 by Kiory