UV Maps, Normal Maps, and Texturing / Unofficial Thesis Dissertation By James Nichols How do normal maps work? They work by adjusting the normal on every pixel on a face, according to the normal information saved in the texture given the UV coordinates. What does that all mean? To start, let’s dive head first into 3D shading 101. Every geometric face in a 3D model has a "normal" which is a vector describing a line. A vector is simply 3 coordinates put into one data type. The normal depicts a line facing perpendicular to the face surface and is used to tell the 3D renderer how bright to light a face. A vector is also drawn from the face surface to a light source in the area (for simplicity's sake we're talking about one light here). The narrower the angle between the light vector and the face's normal vector, the brighter the surface will be lit. Back in the old days of CG, this lighting model (the lighting model that depicts a lambert shader) was used on a per-face basis which gave the impression of faceted lighting. It was hard to make an object appear "soft". Goraud Lighting introduced normals on the vertices, where light calculations would be found at the normal of each vertex (the normals would be facing away from the average of the face normals) and then the calculations would be interpolated over the face, from vertex to vertex. This led to more realistic lighting, and could sell smooth objects, but left strange artifacts behind. Per Pixel Lighting is where we start seeing bump maps, specular maps, and normal maps being used, since instead of looking at the geometry for lighting, the light calculations would run on each pixel on the screen, making for much more realistic lighting. Per Pixel Lighting allowed for bump maps and more recently normal maps. Bump Maps are used to adjust the brightness level at each pixel based on its grayscale texture. Normal maps are used to adjust the normal vector at every pixel based on the RGB channels on its colored texture. Normal maps are not only glorified bump maps, they work in a completely different, and "more 3D" way, allowing for much better results.
Why should you use normal maps? First and foremost, they just take less processing time to render. Rendering a turntable of a low poly model with a normal map being used to sell the highfrequency detail will take a fraction of the time it will take to render out a turntable of a high poly model. When you’re in a large animation studio complete with renderfarms, you might not need to use this technique, but for our purposes normal maps can help out a lot, even if you're just using them on scenery in the background. Normal maps in the game industry are now becoming a standard; almost every recent title has made use of them. The technique of using displacement maps to tessellate geometry on the fly is starting to make its way into the game industry, but right now normal maps are where it is at. If you're an animator, you can get away without using normal maps, but if you learn how to use them effectively, it will get you far. But don't take my word for it, do your own research. I'm going to go over 3 different techniques that can be used alone or in conjunction to make normal maps. I will also touch on high poly workflow, UV mapping, and texturing. The 3 techniques I'm going to go over include height map textures, Maya renders, and transfer maps. Normal Maps via Height Textures: It's easy enough to make any grayscale heightmap into a normal map. There are plenty of tools out there that will do this for you, so try googling it. Two popular methods that exist are CrazyBump and nVidia's Normal Map filter for Photoshop CS3. I prefer using the Normal Map Filter since I've grown accustomed to it. CrazyBump appears to generate a displacement map from your normal map and both effects are shown at the same time. Even when turning displacement off, the normal maps (at least on hard surface models) never show up right for me. I've had better luck with organic models, and some people swear by CrazyBump, so I digress. I encourage you to try using CrazyBump if you're interested. But I choose the nVidia Normal Map Filter over it since I work on textures primarily in photoshop anyway, and the filter is free. The filter is one of the only reasons I keep CS3 around, besides the fact that CS4 and CS5 bog my machine down. I should mention that the normal map filter works alright if you just run it on your diffuse maps, but ideally you should use a height map. The reason is often diffuse maps will emulate lighting, and specular highlights. Game models are almost never lit by the traditional 3-point lighting setup we see in renders, so to make up for it, a lot of the lighting information for game models need to be emulated in the diffuse texture. So many areas in a diffuse texture are lighter than what they "should be" (like the gloss on a sniper scope, for example). These areas will appear bulged in the resulting normal map if you were too run the filter. The same would happen to white painted on text on a metal plate- although the plate should be smooth, that text will show up embossed. So if anything, remember to take your diffuse map and make it more accurately display the actual depth before running the normal map filter on it. Also- remember to save out your diffuse map, or height map, whatever you're using- as a seperate file, to avoid saving over your already made files! Assuming you've done this, and have Photoshop CS3 with the normal map filter installed (it's on the nVidia website, a few clicks away), you now flatten your image into one layer. Now go Filter > nVidia Tools > NormalMapFilter. A dialog box should appear.
The dynamic preview will show you a thumbnail of what the resulting normal map will look like. If you want to preview how the normal map will look when lit on a plane, you can click the 3D preview button. In the 3D preview, left mouse is rotate, middle mouse is zoom, and right mouse is pan. There are a few options in the normal map dialog box to change how the 3D view works. By default, the light should be animated- that simply means it will rotate around the plane in the 3D view to show you how the normal map will look from all lighting angles. From here there's not much you need to know. I keep the filter type the same- "4 sample". The scale will 'scale' the depth of the normal map and make it appear deeper or shallower. I normally keep it at 5, but have gone as low as 2 and as high as 10- it really depends on what the contrast of the image is, and the effect you're aiming for. The "Average RGB" should almost always be selected as your height source, as opposed to the alternate conversions. The only other radio button you'll ever select using this tool is the "Normalize Only" button, and you'd run that on an already existing normal map, but I'll get into that later. Otherwise, keep all the other options the same. And that's all there is to it. I'd suggest that if you're running this tool on a terrain texture, make sure your terrain texture is seamless. This goes without saying that you want your texture to be seamlessly looping, but once you apply a normal map to a terrain, if the texture doesn't loop already, it will be even more obvious. Also, a pro-tip: I create a library of diffuse and normal textures that emulate different materials for my own use later down the road. It saves a lot of time.
Normal Maps via Maya Renders: There is another neat trick you can do inside Maya to render out normal maps on the fly and use them as normal map textures or to embellish other normal maps you've made. Start out by opening the Maya hypergraph and tying a sampler info node to a surface shader node. Choose "Other" as the input type so the dialog box comes up. For the sampler info output, choose "normalCamera", and choose "outColor" as the surface shader output. Create a new "Top" orthographical view and lock it's position, rotation, and orthographic width- this effectively keeps the view of the top view static. Now build a plane on the grid and fit it into the top viewport.
Now it's as easy as building and defining geometry that you'd like to appear in a normal map, right above the plane. Assign the surface shader to the plane and all of the components resting on top of it, and render from this new top view at the resolution you want the texture to be. The result is a normal map, but it's not quite what you want yet- the colors are a bit too dark for the corresponding normal values we want. Load the texture into photoshop, and hit CTRL+U - this is the shortcut for the hue/saturation bar inside photoshop. If you crank up the saturation, and bring the lightness up by 50, the colors will be correct, and the normal map will work correctly. So now you have your texture. This can be very useful for making terrain/surface textures (for example, like a brick wall), but I made this specific normal map for a specific purpose- I'll explain later.
Normal Maps via Transfer Maps: Transfer mapping is what people normally think of when talking about normal mapping. It involves taking a high resolution model and transferring its lighting information to the low poly model through projection. Basically this means that at each pixel of the mesh's UV map, a ray is sent upwards from the poly surface until it hits the high poly mesh or an encasing envelope. Once the ray hits the high poly model, the normal of the surface at that specific point of the high resolution model is read, and the difference in the two normals for that pixel is transcribed into a texture based on the UV map. A good UV map is essential to a good transferred normal map. In fact, and this goes without saying, your final product is only as strong as your weakest link in the workflow. Low poly modeling, UV mapping, high poly modeling, texturing, are all very essential to getting the final product looking good. That being said, it doesn't matter how good your high poly model looks if you have a bad UV map or haven't planned for the projection in your low poly model. You can edit the normal map texture after projection- and likely you will have to, there are problems in projection that just can't be helped no matter what you do. But the more accurate you can make your transfer map emulate the high poly onto the low poly from the get-go, the easier it will make your life when it comes to editing. Imagine you took your low poly model and applied a half inch layer of clay over the entire model, and then started carving details into it, and building up small areas here and there. This is a good way to look at normal maps. Normal maps can change the lighting drastically on your model to make it appear as if it has a lot more detail, but only where the camera can see it. Normal maps cannot break the low poly's silhouette- that is why it's imperative that if you're using high poly models with transfer maps to derive your normal map, that your high poly matches the silhouette of your low poly model as closely as possible from every angle. It’s also good practice to do this to make sure the normal map projects correctly. A good edge flow on your low poly model is also imperative. There are a variety of methods you can use to create your high-poly model, based on what the model is. But normally, for any normal map, you are going to start from your low poly model- and this is a model that has the rough shape defined, with the polygons spaced evenly out, roughly. The low poly model should also have good edge flow, especially if it should animate. Making sure your low poly model has all of these will make your life much simpler down the road. Some people UV map the low poly model now, but I wait until I finish the high poly model- I'll explain why later. For organic shapes, it's advisable to use a sculpting program such as Zbrush or Mudbox to sculpt in details, such as muscles, veins, wrinkles, and the like. I'll lightly touch on what you'd do for an organic normal. You will subdivide the mesh likely 2 or 3 times to get the amount of detail you need. Once you finish sculpting in the program, you will have to run the transfer maps- either in Maya, or inside the sculptor you're using. Sometimes running the transfer maps inside the sculpting program is preferred, but the thing is the sculpting programs only run the transfer maps between different levels of the sculpt. This sometimes is satisfactory, but more often than not you'll want to go into Maya and hand-tweak the
verts of the low poly model to better match the high poly model in Maya, and then run the transfer maps from there. This is why it's advisable to always run the transfer inside Maya. In my example, however, I am making a hard-surface model. A gun. For hard-surface assets, I model the high poly right inside Maya. You can use a sculpting program- and some people are very good at doing this- but it takes a lot of finessing to make look right. You'd be doing yourself a favor by modeling the high poly, at least as far as hard surface models are concerned. For the high poly model, I duplicate the layer that has my low poly model in it, and rename this duplicate "HighPoly". I then start breaking the new "high poly" model into patches and extrude the sides of the patches inwards (towards the center of the object, normally), and then edge project the extrusions by adding a few loops in- one near the end of the extrusion, and one halfway down the extrusion. Once you've done this on two adjacent patches you can already see that a good 'solid' edge is sold when you turn on smooth preview by hitting '3'. Keep in mind that if your low-poly mesh consists of multiple pieces that will animate together, you need to 'blow up' the low res model before duplicating it and doing your patch modeling- essentially move the pieces away from their sitting position. Move the pieces before duplicating the low res layer. This is to ensure the normal map transfer will bake in the details from all angles of these moveable parts. If the parts weren't moved, none of the normal map information would transfer over at all where the pieces occlude each other. In this case as you can see, I know that my ammo canister, the barrel, and the energy tanks on the sides will be animating- so I moved them out of the way. Thing with the high poly model is that you're making it solely for the purpose of projecting a normal map onto the low poly model. For this reason, you can get away with a few things. You can completely mangle the topology of the mesh as long as it’s smoothing correctly. You can also make use of floating geometry above your surface geometry, and as long as it’s read as part of the same surface, you do not have to worry about building anything into the geometry. One practice that can be useful for normal map creation is pre-production of the floating assets that you are planning on using over and over again. Things like rivets, vents, and caps would be good candidates for this- generally just anything that your design calls for. I personally keep a library of these things in a single maya file that I reuse over and over again. Every time I need a certain rivet or screw, I will load the project up, select the object and copy it, and paste it into my project. With floating pieces, make sure the part doesn't extrude too far from the surface, and that it is flush to the surface as much as possible. Remember normal maps are projected outwards from the low poly mesh? Your high poly detail should be beveled inwards a bit. If you have any geometry that extrudes straight upwards, it will not be projected onto the low poly model correctly. Keep all of your edges soft. Extrusions outwards should bevel inwards from anywhere between 30 and 45 degrees. Think of it in terms of pixel-space. If your extrusions are wider, they will take up more pixel space- more pixels will be used to sell the surface data. And this should go without saying- high poly edges should NEVER be sharp.
Make sure all your geometry normals are facing outwards. I'm not going to explain how to do this, you guys likely know how. But once I'm done with my high poly patches and I know they are facing the right way, I smooth them all by selecting all of them and preview smoothing, and go Modify > Convert > Mesh Smooth Preview to Polygons. This gives you exactly what you see in the mesh smooth preview, as opposed to going Mesh > Smooth. It's just one more thing I do that saves time and removes guestimating how a model will look. I then combine all of the patches together into one object- right now it's likely very poly heavy.
Readjusting The Low Poly: Now that you have a high poly model made, it's likely you've changed the silhouette of the low poly a bit. Or if you've gone overboard with embellishing the high poly model, you might've changed the silhouette a lot. Either way, you're going to want to adjust your low poly to better "hold" the information of the high poly model, before you UV map. You can do this by making your high poly model "Live" by hitting the magnet button on top of the screen, then pulling points of your low poly model around to better match the high poly. If you've added huge chunks to your high poly model that aren't in the low poly, obviously you're going to want to add that geometry into the low poly model too. (Excluding floating geometry, of course.) Another thing you need to do to the low poly is select the whole thing and soften the normal edges. (Polygons Tab > Normals > Soften Edges). Normally, softening and hardening specific edges on low poly models is great for creating 'lighting seams' and making certain edges appear hard and others appear soft. But these "smoothing groups" (as they are called in 3DSmax) are the devil when it comes to normal mapping- it will take the lighting information a normal map is applying to the surface of a model and skew it. This is because a hard edge is pretty much telling the lighting information not to be calculated over that edge, and makes it "faceted"- so the light calculation
ends at the edge and begins again on the other side of it. Whenever you use a normal map on a model, all of the edges should be soft. You'll notice that when you soften all the edges, the model appears very splotchy- the lighting is being interpolated from edge to edge over the entire model, so it appears very organic. This is not a problem; the normal map will fix this. But there's one thing you need to be wary of, and that's corner edges (or any sharp edges in the silhouette). High poly models almost never project correctly onto a sharp edge on the low poly models. So you have to select all of your sharp edges and bevel them, if only by a little bit- this gives the transfer maps 3 faces in which to project the high poly model onto as opposed to just two meeting at an extreme angle. If you double click a hard edge, normally the entire edge loop will be selected. Try to select all the sharp edges that are touching at the same time before beveling. If you select a double of edges and bevel, and then a couple more and bevel, normally a bunch of crazy geometry will ensue. It's much better to select all the hard edges and bevel in one go. There is a correlation between how creative you are when making your high poly detail and how much you'll need to go back and revise the low poly later. Since I feel pretty comfortable with my low poly flow, I usually go crazy when modeling high poly details, since I am confident that I can quickly get the low poly equivalent made. Some people build the high poly before they build the low poly even, but I find that if you start out with some sort of low poly model first to build the high poly out of and edit the low poly later, it makes things much faster. As you can see, the topology of the low poly model has changed a lot to ensure the normal map projects correctly. This is why we're going to make the UV map now, and not sooner. UV Mapping: I'm going to imagine you all know what a UV map is- it essentially takes a 3D object and "unwraps" its surface information to a 2D plane for the sake of texturing. I also wouldn't be surprised if it was the most revered part of Maya for some people. As I said before a good UV map is essential to a good texture, and therefore a good looking model. Essentially, you want your UVs to run straight up and down and left to right as much as possible- this just makes the texturer's job that much easier since many texturing operations run up and down and left to right. As well, unless specified the UV shells should lie in the 0-1 box region specified and be scaled to fit. You shouldn't worry about the scale of things until the very end of UV mapping though. Good UV maps also make use of overlapping UVs for symmetrical parts and scaling down shells you'll hardly ever see, to save on UV space and allow more pixels to be dedicated to any one shell at one time. The thing is, most people like to approach UVmapping like they approach modeling- on a faceto-face, or edge-to-edge basis. At first this would make sense since we model from face-to-face, performing operations such as extruding or inserting edge loops or moving faces or verts, moving rapidly around a model until we get what we want. But there is a saying a wise person once told me, "Think large to small" when approaching any task. Even when we model, we have the bigger picture of the model in mind- that's why it's advisable that we move from area to area when first blocking out models, and not to focus on one area. Working big to small is no exception with UVmapping.
So to begin, I select the whole object and run a planar projection on it- this puts the object into one shell for simplicity's sake. From here we will start detaching patches of geometry from the main shell, so whenever we select the main shell and change the selection to faces, we can see what we have left to UV map- this can be helpful when you know you have stuff left to UV map but are not sure where specifically the faces are. The next thing I'll show you is paramount to seeing where your shells are. Turn this on: Display > Polygons > Texture Border Edges This will bolden every edge on the model where there is a seam. Since the model is all in one shell there should be none of these edges shown. But now, if we open our UV texture editor (Polygons>Edit UVs>UV Texture Editor), we can see a button with a scissor icon on the left side of the buttons bar. If we were to select an edge loop by double-clicking an edge on our model that we knew wrapped around the model, and hit this button, we'd see the entire loop now boldened. You'd also see we know have two separate shells we can pull around in the UV texture editor. Remember, if a seam wraps all the way around a model in any direction, it detaches the geometry in UV space completely from the rest of the model- this creates two separate shells. The next tool I'll show you is the Smooth UV tool. If you select UVs in the texture editor and go Tools > Smooth UV Tool, a gizmo will pop up in the texture editor with two buttons: "Smooth" and "Unfold". By click-dragging left and right, you can see the model smooth and unfold interactively right before you. This takes so much of the busy work out of UV mapping. One option you may need to set (by selecting the option box) is to turn on "UVspace" under space, instead of "World"- this looks at the distance of UVs relative to one another as opposed to where they are spaced out in the 3D world- and generally, things unfold better with this option selected. You may also at one point or another want to "Pin Borders"- this essentially makes the smooth tool automatically deselect any UVs that lie on the seams, so when you unfold and smooth it only does it from within the seam edges, which can be very handy. You have to remember to turn this off when you need to select seams again. Usually I never have this option selected, but it can be helpful. One problem people have with UV mapping from the beginning is how the UV tools work with different selection types. You cannot run the Smooth UV Tool on anything other than UVs, and you cannot split edges and create seams on anything other than edges. During UV mapping you will find yourself jumping back and forth between faces, edges, shells, and UVs often. So, remember CTRL+RMB. CTRL+RMB inside a Maya viewport or inside the UV texture editor will give you a hotbox where you can convert a selection type into another selection type. To actually move and edit a certain select, you need to RMB and select the given type you want to be working with. This is where a lot of people get hung up on UV mapping, and believe me, once you get around this functionality you'll use this often. For example, if you screw up the mapping of a certain patch and just want to redo it, you can select a face inside the UV texture editor and CTRL+RMB > To Shell, and then go Create UVs > Planar Mapping and voila, you have a new patch, free of any seams.
Another thing I'd like to mention is the "End Command" shortcut, Q. Pressing Q will get you out of any command you're currently running, whether inside the UV Texture Editor or out. Again, very helpful. The way I UV map things is by building seams by selecting edges, cutting them, and then unfolding smoothing out the resulting UV patches. I find it to be the most fast and efficient way to UV map things. Think about it, so many edges where faces meet aren't on the seam and are contained within a shell, and the Smooth UV tool moves these edges for you. Instead of performing countless UV operations on faces and moving and sewing UVs together until you get what you want, doesn't it make more sense to plan out your patches and seams beforehand and then unfolding from there? This is what I mean when I say think big to small- think about where your patches should be, where your seams should be, and then unfold, smooth, and tweak from there. Following my workflow doesn't mean you won't be using the sewing tools and doing other operations, but you will be doing these a lot less. So seams, where should they be? As I said before, seams should wrap around the object where you want two separate patches- an obvious example would be an arm extruding out of a torso. Any part that extrudes out of another part normally is a good candidate for a seam. Seams should also be tucked away from view when they can be. One example you might be familiar with is having a seam run down the length of the arm on the underside of the arm. Also, if you have an object that will normally be viewed from a certain angle, have the seams on the backside of that angle. Seams should also be used inside patches where there are corners, or wherever there are more than 2 faces meeting at a steep angle. Also wherever there are 3 sides meeting together (3-point star), or 5 sides meeting together (a 5-point star or "web"), you may want to use seams. Basically, building seams into your patches stops pinching. When you’re unfolding, and things will unfold to be straight and tidy when pinching is taken care of. There are certain geometry types you'll run into that need their seams to be put in certain places- and recognizing this pattern will save you a boat load of time. The UV operations offered to you under "Create UVs" place these seams for you interactively- tools such as "Spherical Mapping" or "Cylindrical Mapping" but using these tools requires you to translate rotate a mapping gizmo aroundsomething that takes a lot of time if done right. As you'll see, my method is much faster. "Cylinder Mapping" - Use this if you have an object that has 4 or more faces meeting into one (like a cube). The face that all of the other faces meet up to is your "end"- you should isolate this face by cutting seams in all but one of the sides. This makes it a "peninsula". From there, select the edge of one of the faces that run up to it and cut a seam in. That's all there is to it, really. If there is a bottom "end", do the same to the bottom as you did to the top. From where, everything should unfold correctly when using the Smooth UV tool. You want to use Cylinder Mapping when you want the texture that runs along multiple faces to be seamless (such as a cylinder, thus why I call it that). Cylinder Mapping is also good for keeping your faces together tightly- other methods of mapping will take up much more UV space since there will be multiple UV "wings".
Normally UV shells fall under the “cylinder mapping” category, or the “planar mapping category”. If you find a group of faces have no sharp edges or pinching going on, simply running a planar projection to get them into one shell normally does the trick. Run the smooth UV tool afterwards, and if no pinching exists, the shell should just melt apart. If you see some pinching going on, sometimes just splitting at the edges that are pinching solves the problem- just make sure where you are splitting your seam isn’t an area that people will see a lot. Sometimes the Smooth UV Tool misbehaves, even after you've taken care of pinching and geometry problems, the geometry will sometimes fold in on itself. This is just the nature of the beast, and there are many things you can do to get the smooth UV tool to work for you. First, unfold before you smooth- unfolding will often get rid of geometry folding in on itself. Second, flip your UVs- there is a button that will flip your geometry in the horizontal and vertical direction. There is a button on the right side of the UV Texture Editor palette that will color your geometry based on where the UVs are facing- if they are facing the wrong way, they will be red and can cause this unfolding problem. Flipping the geometry by using one of the buttons on the left side can fix it. Yet another way of getting the Smooth UV tool to work is to scale up your selected patch to be very big, especially if you do it extremely in one direction. When you unfold a patch that is skewed like this, often the folding issue works itself out as you are dialing the smoothing. At this point, don't be worried about the scale of things. When the Smooth UV tools runs correctly, the UVs in an edge loop should follow the same line. Sometimes the result isn't exact, and sometimes the shell is rotated in a strange angle. If you select your shell (either by selecting a face and converting "To Shell", or by selecting a single UV and using the Move Shell Tool and ending the command by pressing "Q"), and rotate the UVs so that a loop lies roughly horizontally or vertically, it makes it much easier to get your UVs lined up. From here you can marquee select the UVs you want to line up and either scale them, or use one of the align tools offered in the middle of the buttons palette in the UV texture editor. There might be a thing or two I missed here, but everything I've explained should be enough for you to make awesome UV maps with a fraction of the effort it takes otherwise. Some people prefer using the Automatic Mapping Tool and then sewing the patches together- but there is no deliberateness here. Once you've Automatic Mapped stuff, you have no idea where one part ends and another begins inside the UV texture editor, and you have to jump between the viewport and the Texture Editor countless times to see where things shells meet and where you need to merge them. At the end of merging seams might not be where you want them to be, so then you have to go in and split off faces and remerge them into other shells. It's not like I'm saying this is wrong by any extent- there is no wrong way to do things inside Maya. But I personally believe my way is more efficient and faster. But yes, the last step the UV map creation is to layout the shells into the 0-1 area. Inside the Texture Editor, go Polygons > Layout with Options. For the Layout Options, select Single or Multiple Objects, and set Prescale to World. Set the rotate to 90 degrees, and select the resolution map you’re planning on using for spacing presets. Once you do this, all of the shells should be scaled right, and put into the 0-1 area. You're not done yet though. You should scale up certain pieces that will be seen by the
eye more, and scale down pieces that aren't seen as much. And do some rearranging- by default Maya hardly ever spaces things out the most efficient way, unless you have less than 6 or so patches. The last thing you want to do is render out a UV snapshot, the same dimensions as the texture you're going to make. Later on inside Photoshop, we will place the UV map as the top layer of our normal and diffuse textures, with a blending mode of "Difference". With white lines on a black background, this will make everything under the white lines flip its color information, and the black will be transparent. This is the best technique I've found for using UV snapshots as guides for texturing. This concludes my lengthy but brief overview of UV mapping. It's not rocket science, but it takes some getting used to. The Transfer Map: Now the moment we've all been waiting for- the transfer! You can find the transfer map tool in the Rendering Tab in Maya. Go Lighting/Shading > Transfer Maps... and a dialog box should appear. If you had your low poly mesh selected before opening this tool, it should appear in the Target Meshes tab in the dialog box. If not, select it and click "Add Selected". You'll see a few things listed in the Target Meshes tab. One is a dropdown menu for the "Output UV set". This is a good time to bring up UV overlapping. Having shells overlap is a great way to save on pixel space, but if you try to render out transfer maps with the shells overlapping, the normal data will be multiplied and you'll get a narly result. What you need to do to correct this is copy all of your UVs into a new UV set and move one of the overlapping pieces off the 0-1 area. This UV set you'll use for baking the transfer maps, and then you'll use the normal UV set to get the flipped piece back. Open your UV Texture Editor and go Polygons > Copy UVs to UV Set > Copy into New UV Set. Name the UV set something specific that works for you- I simply call mine "UVflipped". Once you’re in this new UV set, move any overlapping shells off the grid. Now, go back inside your Transfer Maps dialogue box and select the new UV set as the one for transfer. So, moving on... Now let’s move to the Source Meshes tab. Select your high poly mesh and click "Add Selected". Make sure your low poly and high poly mesh lie directly on top of one another. Now moving back to the target mesh tab, let's cover the display and envelope options. Display simply allows you to select what you're going to preview in the viewport for the sake of fine-tuning the baking envelope. You can select whether to just see the mesh, the mesh and envelope, and envelope. The envelope size is displayed next to this menu. I usually keep this value around 0.2, and no bigger than 0.5. The envelope size determines how far the rays are going to be cast away from the low poly object. If your high poly model is very snuggly fit to your low poly model, this value shouldn't need to be that big. And having a large envelope will cause weird artifacts in the final normal map, so pretty much the lower this value the better. Output Maps- you should know transfer maps don't only have to transfer out normal information, but this case that is what we want. Click on the normal sphere, and this tab should be
expanded with normal information. Select a folder and file name for your normal map. As for the file format, I'd personally suggest using a .png or .tga map- there are the most accessible by other programs. Psd files cannot be rendered in mental ray without the program hiccupping. And never save out as a .jpg- seriously. I do not understand why this option is given, and why so many people use it, including teachers. JPEGs are not lossless, and they were made for the web, that's where they should stay. But anyway, that aside. Include materials can be checked or unchecked, it doesn't really matter much. The map space should be "Tangent Space"- and you should use Maya Common Settings. Connect your output map to a new shader if you’re making a new normal map- if your normal map ends up being really screwed up and you want to rerun the command again, you should use "assigned shader" to keep the normal map in the same layer. Inside your Maya Common Output tab, specify your map width and height. 512 works fine, 1024 is ideal, lately I've been boosting mine to 2048 just to get that much more detail. Keep in mind the resolution you choose here will likely also be the size of your diffuse and specular maps if you decide to make them. Transfer the map in World Space. If your just previewing how the map should look like, the sampling quality does not need to be high, but for the final production purposes, you should set this to high. The filter size is usually fine at 2, sometimes it needs to be boosted up- the filter type I keep at gaussian. Now, the fill texture seams- this pretty much gives you a pixel buffer at the border of every shell. If you don't set this high enough, black borders around every shell will be visible when you're looking at the model from a distance. At smaller resolutions this doesn't need to be so big, but at 1024x1024 I set this to 10, and at 2048x2048 I set it to 15, and I've never had a problem with this values. If your shells are very close together, you might need to decrease this to avoid overlap. Inside the advanced options you are given a search method and a matching option. I've gotten different results using different things, but normally I set the search method to be Closest to Envelope, or Inside Envelope then Outside- never just inside. This is because normally certain parts of the high poly mesh always fall outside the envelope and should still be picked up. The matching options I almost always keep at Surface Normals, and not Geometry Normals- I tend to get better results this way. Then you let the transfer map run- this could take a while at the production level. Don't expect a clean normal map from the get-go. Most people never get a clean normal map at the beginning. There are always artifacts that need to be cleaned up. Sometimes different settings will make certain parts of the model render out fine, and other parts mess up royally. That's why it's advisable to save multiple maps and splice them together in photoshop afterwards. Once the transfer map runs, if you turn on High Quality Rendering inside the viewport you should be able to see the normal map on the model. Normal maps render out fine in Maya Hardware (what the viewports use) and in Mental Ray, but Maya Software doesn't show normal maps correctly by default, so don't use it to preview your normal maps. An error that happens often is that certain parts will appear black- and this is a tell-tale sign of either your low poly or high poly models having their normals flipped the wrong way. If you turn on Default Quality Rendering in the viewport, and then turn off Two Sided Lighting in the same viewport,
any area that appears black has its normals flipped- so go ahead, select those faces and reverse the normals, and run the transfer again. Fixing and Polishing Normal Maps: Undoubtedly you're going to have issues with your normal map at the end, even if you get it to a point of near perfection from transfer mapping. Since I've been doing this for a while, I don't have so many flaws, but they do exist. Since each color channel does something different to the normals, it can be challenging to simply paint problems out. But there are a few techniques you can use to fix the normal maps. Whatever you do to the normal map, make sure you are performing the techniques on new layers, so that they can always be removed later if you did something wrong. Make a new layer by hitting CTRL+SHIFT+N. The healing and the smudging tool are the tools I normally use for fixing up normal maps. To get the tools to work on the new layer, while pulling the color data from layers below it, make sure "Sample All Layers" is selected in the tool options on the top of the screen. For the healing brush, make sure "Current and Below" is selected for the sampling. With the healing brush tool selected if you hold ALT and click on a location on the texture, it will choose that location as the place where you're healing will come from. Then if you click anywhere else on the texture, the healing tool will try it's best to blend the color information from your source area into your target area. This tool is used commonly for removing birthmarks in photos of people, but in this case we're using it to fix hiccups in the surface and on the sides of the UV shells. It's best used for small areas. The same goes for the smudge tool. The smudge tool is best used in many short strokes (but a warning: this will fill up your tool history fast). The blur tool is also handy for taking care of parts where weird aliasing occurs from the transfer- simply brushing up and down with the blur tool at a low opacity can get rid of the glitchy values pretty fast. Remember how I said earlier that a color with a 128 red and green channel and 255 blue channel reads as no normal change? If you paint this color with a very low opacity onto parts of the normal map that have glitches, and then smooth out the colors around that color, you can smooth out the glitches pretty fast. Where should you use these tools? Wherever you see a noticeable glitch in the model when you preview it with high quality rendering in the viewport. It's good to build a mental memory of where patches on the UV map relate to parts of the model, make sure to do this as you are UV mapping. A tell-tale sign of a normal glitch is where the colors on a normal map seem to change drastically for no reason at all. It takes a little bit of training your eye to point these out, but they are pretty noticeable. As I said before, my normal map didn't really have too many problems- but I did notice the scope needs a little bit of loving. After using the techniques I just explained, I got the following results. It
still needs a bit TLC, but it's a lot cleaner than what it used to be.
Another thing I do to my normal maps is overlay textures onto it to embellish and polish it. Remember the normal map I made in Maya earlier? There it is, right on the gun. I did this by loading up the maya render I made earlier and placing it into the normal map with a blending style of "overlay". I removed some of the background I didn't need, and translated and rotated it into place. CTRL+T turns on free transform, which gives you a gizmo for rotating and gizmo which is pretty easy to use. Holding shift while rotating will snap the rotate to useable increments. You can also overlay textures you've made with the nVidia normal map tool, too! Remember how I kept a library of different textures? Normally as I'm modeling, I'm also thinking about what materials different things are going to be, and thinking what I can use in my library to create that material. I have a rubberized leather texture I can use for the grips, for example. I simply drag my texture onto the normal map, make a selection of where the grip is going to be, invert the selection (CTRL+SHIFT+I) and hit delete- now the texture is only where it needs to be. I flip this texture on overlay too. Using these maya renders and normal map textures can help me get more high frequency detail in a fraction of the time it would have taken to model it all into the mesh.
One thing you need to do if you're overlaying things is run a normalize function of the normal map. Normalizing means bringing the normal values the red, green, and blue values translate to back to a value of 1- effectively dampening overdriven values. This is because once you've overlayed so many normal maps on top of one another, the vector information that is saved ends up being too powerful or
conflicting, giving you strange results. You fix this at the end of normal map creation- once you're confident with the normal map you've made, you flatten the image and run nVidia's normal map filter again. This time, you choose the radio option "Normalize Only"- this will simply normalize the map. The scale and sample size doesn't matter here; the only function running is the normalization of the map. And just FYI, Crazybump renormalizes a normal map every time you load one in. So if you load up a normal map, then simply save it, it will be renormalized. Interesting trick for the CrazyBump fans. If you’re making this mesh for the Unreal Engine or any other engine that uses Z as its vertical axis as opposed to Y, you need to invert the green channel's values, the red channel's values, or sometimes both. You do this by selecting the channel and hitting CTRL+I. This will make the normal map appear correctly in those engines. Texturing: I was going to get into texturing the diffuse but I've run out of time. But I can give you a quick rundown of what I do when I texture. First I start off with a baked out ambient occlusion layer. I then create a group of color layers, each color getting its own layer. I select portions of the mesh I want to be a certain color and simply fill in the selections. I set this whole layer group to multiply. I think load up different grayscale diffuse textures of different materials I have in my library and make a group of "Materials" using them. I use the same color selections to mask out the materials- normally each color gets one corresponding material. I set this group to "overlay". I think embellish the diffuse with dust, scratches, paint, rivets, logos, general wear and tear. And of course, the UV map on top set to "Difference", for reference only. It gets hidden during the final save of the texture. Don't ever think as a texture artist that you can get away with not using layers. Photoshop's history only holds so much, and the second you bury a mistake in history, or close the program, you have to deal with that mistake which can be painstaking. Sometimes you'll texture things a certain way and your client will want the gun in cornflower blue instead of red, and all the neat tools in Photoshop will not be able to save you then. I really shouldn't have to argue this, just keep it in mindalways use layers!
One thing I'll mention about ambient occlusion is that you can perform an AO transfer from high poly to low poly inside Maya- but it often doesn't work. I'll share with you a neat trick though. If you bake out the low poly's ambient occlusion instead, and then place the blue channel of your normal map onto it with a blending mode of "Multiply" you'll get something similar to what the transfer map would have given you. This is because the blue channel of a normal map, if you recall, deals with the depth information. Thus concludes my rather lengthy foray into explaining my workflow. It seems all simple to me, but I know its not- there is a lot to it. But I know after a while of practice and training, this stuff all becomes second nature. I also know that one day this information will become more second-hand knowledge and/or easier ways will be invented to do the same things I’ve explained. But now you have in your grasp the one thing I feel I know the most about in the academic world of 3D- so go out there, try it, and enjoy your work!
Sources: "Resources - Tutorials." Ben Cloward - Character Animator. Web. 08 Apr. 2011. . "Tutorial - Tips & Tricks for Normal Mapping." GameArtisans. Web. 08 Apr. 2011. . "Ben Mathis - Tutorials." Ben Mathis - Freelance Artist. Web. 08 Apr. 2011. .
