Making My Game's Textures Can't Be That Hard, Right? Right??

21 Sep, 2024

A Stubborn Programmer's Overcomplicated Art Pipeline

I want to brag about an art pipeline I made. Not only because I'm proud of it, and that's rad, but also because I think I solved some interesting problems with solutions worth sharing. I'll be covering the process I went through to create this pipeline, making sure to highlight any optimizations I made to save myself time.

If you're the same flavor of nerd as I am, you'll probably find this fun. And even if you're not, keep reading! You might be surprised by the the ordeal I went through to make these textures. Buckle up. It's a slow burn.

But first, some context

This game is a solo project. Thankfully, that lowers the bar for pipeline development. The tools I make don't have to be streamlined so that anyone can use it. Just streamlined enough so that me-in-four-months can.

Also, while I aspire to be good at art, I recognize that my strengths lie in programming. One of the key pillars of this project is simply to finish it; leaning into my skills aligns with that goal. However! I also refuse to make something that doesn't look good.

This game's geometry is extremely simple (flat/sharp) due to the level editing process. To compensate, I wanted the materials to be visually complex. My proposed solution was to focus on creating PBR materials. The game's aesthetic pulls heavily from ancient Roman architecture. Lots of stone, brick, and tile. PBR materials can provide a ton of that visual complexity as long as the input textures are sufficiently detailed. Plus there are a lot of free PBR resources available online.

So what do I even want?

Here are the requirements:

• Input textures should be simple patterns. I'm no artist, but I can make a pattern. The pipeline should handle the rest.
• They should also be material agnostic. The same input texture should be able to create textures for different materials/colors with little additional work. i.e. one input texture can be used to make concrete or marble variants easily.
• Output textures should be tileable and/or seamless. This is a requirement for use in the level creation process.
• The output should consist of several PBR textures. Albedo, Normal, and Roughness textures are required. Optionally include Displacement textures.

Perfect. I know what should go in and come out. Now, I need to actually make it.

The first attempt

I started by reading up on the feasibility of procedural materials in Blender. In a professional environment, this would normally be done in Adobe's Substance Designer but I'm not exactly working with an Adobe-sized budget and I really don't want to learn a new piece of software (also Adobe is icky). I have nearly ten years of being awful at Blender under my belt, so let's go with that.

Fortunately for me, all I need to do is make a quad, assign a material, and then work within the visual shader editor. I have a good enough grasp on how shaders work; enough to be able get my ideas into reality without too much struggle.

I don't have a screenshot of the original node graph but, let me tell you, it was bad. It was complete spaghetti, hard to debug, and extremely inefficient. But it did work! For my concrete material, I could create a combined input texture with a grout mask in the red channel and a height mask in the green channel. I setup a Blender shader that took those masks and computed the PBR values. Using the SimpleBake Blender plugin, I could easily export the material as separate textures.

Functionally, it worked! But it was awful to work with. There was a lot wrong with my Blender flow. Let's run through the major problems and my solutions.

Blender Improvements

Problem: I could only export one material at a time. Exporting a different material required manually loading new mask textures into the shader.
Solution: SimpleBake supports batch exporting! I set up multiple quads with separate materials in the same Blender file.

SimpleBake lets me press a single button to export each material's PBR textures into destination folders with automatic naming. If I want to create a new material variant, I just add another quad, setup its shader, and add it to SimpleBake.

Problem: Performance. I was heavily relying on procedural noise; this slowed Blender down to a crawl.
Solution: Just use pre-baked noise textures. They didn't need to be super high resolution or infinitely unique. I also used freely available PBR textures as a base for my materials instead of generating my own concrete from scratch. Swapping over to these was a night-and-day difference for performance and there was no drop in the material's visual quality.

Problem: Spaghetti Code. My shader nodes were so tangled that it was arduous to make any changes.
Solution: Learn a bit more Blender. I rebuilt the entire shader from scratch, this time focusing on organization. Blender has two features called Reroutes and Frames. Frames act as big comment boxes, allowing you to organize nodes under a header. Reroutes allow node connections to have extra pivots. This greatly helps keep lines straight and nodes organized.

Problem: Each material needed its own copy of the shader. I did not want to have to duplicate changes across each one.
Solution: Learn more Blender again. Blender shaders have a feature called Groups. They essentially behave like a function in programming. You can define input parameters, and output return values; just put a bunch of code inside and re-use it wherever you want. This meant that all my materials now look like this:

Pretty simple now! Just import the specific mask texture and send it into the group. The group does all the concrete-ifying and outputs directly to Blender's built-in shader nodes. It also exposes some other parameters, like the concrete's tint. (I want the wall and ceiling materials to be slightly different colors). I also use groups to handle global properties too. Changing the color in the Wall Properties group will output the same color in all the individual material variants that use it. Now, no matter what, I only have to change something in one place and it'll propagate to all the materials.

Great! Much better

With all these improvements, my Blender experience is snappier, easier, and more organized. I can create new material variants, adjust existing ones, and globally modify the shader without any significant friction. Most importantly, these improvements stopped material creation from being a hassle.

Demotivators are super dangerous in solo development. For me, if I need to do something but it's going to be annoying, I'm very likely to just not work on it at all. Getting this mission-critical process out of that danger zone was super important for the project's health.

Alright, so this process works great for simple materials. It's not hard to make marble once I've figured out concrete or stone. However, one material proved too complex to handle with just masks and shaders.

Mosaic-n' me crazy

The Romans just had to love their mosaics. I was hoping to avoid this issue by sticking to marble or tile floors, but. every. single. reference photo I found just had to have mosaics in it.

Technically speaking, mosaic tiles are pretty straight forward. They're a bunch of smooth flat pieces held in place with some grout. Honestly, that's no different from the bricks I'm already making. Just swap out the concrete for some ceramic and we're done, right?

Wrong.

For other materials, I was drawing the grout by hand. This gave me the freedom to make interesting brick designs. It was faster to lay out those lines individually than make them procedurally. For mosaics, it was the opposite. There was absolutely no way I was going to draw out each grout line for a mosaic texture.

Some cursory googling resulted in a few really hideous materials that relied on basic procedural grid generators. These materials looked like sliced soap in those satisfying™-style TikToks; Roman mosaics have an organic curvature to the tile placement that these couldn't replicate. But I felt like this had to be a solved problem. It was too complex for me to think of a DIY solution, but that doesn't mean others haven't. My hunch was that I'd need to create some sort of image processing step that would take my input mask texture and add mosaic grout lines to it. This felt machine learning adjacent (I knew I didn't want neural style transfer, but I wanted something similar to it), so I assumed Python would end up being my tool of choice.

A search for "python mosaic generation" later and… wow. This is a lot more complicated than I thought. I know this because I found who someone did all the hard work already and uploaded it to github all under the MIT license. Truly a godsend.

Let's give it a shot.

My machine's Python install was totally borked and the git repo was out of date. It took an evening or two (fortunately I found another incredible human who had published their own fork of the repo with updates) but eventually, I got it up and running. A simple mask texture goes in, mosaic grout version comes out. Super cool

This worked great! I was still allowed to create simple input textures, and the output exceeded my bar of quality. The grout (mostly) manages to replicate the organic tile placement of real Roman mosaics. And at the very least, it's much better than the alternatives I found online.

But! Just like my first attempt in Blender, this tool wasn't quite ready for use. You know what that means…

Grout Generator Improvements

Problem: The grout texture wasn't seamless. The tiles would generate up to the edge of the texture, leaving ugly seams when the material had to repeat.
Solution: We're in Python land. If you have a problem, there's probably a package to fix it for you. In this case, I added another step to the Python script which used img2texture to blend the outer edges and make it seamless. However, this did not play well with the grout. Blending the outer 1% was just enough to strike a balance between seamlessness and grout integrity.

Problem: The grout texture was very low resolution. Computing the grout lines is not an easy task. Keeping it low resolution keeps the python script from chugging. However, Blender needs more pixels to properly create the PBR textures.
Solution: Same as above. No need to reinvent the wheel. I added another step to the script after the seamless-ifying to upscale the grout texture by 2x using the super-image package.

Problem: The script had to be run from PowerShell. This was clunky and required weird configuration steps to correctly point to the input file. Defining the output directory was equally obtuse.
Solution: I made a few more tweaks to the code to enable drag and drop execution. This allows me to drop an image file onto the script in the file explorer. It will automatically grab the file, convert it, and output a grout version next to where the input file originated. This is a huge time saver.

The Big Picture

The final flow ends up working like this:

1. Create a simple mask texture
2. Drag the mask texture onto the grout generator script in the file explorer. This will create a grout-ified version
3. In Blender, create a new quad and material, referencing the mask texture. Add the quad to SimpleBake and export all the PBR textures
4. Import the PBR textures into the game engine and assign them to a material

Wrapping up

That was quite the adventure, wasn't it? All that work just to make some concrete.

Honestly, this sort of stuff is why I love game development. Even seemingly simple tasks can contain complex problems with a myriad of possible solutions.

If you asked me if I liked pipeline development, I'd probably say no. But I'd be lying. There's absolutely a part of creating a thing that makes making easier that really lights up my brain. Video game development is like trying to chop down a tree with a thousand papercuts. A well made tool might make a few of those cuts slightly sharper.

If you're a non-dev reading this and are still here for the end, I applaud your interest in learning how the sausage gets made. I hope this post conveyed to you how much love goes into every single pixel you see in a game. If you're a dev reading this, I hope you've found something you want to try (or avoid) for a future project. At the very least, I hope this post was even a fraction as interesting as this pipeline was to make.

Thanks for reading,

-Saul

#gamedev