Filtering, MIP Maps, Alpha Blending, and Animation
Introducing textures into a graphics pipeline also comes with several new problems. However, several new possibilities come with textures as well!
Nearest Neighbor & Bilinear Filtering
One of the main problems textures come with is that they are an image with a certain resolution. This is fine if we are drawing the texture on a window that's the exact same resolution, but in real-time simulations that's rarely the case. If we move up really close to a texture, we could run into the problem of needing more texels than the base texture can provide.
One solution to this problem is sampling the texel that is closest to the fragment we're drawing. This solution is called nearest neighbor, but results in a very pixelated look. This is because it can be thought of as expanding the size of the texels. Below is an example of what this looks like. Notice how it's very clear the bricks are made out of individual texels.
Another solution is having every fragment take an average color of the texels around it. This is called bilinear filtering and gives a blended look.
The resolution of a texture can cause problems in a similar way by providing too many texels! If we need to render a texture that has more texels than we have pixels to draw them, our texture will appear incorrect.
The main solution to this problem is MIP Maps! MIP Maps are smaller resolutions of our base texture that we generate at build time. When we draw our base texture at a smaller resolution than it, we can swap it out with a generated MIP Map!
To get a sense of this, lets first look at the brick texture with MIP Maps enabled. Below is a brick texture up close that looks relatively normal.
Here's the same brick texture from far away. Nothing too surprising here as it might be what you expect the bricks to look like from far away.
Now that we've seen what we expect textures to look like in a real-time simulation, lets look at what they look like with MIP Maps turned off!
Notice how the same texture at the same distance looks much more pixelated and a little inaccurate.
With alpha transparency accounted for in draw call order, we can use the alpha channel in textures to get really detailed meshes! Notice in the image below how a simple quad can almost appear like a window.
If we zoomed in close to our transparent quad, we can see the detail of our alpha blended texture. However, the texture's alpha channel is also a powerful tool to convert our dependent transparent draw call into an independent one!
This is done by simply discarding fragments that have an alpha value below a certain threshold. This would allow us to achieve binary transparency without needing to treat the call as a dependent one.
Finally, one of the most visually impactful features of textures is the ability to create animations. Since textures are drawn by providing shaders coordinates for sampling textures, we can edit those coordinates over time to make it appear the texture is dynamic.
Below, in the vertex shader I simply add the simulation's elapsed time to the texture's coordinate's u-value. If the texture is set to Tile, the texture will repeat as we sample a coordinate outside of the base textures horizontal resolution. Thus, this technique requires a texture that tiles. The same calculation could happen in the fragment shader, but it would be more expensive as there are usually more fragments than vertices.
Below is another example of scrolling through our texture, but in two directions at once! However, instead of just adding the simulation's elapsed time to the texture coordinates, I added the sine and cosine of elapsed time. This creates an oscillation effect, where it almost seems the mesh is acting as a window into a larger world full of water.