How can I pack repeated textures into one texture efficiently?

Question

I have an obj mesh with some textures, several of them are repeated(uvs are not within [0,1]). I'd like to merge all those textures into one texture and transform the uvs of obj.

Answer 1

Since the question is about "efficiently" packing a mix of repeated and non-repeated textures into a texture atlas, the answer is: most likely no, you can't. As DMGregory said Texture Arrays are probably the way to go instead.

Why:

If you want to go the atlas way you will have to slice triangles on UV repeat edges (edges in texture space) but this may significantly increase your mesh complexity, increasing geometry workload.

And this has repercussions for linear interpolation, including mip-maps.

This means you'll need to add a repeated border at least 2^(n+1) pixels thick where n is the number of mip-map levels you want.

Unless you're making a voxel engine or GUI engine that would otherwise cause massive amount of texture switches there isn't a gain to be had. As long as the GPU can handle the number of textures being sampled in a single object there isn't a real benefit to creating an atlas.

At the shader level sampling 4 different texture or sampling a single texture atlas 4 times makes no difference to a modern GPU of at least the last 10 years.

The only saving might be the removal of 3 tiny "Set Texture Sampler {X} to Texture {Y}" commands from the GPU command stream.

If you gain any benefit at all from this you will likely gain a higher benefit by making sure all your objects using the same material are rendered at once without switching materials / GPU render states. Making the atlas packing gains insignificant.

Large texture atlases also prevent some GPUs from efficiently managing GPU RAM as some need to transfer entire textures to GPU RAM before running the shader.

So while automatically-generated texture atlases make sense for GUI text or 2D tile / 3D voxel massively-tile-based renderer, it does not make so much sense for characters.

The usual way to go is to bake (combine) the complex textures layers into simpler unwrapped textures using a capable 3D modeling/painting software.