3
\$\begingroup\$

Since Unity 2021.2.0b7 (that is, URP 12.0.0), there has been a well known problem with absurd number of shader variants wanting to be compiled for even a somewhat simple shader (i.e. shaders generated by Shader Graph, e.g. basic Lit Shader Graph, or even a copy of basic URP Lit shader), due to the amount of new keywords processed with #pragma multi_compile - with exact numbers being in the range of tens of thousands to millions shader variants. While the related Unity dev team tried and still tries to fix this issue, there is no official solution available so far, and the problem still affects all the users of URP >= 12, including (but not limited to) 2021.3.x LTS, 2022.1.x stable branch and all current alphas/betas, up to at least the semi-official 2023.1.0a4 (last one I tested).

Since the actual compile times for such amounts of shader variants go into hours-days-months (depending on actual shader complexity and build power available) which is often prohibitive to normal development (i.e. for me to test my shader code), what can I do now to reduce the amount of variants (and thus build time) to a reasonable value until the fix is available (which may be e.g. somewhere in 2023 in some alpha build etc.), and to possibly circumvent this problem in other similar cases?

\$\endgroup\$
0

1 Answer 1

5
\$\begingroup\$

Note: Variant Prefiltering landed in 2023.1.0a14 and is being backported to other LTS versions. It's not perfect yet (a lot of variants are processed anyway), but you will notice a reduction of time of about one or two orders of magnitude after having it.

Currently, the issue with too many variants and abysmal build times can happen because of two reasons:

  1. you actually have too many shader variants needed for your scene, due to e.g. different quality settings spanning through too many options (quite unlikely, and mostly unsolvable in a way other than optimizing those options),
  2. you're unlucky enough to use a stock Unity shader code, either by using Shader Graph or #including / copy-pasting some of their shader code.

While the amount of permutations for a decent range of quality settings etc. can be indeed high (note that this is usually a multiplicative process), this particular problem usually happens because Unity Editor doesn't recognize the keywords actually (un)used in your scene and doesn't strip them properly from the build. An example offending code from Complex Lit looks like this:

            #pragma multi_compile _ _MAIN_LIGHT_SHADOWS _MAIN_LIGHT_SHADOWS_CASCADE _MAIN_LIGHT_SHADOWS_SCREEN
            #pragma multi_compile _ _ADDITIONAL_LIGHTS_VERTEX _ADDITIONAL_LIGHTS
            #pragma multi_compile_fragment _ _ADDITIONAL_LIGHT_SHADOWS
            #pragma multi_compile_fragment _ _REFLECTION_PROBE_BLENDING
            #pragma multi_compile_fragment _ _REFLECTION_PROBE_BOX_PROJECTION
            #pragma multi_compile_fragment _ _SHADOWS_SOFT
            #pragma multi_compile _ _MIXED_LIGHTING_SUBTRACTIVE
            #pragma multi_compile_fragment _ _SCREEN_SPACE_OCCLUSION
            #pragma multi_compile_fragment _ _DBUFFER_MRT1 _DBUFFER_MRT2 _DBUFFER_MRT3
            #pragma multi_compile_fragment _ _LIGHT_LAYERS
            #pragma multi_compile_fragment _ _LIGHT_COOKIES
            #pragma multi_compile _ _FORWARD_PLUS
            #pragma multi_compile_fragment _ _WRITE_RENDERING_LAYERS

As one can easily calculate, that by itself gives a multiplier of 2^10 * 3 * 4 * 4, i.e. 49152 (roughly 50k), to the amount of needed variants, if the keywords aren't stripped before the build. For a reason (yet) unknown to me, the actual stripping doesn't happen for them (regardless of what the official post of Unity shader team said here), even if the keyword is indeed always on or always off in the scene/URP settings.

The most efficient solution would be to either use a plugin/asset that allows to handle this (or even write a stripper plugin oneself) automatically (e.g. via IPreprocessShaders or similar). Still, if you're OK with fixing the problem by hand, the actual solution is quite simple:

  1. determine the actual keywords applicable to your shader in your scene, and which of them are fixed, and which ain't (e.g., if you don't have SSAO enabled and don't want to enable it - or if you have it enabled, and want to always have it like that),
  2. try to reduce the amount of the actual multi-keywords (variants) to a reasonable number, taking into account their multiplicative nature (i.e. a single keyword and on/off multi gives *2 variants and build time), and bear in mind that you'll have to have about 10-100 variants anyway, even with the aforementioned keyword completely fixed, depending on your scene & build, and that you can have troubles compiling more than 100k variants regardless of the build machine specs,
  3. remove/comment out the multi_compile lines with keywords that you don't want to have enabled at all; remove/comment out the variants you don't want/need and/or replace the #pragma multi_compile (or multi_compile_fragment etc.) with #pragma shader_feature etc. (see the docs for more info)
  4. ???
  5. profit.

In my case e.g., it would look like this after the edits:

            // -------------------------------------
            // Universal Pipeline keywords
            #pragma multi_compile _ _MAIN_LIGHT_SHADOWS _MAIN_LIGHT_SHADOWS_CASCADE /* _MAIN_LIGHT_SHADOWS_SCREEN */
            #pragma multi_compile /* _ _ADDITIONAL_LIGHTS_VERTEX */ _ADDITIONAL_LIGHTS
            #pragma multi_compile_fragment /* _ */ _ADDITIONAL_LIGHT_SHADOWS
            // #pragma multi_compile_fragment _ _REFLECTION_PROBE_BLENDING
            // #pragma multi_compile_fragment _ _REFLECTION_PROBE_BOX_PROJECTION
            #pragma multi_compile_fragment /* _ */ _SHADOWS_SOFT
            // #pragma multi_compile _ _MIXED_LIGHTING_SUBTRACTIVE
            #pragma multi_compile_fragment /* _ */ _SCREEN_SPACE_OCCLUSION
            // #pragma multi_compile_fragment _ _DBUFFER_MRT1 _DBUFFER_MRT2 _DBUFFER_MRT3
            // #pragma multi_compile_fragment _ _LIGHT_LAYERS
            // #pragma multi_compile_fragment _ _LIGHT_COOKIES
            // #pragma multi_compile _ _FORWARD_PLUS
            // #pragma multi_compile_fragment _ _WRITE_RENDERING_LAYERS

(note that in case of the stock Lit/Complex Lit shader, you'll have to do this in two places, once for ShaderModel 2.0 block and once for ShaderModel 4.5 block)

which gave me 24 variants for both Vulkan and OpenGL Core instead of 196608 variants for Vulkan and 24576 variants for OpenGL Core without the manual stripping, for a Complex Lit-based shader.


If you're unlucky enough to be using Shader Graph, it's just a bit more tricky: you'll have to, after each SG change, manually export your SG to a "normal" shader using "View Generated Shader", and then do the above steps manually. It's theoretically possible to "force" a SG shader to be generated in a different way, via injecting custom processing of pass.keywords (stored in PassDescriptor pass) when GenerateShaderPass method is called from Generator() c-tor in UnityEditor.ShaderGraph.Generator.cs. They are sourced from KeywordCollection Forward etc. in e.g. UniversalLitSubTarget.cs. Hard to say how worthwhile it is to go this way (note that SG package is tightly coupled with URP package which is tightly coupled with Unity version by itself...) vs just manual (or semi-automatic) preprocessing of shaders instead.

(also, note that even Unity devs considered current multi_compile / shader_feature #pragma system broken and problematic a couple of years before this issue finally hit us big, although for slightly-yet-not-completely-different reasons)

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .