I solved this problem. The issue is actually related to variable declaration. So I moved all the variable declarations in to the constructor and that fixed the problem. I don't know whether that is a common practice or not anyway.
Not only how tall and big walls should be, how thick is also very important as I started seeing this:
[Physics.PhysX] cleaning the mesh failed
Wall model is the following:
a wall is made of points, at least two, in the end only p0 is needed for a wall segment
p1 is next segment, p2 and p3 are simply p0 + Vector3.up
How were thick walls in gray built in a ...
There really is just one realistic option, and that's a Mesh Collider. Make sure you mark the object as "Static" and enable mesh cooking on your mesh collider in order to improve performance. The alternative would be to manually create a separate Box Collider for every single wall in the level, which would be a very tedious and error-prone task.
Like DMGregory said: it's very unlikely Total War is actually simulating physics for all those arrows. There's probably a big separation from the battle simulation and the actual animations in game.
Here's a GDC talk about "predictable projectiles" which could be used for the animation side of things.