I've been using Godot for my game, but it has a unfortunate quirks with tilemap collision where physics objects can bounce weirdly near tile seams and kinematic bodies can often get stuck in seams as well. For early alpha work, I've been able to get around this by not using tilemap collision and then tracing the contours by hand... Well this isn't going to scale well to 200 rooms (or whatever I end up having) and could easily lead to mismatched collision data, so I've decided that this process needs to be automated.
Tilemaps have a few different tile types - fully solid, diagonal slopes, and 2:1 slopes.
My first attempt matched each of these types to a half-tile size, marking each vertex as solid or not solid, and then combining all the vertices (with their overlap) into a single grid of vertices. I could then use all of these together by marching along the perimeter and probing ahead to detect corners. Turns out this works well for convex corners but has serious issues in concave corners, plus there isn't enough data to detect slopes properly, so it often resulted in extra slopey parts or cutting into slopes just slightly.
From there I decided to add some extra information on the vertex grid, such as whether a vertex is a slope edge and doing clever things surrounding the properties of how it overlaps, but it's quickly blowing up into a lot of specific cases.
I spent much of today thinking about adding vertex normals to each tile definition, but I realized it wasn't going to work without adding a bunch of different tile solidity types.
Are there better approaches than what I'm thinking that don't involve brittle/lossy intermediate representations or ridiculous amounts of special cases?