My question is a bit involved so I'm going to split it into parts and attach a picture.

1) In Unity I'd like to be able to draw a line on a 3D worldspace mesh with the mouse.

2) As I'm drawing on the 3D worldspace mesh, I need to have a 2D UI Image component that represents the same 3D worldspace mesh surface. The UI Image component needs to show the line drawing update in real-time on the UI.

3) I need to be able to also draw a line on the 2D UI Image component and have the line show up on the 3D worldspace mesh.

Currently, I have implemented part 1 by using raycasting and drawing my line on the world coordinates of the returned raycasthit. This works great.

Now, I'm a little at a loss as how to correctly map the world coordinates of the line to the 2D UI Image component's coordinates. I know if I can figure out how to complete step 2, step 3 would simply be reversing the process for step 2.

Here's a picture visualizing what I'm trying to achieve: Here's a picture visualizing what I'm trying to achieve.

  • \$\begingroup\$ as you alredy done step 1) , can you edit mesh texture using mesh.uv ? \$\endgroup\$ Aug 26, 2015 at 7:24
  • \$\begingroup\$ I dont know if you can draw on the mesh directly like you say. I think you need to draw on its texture. There are methods fro drawing on a 2DTexture by setting the pixel color. There is also a SetPixels that chage a whole block. The size of the block would be your line width. And you can map your world coords to a x and y and them call SetPixels on both the ui image and the mesh texture. Ui will have to be a raw image for this to work I think. docs.unity3d.com/ScriptReference/Texture2D.SetPixel.html \$\endgroup\$
    – Uri Popov
    Apr 28, 2016 at 8:13

2 Answers 2


Unfortunately I cannot serve you with code, but hopefully with some hints:

For 1: First you need the coordinate of the mouse on the texture which surrounds this 3D mesh. You can use the Unity's ray class and send one from the camera origin towards the mouse and therefore detect where it collides with the mesh. Once you have that coordinate you should be able to transform it into the texture coordinate as you know the transformation of the mesh and you also know the UV coordinates of the texture on the mesh (assuming that you assigned those yourself).

The you can use the Texture2d class which has functions like setPixel to modify or replace the texture (google for "unity modify texture runtime" for more information).

2. Just asign the Texture (which is part of the material) of the 3D mesh onto some GUI element. If I'm not wrong it will actually change both at the same time this way because what you asign is just a reference to the actual texture which is handled by Unity. Otherwise just copy it after every change.

3. Again if you draw onto the 2d representation, the 3D representation will be modifed as well: Question is, is the 3D mesh actually some strange, perhaps even concave figure? If so, how do you actually want its surface to look like in 2D? Because the user wouldn't really expect the 3D result if he draws in 2D. Or is the 3D mesh just a plain, flat wall (like in your picture)? If so, the method I describe should work well. A cube would probably work too if you display the edges onto the 2D representation as a guidance for the user. Curved figures wouldn't really make sense.

Good luck!


I have a basic solution for Step 2, but it's not reversible for Step 3.

Simply make a isometric camera and make the display at the bottom of the view

Creating an Isometric Camera in Unity
2 cameras renderring at the same time

Another method (my imagination) is make the Line Drawing to "modify" the material of the mesh, and then use the same material on the 2D UI Image. Hence when the material is changed, both the Mesh and the Image will change their materials too.

  • \$\begingroup\$ The first suggestion won't work in my case because the 3D mesh can be non-rectangular while the texture is rectangular. But I'll look into the 2nd suggestion. I think I tried that, but there was an issue. \$\endgroup\$ Jun 29, 2015 at 16:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .