# Getting a pixel of Sprite at the cursor position in Unity

Summary: I've been trying to figure out a way to use simple box colliders as a nice and efficient "first pass" collider. For the "second pass", I want to check that the area clicked has an alpha value > 10%. I know this is generally possible with graphics programming, but I'm struggling to find a way to do this in Unity2D with packed sprites (packed via the native Sprite Packer tool).

What I was hoping to do (but Unity will not allow) would've been something like...

// Check to see if a point intersects with a solid color
private bool CheckPoint()
{
Vector2 cursorWorld = Camera.main.ScreenToWorldPoint(Input.mousePosition);

//foreach (Collider2D col2D in Physics2D.OverlapPointAll(cursorWorld))
foreach (RaycastHit2D hit in Physics2D.RaycastAll(cursorWorld, Vector2.zero))
{
if (hit.collider.gameObject != gameObject) continue;

foreach (SpriteRenderer sprRend in hit.collider.gameObject.GetComponentsInChildren<SpriteRenderer>())
{
Texture2D tex = (Texture2D)sprRend.sprite.texture;

// this is the missing link; easy with 3d, but impossible with 2d?
Vector2 pixelUV = hit.??????;

int uvX = (int)(pixelUV.x * tex.width);
int uvY = (int)(pixelUV.y * tex.height);

Color hitColor = tex.GetPixel(uvX, uvY);

if (hitColor.a > 0.1) return true;
}
}

return false;
}


So far: I've been able to find some solutions for doing this with (3D) meshes and 3D colliders. However, the 2D colliders do not include the UV data that would be needed to make this work.

Someone was also able to get this working for their specific game using SpriteManager2 (which we're not using since Unity 4.3 makes SM2 mostly unnecessary) and posted their solution here... http://answers.unity3d.com/questions/17903/detecting-mouseclick-on-pixels-in-sprite-using-sm2.html and I think the same approach might work with minor adaptations for Unity2D, but I'm thinking there might be a more robust approach.

I'm also looking at trying to do something like http://answers.unity3d.com/questions/687116/what-is-the-easiest-way-to-convert-raycasthit2d-to.html but even if I can calculate the local UV, that wont help me with the real UV for the packed sprite unless I can somehow get the sprite's UV offset into the sheet.

I can't give exact code because I've not done this myself yet (may make a good exercise with the new 2D functionality) but I can see the outlines of how you would implement this.

First off, detect which sprite was clicked either using good ol' OnMouseDown. Use Input.mousePosition along with the sprite's transform to determine where the cursor is on the sprite.

With that, use properties of the sprite to figure out the exact pixel; I'm not sure off the top of my head which ones you'd use, but the script reference for Sprite mentions stuff like 'rect' and 'textureRectOffset'

Finally get the Texture2D with the sprite's 'texture' property and call GetPixel()

• This is pretty much what I'm doing. I put in another attempt after posting this question. The only real issue now is that I cannot call textureRectOffset on a Sprite unless it's packed into rectangles (not tightly packed) which means wasting TONS of texture memory. – Wisteso Sep 9 '14 at 4:47
• How about just 'rect' and not 'textureRectOffset'? Does that work? Even if it does, I suppose that creates potential issues of reading pixels from other sprites – jhocking Sep 9 '14 at 12:01
• Neither gives the right offset. I'm able to get the size of the sprite in the atlas, but not the position in the atlas. I think what I'm going to do is resort to polygon colliders that wont be exact, but it works and actually doesn't degrade performance much if the colliders are kept simple. – Wisteso Sep 10 '14 at 3:58
• I did this not long ago... I will post the code when im at my computer next... – Savlon Jul 6 '15 at 12:21
• Note that as @jhocking seems to propose it, you don't even need a collider to do what you're trying to achieve. Unity' Sprites have boundary informations. Using them with transform.position will let you know if the click is contained by a certain sprite. The a simple GetPixel() on the sprite will give you the information you try to get. – lvictorino Jul 6 '15 at 16:28

Below I've posted something I mocked up going for deformable pixel terrain. You may be able to get something from it... You may not... Either way, It can obtain the pixels of an image around the mouse position. The names are pretty self explanatory but if you don't understand something, post a comment here.

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

[RequireComponent (typeof (SpriteRenderer))]
{
public Texture2D spriteTexture;
public int diameter = 50;

private float _width;
private float _height;
private Sprite _image;
private SpriteRenderer _spriteRenderer;

private Color[] tempColors;

void Start ()
{
_spriteRenderer = GetComponent <SpriteRenderer> ();

_spriteRenderer.sprite = _image;

_width = _image.bounds.size.x * _image.pixelsPerUnit;
_height = _image.bounds.size.y * _image.pixelsPerUnit;
}

void Update ()
{
Vector2 mousePos = Camera.main.ScreenToWorldPoint (Input.mousePosition);

mousePos += (Vector2)_spriteRenderer.sprite.bounds.extents;
mousePos *= _spriteRenderer.sprite.pixelsPerUnit;

if (Input.GetMouseButton (0))
{
ChangePixels ((int)mousePos.x, (int)mousePos.y, diameter, Color.black);
}

if (Input.GetMouseButtonDown (1))
{
ChangePixels ((int)mousePos.x, (int)mousePos.y, 50, Color.white);

}
}

{
Texture2D texture = new Texture2D (spriteTexture.width, spriteTexture.height);
Color[] spriteColors = spriteTexture.GetPixels ();

tempColors = new Color[spriteTexture.width * spriteTexture.height];

for (int y = 0; y < maskTexture.height; y++)
{
for (int x = 0; x < maskTexture.width; x++)
{
bool masked = (maskColors[x + y * maskTexture.width]) == Color.white; // change to white and change actual image

{
tempColors[x + y * maskTexture.width] = new Color (0, 0, 0, 0);
}
else
{
}
}
}
texture.SetPixels (tempColors);
texture.Apply ();

return Sprite.Create (texture, new Rect (0, 0, texture.width, texture.height), new Vector2 (0.5f, 0.5f));
}

void ChangePixels (int centreX, int centreY, int diameter, Color color)
{
int radius = diameter / 2;
int a;
int b;

for (int y = centreY - (diameter/2); y < centreY + (diameter/2); y++)
{
for (int x = centreX - (diameter/2); x < centreX + (diameter/2); x++)
{
if (x < 0 || y < 0 || x >= _width || y >= _height) continue;

a = x - centreX;
b = y - centreY;

if ((a * a) + (b * b) <= (radius * radius))
}
}

}

void ChangeSprite (Sprite updatedSprite)
{
_spriteRenderer.sprite = updatedSprite;
}

}


Basically it takes in a spriteTexture and a maskedTexture. The masked texture gets altered to either black or white around the mouse position when a click is detected. The masked texture is then used to determine whether pixels from the sprite Texture should be shown, obviously based on the colour.

It may not exactly be what you're looking for but its a start in the right direction. Oh yeh and its not optimized and I have no idea about the bugs...

Goodluck.