Colouring greyscale images/sprites for HTML canvas

Question

What is the best method to colour in sprites at runtime using the canvas element and javascript? I know there are various blending modes using globalCompositeOperation, but I don't know how best to utilise this.

I'll break my question down into the following:

• Is there an optimum way to export greyscale images so they can be easily re-coloured later? (At the moment, the images are created with colour in Photoshop or a similar package and then turned into a greyscale and exported from there before I even touch them)
• With these greyscale images in place, what method makes colouring them with any colour possible? (They must retain the shading etc)

I've tagged this with HTML5 and Javascript, but any advise from other platforms would be helpful.

Edit: Here's an example greyscale output of the image on the left. I want to be able to restore it to the same colour, but also be able to blend it with other colours for different shades of skin.

Answer 1

There are several ways to achieve what you want. The easiest way would be to paint a colorized rectangle above your grayscale image and use one of the following blend-modes: multiply, screen or overlay (which is a combination of screen and multiply).

Here's how these blend-modes look (red rectangle over grayscale image, using multiply, screen and overlay):

The blend modes don't seem to be supported in all browsers, so maybe you'd have to implement them yourself. This should be quite straight-forward by iterating through all pixels and mix them with your desired color. Here's a list of formulas for the most common blend-modes.

Answer 2

Getting the raw pixel data

You can get the raw pixel data with the function context.getImageData(x, y, width, height). The return value is an ImageData structure. This structure will have a field data which is an UInt8ClampedArray where each pixel is represented by four values between 0 and 255 (red, green, blue, alpha).

Converting an image to greyscale

In order to reduce three color channels (R, G, B) to one, there are different approaches. Which one is the best depends on your use-case.

• Using the highest value (intensity)
• Using the average between the highest and the lowes value (luminosity)
• Using the average of all three (brightness)
• A color-psychological approach: For the human eye, yellow (255, 255, 0) seems brighter than blue (0, 0, 255) and green (0, 255, 0) seems brighter than red (255, 0, 0). You could experiment with a conversion-function which accounts for this.

Converting greyscale back to color

Depending on which algorithm you decided to use, you will now have a greyscale value between 0 and 255 for each pixel. To converte these to RGB-values depending on your target-colors RGB, there are also different methods.

• Multiplication: A greyscale value of 0 is black, a greyscale value of 255 is equal to your target-color
• Overlay: A greyscale value of 0 is black, a value of 127 is your target-color and a value of 255 is white
• Screen: A greyscale value of 0 is your target-color, a greyscale value of 255 is white

The values in between are interpolated linearly. The best method depends on the effect you are aiming for, what algorihm you've chosen for greyscaling and what your target-color is.

Writing the raw pixel data back to the canvas

To copy the imagedata structure with the modified array back to the canvas, you can use context.putImageData(imagedata, x, y).

Some performance considerations

Keep in mind that working with raw image-data is an expensive operation. Usually all canvas operations can be done on the GPU. But in this case the canvas texture must be serialized, copied to RAM, processed by the CPU and then copied back to the video memory. This is quite slow. Usually too slow to do it with everything on the screen and still keep a steady framerate. So when you intend to draw the resulting colored sprite more than once, you should buffer the result of the recoloring on an invisible background-canvas and use this as an image source for context.drawImage when the recolored sprite needs to be drawn again.

Answer 3

Adding to what @Philipp said (especially the "caching blended result"). You can reverse calculate the second image from the blend(grey,????) = colored equation for the multiply,overlay and screen functions. The problem is that the range of the ????? image may be out of the range of 0-255. So you might need to save them as a different format or encode something like a signed int into the 4 bytes of the RGBA values in one pixel for each channel.

As others said you can also use other color-models for help. HSV is the one that appears to be the obvious choice and it will look like this:

But as you can see the separated saturation and hue image still contains a good amount of the shading information, so a color model like YUV might seem more appropriate. It would look like this:

The problem here is that the inter-relationship that the RGB values in your blending source have with the final image might not always be so apparent.

Using just the raw components of these color models though has the issue of not being able to make image darker whatsoever. So I'd say use the reverse blending modes or combine the other color-spaces with the blending modes to get it done.

Edit: to visualize what I meant with the multiplication look at these examples:

The top 2 use multiplication with the regular color spectrum (0-255 mapped to 0-1), you can see that the image can only be made darker, making the red channel bright enough for a natural skin tone is basically impossible within the default spectrum. However if you look at the lower example of brightening it, then you can see that if you multiply it with values over 1 you can once again recreate the original result

Btw. if you're worried about the underwear being colored too, you need to mask it out (just use the grey base image where you don't want any coloration):

To calculate the scale value lets take for example the gray scale has the color [120 , 120, 120] at one point and the colored image has the color [152 107 74] at the same spot then the scale factors would be s_red = 152/120 = 1.27 , s_green = 107/120 = 0.892 and s_blue = 74/120 = 0.617, so [1.27 0.892 0.617].

You could reasonably map the re-color RGB value from [0-255] to [0-2] or something like that to easier use existing structures as well as limit the brightening to reasonable values. So your setup would look something like this (with the masking applied of course):

Don't forget to clamp your resulting image to [0-255] though.