First of all, this sort of thing is easier to do if you convert your pixel values to floating-point [0, 1] rather than integer [0, 255] values:
// convert 8-bit int to [0, 1] float
float r = r_int / 255.0f;
float g = g_int / 255.0f;
float b = b_int / 255.0f;
// ... do your processing on the float rgb values ...
// at the end, convert back to 8-bit int (with clamping and rounding)
r_int = int(min(max(r, 0.0f), 1.0f) * 255.0f + 0.5f);
g_int = int(min(max(g, 0.0f), 1.0f) * 255.0f + 0.5f);
b_int = int(min(max(b, 0.0f), 1.0f) * 255.0f + 0.5f);
You can scale the luminance of the image by simply multiplying the RGB values by some constant. Constants less than one will make it darker, and constants greater than one will make it brighter. There's no need to convert to YUV or YCrCb, or even calculate the luminance, if all you want is to make the image brighter or darker.
(You should also technically convert the RGB values from gamma space to linear using the sRGB transformation before operating on them, and convert back afterward...but I'd forgive you if you didn't want to deal with that complication at this point. :) Also, if you do need to calculate the luminance, it's best to convert to linear and average the RGB using the Rec. 709 luma coefficients rather than just doing
(r + g + b) / 3. Then convert back to gamma space before displaying the grayscale value on screen.)