2
\$\begingroup\$

In SDL_ttf, there are a few functions provided that handles non-ASCII string literals, such as:

  • TTF_RenderUNICODE_Solid
  • TTF_RenderUTF8_Solid

and etc. I'll be using only the two functions as part of my question.

Currently, I'm fussing over whether to stick to using TTF_RenderUNICODE_Solid or TTF_RenderUTF8_Solid, mainly because I wanted my program to be able to work flawlessly on both Windows and Linux. (No plans for Mac, because of lack of hardware).

I know for a fact, on Windows, they used UTF-16 for handling wide characters. It's pretty straightforward to use Japanese for SDL, by using TTF_RenderUNICODE_Solid and the use the following code, that I'm currently using, to convert from wchar_t string literals to Uint16* type that the SDL_ttf uses:

static inline Uint16* Convert(wchar_t LString[]) {
    return reinterpret_cast<Uint16*>(&(LString[0]));
}

Thus, I can easily do:

SDL_Surface* textSurface = TTF_RenderUNICODE_Solid(font, Convert(L"い"), this->color);

But, given that any string literals, L"" is a wchar_t[] type, it makes things more complicated for me if I were to port this over to Linux. The only way I know of compiling code for Linux is to use GCC, and GCC clearly defines wchar_t to be of 32 bits long.

Of course, I can just straight up use the UTF8 versions:

  • TTF_RenderUTF8_Solid

And just use string literals, such as u8"あ", or u8"いいえ", and they will all work in most cases. The problem here is, when I store these string literals and then inspect the variables in the debugger, I would not get the values I wanted. The stored data would suffer from integer overflow, like so:

Where at the top, I used u8"あ" and for bottom, I used L"あ". You can see how the value is no longer preserved when using UTF8 strings, but the value is preserved when using wchar_t type strings.

Thus, I really can't use TTF_RenderUTF8_Solid in SDL_ttf to help me with Japanese encoding.

May I ask what is the best way to go about this? And how I should correctly handle Japanese on both Windows and Linux platforms?

Thanks in advance.

\$\endgroup\$
2
\$\begingroup\$

I'm an idiot.

In C++, the compiler knows what a u8"" string literal is, and will convert that string literal appropriately to an UTF-8 compatible string.

And since it's UTF-8, it means each character is separated into 8-bits long bytes. With this in mind, you can just point a char* at the string literal, and it will work as if it's an ASCII char* string.

The Visual Studio's Immediate Window only displays ASCII characters, and will convert some parts of UTF-8 compatible characters into multiple bytes, then display each byte as if they are ASCII characters. This will create an allusion that the values are getting integer overflowed. I was merely tricked by this until now.

Thus, the best way to render text is to use the provided UTF-8 methods:

SDL_Surface* textSurface = TTF_RenderUTF8_Solid(font, u8"い", this->color);

And then it should work from there. No need to do any conversion whatsoever.

\$\endgroup\$
  • 1
    \$\begingroup\$ Given the PITA that text encoding are, especially when tools mislead you; and the fact that you walked through every detail clearly, I have to disagree with the first sentence. downvote \$\endgroup\$ – Quentin Dec 26 '17 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.