thanks to your great effort

Thank you for this article. It is really great. It helped me a lot to setup my site. Now I have a question. My site works in two languages, English and Spanish. How can I switch between two different header/footer files depending on the current language? My sites are http://www.9ilc.co and http://www.9ib.co. They are developed in html5 and CSS3.

Thanks again!

Anyway, we both agree that the internal representation of strings should be consistent. If you choose an internal UTF-16-encoded std::wstring representation, then you would need conversions when interacting with those parts of the outside world (like third-party libraries or network protocols) that require UTF-8, and in the approach I describe, it is the other way around. I think the difference in approach hinges on what we regard as “the outside world”. In multi-platform projects, it makes sense to think about Windows and Linux (or Mac or whatever) as different kinds of “outside world” that the core modules can be plugged into. This was the situation at the company where I was working before (GeoVirtual, in Barcelona, which closed down a couple of years ago), where our code consisted of a large number of modules that was mostly mathematical and graphical stuff (for geographic data, with an OpenGL viewport for rendering) and mostly platform independent. We basically wanted our core code to build under both Windows and Linux. We were using quite a lot of third-party libraries and had some facade classes for platform-dependent things. In that way, calls to Windows were mostly isolated through those facade classes. We actually had several different applications with different styles of GUI built on the same core of libraries. For a long time, the code was not able to handle Unicode because of the use of many char-based functions where the local code page was assumed. We tried to turn all the std::strings into std::wstrings, but didn’t manage to adapt a lot of the code using file streams. Eventually, I decided to revert the std::wstring changes and use std::strings and assume UTF-8 internally. We had to send data through the network and we interacted with the input data through the network and through configuration XML files that were UTF-8-encoded. In such a scenario, complicating the facade classes to interact with Windows (and with the wxWidgets GUI part, if I remember correctly) with the additional character conversions turned out to be simpler and more effective. All the parts reading data form the XML files and from the network were based on essentially the same code in Windows and Linux, and the code interacting with the mathematical and graphical third-party libraries didn’t require many changes (just adding the UTF-8 wrappers for file streams and fopen, which you would also need if you were using UTF-16 internally). I wrote this post with my experience during that time in mind. I agree that in a project where everything is Windows it may make more sense to use std::wstrings, but don’t underestimate the benefits of aiming at portability, even when it is not an immediate goal.

typedef std::basic_string<TCHAR> tstring;

And others as required:

typedef std::basic_istringstream<TCHAR> tistringstream;
typedef std::basic_ostringstream<TCHAR> tostringstream;
typedef std::basic_stringstream<TCHAR> tstringstream;

etc.

typedef std::basic_string tstring;

Please understand, that I do admire your goal, which is to make UTF-8 your native type and only work with UTF-16 when interacting with Windows functions (cleanly separating the two), but for most Windows applications that approach is backwards. Everything should normally be native UTF-16 and then convert to UTF-8 only when needed. It eliminates all the extra work you’ve created for yourself, and all the problems that come with it. You seem to be playing those problems down but I know from vast experience that it would be a serious problem in most shops.

BTW, the situation with the native exception classes is another story altogether, but only because they work natively with “char” as you pointed out. Some might consider this a “mistake” in the standard itself (why not make them template-based) but I’ve seen arguments to the contrary (long story). In any case, the topic of exception encoding is another story (since it gets into deeper issues not worth discussing here).