WChars, Encodings, Standards and Portability

Context: The question explores the understanding and approach to character handling in C, focusing on the relationship between portability, serialization, and encodings.

Understanding of Character Handling in C:

• Portability: C provides the wchar_t type and functions for manipulating character sequences, which can represent all system characters. However, C doesn't specify any encodings or how these characters should be interpreted.
• Serialization: Character data needs to be serialized for storage or transmission, and there are standardized encodings (e.g., UTF-8, UTF-16, UTF-32) for this purpose. Iconv library is used for transcoding between these encodings.

Proposed Approach:

The question suggests using wchar_t internally, interfacing with CRT via wcsrtombs() for serialization, and iconv() for conversion to and from UTF formats. This approach aims to maintain portability while allowing for encoding-agnostic character handling.

Answer:

While the proposed approach can work on some platforms, it falls short on Windows.

Windows-Specific Considerations:

• Windows mandates the use of wchar_t even for command line arguments, deviating from the C standard.
• File and console I/O in Windows should be handled with Microsoft extensions or wrapper libraries.
• Filenames on Windows can use different encodings than the OS uses internally.

Portability and Encoding Agnosticism:

Achieving true portability with Unicode support in C/C is challenging:

• File systems and file names can use platform-specific encodings.
• Some platforms (e.g., Linux) may use UTF-8 for char type, while others (e.g., Windows) use UTF-16 for wchar_t.

Conclusion:

While the C/C standards provide some tools for character handling, portability and encoding-agnosticism require additional effort and platform-specific considerations. It is crucial to use appropriate extensions and wrapper libraries to address these challenges and ensure proper support for Unicode across different systems.