[vtkusers] VTK and Unicode

[David Gobbi]

That's a pretty good summary of the situation. Since
there is no option in VTK to use unicode filenames,
you're at the mercy of whatever codepage is active
on the user's computer.

  David


On Fri, Jun 6, 2014 at 11:03 AM, Maarten Beek <beekmaarten at yahoo.com> wrote:
> So I program I make in Canada, would work normally in China? However as soon
> as I use Chinese characters or the user in China uses Russian characters,
> things will break in Windows?
>
> So before I try to read a file, I'll have to guess the code page first?
> In other words, apply WideCharToMultiByte() with different codepages until I
> can successfully read the file?
>
> Now I have to figure out how to set a codepage...
>
> Maarten
>
>
>
> On Friday, June 6, 2014 12:30:15 PM, David Gobbi <david.gobbi at gmail.com>
> wrote:
>
>
> Well, in China people use the Chinese edition of Windows which defaults
> to a Chinese codepage... so everything works fine for them as long as
> they stick to Chinese.
>
> Things only get complicated when you try to mix several languages on the
> same computer.  Working with one language in isolation is never a problem.
>
> OS X was very smart to standardize of utf-8.  Linux has pretty much done
> the same.  But Windows still uses a different 8-bit encoding for every
> language.
>
>   David
>
>
> On Fri, Jun 6, 2014 at 10:08 AM, Maarten Beek <beekmaarten at yahoo.com> wrote:
>> Hi David,
>>
>> You're the only one replying to this thread and as long as you give me new
>> suggestions I keep poking ;-)
>>
>> I started using WideCharToMultiByte() directly as well, but this returns a
>> '?' on the spot of my Chinese character. VtkImageReader2 (and I think all
>> other readers as well) cannot find a file with this file name.
>>
>> How do people use VTK in China (or Russia, Greece...)?
>>
>> Maarten
>>
>>
>> On Friday, June 6, 2014 11:41:22 AM, David Gobbi <david.gobbi at gmail.com>
>> wrote:
>>
>>
>> Hi Maarten,
>>
>> I've never used ToNarrow() myself, so I think you're relying a bit too
>> much on my expertise ;)
>>
>> However, I have used WideCharToMultiByte() directly, and if given
>> CP_UTF8 as the first argument, it converts to UTF8.  It simply does
>> the conversion according to the codepage passed in the first argument.
>>
>> - David
>>
>> On Fri, Jun 6, 2014 at 9:14 AM, Maarten Beek <beekmaarten at yahoo.com>
>> wrote:
>>> Hi David,
>>>
>>> After copying the ushort string into a wchar_t string, ::ToNarrow() would
>>> return nonsense.
>>>
>>> I gave the variable KWSYS_ENCODING_DEFAULT_CODEPAGE the value CP_UTF8
>>> (defaults to CP_ACP in the vtksys CMakeLists.txt if nothing is defined)
>>> by
>>> using the add entry button in CMake.
>>> After rebuilding VTK, this didn't solve the problem.
>>>
>>> Would the codepage setting of the computer have to be set to CP_UTF8 at
>>> the
>>> moment of file creation for this to work? Although, this wouldn't affect
>>> my
>>> problem of getting the correct char* into the SetFileName() function.
>>> Would
>>> it?
>>>
>>> Maarten
>>>
>>>
>>> On Thursday, June 5, 2014 6:45:38 PM, David Gobbi <david.gobbi at gmail.com>
>>> wrote:
>>>
>>>
>>> Hi Maarten,
>>>
>>> Yes, vtksys::Encoding::ToNarrow() requires a wchart_t *.  If you like to
>>> live
>>> dangerously, you could just use a reinterpret_cast to convert your ushort
>>> *
>>> to a wchar_t *, but to be safe you should allocate a wchar_t string of
>>> the
>>> right size and then copy your ushort string into it.
>>>
>>> VTK doesn't have any cmake options related to unicode.  All file-level
>>> stuff
>>> in VTK always uses 8-bit strings no matter what build options you choose.
>>>
>>>  David
>>>
>>>
>>> On Thu, Jun 5, 2014 at 4:20 PM, Maarten Beek <beekmaarten at yahoo.com>
>>> wrote:
>>>> It's complaining about the ToNarrow() function.
>>>> (... symbol "__declspec(dllimport) <bla-bla> __cdecl
>>>> vtksys::Encoding::ToNarrow(<bla-bla>) referenced in function <bla-bla> )
>>>>
>>>> I got the VS solution from svn before my unicode changes and added the
>>>> ToNarrow() function and everything builds fine.
>>>> I guess if I build my app with unicode, I should do this with VTK as
>>>> well
>>>> (how? don't see such an option in cmake...).
>>>>
>>>>
>>>> However the app crashes in Debug, but not in Release.
>>>> In another app, I need to set wchar_t as a build-in type to avoid the
>>>> unresolved external symbol error, but then an ushort* (returned by
>>>> QString::utf16()) cannot be cast to a wchar_t*... I guess
>>>> ::ToNarrow(const
>>>> wchar_t*) needs a ::ToNarrow(const ushort*) overload?
>>>>
>>>> Maarten
>>>>
>>>>
>>>>
>>>> On Thursday, June 5, 2014 5:03:56 PM, David Gobbi
>>>> <david.gobbi at gmail.com>
>>>> wrote:
>>>>
>>>>
>>>> Hi Maarten,
>>>>
>>>> I use the functions in vtksys all the time.  Not specifically the
>>>> Encoding functions, but they shouldn't be any different.
>>>>
>>>> You said it reported an unresolved symbol error... what symbol
>>>> was it complaining about?
>>>>
>>>>  David
>>>>
>>>>
>>>> On Thu, Jun 5, 2014 at 2:56 PM, Maarten Beek <beekmaarten at yahoo.com>
>>>> wrote:
>>>>> I am linking my app with vtksys.lib.
>>>>>
>>>>> But your answer also tells me I can use this function in my app.
>>>>>
>>>>> So it must be something else... maybe the fact that my vtk is built
>>>>> with
>>>>> multibyte and the app with unicode?
>>>>> Should find a function in vtksys that is unrelated to this to test.
>>>>>
>>>>> Maarten
>>>>>
>>>>>
>>>>> On Thursday, June 5, 2014 4:31:20 PM, David Gobbi
>>>>> <david.gobbi at gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> An unresolved symbol just means that you have to link to the library,
>>>>> e.g. if you have a target_link_libraries() call in your CMakeLists.txt,
>>>>> make sure that "vtksys" is listed.
>>>>>
>>>>> On Thu, Jun 5, 2014 at 2:20 PM, Maarten Beek <beekmaarten at yahoo.com>
>>>>> wrote:
>>>>>> Hi David,
>>>>>>
>>>>>> Thanks for the quick reply.
>>>>>>
>>>>>> Sounds complicated, but I'll browse through the windows docs.
>>>>>>
>>>>>> In my search for a solution I also bumped into
>>>>>> vtksys::Encoding::ToNarrow(),
>>>>>> however this gives me an 'unresolved external symbol' link error. I
>>>>>> have
>>>>>> never used stuff in vtksys (and similar libs like vtkpng, vtktiff)
>>>>>> before,
>>>>>> so I am not really sure I am supposed to, i.e., these are just
>>>>>> functions
>>>>>> used by cmake and/or in just the vtk build.
>>>>>>
>>>>>> Maarten
>>>>>>
>>>>>>
>>>>>> On Thursday, June 5, 2014 3:44:46 PM, David Gobbi
>>>>>> <david.gobbi at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Maarten,
>>>>>>
>>>>>> VTK just uses C++ streams in most of its readers/writers.  On OS X,
>>>>>> you'll find that you can use utf-8 filenames just fine.  Same for
>>>>>> linux,
>>>>>> for the most part at least.
>>>>>>
>>>>>> For Windows, you'd have to set the codepage to 65001 (utf-8).  I've
>>>>>> done this successfully for console I/O (via SetConsoleOutputCP()),
>>>>>> but you'll have to look through the Windows docs to see what function
>>>>>> is needed to change the codepage used by CreateFileA.
>>>>>>
>>>>>> - David
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 5, 2014 at 1:17 PM, Maarten Beek <beekmaarten at yahoo.com>
>>>>>> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I was wondering if there is a multi-platform way of loading a file
>>>>>>> (e.g.
>>>>>>> tif, stl) with chinese, russian characters in the file path in VTK.
>>>>>>> How would I build VTK with Unicode characters?
>>>>>>> Can I use a non-unicode VTK in a unicode app?
>>>>>>>
>>>>>>> Thanks - Maarten
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>