[Insight-developers] Unicode filenames, VS6 and fdstream (was:Unicode filenames and borland issue)

[Tom Vercauteren]

Hi Bill,

I have updated the test on the bug tracker.
http://public.kitware.com/Bug/file_download.php?file_id=2649&type=bug

I tested it on the same four compilers:
* visual c++ 2008 express
* mingw with gcc 3.4
* borland 5.5
* cygwin 1.7 with gcc 4.3
but tried to accommodate the code for VS 6.0 and VS 7.x according to:
* the information you gave me
* http://msdn.microsoft.com/en-us/library/ae1k9a9f%28VS.71%29.aspx and
http://msdn.microsoft.com/en-us/library/70bb7saf.aspx
* http://www.cdash.org/CDash/testDetails.php?test=36176221&build=469256

Could you give it a try on VS6?

Thanks a lot,
Tom

On Wed, Nov 11, 2009 at 11:23, Tom Vercauteren <tom.vercauteren at m4x.org> wrote:
> All right, the easiest would then be to not allow unicode filenames on VS6.0.
>
> I'll try an get something for tomorrow or Friday.
>
> Tom
>
> On Wed, Nov 11, 2009 at 06:01, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>> VS6 does not have a wide string constructor for (i/o)fstream. I'm not
>> sure in what VS version they added thw wide string constructor.
>>
>> Bill
>>
>>
>> On Tue, Nov 10, 2009 at 8:41 PM, Tom Vercauteren
>> <tom.vercauteren at m4x.org> wrote:
>>> Hi Bill,
>>>
>>> I just finished implementing your suggestion. I created some stream
>>> classes that inherits from (i/o)fstream on all systems except for
>>> mingw and borland where
>>> 1) they inherit from (i/o)stream
>>> 2) they implement some functions that are only in (i/o)fstream to get
>>> a uniform API
>>> 3) use the c-style open and code from fdstream to allow unicode filenames
>>>
>>> On MSVC, the new implementation takes advantage of the MSVC-specific
>>> wide string constructor of (i/o)fstream [1]. This could thus be easier
>>> for VS6.
>>>
>>> The file is here:
>>> http://www.itk.org/Bug/file_download.php?file_id=2646&type=bug
>>>
>>> I tested it on windows xp with
>>> * visual c++ 2008 express
>>> * mingw with gcc 3.4
>>> * borland 5.5
>>> * cygwin 1.7 with gcc 4.3
>>>
>>> Could you give it a try on VS6?
>>>
>>> Thanks,
>>> Tom
>>>
>>> [1] This constructor is not available on both borland and mingw which
>>> implies that stdio_filebuf is not sufficient and we need a more
>>> portable solution such as fdstream.
>>>
>>> On Tue, Nov 10, 2009 at 22:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>> fdstream is the problem on vs6 I think. We might be able to modify the
>>>> fdstream header with ifdef's for VS6. Really make fdstream look like
>>>> (i/o)stream. Maybe with (uggh!) #defines to redefine fd(i/o)stream to
>>>> be (i/o)stream only on VS6 (and maybe VS7).
>>>>
>>>> On Tue, Nov 10, 2009 at 4:11 PM, Tom Vercauteren
>>>> <tom.vercauteren at m4x.org> wrote:
>>>>> Hi Bill,
>>>>>
>>>>> Thanks for taking a look at this!
>>>>>
>>>>> Are the problems you found related to the use of _wfopen and _wopen or
>>>>> to the use of fdstream? This is an important distinction for the
>>>>> unicode filename porting because the plan I had in mind relies on
>>>>> fdstream being used in all cases (ITK_USE_REVIEW_UTF8_STRINGS on or
>>>>> off).
>>>>>
>>>>> Indeed here is what I found (VS6 not taken into account here):
>>>>>
>>>>> 1) c-style file IO can be ported relatively easily to utf8 on windows
>>>>> (msvc, borland, mingw). Just replace fopen by _wfopen and open by
>>>>> _wopen together with some string conversion
>>>>>
>>>>> 2) c++-style IO is more involved. There is a wide string version of
>>>>> ofstream.open() and ifstream.open() on msvc but not on mingw (did not
>>>>> try with on borland).
>>>>>
>>>>> Since the fstreams cannot be opened with a unicode filename on mingw,
>>>>> the workaround is to open the file using c-style functions and then
>>>>> create a stream from the result of the c-style open.
>>>>>
>>>>> A gcc specific extension exists for that (stdio_filebuf) but it is
>>>>> obviously non-portable. fdstream does the same thing but appeared to
>>>>> be portable.
>>>>>
>>>>> So to get unicode filename enabled streams, we need to replace code like
>>>>>
>>>>> A)
>>>>> std::ifstream readstream(filename, std::ios::binary | std::ios::in)
>>>>>
>>>>> by code like
>>>>>
>>>>> B)
>>>>> int fd = _wopen(filename, _O_RDONLY);
>>>>> boost::fdistream readstream(fd);
>>>>> [...]
>>>>> close(fd)
>>>>>
>>>>> We might of course pepper the code with ifdefs each time a stream is
>>>>> used (as is already done to workaround a sgi bug in metaImage.cxx
>>>>> http://public.kitware.com/cgi-bin/viewcvs.cgi/Utilities/MetaIO/metaImage.cxx?root=Insight&view=markup
>>>>> ) but I wanted to avoid that by restricting these ifdefs to live in a
>>>>> specified file/module. Hence my plan to wrap only the c-style (f)open
>>>>> functions and use fdstreams everywhere.
>>>>>
>>>>> Note that since we need to close the file descriptor and because
>>>>> fd(i/o)streams do not inherit from (i/o)fstream but only from
>>>>> (i/o)stream, it is not obvious to have a function that uses either the
>>>>> A mode or B mode above.
>>>>>
>>>>> Sorry for this long email,
>>>>> Tom
>>>>>
>>>>> P.S.: some information is also available on the bug tracker:
>>>>> http://public.kitware.com/Bug/view.php?id=9623
>>>>>
>>>>> On Tue, Nov 10, 2009 at 21:11, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>>>> My first try with VS6 is not encouraging. I suggest that we instrument
>>>>>> the code so that VS6 ITK cannot use Unicode.
>>>>>
>>>>
>>>
>>
>