wx3.0.1 and decimal point

[Carsten A. Arnholm]

This is rather frustrating....

I am Kubuntu 14.04 and I am in Norway. The KDE locale has been set to
use only decimal point ("full stop") instead of comma which some "wise
men" have decided should be the standard in this country.

Now I am trying to write a small console program writing an XML file
using wx3.0.1, this file shall not use anything else but the "C" locale,
as the files will be copied around the world, and it contains numerical
values.

double temp = 29.862;
wxString string_temp = wxString::Format("%f",temp)

No matter what I try, I get "29,862000" as value of the string_temp, and
it breaks my application completely. I also don't understand how this
can happen given both the settings of the program and my system settings.

I used to have this problem with wx2.8 a couple of years ago, and the
work-around then was to include the following in the main program

wxLocale locale(wxLANGUAGE_ENGLISH);

Note I really don't want an english locale, I want the "C" locale. In
fact, all I want is to be able to read and write decimal numbers using
full stop as the decimal separator. Now the work around does not work
anymore. What am I doing wrong? this should really not be so hard.

Out of desperation I also tried
std::setlocale(LC_NUMERIC,"C");

But it has no effect in wxWidgets
.
Please can you explain what is going on, and how to read and write
numbers using "C" locale specification?

----

Hmmm I did a bit of research, here is a snippet from my main program


int main(int argc, char **argv)
{
// std::setlocale(LC_NUMERIC,"C");

// initialise wxWidgets library
wxInitializer initializer(argc,argv);
wxLocale locale(wxLANGUAGE_ENGLISH);

double ttemp = 29.862;
char buf[80];

// parse command lin
wxCmdLineParser parser(cmdLineDesc);
parser.SetSwitchChars(wxT("-"));
sprintf(buf,"one %f",ttemp);
cout << buf << endl;

parser.SetCmdLine(argc,argv);
sprintf(buf,"two %f",ttemp);
cout << buf << endl;

if(parser.Parse() != 0) {
// command line parameter error
return 1;
}


// etc...
]

The output from this test is

one 29.862000
two 29,862000


I find it rather problematic that a call to parser.SetCmdLine(argc,argv);

changes my program locale silently?

Carsten Arnholm

[Vadim Zeitlin]

On Wed, 03 Sep 2014 00:53:39 +0200 Carsten A. Arnholm wrote:

CAA> This is rather frustrating....

Welcome to the locales.

CAA> double temp = 29.862;
CAA> wxString string_temp = wxString::Format("%f",temp)

Use wxString::FromCDouble() (and ToCDouble() in the other direction).

CAA> I find it rather problematic that a call to parser.SetCmdLine(argc,argv);
CAA> changes my program locale silently?

This is a bug :-( It's supposed to restore it, but it doesn't do it
because it doesn't call setlocale() correctly to query the original locale.
This (just committed as r77542 to 3.0.x branch) should correct it:

diff --git a/src/common/cmdline.cpp b/src/common/cmdline.cpp
index a9020f1..1362a65 100644
--- a/src/common/cmdline.cpp
+++ b/src/common/cmdline.cpp
@@ -278,7 +278,8 @@ void wxCmdLineParserData::SetArguments(int argc, char **argv)
// temporarily change the locale here. The only drawback is that changing
// the locale is thread-unsafe but precisely because we're called so early
// it's hopefully safe to assume that no other threads had been created yet.
- char * const locOld = SetAllLocaleFacets("");
+ char * const locOld = SetAllLocaleFacets(NULL);
+ SetAllLocaleFacets("");
wxON_BLOCK_EXIT1( SetAllLocaleFacets, locOld );

for ( int n = 0; n < argc; n++ )

Thanks for reporting,
VZ

[Carsten A. Arnholm]

I am reposting 2 posts from a few days ago in one. The originals have

Post by Vadim Zeitlin
CAA> wxString string_temp = wxString::Format("%f",temp)
Use wxString::FromCDouble() (and ToCDouble() in the other direction).

Thanks a lot Vadim. I had forgotten about this. It is obviously the
correct solution. It is only 8 months since I moved from wx2.8 to wx3.0
so it wasn't available to me before and therefore forgotten. But I will
certainly use it from now on...!

Post by Vadim Zeitlin
CAA> I find it rather problematic that a call to

parser.SetCmdLine(argc,argv);

Post by Vadim Zeitlin
CAA> changes my program locale silently?
This is a bug It's supposed to restore it, but it doesn't do it
because it doesn't call setlocale() correctly to query the original locale.

Thank you for the confirmation! I thought as much.

All is well then. I will use FromCDouble()/ToCDouble() and the bug is
being handled.

===

As mentioned, the main problem has been solved by restoring the locale
correctly, but I had to think again about the issue as i could not quite
figure out why the locale had been chenged in the first place. So I
checked the code (before you fixed it), e.g. at

http://review.wxwidgets.org/r/557/diff/2/download/7484/new/

The logic explained in the comments of function
wxCmdLineParserData::SetArguments(int argc, char **argv) is

"Command-line arguments are supposed to be in the user locale encoding
(what else?) but wxLocale probably wasn't initialized yet as we're
called early during the program startup and so our locale might not have
been set from the environment yet. To work around this problem we
temporarily change the locale here."

I do have to disagree fundamentally with this reasoning, even when done
correctly after the fix of locale setting/restoration.

What this implies is that wxWidgets silently enforces user locale for
numerical command line arguments, regardless of what the application
programmer explicitly tries to achieve. Sometimes this may be ok, but
not always.

For the types of programs I am writing and using, scripts and input
files are shared across continents. There is absolutely no way we can
manage anything but C locale everywhere for numerical values, including
command line arguments. Most users cannot be relied upon to set up their
locale consistently with others, and I don't want to try and force it on
them either. But I need to be able to tell them when asked how to use
our programs consistently, regardless of where they are in the world.

The way this is set up now, a script written in the UK and calling a
wxWidgets program might work correctly, but when copied unchanged to
Norway it could fail or even worse: Give a different result. This is bad.

To avoid this, I will have to apply yet another work-around, i.e. never
ever use wxCMD_LINE_VAL_DOUBLE but use wxCMD_LINE_VAL_STRING only
instead and parse it myself using wxString::FromCDouble(). But I think
it is wrong to have have to keep chasing work-arounds for these things....

In conclusion, I think it is fundamentally wrong for
wxCmdLineParserData::SetArguments(int argc, char **argv) to change the
locale, even temporarily. I strongly suggest that "what else?" is the
correct alternative: It should always be the application programmer's
responsibility to decide/set locale before calling SetArguments(..), and
such settings should not be overruled silently. Following this line of
reasoning, the changing and restoration of locale should be removed from
SetArguments(...).


Best regards
Carsten Arnholm

[Vadim Zeitlin]

On Tue, 09 Sep 2014 22:12:24 +0200 Carsten A. Arnholm wrote:

CAA> The logic explained in the comments of function
CAA> wxCmdLineParserData::SetArguments(int argc, char **argv) is
CAA>
CAA> "Command-line arguments are supposed to be in the user locale encoding
CAA> (what else?) but wxLocale probably wasn't initialized yet as we're
CAA> called early during the program startup and so our locale might not have
CAA> been set from the environment yet. To work around this problem we
CAA> temporarily change the locale here."
CAA>
CAA> I do have to disagree fundamentally with this reasoning, even when done
CAA> correctly after the fix of locale setting/restoration.
CAA>
CAA> What this implies is that wxWidgets silently enforces user locale for
CAA> numerical command line arguments, regardless of what the application
CAA> programmer explicitly tries to achieve. Sometimes this may be ok, but
CAA> not always.

Yes, you're right. However notice that this change wasn't done for
numerical arguments but rather for string ones. Because without it,
wxWidgets programs couldn't accept any strings containing non-ASCII
characters as command line arguments: converting them to "C" locale would
fail and result in empty strings. And this is a much worse problem than the
decimal point/comma one.

CAA> For the types of programs I am writing and using, scripts and input
CAA> files are shared across continents. There is absolutely no way we can
CAA> manage anything but C locale everywhere for numerical values, including
CAA> command line arguments. Most users cannot be relied upon to set up their
CAA> locale consistently with others, and I don't want to try and force it on
CAA> them either. But I need to be able to tell them when asked how to use
CAA> our programs consistently, regardless of where they are in the world.
CAA>
CAA> The way this is set up now, a script written in the UK and calling a
CAA> wxWidgets program might work correctly, but when copied unchanged to
CAA> Norway it could fail or even worse: Give a different result. This is bad.

Yes, I agree this is not ideal.

CAA> To avoid this, I will have to apply yet another work-around, i.e. never
CAA> ever use wxCMD_LINE_VAL_DOUBLE but use wxCMD_LINE_VAL_STRING only
CAA> instead and parse it myself using wxString::FromCDouble(). But I think
CAA> it is wrong to have have to keep chasing work-arounds for these things....

I think the best would be to add wxCMD_LINE_VAL_CDOUBLE which would parse
doubles using wxString::ToCDouble() instead of ToDouble(). If you are
motivated to make a patch doing this, it would be welcome.

CAA> In conclusion, I think it is fundamentally wrong for
CAA> wxCmdLineParserData::SetArguments(int argc, char **argv) to change the
CAA> locale, even temporarily. I strongly suggest that "what else?" is the
CAA> correct alternative: It should always be the application programmer's
CAA> responsibility to decide/set locale before calling SetArguments(..), and
CAA> such settings should not be overruled silently. Following this line of
CAA> reasoning, the changing and restoration of locale should be removed from
CAA> SetArguments(...).

The vast majority of programmers won't bother setting the locale before
the command line is parsed though, so the most likely result of doing this
would be that the program would work with ASCII command line arguments
(e.g. file names) but completely fail with non-ASCII ones, which would be
pretty bad IMO.

Regards,
VZ

[Carsten A. Arnholm]

Post by Vadim Zeitlin
I think the best would be to add wxCMD_LINE_VAL_CDOUBLE which would parse
doubles using wxString::ToCDouble() instead of ToDouble(). If you are
motivated to make a patch doing this, it would be welcome.

I agree this is a solution that is well defined and acceptable.

I'll try to find the motivation and figure out how to do this. It might
take some time as I am not familiar with making patches. Could be an
opportunity to learn it.

Best regards
Carsten Arnholm