Details Ticket 3591


Comment | Reply | Take | Open


Serial Number 3591
Subject d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
Area wish6
Queue grass
Requestors mlennert@club.worldonline.be
Owner none
Status resolved
Last User Contact Thu Sep 1 17:30:27 2005 (3 yr ago)
Current Priority 30
Final Priority 70
Due No date assigned
Last Action Wed Oct 18 12:26:56 2006 (2 yr ago)
Created Tue Aug 30 14:42:03 2005 (3 yr ago)

Transaction History Ticket 3591


Tue, Aug 30 2005 14:42:03    Request created by guest  
Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not used in
my Debian

Platform: GNU/Linux/i386
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: cvs20050830

This is not really a GRASS bug, but it has important consequences for d.vect.thematic
and, therefore, is worth mentioning.

On my Debian testing/unstable machine with a grass package created using libc6
debian package number 2.3.5-5, the individual LC settings cannot be changed.
For example:

> locale
LANG=fr_BE
LC_CTYPE="fr_BE"
LC_NUMERIC="fr_BE"
LC_TIME="fr_BE"
LC_COLLATE="fr_BE"
LC_MONETARY="fr_BE"
LC_MESSAGES="fr_BE"
LC_PAPER="fr_BE"
LC_NAME="fr_BE"
LC_ADDRESS="fr_BE"
LC_TELEPHONE="fr_BE"
LC_MEASUREMENT="fr_BE"
LC_IDENTIFICATION="fr_BE"
LC_ALL=fr_BE

> export LC_NUMERIC=C
> locale
LANG=fr_BE
LC_CTYPE="fr_BE"
LC_NUMERIC="fr_BE"
LC_TIME="fr_BE"
LC_COLLATE="fr_BE"
LC_MONETARY="fr_BE"
LC_MESSAGES="fr_BE"
LC_PAPER="fr_BE"
LC_NAME="fr_BE"
LC_ADDRESS="fr_BE"
LC_TELEPHONE="fr_BE"
LC_MEASUREMENT="fr_BE"
LC_IDENTIFICATION="fr_BE"
LC_ALL=fr_BE

export LC_ALL=C works as expected.

I have found a bug report in Debian about this, which was marked as solved for
a previous version of libc6: http://lists.debian.org/debian-glibc/2002/10/msg00333.html
Apparently it is not solved, and I'll report this to Debian, but this means that
we maybe should think about using LC_ALL=C in scripts, instead of LC_NUMERIC.
Unless this is a specific problem to my config.
Can other Debian users confirm this ?

Moritz
Wed, Aug 31 2005 00:26:43    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17172.56608.170716.433988@cerise.gclements.plus.com>
Date Tue, 30 Aug 2005 23:26:40 +0100
To Request Tracker <grass-bugs@intevation.de>
Cc grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
In-Reply-To <20050830124203.30E8B1005A3@lists.intevation.de>
References <20050830124203.30E8B1005A3@lists.intevation.de>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Request Tracker wrote:

> this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> -------------------------------------------------------------------------
> 
> Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> used in my Debian

GRASS doesn't use any of the locale categories except for LC_MESSAGES
(if it was built using --with-nls).

Making it use LC_NUMERIC would be a lot of work. You can't just
blindly call setlocale(LC_NUMERIC, "") on startup as a lot of file
formats have to use a dot (period) as the decimal separator. Instead,
you would have to identify every use of printf("%f") etc and decide
whether or not it should use the locale's numeric conventions.

Also, there are plenty of ambiguous cases, where output is intended to
be both human-readable and machine-readable. This isn't limited to
GRASS; in locales which use a comma as the decimal separator,
programmers often set Windows to use the US locale as, if you use the
correct locale, you can't paste the output from programs into source
code.

-- 
Glynn Clements <glynn@gclements.plus.com>


Wed, Aug 31 2005 10:24:51    Mail sent by mlennert@club.worldonline.be  
Return-Path <mlennert@club.worldonline.be>
Delivered-To grass-bugs@lists.intevation.de
Message-ID <4315692D.4000908@club.worldonline.be>
Date Wed, 31 Aug 2005 10:24:13 +0200
From Moritz Lennert <mlennert@club.worldonline.be>
User-Agent Debian Thunderbird 1.0.2 (X11/20050602)
X-Accept-Language en-us, en
MIME-Version 1.0
To Glynn Clements <glynn@gclements.plus.com>
Cc Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com>
In-Reply-To <17172.56608.170716.433988@cerise.gclements.plus.com>
X-Enigmail-Version 0.91.0.0
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00, UPPERCASE_25_50
X-Spam-Level
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Glynn Clements wrote:
> Request Tracker wrote:
> 
> 
>>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
>>-------------------------------------------------------------------------
>>
>>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
>>used in my Debian
> 
> 
> GRASS doesn't use any of the locale categories except for LC_MESSAGES
> (if it was built using --with-nls).

mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
d.correlate/d.correlate:export LC_NUMERIC=C
d.out.png/d.out.png:export LC_NUMERIC=C
d.resize/d.resize:export LC_NUMERIC=C
d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
i.image.mosaic/i.image.mosaic:export LC_NUMERIC=C
i.in.spotvgt/i.in.spotvgt:export LC_NUMERIC=C
i.oif/i.oifcalc:export LC_NUMERIC=C
r.fillnulls/r.fillnulls:export LC_NUMERIC=C
r.in.srtm/r.in.srtm:export LC_NUMERIC=C
r.plane/r.plane:export LC_NUMERIC=C
r.reclass.area/r.reclass.area:export LC_NUMERIC=C
r.regression.line/r.regression.line:export LC_NUMERIC=C
r.univar.sh/r.univar.sh:export LC_NUMERIC=C
v.db.addcol/v.db.addcol:export LC_NUMERIC=C
v.db.addtable/v.db.addtable:export LC_NUMERIC=C
v.db.droptable/v.db.droptable:export LC_NUMERIC=C
v.db.update/v.db.update:export LC_NUMERIC=C
v.in.e00/v.in.e00:export LC_NUMERIC=C
v.in.garmin/v.in.garmin:export LC_NUMERIC=C
v.in.garmin/v.in.garmin:export LC_NUMERIC=C
v.in.gns/v.in.gns:export LC_NUMERIC=C
v.in.mapgen/v.in.mapgen:export LC_NUMERIC=C
v.rast.stats/v.rast.stats:export LC_NUMERIC=C
v.univar.sh/v.univar.sh:export LC_NUMERIC=C

So it is used quite a lot in scripts.

But I received an answer to the question on why it didn't work for me:

At Tue, 30 Aug 2005 10:06:37 -0400,
Daniel Jacobowitz wrote:

>>> > LC_ALL=fr_BE
>
>>
>> Note: this means that LC_ALL is already set in your environment.
>> LC_ALL is documented to override all the other environment variables.
>> So it's completely expected that changing LC_NUMERIC has no effect.

So, whenever a user has LC_ALL set, any other LC settings won't
work. We should either indicate this in the man pages, or find
another solution...

Moritz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDFWktrIrMbm76jD8RAjj/AJ4+afU4qV4TizQQIAVB3bdlpO2hjACeJKEy
gR4INjSzhMGln1r2ccvgyi0=
=S44p
-----END PGP SIGNATURE-----


Wed, Aug 31 2005 15:57:15    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Wed, 31 Aug 2005 15:57:08 +0200
From Markus Neteler <neteler@itc.it>
To Moritz Lennert <mlennert@club.worldonline.be>
Cc Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
Message-ID <20050831135708.GB25187@thuille.itc.it>
Mail-Followup-To Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com> <4315692D.4000908@club.worldonline.be>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <4315692D.4000908@club.worldonline.be>
User-Agent Mutt/1.4.1i
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Wed, Aug 31, 2005 at 10:24:13AM +0200, Moritz Lennert wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Glynn Clements wrote:
> > Request Tracker wrote:
> > 
> > 
> >>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> >>-------------------------------------------------------------------------
> >>
> >>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> >>used in my Debian
> > 
> > 
> > GRASS doesn't use any of the locale categories except for LC_MESSAGES
> > (if it was built using --with-nls).
> 
> mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
> d.correlate/d.correlate:export LC_NUMERIC=C
> d.out.png/d.out.png:export LC_NUMERIC=C
> d.resize/d.resize:export LC_NUMERIC=C
> d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
> i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
> i.image.mosaic/i.image.mosaic:export LC_NUMERIC=C
> i.in.spotvgt/i.in.spotvgt:export LC_NUMERIC=C
> i.oif/i.oifcalc:export LC_NUMERIC=C
> r.fillnulls/r.fillnulls:export LC_NUMERIC=C
> r.in.srtm/r.in.srtm:export LC_NUMERIC=C
> r.plane/r.plane:export LC_NUMERIC=C
> r.reclass.area/r.reclass.area:export LC_NUMERIC=C
> r.regression.line/r.regression.line:export LC_NUMERIC=C
> r.univar.sh/r.univar.sh:export LC_NUMERIC=C
> v.db.addcol/v.db.addcol:export LC_NUMERIC=C
> v.db.addtable/v.db.addtable:export LC_NUMERIC=C
> v.db.droptable/v.db.droptable:export LC_NUMERIC=C
> v.db.update/v.db.update:export LC_NUMERIC=C
> v.in.e00/v.in.e00:export LC_NUMERIC=C
> v.in.garmin/v.in.garmin:export LC_NUMERIC=C
> v.in.garmin/v.in.garmin:export LC_NUMERIC=C
> v.in.gns/v.in.gns:export LC_NUMERIC=C
> v.in.mapgen/v.in.mapgen:export LC_NUMERIC=C
> v.rast.stats/v.rast.stats:export LC_NUMERIC=C
> v.univar.sh/v.univar.sh:export LC_NUMERIC=C
> 
> So it is used quite a lot in scripts.
> 
> But I received an answer to the question on why it didn't work for me:
> 
> At Tue, 30 Aug 2005 10:06:37 -0400,
> Daniel Jacobowitz wrote:
> 
> >>> > LC_ALL=fr_BE
> >
> >>
> >> Note: this means that LC_ALL is already set in your environment.
> >> LC_ALL is documented to override all the other environment variables.
> >> So it's completely expected that changing LC_NUMERIC has no effect.
> 
> So, whenever a user has LC_ALL set, any other LC settings won't
> work. We should either indicate this in the man pages, or find
> another solution...

Stupid idea: Can you try to add

unset LC_ALL

before the LC_NUMERIC change in d.vect.thematic?

Markus


Wed, Aug 31 2005 16:45:26    Mail sent by mlennert@club.worldonline.be  
Return-Path <mlennert@club.worldonline.be>
Delivered-To grass-bugs@lists.intevation.de
Message-ID <4315BDC7.7090200@club.worldonline.be>
Date Wed, 31 Aug 2005 16:25:11 +0200
From Moritz Lennert <mlennert@club.worldonline.be>
User-Agent Debian Thunderbird 1.0.2 (X11/20050602)
X-Accept-Language en-us, en
MIME-Version 1.0
To Markus Neteler via RT <grass-bugs@intevation.de>
Subject Re: [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
References <20050831135715.BF13D1006C4@lists.intevation.de>
In-Reply-To <20050831135715.BF13D1006C4@lists.intevation.de>
X-Enigmail-Version 0.91.0.0
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Markus Neteler via RT wrote:
> On Wed, Aug 31, 2005 at 10:24:13AM +0200, Moritz Lennert wrote:
> 
>>-----BEGIN PGP SIGNED MESSAGE-----
>>Hash: SHA1
>>
>>Glynn Clements wrote:
>>
>>>Request Tracker wrote:
>>>
>>>
>>>
>>>>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
>>>>-------------------------------------------------------------------------
>>>>
>>>>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
>>>>used in my Debian
>>>
>>>
>>>GRASS doesn't use any of the locale categories except for LC_MESSAGES
>>>(if it was built using --with-nls).
>>
>>mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
>>d.correlate/d.correlate:export LC_NUMERIC=C
>>d.out.png/d.out.png:export LC_NUMERIC=C
>>d.resize/d.resize:export LC_NUMERIC=C
>>d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
>>i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
>>i.image.mosaic/i.image.mosaic:export LC_NUMERIC=C
>>i.in.spotvgt/i.in.spotvgt:export LC_NUMERIC=C
>>i.oif/i.oifcalc:export LC_NUMERIC=C
>>r.fillnulls/r.fillnulls:export LC_NUMERIC=C
>>r.in.srtm/r.in.srtm:export LC_NUMERIC=C
>>r.plane/r.plane:export LC_NUMERIC=C
>>r.reclass.area/r.reclass.area:export LC_NUMERIC=C
>>r.regression.line/r.regression.line:export LC_NUMERIC=C
>>r.univar.sh/r.univar.sh:export LC_NUMERIC=C
>>v.db.addcol/v.db.addcol:export LC_NUMERIC=C
>>v.db.addtable/v.db.addtable:export LC_NUMERIC=C
>>v.db.droptable/v.db.droptable:export LC_NUMERIC=C
>>v.db.update/v.db.update:export LC_NUMERIC=C
>>v.in.e00/v.in.e00:export LC_NUMERIC=C
>>v.in.garmin/v.in.garmin:export LC_NUMERIC=C
>>v.in.garmin/v.in.garmin:export LC_NUMERIC=C
>>v.in.gns/v.in.gns:export LC_NUMERIC=C
>>v.in.mapgen/v.in.mapgen:export LC_NUMERIC=C
>>v.rast.stats/v.rast.stats:export LC_NUMERIC=C
>>v.univar.sh/v.univar.sh:export LC_NUMERIC=C
>>
>>So it is used quite a lot in scripts.
>>
>>But I received an answer to the question on why it didn't work for me:
>>
>>At Tue, 30 Aug 2005 10:06:37 -0400,
>>Daniel Jacobowitz wrote:
>>
>>
>>>>>>LC_ALL=fr_BE
>>>
>>>>Note: this means that LC_ALL is already set in your environment.
>>>>LC_ALL is documented to override all the other environment variables.
>>>>So it's completely expected that changing LC_NUMERIC has no effect.
>>
>>So, whenever a user has LC_ALL set, any other LC settings won't
>>work. We should either indicate this in the man pages, or find
>>another solution...
> 
> 
> Stupid idea: Can you try to add
> 
> unset LC_ALL
> 
> before the LC_NUMERIC change in d.vect.thematic?
> 

Tried it and it works.
I've committed to CVS. Bug can be closed. (Or can someone give me
access to the bug tracker so that I can do it myself ?)

Should we apply this to all scripts ?

Moritz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDFb3HrIrMbm76jD8RAgj+AJ4tMkprKj9L2ekwZiS4tRapAqOLmgCeKuoF
SP5/eSvr43PD85aYMIOcQ04=
=sqdG
-----END PGP SIGNATURE-----


Wed, Aug 31 2005 17:15:01    Mail sent by mlennert@club.worldonline.be  
Return-Path <mlennert@club.worldonline.be>
Delivered-To grass-bugs@lists.intevation.de
Message-ID <4315BDC7.7090200@club.worldonline.be>
Date Wed, 31 Aug 2005 16:25:11 +0200
From Moritz Lennert <mlennert@club.worldonline.be>
User-Agent Debian Thunderbird 1.0.2 (X11/20050602)
X-Accept-Language en-us, en
MIME-Version 1.0
To Markus Neteler via RT <grass-bugs@intevation.de>
Subject Re: [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
References <20050831135715.BF13D1006C4@lists.intevation.de>
In-Reply-To <20050831135715.BF13D1006C4@lists.intevation.de>
X-Enigmail-Version 0.91.0.0
Content-Type text/plain; charset=ISO-8859-1
Content-Transfer-Encoding 7bit
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Markus Neteler via RT wrote:
> On Wed, Aug 31, 2005 at 10:24:13AM +0200, Moritz Lennert wrote:
> 
>>-----BEGIN PGP SIGNED MESSAGE-----
>>Hash: SHA1
>>
>>Glynn Clements wrote:
>>
>>>Request Tracker wrote:
>>>
>>>
>>>
>>>>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
>>>>-------------------------------------------------------------------------
>>>>
>>>>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
>>>>used in my Debian
>>>
>>>
>>>GRASS doesn't use any of the locale categories except for LC_MESSAGES
>>>(if it was built using --with-nls).
>>
>>mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
>>d.correlate/d.correlate:export LC_NUMERIC=C
>>d.out.png/d.out.png:export LC_NUMERIC=C
>>d.resize/d.resize:export LC_NUMERIC=C
>>d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
>>i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
>>i.image.mosaic/i.image.mosaic:export LC_NUMERIC=C
>>i.in.spotvgt/i.in.spotvgt:export LC_NUMERIC=C
>>i.oif/i.oifcalc:export LC_NUMERIC=C
>>r.fillnulls/r.fillnulls:export LC_NUMERIC=C
>>r.in.srtm/r.in.srtm:export LC_NUMERIC=C
>>r.plane/r.plane:export LC_NUMERIC=C
>>r.reclass.area/r.reclass.area:export LC_NUMERIC=C
>>r.regression.line/r.regression.line:export LC_NUMERIC=C
>>r.univar.sh/r.univar.sh:export LC_NUMERIC=C
>>v.db.addcol/v.db.addcol:export LC_NUMERIC=C
>>v.db.addtable/v.db.addtable:export LC_NUMERIC=C
>>v.db.droptable/v.db.droptable:export LC_NUMERIC=C
>>v.db.update/v.db.update:export LC_NUMERIC=C
>>v.in.e00/v.in.e00:export LC_NUMERIC=C
>>v.in.garmin/v.in.garmin:export LC_NUMERIC=C
>>v.in.garmin/v.in.garmin:export LC_NUMERIC=C
>>v.in.gns/v.in.gns:export LC_NUMERIC=C
>>v.in.mapgen/v.in.mapgen:export LC_NUMERIC=C
>>v.rast.stats/v.rast.stats:export LC_NUMERIC=C
>>v.univar.sh/v.univar.sh:export LC_NUMERIC=C
>>
>>So it is used quite a lot in scripts.
>>
>>But I received an answer to the question on why it didn't work for me:
>>
>>At Tue, 30 Aug 2005 10:06:37 -0400,
>>Daniel Jacobowitz wrote:
>>
>>
>>>>>>LC_ALL=fr_BE
>>>
>>>>Note: this means that LC_ALL is already set in your environment.
>>>>LC_ALL is documented to override all the other environment variables.
>>>>So it's completely expected that changing LC_NUMERIC has no effect.
>>
>>So, whenever a user has LC_ALL set, any other LC settings won't
>>work. We should either indicate this in the man pages, or find
>>another solution...
> 
> 
> Stupid idea: Can you try to add
> 
> unset LC_ALL
> 
> before the LC_NUMERIC change in d.vect.thematic?
> 

Tried it and it works.
I've committed to CVS. Bug can be closed. (Or can someone give me
access to the bug tracker so that I can do it myself ?)

Should we apply this to all scripts ?

Moritz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDFb3HrIrMbm76jD8RAgj+AJ4tMkprKj9L2ekwZiS4tRapAqOLmgCeKuoF
SP5/eSvr43PD85aYMIOcQ04=
=sqdG
-----END PGP SIGNATURE-----


Wed, Aug 31 2005 22:00:23    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17174.3153.868078.95493@cerise.gclements.plus.com>
Date Wed, 31 Aug 2005 21:00:17 +0100
To Moritz Lennert <mlennert@club.worldonline.be>
Cc Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
In-Reply-To <4315692D.4000908@club.worldonline.be>
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com> <4315692D.4000908@club.worldonline.be>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Moritz Lennert wrote:

> >>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> >>-------------------------------------------------------------------------
> >>
> >>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> >>used in my Debian
> > 
> > 
> > GRASS doesn't use any of the locale categories except for LC_MESSAGES
> > (if it was built using --with-nls).

Actually, I've just noticed that it also uses LC_CTYPE (again, only if
built using --with-nls):

	revision 1.8
	date: 2004/08/27 11:16:51;  author: markus;  state: Exp;  lines: +1 -0
	added LC_TYPE for japanese

This looks pretty dubious, as it changes the behaviour of many of the
<ctype.h> functions (isalpha(), tolower() etc) as well as
strcasecmp(), but GRASS' built-in string handling always assumes ASCII
(e.g. G_strcasecmp() only considers A-Z/a-z regardless of the locale).

> mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
> d.correlate/d.correlate:export LC_NUMERIC=C
> d.out.png/d.out.png:export LC_NUMERIC=C
> d.resize/d.resize:export LC_NUMERIC=C
> d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
> i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C

[snip]

> So it is used quite a lot in scripts.

In all cases, it's forced to "C", which is the same as it not being
used. Scripts do this to prevent other programs (e.g. awk) from using
the user's LC_NUMERIC settings, so that they don't end up passing
floating-point values which use a comma to programs which expect a
dot.

-- 
Glynn Clements <glynn@gclements.plus.com>


Wed, Aug 31 2005 22:22:47    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Wed, 31 Aug 2005 22:22:40 +0200
From Markus Neteler <neteler@itc.it>
To Glynn Clements <glynn@gclements.plus.com>
Cc Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
Message-ID <20050831202240.GA29081@thuille.itc.it>
Mail-Followup-To Glynn Clements <glynn@gclements.plus.com>, Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com> <4315692D.4000908@club.worldonline.be> <17174.3153.868078.95493@cerise.gclements.plus.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <17174.3153.868078.95493@cerise.gclements.plus.com>
User-Agent Mutt/1.4.1i
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Wed, Aug 31, 2005 at 09:00:17PM +0100, Glynn Clements wrote:
> 
> Moritz Lennert wrote:
> 
> > >>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> > >>-------------------------------------------------------------------------
> > >>
> > >>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> > >>used in my Debian
> > > 
> > > 
> > > GRASS doesn't use any of the locale categories except for LC_MESSAGES
> > > (if it was built using --with-nls).
> 
> Actually, I've just noticed that it also uses LC_CTYPE (again, only if
> built using --with-nls):
> 
> 	revision 1.8
> 	date: 2004/08/27 11:16:51;  author: markus;  state: Exp;  lines: +1 -0
> 	added LC_TYPE for japanese

To what file do you refer?

Aha:
2004-08-27 13:16  markus
	* lib/gis/locale.c: added LC_TYPE for japanese

The change was actually

~/grass61/lib/gis > cvs diff -r 1.7 -r 1.8 locale.c
Index: locale.c
===================================================================
RCS  /home/grass/grassrepository/grass6/lib/gis/locale.c,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -r1.7 -r1.8
--- locale.c    13 May 2002 07:02:19 -0000      1.7
+++ locale.c    27 Aug 2004 11:16:51 -0000      1.8
@@ -35,6 +35,7 @@

        if (!initialized)
        {
+               setlocale(LC_CTYPE, "");
                setlocale(LC_MESSAGES, "");
                initialized = 1;
        }


> This looks pretty dubious, as it changes the behaviour of many of the
> <ctype.h> functions (isalpha(), tolower() etc) as well as
> strcasecmp(), but GRASS' built-in string handling always assumes ASCII
> (e.g. G_strcasecmp() only considers A-Z/a-z regardless of the locale).

So, should we remove LC_CTYPE?


> > mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
> > d.correlate/d.correlate:export LC_NUMERIC=C
> > d.out.png/d.out.png:export LC_NUMERIC=C
> > d.resize/d.resize:export LC_NUMERIC=C
> > d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
> > i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
> 
> [snip]
> 
> > So it is used quite a lot in scripts.
> 
> In all cases, it's forced to "C", which is the same as it not being
> used. Scripts do this to prevent other programs (e.g. awk) from using
> the user's LC_NUMERIC settings, so that they don't end up passing
> floating-point values which use a comma to programs which expect a
> dot.
> 

I have added it everywhere upon your suggestion:
 http://grass.itc.it/pipermail/grass5/2005-March/017637.html

Maybe a misunderstanding. What do do now?

Markus


Thu, Sep 1 2005 14:08:22    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17174.61225.525054.833411@cerise.gclements.plus.com>
Date Thu, 1 Sep 2005 13:08:09 +0100
To Markus Neteler <neteler@itc.it>
Cc Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
In-Reply-To <20050831202240.GA29081@thuille.itc.it>
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com> <4315692D.4000908@club.worldonline.be> <17174.3153.868078.95493@cerise.gclements.plus.com> <20050831202240.GA29081@thuille.itc.it>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Markus Neteler wrote:

> > > >>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> > > >>-------------------------------------------------------------------------
> > > >>
> > > >>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> > > >>used in my Debian
> > > > 
> > > > 
> > > > GRASS doesn't use any of the locale categories except for LC_MESSAGES
> > > > (if it was built using --with-nls).
> > 
> > Actually, I've just noticed that it also uses LC_CTYPE (again, only if
> > built using --with-nls):
> > 
> > 	revision 1.8
> > 	date: 2004/08/27 11:16:51;  author: markus;  state: Exp;  lines: +1 -0
> > 	added LC_TYPE for japanese
> 
> To what file do you refer?
> 
> Aha:
> 2004-08-27 13:16  markus
> 	* lib/gis/locale.c: added LC_TYPE for japanese

Yep.

> +               setlocale(LC_CTYPE, "");
>                 setlocale(LC_MESSAGES, "");

> > This looks pretty dubious, as it changes the behaviour of many of the
> > <ctype.h> functions (isalpha(), tolower() etc) as well as
> > strcasecmp(), but GRASS' built-in string handling always assumes ASCII
> > (e.g. G_strcasecmp() only considers A-Z/a-z regardless of the locale).
> 
> So, should we remove LC_CTYPE?

I don't know. What was the reason for adding it? I know the commit
message says "for japanese", but which functions need to use the
locale's LC_CTYPE?

The problem with C's locale mechanism is that it's far too coarse. It
was a hack added to deal with the fact that the existing API functions
had US/English conventions hardcoded into them.

Proper localisation requires considering the "audience" for each
individual piece of information. If you're writing something to a file
in a standardised format, it needs to use that format's conventions
regardless of the user's locale. If it's part of the interface text,
it should use the user's locale.

Other cases are less clear; e.g. what conventions (or language) should
be used for history text? The user's locale is a reasonable guess, but
it could well be wrong if their native language isn't the same as the
rest of their team.

> > > mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
> > > d.correlate/d.correlate:export LC_NUMERIC=C
> > > d.out.png/d.out.png:export LC_NUMERIC=C
> > > d.resize/d.resize:export LC_NUMERIC=C
> > > d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
> > > i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
> > 
> > [snip]
> > 
> > > So it is used quite a lot in scripts.
> > 
> > In all cases, it's forced to "C", which is the same as it not being
> > used. Scripts do this to prevent other programs (e.g. awk) from using
> > the user's LC_NUMERIC settings, so that they don't end up passing
> > floating-point values which use a comma to programs which expect a
> > dot.
> 
> I have added it everywhere upon your suggestion:
>  http://grass.itc.it/pipermail/grass5/2005-March/017637.html
> 
> Maybe a misunderstanding. What do do now?

Find out why the setlocale(LC_CTYPE, "") was needed for Japanese. If
it's due to a few specific function calls, we should probably be
doing e.g.:

	setlocale(LC_CTYPE, "");
	do_something(...);
	setlocale(LC_CTYPE, "C");

-- 
Glynn Clements <glynn@gclements.plus.com>


Thu, Sep 1 2005 14:34:10    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Thu, 1 Sep 2005 14:33:59 +0200
From Markus Neteler <neteler@itc.it>
To Glynn Clements <glynn@gclements.plus.com>
Cc Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it, Venkatesh Raghavan <raghavan@media.osaka-cu.ac.jp>
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
Message-ID <20050901123359.GD1798@thuille.itc.it>
Mail-Followup-To Glynn Clements <glynn@gclements.plus.com>, Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker <grass-bugs@intevation.de>, grass5@grass.itc.it, Venkatesh Raghavan <raghavan@media.osaka-cu.ac.jp>
References <20050830124203.30E8B1005A3@lists.intevation.de> <17172.56608.170716.433988@cerise.gclements.plus.com> <4315692D.4000908@club.worldonline.be> <17174.3153.868078.95493@cerise.gclements.plus.com> <20050831202240.GA29081@thuille.itc.it> <17174.61225.525054.833411@cerise.gclements.plus.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <17174.61225.525054.833411@cerise.gclements.plus.com>
User-Agent Mutt/1.4.1i
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
(CC Venkatesh Raghavan, Japan)

[@Venka: we try to find out how to deal with LOCALE. While merging
  the GRASS 5 FOSS4G efforts into GRASS 6, I probably made a
  mistake. May I ask you to help us to figure out below issue?
]

On Thu, Sep 01, 2005 at 01:08:09PM +0100, Glynn Clements wrote:
> 
> Markus Neteler wrote:
> 
> > > > >>this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
> > > > >>-------------------------------------------------------------------------
> > > > >>
> > > > >>Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
> > > > >>used in my Debian
> > > > > 
> > > > > 
> > > > > GRASS doesn't use any of the locale categories except for LC_MESSAGES
> > > > > (if it was built using --with-nls).
> > > 
> > > Actually, I've just noticed that it also uses LC_CTYPE (again, only if
> > > built using --with-nls):
> > > 
> > > 	revision 1.8
> > > 	date: 2004/08/27 11:16:51;  author: markus;  state: Exp;  lines: +1 -0
> > > 	added LC_TYPE for japanese
> > 
> > To what file do you refer?
> > 
> > Aha:
> > 2004-08-27 13:16  markus
> > 	* lib/gis/locale.c: added LC_TYPE for japanese
                                  ^^^^^^ - LC_CTYPE correctly
 
> Yep.
> 
> > +               setlocale(LC_CTYPE, "");
> >                 setlocale(LC_MESSAGES, "");
> 
> > > This looks pretty dubious, as it changes the behaviour of many of the
> > > <ctype.h> functions (isalpha(), tolower() etc) as well as
> > > strcasecmp(), but GRASS' built-in string handling always assumes ASCII
> > > (e.g. G_strcasecmp() only considers A-Z/a-z regardless of the locale).
> > 
> > So, should we remove LC_CTYPE?
> 
> I don't know. What was the reason for adding it? I know the commit
> message says "for japanese", but which functions need to use the
> locale's LC_CTYPE?

Last year I merged the GRASS 5.0/japanese changes from http://www.foss4g.org/
into GRASS 6 to enable Asian fonts etc. In GRASS 5.0 they made this
addition, so I inserted it as well in GRASS 6.
 

> The problem with C's locale mechanism is that it's far too coarse. It
> was a hack added to deal with the fact that the existing API functions
> had US/English conventions hardcoded into them.
> 
> Proper localisation requires considering the "audience" for each
> individual piece of information. If you're writing something to a file
> in a standardised format, it needs to use that format's conventions
> regardless of the user's locale. If it's part of the interface text,
> it should use the user's locale.
> 
> Other cases are less clear; e.g. what conventions (or language) should
> be used for history text? The user's locale is a reasonable guess, but
> it could well be wrong if their native language isn't the same as the
> rest of their team.

Right. E.g., here at IRST we have as many nations as people in our small
group :-) You never know which locale is used.

> > > > mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
> > > > d.correlate/d.correlate:export LC_NUMERIC=C
> > > > d.out.png/d.out.png:export LC_NUMERIC=C
> > > > d.resize/d.resize:export LC_NUMERIC=C
> > > > d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
> > > > i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
> > > 
> > > [snip]
> > > 
> > > > So it is used quite a lot in scripts.
> > > 
> > > In all cases, it's forced to "C", which is the same as it not being
> > > used. Scripts do this to prevent other programs (e.g. awk) from using
> > > the user's LC_NUMERIC settings, so that they don't end up passing
> > > floating-point values which use a comma to programs which expect a
> > > dot.
> > 
> > I have added it everywhere upon your suggestion:
> >  http://grass.itc.it/pipermail/grass5/2005-March/017637.html
> > 
> > Maybe a misunderstanding. What do do now?
> 
> Find out why the setlocale(LC_CTYPE, "") was needed for Japanese.

Will do.

> If it's due to a few specific function calls, we should probably be
> doing e.g.:
> 
> 	setlocale(LC_CTYPE, "");
> 	do_something(...);
> 	setlocale(LC_CTYPE, "C");
> 
> -- 
> Glynn Clements <glynn@gclements.plus.com>

OK, thanks.

Maybe Venkatesh has an idea or can contact the person who inserted
LC_CTYPE in lib/gis/locale.c for japanese.


Markus


Thu, Sep 1 2005 17:30:27    Mail sent by michael.barton@asu.edu  
Return-Path <Michael.Barton@asu.edu>
Delivered-To grass-bugs@lists.intevation.de
Date Thu, 01 Sep 2005 08:29:41 -0700
From Michael Barton <michael.barton@asu.edu>
Subject Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts): LC_NUMERIC setting not used in my Debian
In-reply-to <20050831202240.GA29081@thuille.itc.it>
To Markus Neteler <neteler@itc.it>, Glynn Clements <glynn@gclements.plus.com>
Cc Moritz Lennert <mlennert@club.worldonline.be>, Paolo Cavallini via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
Message-id <BF3C6C75.2B44%michael.barton@asu.edu>
MIME-version 1.0
Content-type text/plain; charset=US-ASCII
Content-transfer-encoding 7bit
User-Agent Microsoft-Entourage/11.1.0.040913
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
My understanding of the main reason for using LC_NUMERIC in d.vect.thematic
and other scripts is that without it, the math chokes on European notation
(i.e., using a comma rather than a period/fullstop to signal decimal
places).

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton



> From: Markus Neteler <neteler@itc.it>
> Date: Wed, 31 Aug 2005 22:22:40 +0200
> To: Glynn Clements <glynn@gclements.plus.com>
> Cc: Moritz Lennert <mlennert@club.worldonline.be>, Request Tracker
> <grass-bugs@intevation.de>, <grass5@grass.itc.it>
> Subject: Re: [GRASS5] [bug #3591] (grass) d.vect.thematic (and other scripts):
> LC_NUMERIC setting not used in my Debian
> 
> On Wed, Aug 31, 2005 at 09:00:17PM +0100, Glynn Clements wrote:
>> 
>> Moritz Lennert wrote:
>> 
>>>>> this bug's URL: http://intevation.de/rt/webrt?serial_num=3591
>>>>> -------------------------------------------------------------------------
>>>>> 
>>>>> Subject: d.vect.thematic (and other scripts): LC_NUMERIC setting not
>>>>> used in my Debian
>>>> 
>>>> 
>>>> GRASS doesn't use any of the locale categories except for LC_MESSAGES
>>>> (if it was built using --with-nls).
>> 
>> Actually, I've just noticed that it also uses LC_CTYPE (again, only if
>> built using --with-nls):
>> 
>> revision 1.8
>> date: 2004/08/27 11:16:51;  author: markus;  state: Exp;  lines: +1 -0
>> added LC_TYPE for japanese
> 
> To what file do you refer?
> 
> Aha:
> 2004-08-27 13:16  markus
> * lib/gis/locale.c: added LC_TYPE for japanese
> 
> The change was actually
> 
> ~/grass61/lib/gis > cvs diff -r 1.7 -r 1.8 locale.c
> Index: locale.c
> ===================================================================
> RCS  /home/grass/grassrepository/grass6/lib/gis/locale.c,v
> retrieving revision 1.7
> retrieving revision 1.8
> diff -u -r1.7 -r1.8
> --- locale.c    13 May 2002 07:02:19 -0000      1.7
> +++ locale.c    27 Aug 2004 11:16:51 -0000      1.8
> @@ -35,6 +35,7 @@
> 
>         if (!initialized)
>         {
> +               setlocale(LC_CTYPE, "");
>                 setlocale(LC_MESSAGES, "");
>                 initialized = 1;
>         }
> 
> 
>> This looks pretty dubious, as it changes the behaviour of many of the
>> <ctype.h> functions (isalpha(), tolower() etc) as well as
>> strcasecmp(), but GRASS' built-in string handling always assumes ASCII
>> (e.g. G_strcasecmp() only considers A-Z/a-z regardless of the locale).
> 
> So, should we remove LC_CTYPE?
> 
> 
>>> mlennert@moritz:~/CVS/GRASS/grass6/scripts$ grep -R LC_NUMERIC *
>>> d.correlate/d.correlate:export LC_NUMERIC=C
>>> d.out.png/d.out.png:export LC_NUMERIC=C
>>> d.resize/d.resize:export LC_NUMERIC=C
>>> d.vect.thematic/d.vect.thematic:export LC_NUMERIC=C
>>> i.fusion.brovey/i.fusion.brovey:export LC_NUMERIC=C
>> 
>> [snip]
>> 
>>> So it is used quite a lot in scripts.
>> 
>> In all cases, it's forced to "C", which is the same as it not being
>> used. Scripts do this to prevent other programs (e.g. awk) from using
>> the user's LC_NUMERIC settings, so that they don't end up passing
>> floating-point values which use a comma to programs which expect a
>> dot.
>> 
> 
> I have added it everywhere upon your suggestion:
>  http://grass.itc.it/pipermail/grass5/2005-March/017637.html
> 
> Maybe a misunderstanding. What do do now?
> 
> Markus
> 


Wed, Oct 18 2006 12:26:56    Status changed to resolved by msieczka  
Comment | Reply | Take | Open

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request