Thu, Jun 27 2002
11:42:42
|
|
Request created by guest
|
|
Subject: Non-portable snprintf() used in several programs
Platform: Irix
grass obtained from: CVS
grass binary for platform: Compiled from Sources
GRASS Version: CVSreleasebranch_26_april_2002_5_0_0 grass checked out 20020618
I have already mentioned this problem on the grass5 mailing list, but was just
reading the descriptions of types of bugs and realised this is a release-critical
one, since it wasn't present in earlier versions and has only appeared in the
last six months or so since I started using GRASS. Therefore I am obliging by
reporting it. There are probably more important things GRASS developers could
be working on fixing but I think attention to detail and overall code quality
and portability is very important.
The following programs contain references to the snprintf() function and therefore
fail to compile on the IRIX 6.2 system I am using.
Compilation error in module: src/libes/dbmi/clients (ignored)
Compilation error in module: src/display/d.db (ignored)
Compilation error in module: src/display/d.what.db (ignored)
Compilation error in module: src/display/d.what.vect (ignored)
Compilation error in module: src/mapdev/v.db.reclass (ignored)
Compilation error in module: src/mapdev/v.in.mif (ignored)
Compilation error in module: src/mapdev/v.in.shape (ignored)
Compilation error in module: src/mapdev/v.out.shape (ignored)
Compilation error in module: src/mapdev/v.to.db (ignored)
Compilation error in module: src/raster/r.in.shape (ignored)
Compilation error in module: src/sites/s.in.dbf (ignored)
Compilation error in module: src.contrib/GMSL/NVIZ2.2 (ignored)
(This list may not be exhaustive; I didn't attempt to compile every module.)
It was also mentioned in the mailing list before that even on systems that do
provide snprintf(), the return value when an error occurs is not standardised,
so that is another reason for not relying on the system library version.
Paul Kelly |
|
Tue, Oct 8 2002
10:07:38
|
|
Mail sent by mneteler
|
|
The full list is:
find . -type f -name "*.c" -exec grep -l snprintf {} \;
./libes/dbmi/drivers/odbc/cursor.c
./libes/dbmi/drivers/odbc/db.c
./libes/dbmi/drivers/odbc/describe.c
./libes/dbmi/drivers/odbc/error.c
./libes/dbmi/drivers/odbc/execute.c
./libes/dbmi/drivers/odbc/select.c
./libes/dbmi/drivers/odbc/table.c
./libes/dbmi/lib/dbmscap.c
./libes/dbmi/lib/reclass.c
./libes/gis/extracted-funcs/_make_toplevel.c
./libes/gis/user_config.c
./mapdev/v.in.mif/main.c
./mapdev/v.in.mif/write_lines.c
./mapdev/v.in.shape/lines.c
./mapdev/v.in.shape/vmap_import.c
./mapdev/v.in.shape/test/arctest1.c
./mapdev/v.to.db/update.c
./display/d.db/main.c
./display/d.what.db/attr.c
./display/d.what.db/main.c
./display/d.what.vect/cmd/attr.c
./raster/r.in.shape/main.c
./sites/s.in.dbf/dump.c
Is the problem fixed for you?
Markus
|
|
Tue, Oct 8 2002
16:49:35
|
|
Mail sent by paul-grass@stjohnspoint.co.uk
|
|
Return-Path |
<paul-grass@stjohnspoint.co.uk>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Tue, 8 Oct 2002 15:49:22 +0100 (BST)
|
From |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
X-X-Sender |
paulk@agrippa.ukshells.co.uk
|
To |
Markus Neteler via RT <grass-bugs@intevation.de>
|
Subject |
Re: [bug #1140] (grass) Transaction (mneteler)
|
In-Reply-To |
<20021008080738.0139C13AB4@lists.intevation.de>
|
Message-ID |
<Pine.LNX.4.44.0210081511210.13091-100000@agrippa.ukshells.co.uk>
|
MIME-Version |
1.0
|
Content-Type |
TEXT/PLAIN; charset=US-ASCII
|
X-Spam-Status |
No, hits=-3.8 required=5.0 tests=IN_REP_TO,SPAM_PHRASE_00_01,USER_AGENT_PINE version=2.41
|
X-Spam-Level |
|
Hello Markus
>
> Is the problem fixed for you?
>
> Markus
>
Well, I have a workaround for it, which is to compile a 3rd-party snprintf
library (obtained from http://www.ijs.si/software/snprintf/ ) and add it
to the end of the GISLIB line in src/CMD/generic/make.mid
But I don't really regard the bug as cleared. There were some interesting
messages on the mailing list and I thought somebody was going to do
something about it, e.g.
http://grass.itc.it/pipermail/grass5/2002-May/002936.html
http://grass.itc.it/pipermail/grass5/2002-April/002733.html
So it is not urgent but I think you should keep it there with lower
priority and someone will sort it out eventually.
Paul
|
|
Sun, May 8 2005
08:12:03
|
|
Taken by bdouglas
|
|
Tue, Jul 4 2006
14:38:13
|
|
Mail sent by mneteler
|
|
Hi,
the snprintf() is used as of today in:
./raster3d/r3.in.ascii/main.c
./db/drivers/dbf/dbfexe.c
./raster/r.support/front/front.c
./raster/r.support/front/check.c
./raster/r.support/front/run.c
./raster/r.support/modhead/check_un.c
./raster/r.support/modhead/modhead.c
./raster/r.support/modhead/ask_format.c
./lib/init/clean_temp.c
./lib/db/dbmi_client/select.c
./lib/gis/user_config.c
./lib/vector/dglib/examples/components.c
I suggest to implement G_snprintf() and use that
function everywhere. Then we can fix in a single place
if needed.
Markus
|
|
Tue, Jul 4 2006
15:09:02
|
|
Mail sent by paul-grass@stjohnspoint.co.uk
|
|
Return-Path |
<paul-grass@stjohnspoint.co.uk>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Tue, 4 Jul 2006 14:08:48 +0100 (BST)
|
From |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
X-X-Sender |
paulk@agrippa.ukshells.co.uk
|
To |
Markus Neteler via RT <grass-bugs@intevation.de>
|
Cc |
grass-dev@grass.itc.it
|
Subject |
Re: [bug #1140] (grass) Non-portable snprintf() used in several programs
|
In-Reply-To |
<20060704123813.F3AE41005CA@lists.intevation.de>
|
Message-ID |
<Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk>
|
References |
<20060704123813.F3AE41005CA@lists.intevation.de>
|
MIME-Version |
1.0
|
Content-Type |
TEXT/PLAIN; charset=US-ASCII; format=flowed
|
X-SA-Do-Not-Run |
Yes
|
X-SA-Exim-Connect-IP |
217.10.143.90
|
X-SA-Exim-Mail-From |
paul-grass@stjohnspoint.co.uk
|
X-SA-Exim-Scanned |
No (on mail.ukshells.net); SAEximRunCond expanded to false
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.741 tagged_above=-999 required=4 tests=[AWL=1.259, BAYES_00=-5]
|
X-Spam-Level |
|
Hello Markus!
I contend that most of those recent introductions of snprintf() probably
don't need it. E.g. the first one (raster3d/r3.in.acii/main.c) can
calculate the length of the string to be malloc'ed using strlen() so can
guarantee to leave enough space. The second one (db/drivers/dbf/dbfexe.c)
can use a fixed field width specifier for the %d format string to fix the
length also - again predictable.
So I will work through those and see if I can change them not to use
snprintf(). Started it now. I feel if there's something where the length
of the resulting string really cannot be predicted, then we should use
G_asprintf() because it's there. It does seem to be a matter of personal
preference and philosophy though! But I think as Glynn has said before,
most of the places snprintf() is used the return value is not checked. So
if an overflow was prevented, and the string was not the expected value,
the program would just ignore it!
So to summarise - G_snprintf() would be useful to have to easily fix
the places that snprintf() was erroneously introduced - so you could put
it in, but I think we should discourage its use in favour of calculating
how long the string will be and allocating enough memory.
Paul
On Tue, 4 Jul 2006, Markus Neteler via RT wrote:
> Hi,
>
>
>
> the snprintf() is used as of today in:
>
>
>
> ./raster3d/r3.in.ascii/main.c
>
> ./db/drivers/dbf/dbfexe.c
>
> ./raster/r.support/front/front.c
>
> ./raster/r.support/front/check.c
>
> ./raster/r.support/front/run.c
>
> ./raster/r.support/modhead/check_un.c
>
> ./raster/r.support/modhead/modhead.c
>
> ./raster/r.support/modhead/ask_format.c
>
> ./lib/init/clean_temp.c
>
> ./lib/db/dbmi_client/select.c
>
> ./lib/gis/user_config.c
>
> ./lib/vector/dglib/examples/components.c
>
>
>
> I suggest to implement G_snprintf() and use that
>
> function everywhere. Then we can fix in a single place
>
> if needed.
>
>
>
> Markus
>
>
>
> -------------------------------------------- Managed by Request Tracker
>
|
|
Tue, Jul 4 2006
15:24:01
|
|
Mail sent by neteler@itc.it
|
|
Return-Path |
<neteler@itc.it>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Date |
Tue, 4 Jul 2006 15:23:57 +0200
|
From |
Markus Neteler <neteler@itc.it>
|
To |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
Cc |
Markus Neteler via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
|
Subject |
Re: [GRASS-dev] Re: [bug #1140] (grass) Non-portable snprintf() used in several programs
|
Message-ID |
<20060704132357.GE21351@bartok.itc.it>
|
Mail-Followup-To |
Paul Kelly <paul-grass@stjohnspoint.co.uk>, Markus Neteler via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
|
References |
<20060704123813.F3AE41005CA@lists.intevation.de> <Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk>
|
Mime-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Disposition |
inline
|
In-Reply-To |
<Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk>
|
X-PGP-Key |
http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
|
X-PGP-Fingerprint |
D4D5 2F80 120E AD60 E2F6 2297 21B3 D02B E1E7 E789
|
User-Agent |
Mutt/1.5.11
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.361 tagged_above=-999 required=4 tests=[AWL=1.373, BAYES_00=-5, FORGED_RCVD_HELO=0.266]
|
X-Spam-Level |
|
Hello Paul,
On Tue, Jul 04, 2006 at 02:08:48PM +0100, Paul Kelly wrote:
> Hello Markus!
> I contend that most of those recent introductions of snprintf() probably
> don't need it. E.g. the first one (raster3d/r3.in.acii/main.c) can
> calculate the length of the string to be malloc'ed using strlen() so can
> guarantee to leave enough space. The second one (db/drivers/dbf/dbfexe.c)
> can use a fixed field width specifier for the %d format string to fix the
> length also - again predictable.
>
> So I will work through those and see if I can change them not to use
> snprintf(). Started it now. I feel if there's something where the length
> of the resulting string really cannot be predicted, then we should use
> G_asprintf() because it's there. It does seem to be a matter of personal
> preference and philosophy though! But I think as Glynn has said before,
> most of the places snprintf() is used the return value is not checked. So
> if an overflow was prevented, and the string was not the expected value,
> the program would just ignore it!
>
> So to summarise - G_snprintf() would be useful to have to easily fix
> the places that snprintf() was erroneously introduced - so you could put
> it in, but I think we should discourage its use in favour of calculating
> how long the string will be and allocating enough memory.
thanks for working on it.
I have submitted now the G_snprintf() wrapper function to
lib/gis/snprintf.c
with the discouragement included on top.
You may use it now if snprintf() cannot be avoided.
Markus
|
|
Tue, Jul 4 2006
21:50:18
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17578.50807.818514.870528@cerise.gclements.plus.com>
|
Date |
Tue, 4 Jul 2006 20:50:15 +0100
|
To |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
Cc |
Markus Neteler via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
|
Subject |
Re: [GRASS-dev] Re: [bug #1140] (grass) Non-portable snprintf() used in several programs
|
In-Reply-To |
<Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk>
|
References |
<20060704123813.F3AE41005CA@lists.intevation.de> <Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk>
|
X-Mailer |
VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.261 tagged_above=-999 required=4 tests=[AWL=1.473, BAYES_00=-5, FORGED_RCVD_HELO=0.266]
|
X-Spam-Level |
|
Paul Kelly wrote:
> I contend that most of those recent introductions of snprintf() probably
> don't need it. E.g. the first one (raster3d/r3.in.acii/main.c) can
> calculate the length of the string to be malloc'ed using strlen() so can
> guarantee to leave enough space. The second one (db/drivers/dbf/dbfexe.c)
> can use a fixed field width specifier for the %d format string to fix the
> length also - again predictable.
Field width specifiers only control the minimum width of a field (i.e.
the amount of padding or the number of leading zeroes); a field can
still be larger than the specified width.
For %d, it's reasonable to assume a 32-bit integer, which means that
it can never be more than 11 characters wide (10 digits, with a
leading minus if negative).
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Tue, Jul 4 2006
22:07:27
|
|
Mail sent by paul-grass@stjohnspoint.co.uk
|
|
Return-Path |
<paul-grass@stjohnspoint.co.uk>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
Message-ID |
<44AACA6D.8030200@stjohnspoint.co.uk>
|
Date |
Tue, 04 Jul 2006 21:07:09 +0100
|
From |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
User-Agent |
Mozilla Thunderbird 1.0.6 (Windows/20050716)
|
X-Accept-Language |
en-us, en
|
MIME-Version |
1.0
|
To |
Glynn Clements <glynn@gclements.plus.com>
|
Cc |
Markus Neteler via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
|
Subject |
Re: [GRASS-dev] Re: [bug #1140] (grass) Non-portable snprintf() used in several programs
|
References |
<20060704123813.F3AE41005CA@lists.intevation.de> <Pine.LNX.4.62.0607041401560.25038@agrippa.ukshells.co.uk> <17578.50807.818514.870528@cerise.gclements.plus.com>
|
In-Reply-To |
<17578.50807.818514.870528@cerise.gclements.plus.com>
|
Content-Type |
text/plain; charset=ISO-8859-1; format=flowed
|
Content-Transfer-Encoding |
7bit
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.333 tagged_above=-999 required=4 tests=[AWL=1.667, BAYES_00=-5]
|
X-Spam-Level |
|
Glynn Clements wrote:
> Paul Kelly wrote:
>
>
>>I contend that most of those recent introductions of snprintf() probably
>>don't need it. E.g. the first one (raster3d/r3.in.acii/main.c) can
>>calculate the length of the string to be malloc'ed using strlen() so can
>>guarantee to leave enough space. The second one (db/drivers/dbf/dbfexe.c)
>>can use a fixed field width specifier for the %d format string to fix the
>>length also - again predictable.
>
>
> Field width specifiers only control the minimum width of a field (i.e.
> the amount of padding or the number of leading zeroes); a field can
> still be larger than the specified width.
Ah yes I realised that when I actually went to implement my proposed change.
> For %d, it's reasonable to assume a 32-bit integer, which means that
> it can never be more than 11 characters wide (10 digits, with a
> leading minus if negative).
Well in this case the existing code was allocating 32 bytes of space and
then using snprintf so I think it should be safe to change the snprintf
a simple sprintf in this case (db/drivers/dbf/dbfexe.c).
|
|
Thu, Jul 6 2006
05:40:49
|
|
Mail sent by glynn@gclements.plus.com
|
|
Return-Path |
<glynn@gclements.plus.com>
|
Delivered-To |
grass-bugs@lists.intevation.de
|
From |
Glynn Clements <glynn@gclements.plus.com>
|
MIME-Version |
1.0
|
Content-Type |
text/plain; charset=us-ascii
|
Content-Transfer-Encoding |
7bit
|
Message-ID |
<17580.34325.326800.333470@cerise.gclements.plus.com>
|
Date |
Thu, 6 Jul 2006 04:40:05 +0100
|
To |
Paul Kelly <paul-grass@stjohnspoint.co.uk>
|
Cc |
Markus Neteler via RT <grass-bugs@intevation.de>, grass-dev@grass.itc.it
|
Subject |
Re: [GRASS-dev] Re: [bug #1140] (grass) Non-portable snprintf() used in several programs
|
In-Reply-To |
<Pine.LNX.4.62.0607051206030.25038@agrippa.ukshells.co.uk>
|
References |
<20060704123813.F3AE41005CA@lists.intevation.de> <Pine.LNX.4.62.0607051206030.25038@agrippa.ukshells.co.uk>
|
X-Mailer |
VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
|
X-Virus-Scanned |
by amavisd-new at intevation.de
|
X-Spam-Status |
No, hits=-3.307 tagged_above=-999 required=4 tests=[AWL=1.427, BAYES_00=-5, FORGED_RCVD_HELO=0.266]
|
X-Spam-Level |
|
Paul Kelly wrote:
> > the snprintf() is used as of today in:
> >
> > ./raster3d/r3.in.ascii/main.c
> > ./db/drivers/dbf/dbfexe.c
> > ./raster/r.support/front/front.c
> > ./raster/r.support/front/check.c
> > ./raster/r.support/front/run.c
> > ./raster/r.support/modhead/check_un.c
> > ./raster/r.support/modhead/modhead.c
> > ./raster/r.support/modhead/ask_format.c
> > ./lib/init/clean_temp.c
> > ./lib/db/dbmi_client/select.c
> > ./lib/gis/user_config.c
> > ./lib/vector/dglib/examples/components.c
>
> Please see attached patch to replace use of snprintf in all those
> modules. I am posting it to the list first, hopefully for some comments,
> before committing.
In cases where the strings being inserted are themselves supposed to
have a maximum length (e.g. map names), I would just check that the
strings don't exceed their maximum length and use a fixed-size buffer.
> Index: lib/init/clean_temp.c
> ===================================================================
> RCS file: /grassrepository/grass6/lib/init/clean_temp.c,v
> retrieving revision 2.5
> diff -u -r2.5 clean_temp.c
> --- lib/init/clean_temp.c 24 Jun 2006 19:33:03 -0000 2.5
> +++ lib/init/clean_temp.c 5 Jul 2006 11:02:49 -0000
> @@ -19,7 +19,6 @@
> *
> * 2006: Rewritten for GRASS 6 by roberto Flor, ITC-irst
> *
> - * TODO (?): Implement snprintf() as G_snprintf() for portability
> **************************************************************/
>
> #include <limits.h>
> @@ -39,7 +38,6 @@
>
> void clean_dir(const char *pathname,uid_t uid,pid_t pid,time_t now,int max_age)
> {
> - char buf[BUF_MAX];
> DIR *curdir;
> struct dirent *cur_entry;
> struct stat info;
> @@ -53,11 +51,16 @@
> /* loop over current dir */
> while ((cur_entry = readdir(curdir)))
> {
> - if ((G_strcasecmp(cur_entry->d_name,".") == 0 )|| (G_strcasecmp(cur_entry->d_name,"..")==0))
> + static char *buf = NULL;
> +
> + if ((G_strcasecmp(cur_entry->d_name,".") == 0 )|| (G_strcasecmp(cur_entry->d_name,"..")==0))
> continue; /* Skip dir and parent dir entries */
> -
> - if ( (pathlen=snprintf(buf,BUF_MAX,"%s/%s",pathname,cur_entry->d_name)) >=
BUF_MAX)
> - G_fatal_error("clean_temp: exceeded maximum pathname length %d, got %d,
should'nt happen",BUF_MAX,pathlen);
> +
> + if(buf)
> + G_free(buf);
> +
> + buf = G_malloc(strlen(pathname) + strlen(cur_entry->d_name) + 2);
> + sprintf(buf, "%s/%s", pathname, cur_entry->d_name);
In this situation, I would start with a reasonably-sized buffer (e.g.
4Kb), then use G_realloc() to enlarge it if necessary.
Both malloc() and free() can be quite computationally expensive, and
repeated use can cause heap fragmentation, particularly if the size
tends to increase with time (as the previously free()d block is too
small to be used by the subsequent malloc()).
The same applies to any kind of loop: re-using a buffer is more
efficient (in terms of both CPU and memory usage) than allocating a
new buffer every time.
--
Glynn Clements <glynn@gclements.plus.com>
|
|
Fri, Jul 14 2006
18:42:46
|
|
Mail sent by mneteler
|
|
Hi,
I have changed
./raster3d/r3.in.ascii/main.c
./raster/r.support/front/front.c
./raster/r.support/front/check.c
./raster/r.support/front/run.c
./raster/r.support/modhead/check_un.c
./raster/r.support/modhead/modhead.c
./raster/r.support/modhead/ask_format.c
./lib/init/clean_temp.c
./lib/db/dbmi_client/select.c
from snprintf() to G_snprintf() to address portability problems
in a single place.
Markus |
|