Details Ticket 4056


Comment | Reply | Take | Resolve


Serial Number 4056
Subject db.test fails on sqlite driver
Area grass6
Queue grass
Requestors tutey@o2.pl
Owner none
Status open
Last User Contact Sat Jul 22 14:58:21 2006 (2 yr ago)
Current Priority 30
Final Priority 70
Due No date assigned
Last Action Wed Jul 26 18:19:40 2006 (2 yr ago)
Created Fri Feb 3 13:17:43 2006 (3 yr ago)

Transaction History Ticket 4056


Fri, Feb 3 2006 13:17:43    Request created by guest  
Subject: db.test fails on sqlite driver

Platform: GNU/Linux/x86
grass obtained from: Trento Italy site
grass binary for platform: Compiled from Sources
GRASS Version: CVS 2005.02.03

Using Grass 6.1 cvs 2005.02.03 anbd sqlite 3.3.3 i get a following error:

$ db.test test=test1 
create table grass_test1 (i1 integer, d1 double precision, c1 varchar(20))
EXECUTE: OK
insert into grass_test1 values ( 1, 123.456, 'abcd' )
EXECUTE: OK
insert into grass_test1 values ( 2, null, 'xxx' )
EXECUTE: OK
select * from grass_test1
EXECUTE: OK
RESULT: OK
select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
DBMI-SQLite driver error:
Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500 / 2 and
i1 <> 2  and c1 ~ 'bc'

near "~": syntax error

EXECUTE: ******** ERROR ********
0a1
> abcd
RESULT: ******** ERROR ********
insert into grass_test1 values ( 3, 0.0, '_''_' )
EXECUTE: OK
alter table grass_test1 add column i2 integer
EXECUTE: OK
update grass_test1 set d1 = 18.6, i2 = 987  where i1 = 2
EXECUTE: OK
select * from grass_test1
EXECUTE: OK
RESULT: OK
drop table grass_test1
EXECUTE: OK



My database connection is as follows:

$ db.connect -p 
driver:sqlite
database:$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db
schema:(null)
group:(null)

Maciek
Fri, Feb 17 2006 00:57:01    Mail sent by mneteler  
According to the SQLite docs, '~' should be supported.

Strange.

Markus
Fri, Feb 17 2006 18:04:33    Mail sent by msieczka  
Radim wrote to the grasslist on 2006.02.17:

---
select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
DBMI-SQLite driver error:
Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500 / 2 and
i1 <> 2  and c1 ~ 'bc'
near "~": syntax error
EXECUTE: ******** ERROR ********

'~' is not SQL standard, this is not a bug.
---

So what to do with it? Good to know it is not a bug in sqlite driver, but it
remains a bug in db.test. It may scare naive users and prevent from using
sqlite, too bad. Well, it scarred me. Gosh, I'm naive.

Maciek
Fri, Feb 17 2006 18:15:09    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Fri, 17 Feb 2006 18:15:05 +0100
From Markus Neteler <neteler@itc.it>
To Maciek Sieczka via RT <grass-bugs@intevation.de>
Cc grass5@grass.itc.it
Subject Re: [bug #4056] (grass) db.test fails on sqlite driver
Message-ID <20060217171505.GA9056@bartok.itc.it>
Mail-Followup-To Maciek Sieczka via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
References <20060217170433.CB99B1005DD@lists.intevation.de>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <20060217170433.CB99B1005DD@lists.intevation.de>
User-Agent Mutt/1.4.1i
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Fri, Feb 17, 2006 at 06:04:33PM +0100, Maciek Sieczka via RT wrote:
> Radim wrote to the grasslist on 2006.02.17:
> 
> ---
> select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
> DBMI-SQLite driver error:
> Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500 / 2 and
> i1 <> 2  and c1 ~ 'bc'
> near "~": syntax error
> EXECUTE: ******** ERROR ********
> 
> '~' is not SQL standard, this is not a bug.
> ---
> 

I consider it as bug as SQLite supports '~':

http://www.sqlite.org/lang_expr.html

"Supported unary operators are these:

 -    +    !    ~
"

Question is if it is probably not passed correctly to sqlite?

Markus


Fri, Feb 17 2006 19:11:09    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17398.4534.605160.291685@cerise.gclements.plus.com>
Date Fri, 17 Feb 2006 18:11:02 +0000
To Markus Neteler <neteler@itc.it>
Cc Maciek Sieczka via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] Re: [bug #4056] (grass) db.test fails on sqlite driver
In-Reply-To <20060217171505.GA9056@bartok.itc.it>
References <20060217170433.CB99B1005DD@lists.intevation.de> <20060217171505.GA9056@bartok.itc.it>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Markus Neteler wrote:

> > ---
> > select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
> > DBMI-SQLite driver error:
> > Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500 / 2
and
> > i1 <> 2  and c1 ~ 'bc'
> > near "~": syntax error
> > EXECUTE: ******** ERROR ********
> > 
> > '~' is not SQL standard, this is not a bug.
> > ---
> > 
> 
> I consider it as bug as SQLite supports '~':
> 
> http://www.sqlite.org/lang_expr.html
> 
> "Supported unary operators are these:
> 
>  -    +    !    ~

Note the use of the word "unary". In the above select statement, the ~
is being used as a /binary/ operator, which SQLite doesn't support:

  SQLite understands the following binary operators, in order from
  highest to lowest precedence:
  
      ||
      *    /    %
      +    -
      <<   >>   &    |
      <    <=   >    >=
      =    ==   !=   <>   IN
      AND   
      OR

-- 
Glynn Clements <glynn@gclements.plus.com>


Sat, Feb 18 2006 19:16:36    Mail sent by neteler@itc.it  
Return-Path <neteler@itc.it>
Delivered-To grass-bugs@lists.intevation.de
Date Sat, 18 Feb 2006 18:41:40 +0100
From Markus Neteler <neteler@itc.it>
To Glynn Clements <glynn@gclements.plus.com>
Cc Maciek Sieczka via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] Re: [bug #4056] (grass) db.test fails on sqlite driver
Message-ID <20060218174140.GB14829@bartok.itc.it>
Mail-Followup-To Glynn Clements <glynn@gclements.plus.com>, Maciek Sieczka via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
References <20060217170433.CB99B1005DD@lists.intevation.de> <20060217171505.GA9056@bartok.itc.it> <17398.4534.605160.291685@cerise.gclements.plus.com>
Mime-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Disposition inline
In-Reply-To <17398.4534.605160.291685@cerise.gclements.plus.com>
User-Agent Mutt/1.4.1i
X-PGP-Key http://www.gdf-hannover.de/neteler/markus_gpgkey.asc
X-PGP-Fingerprint D4D5 2F80 120E AD60 E2F6 2297 21B3 D02B E1E7 E789
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
On Fri, Feb 17, 2006 at 06:11:02PM +0000, Glynn Clements wrote:
> 
> Markus Neteler wrote:
> 
> > > ---
> > > select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
> > > DBMI-SQLite driver error:
> > > Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500 /
2 and
> > > i1 <> 2  and c1 ~ 'bc'
> > > near "~": syntax error
> > > EXECUTE: ******** ERROR ********
> > > 
> > > '~' is not SQL standard, this is not a bug.
> > > ---
> > > 
> > 
> > I consider it as bug as SQLite supports '~':
> > 
> > http://www.sqlite.org/lang_expr.html
> > 
> > "Supported unary operators are these:
> > 
> >  -    +    !    ~
> 
> Note the use of the word "unary". In the above select statement, the ~
> is being used as a /binary/ operator, which SQLite doesn't support:
> 
>   SQLite understands the following binary operators, in order from
>   highest to lowest precedence:
>   
>       ||
>       *    /    %
>       +    -
>       <<   >>   &    |
>       <    <=   >    >=
>       =    ==   !=   <>   IN
>       AND   
>       OR
> 

Ah, I see - but what does it mean for db.test?
Add a sqlite test to skip it?

Markus


Sat, Feb 18 2006 20:23:25    Mail sent by glynn@gclements.plus.com  
Return-Path <glynn@gclements.plus.com>
Delivered-To grass-bugs@lists.intevation.de
From Glynn Clements <glynn@gclements.plus.com>
MIME-Version 1.0
Content-Type text/plain; charset=us-ascii
Content-Transfer-Encoding 7bit
Message-ID <17399.29736.293610.25034@cerise.gclements.plus.com>
Date Sat, 18 Feb 2006 19:23:20 +0000
To Markus Neteler <neteler@itc.it>
Cc Maciek Sieczka via RT <grass-bugs@intevation.de>, grass5@grass.itc.it
Subject Re: [GRASS5] Re: [bug #4056] (grass) db.test fails on sqlite driver
In-Reply-To <20060218174140.GB14829@bartok.itc.it>
References <20060217170433.CB99B1005DD@lists.intevation.de> <20060217171505.GA9056@bartok.itc.it> <17398.4534.605160.291685@cerise.gclements.plus.com> <20060218174140.GB14829@bartok.itc.it>
X-Mailer VM 7.07 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid
X-Spam-Status No, hits=-4.9 tagged_above=-999.0 required=3.0 tests=BAYES_00
X-Spam-Level
Markus Neteler wrote:

> > > > select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 ~ 'bc'
> > > > DBMI-SQLite driver error:
> > > > Error in sqlite3_prepare():select c1 from grass_test1 where d1 < 500
/ 2 and
> > > > i1 <> 2  and c1 ~ 'bc'
> > > > near "~": syntax error
> > > > EXECUTE: ******** ERROR ********
> > > > 
> > > > '~' is not SQL standard, this is not a bug.
> > > > ---
> > > > 
> > > 
> > > I consider it as bug as SQLite supports '~':
> > > 
> > > http://www.sqlite.org/lang_expr.html
> > > 
> > > "Supported unary operators are these:
> > > 
> > >  -    +    !    ~
> > 
> > Note the use of the word "unary". In the above select statement, the ~
> > is being used as a /binary/ operator, which SQLite doesn't support:
> > 
> >   SQLite understands the following binary operators, in order from
> >   highest to lowest precedence:
> >   
> >       ||
> >       *    /    %
> >       +    -
> >       <<   >>   &    |
> >       <    <=   >    >=
> >       =    ==   !=   <>   IN
> >       AND   
> >       OR
> > 
> 
> Ah, I see - but what does it mean for db.test?
> Add a sqlite test to skip it?

As it's an extension (rather than standard SQL), the test should be
disabled by default and only run (if at all) for specific back-ends
which are known to support it.

-- 
Glynn Clements <glynn@gclements.plus.com>


Mon, Apr 10 2006 10:22:59    Mail sent by msieczka  
Glynn Clements wrote:

>   SQLite understands the following binary operators, in order from
>   highest to lowest precedence:
>   
>       ||
>       *    /    %
>       +    -
>       <<   >>   &    |
>       <    <=   >    >=
>       =    ==   !=   <>   IN
>       AND   
>       OR

~ 'string' is the same as LIKE '%string%' in SQL.

And on http://www.sqlite.org/lang_expr.html LIKE is mentioned among 'like-op
::', not among binary operators. So ~ "aka" LIKE is not a binary operator it
seems, and it is supported by SQLITE, as LIKE '%string%'.

But, to confussion, Grass's DBF driver doesn't understand LIKE, only ~. On the
other hand, ogr2ogr DBF driver accepts only LIKE (eg. querries on shapefiles),
while it doesn't accept ~ ...

So maybe to standarise things, all database backends in Grass should be
changed to understand LIKE, leaving DBF understanding ~ for legacy?

And db.test then could skip testing for ~, and do the LIKE test instead for
all backends?

My 0,01 yen.

Maciek
Thu, Jul 20 2006 09:15:05    Mail sent by mneteler  
Hi,

the DBF driver now understands (sort of) 'LIKE'.
Unfortunately the behaviour isn't yet PostgreSQL compliant:

PostgreSQL says:
http://www.postgresql.org/docs/8.1/interactive/functions-matching.html#FUNCTIONS-LIKE
'abc' LIKE 'abc'    true
'abc' LIKE 'a%'     true
'abc' LIKE '_b_'    true
'abc' LIKE 'c'      false

GRASS DBF driver now does:

GRASS 6.1.cvs (pat):~ > v.db.select meteo_stations where="station LIKE 'P0'"
cat|station|east|north
93|P01|1653952|5061546
94|P02|1649511|5063172
95|P03|1640450|5098211

GRASS 6.1.cvs (pat):~ > v.db.select meteo_stations where="station LIKE 'P__'"
cat|station|east|north

GRASS 6.1.cvs (pat):~ > v.db.select meteo_stations where="station LIKE 'P%'"
cat|station|east|north

GRASS 6.1.cvs (pat):~ > v.db.select meteo_stations where="station LIKE 'P'"
cat|station|east|north
93|P01|1653952|5061546
94|P02|1649511|5063172
95|P03|1640450|5098211

GRASS 6.1.cvs (pat):~ >

I don't know where the pattern matching is done for the DBF
driver (db/drivers/dbf/?).

Markus
Sat, Jul 22 2006 14:47:55    Comments added by mneteler  
Glynn Clements glynn at gclements.plus.com
Fri Jul 21 12:47:11 CEST 2006

Markus Neteler wrote:
> The DBF driver now understands (sort of) 'LIKE'.
> Unfortunately the behaviour isn't yet PostgreSQL compliant:

> I don't know where the pattern matching is done for the DBF
> driver (db/drivers/dbf/?).

The actual DBF implementation is in db/drivers/dbf/dbfexe.c.

The LIKE operator (SQLP_MTCH) is simply the strstr() function, i.e. it
indicates whether the RHS is a substring of the LHS.

Efficiently implementing correct semantics for patterns which contain
more than one % is non-trivial.

For a single %, the easiest approach is to match from both ends, i.e. 
check that the beginning of the string matches the part before the %,
the end of the string matches the part after the %, and the two parts
don't overlap (the last part is necessary so that 'abc' LIKE 'ab%bc'
is false).

-- 
Glynn Clements
Sat, Jul 22 2006 14:58:21    Mail sent by mneteler  
Back to the original problem:

Glynn suggested to rewrite the test or disable for sqlite.
Disabling is difficult without a change of db.test.


Another option is to rewrite the test:
 select c1 from grass_test1 where d1 < 500 / 2 and i1 <> 2  and c1 LIKE '%bc%'
works for sqlite, but fails on DBF due to the pattern matching limitations
in the DBF driver (see earlier comments in this bug report).

Not sure what to do.

Markus
Wed, Jul 26 2006 18:19:40    User changed to tutey@o2.pl by msieczka  
Comment | Reply | Take | Resolve

You are currently authenticated as guest.
[Show Configuration] [Login as another user]

Users Guide - Mail Commands - Homepage of RequestTracker 1.0.7 - list any request