This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 87920 - Use case-insensitive entities for Derby
Summary: Use case-insensitive entities for Derby
Status: RESOLVED FIXED
Alias: None
Product: db
Classification: Unclassified
Component: Code (show other bugs)
Version: 5.x
Hardware: All All
: P2 blocker (vote)
Assignee: David Vancouvering
URL: http://wiki.netbeans.org/wiki/view/Da...
Keywords:
: 23287 85449 93700 102621 104227 110093 111324 (view as bug list)
Depends on: 110216 111803
Blocks:
  Show dependency tree
 
Reported: 2006-10-25 03:14 UTC by _ gtzabari
Modified: 2007-10-10 11:13 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description _ gtzabari 2006-10-25 03:14:40 UTC
IBM writes that it is "bad practice" to use case-sensitive names under Derby by
escaping them in quotes, yet this is exactly what Netbeans does by default. Now,
I am not trying to imply that IBM knows what is best for us, but coming from a
MySQL background (since it *is* the most widely deployed database) and there is
no concept of case sensitivity. The first wall I hit when trying to use Derby
under Netbeans is this case-sensitivity.

Please consider using the case-insensitive format by default by removing quotes
around entity names when generating SQL statements. Users can still use quotes
in the UI if they wish.

Source: http://www-128.ibm.com/developerworks/opensource/library/os-ad-trifecta3/
Comment 1 Andrei Badea 2006-10-25 09:11:20 UTC
*** Issue 85449 has been marked as a duplicate of this issue. ***
Comment 2 Andrei Badea 2006-10-25 09:12:30 UTC
You are right. This is usually bad practice on any database (it definitely
causes problems on Oracle too). Will try to implement for 6.0.
Comment 3 Andrei Badea 2007-04-30 13:18:19 UTC
*** Issue 23287 has been marked as a duplicate of this issue. ***
Comment 4 Andrei Badea 2007-05-18 10:53:06 UTC
*** Issue 104227 has been marked as a duplicate of this issue. ***
Comment 5 David Vancouvering 2007-05-18 17:22:19 UTC
I'll say here what I said in the duplicate bug: this is actually a big usability
issue, although I'm not willing to up it to a P2.

DatabaseMetadata provides information to tell you whether a particular vendor
moves everything to lower case, upper case, or supports mixed case.  Perhaps
this could be useful here.

I'm not sure if it can be considered a defect because if we change this, won't
we be breaking existing projects where they have created tables and columns that
are case-sensitive?  In other words, this isn't necessarily something we can fix
after feature freeze...
Comment 6 Andrei Badea 2007-05-18 17:54:36 UTC
The change I intend to make is introduce a checkbox in the Create Table and
Create Column dialogs:

[x] Quote Identifiers

(unchecked by default). So new DDL SQL statements will not quote identifiers by
default, which is what is generally recommended. I don't plan to remove the
quoting in the SELECT statements generated by the View Data action. I guess this
change doesn't break anyone (except those who use invalid characters and spaces
in identifiers and forget to check the check box -- but they will at least get
an error from the database server).
Comment 7 David Vancouvering 2007-05-18 18:27:42 UTC
Thanks, Andrei.

I agree that it doesn't *hurt* to have quotes in the SELECT statements in the
query editor.  But it confuses people (well, it confuses me), because you're not
sure whether you should put quotes in when you modify the command.  This applies
both for the "View Data" option on a table (e.g. SQL editor) and the visual
query editor that is used by visualweb.  

Also, consider the following scenario:

We do CREATE TABLE FOO (int FOO_ID) against Java DB

Java DB, as do most RDBMS, converts FOO to foo and FOO_ID to foo_id

Then we do "View Data" and it generates

SELECT * FROM "APP"."FOO" 

and we get an error "Table 'FOO' does not exist.'

It seems to me we need to be consistent about quoting or not quoting across both
DDL and queries.
Comment 8 Andrei Badea 2007-05-21 13:53:42 UTC
The CREATE TABLE / View Data scenario doesn't fail. Derby, unlike MySQL, but
like Oracle, converts unquoted identifiers to upper case, not lower case.

Quoting in View Data shouldn't at least cause any errors. I guess drivers' impls
of DatabaseMetaData return identifies as they are stored in the database, so
quoting doesn't hurt, it is just superfluous for unuqoted identifiers.

If really needed we might come up with a heuristic for avoiding quoting for some
identifiers, but it might be tricky to get right on all database systems (and
will cause errors when we get it wrong).

FWIW, all DB tools I worked with quoted everything by default and I didn't have
a problem with it. I also think that there is no need to struggle for
consistency at any cost. 

> you're not sure whether you should put quotes in when you modify the command

Well, you should know your database, shouldn't you? You should know whether your
identifiers are quoted or unquoted. That's why you should be able to specify
whether to quote or not in DDL statements, that's definitely needed.
Comment 9 David Vancouvering 2007-05-21 18:19:05 UTC
Let's set aside the question of whether not quoting during DDL causes errors
when quoting during query execution.  My bigger concern is that the user will
see all the quotes everywhere and will feel uneasy about modifying an existing
query without using quotes.  And I am here to testify that putting quotes around
all identifiers is a Big Pain.  

> Well, you should know your database, shouldn't you? You should know 
> whether your identifiers are quoted or unquoted. 
> That's why you should 
> be able to specify whether to quote or not in DDL statements, that's 
> definitely needed.

I'm not sure I understand this.  I have never heard of a database that requires
your identifiers to be quoted.  I think I'm missing your point here.

> If really needed we might come up with a heuristic for avoiding quoting for some
> identifiers, but it might be tricky to get right on all database systems (and
> will cause errors when we get it wrong).

I don't understand this.  If we skip quotes when creating tables and such, then
you should be able to successfully skip quotes when running queries.  If the
user runs DDL with all caps, then we should expect them to use all caps during
queries. 

In terms of making this an option during DDL, the only time I can think of
wanting to do this is if I am migrating an application to a new database, where
for example all your SQL code uses upper case and your new database converts
unquoted identifiers to lower case.  Other than scenarios like this, I don't get
the value/usefulness of quoting during DDL or at any time.

> FWIW, all DB tools I worked with quoted everything by default and I 
> didn't have a problem with it. 

That's funny, this is the only SQL tool I have worked with where quotes are
displayed or used.  I guess for the most part I have used command-line tools,
but I have also used MS Access, SQL Server tools, SQuirreL, and never saw this
quoting thing before.

> I also think that there is no need to struggle for consistency at any cost. 

This is not about consistency (for me anyway), it's about usability.

Comment 10 _ gtzabari 2007-05-21 19:05:11 UTC
I tend to agree that we should be consistent about quoting throughout the entire
process (table creation, query, etc) purely for a usability reason.

I have a question: If one were to always quote everything, would the resulting
behavior be portable across all databases?
Comment 11 Andrei Badea 2007-05-21 21:55:38 UTC
> My bigger concern is that the user will see all the quotes everywhere and
> will feel uneasy about modifying an existing query without using quotes.

Users usually know which kinds of identifiers they have in the database
(database, not database management system). When NetBeans generates:

SELECT * from "MYTABLE";

I'm not confused, because I know how I created the MYTABLE table, and I know
that if I modify the statement I don't need to care about quotes around MYTABLE.
FWIW, I don't think users feel uneasy about this. There is no issue in IZ
requiring that the behavior of View Data be changed. There is an issue about the
behavior of the Create Table dialog though, and that is the real problem.

> I don't understand this.  If we skip quotes when creating tables and such,
> then you should be able to successfully skip quotes when running queries.

And how do you know when to skip quotes and when not to? Suppose there is a
table named "foo bar" (it has a space in the name). View Data has to quote it,
like in

SELECT * FROM "foo bar";

But how should View Data know when to quote and when not to? It should look at
the identifier and decide based on the rules for unquoted identifiers for that
database, which may differ among database systems[1].

> the only time I can think of wanting to do this is if I am migrating an
> application to a new database, where for example all your SQL code uses upper
> case and your new database converts unquoted identifiers to lower case.  

Now it's me who doesn't understand. You do know that you can do

SELECT * FROM MYTABLE;

and have a table named mytable (unquoted) in the database, since unquoted
identifiers are not case-sensitive. Could you please explain the example again?
Why does it matter that you have upper case queries in the application and lower
case identifiers in the database?

> I don't get the value/usefulness of quoting during DDL or at any time.

For example if you want to create a table named "foo bar" or a table containing
national (non-usascii) characters.

> I have used command-line tools, but I have also used MS Access, SQL Server
tools, SQuirreL, and never saw this quoting thing before.

Please check again. MS Access and SQL Server tools (e.g., Enterprise Manager)
put brackets around identifiers in generated SQL (brackets are these tools' way
of quoting). I recall phpMyAdmin did it too (only seen it once, a few years ago)
-- of course, it uses backticks, not quotes.

I don't see a place in SQuirreL which generates SQL, so I can't tell if it
quotes or not. DbVisualizer 5.1.1 doesn't seem to quote at all. I had a table
created with

CREATE TABLE FOO ( FOO_ID INTEGER, "foo bar" INTEGER);

and generated a SELECT for it and the result was

select FOO_ID, 
foo bar from APP.FOO

which, of course, failed to execute.

[1] For View Data we might try a simple heuristic which would avoid quoting for
identifiers containing numbers, us-ascii letters and underscores. A check box is
still needed in the Create Table / Add Column dialogs for reasons explained above.
Comment 12 Andrei Badea 2007-05-21 22:02:05 UTC
gtzabari: I guess it would be, but quoting makes things difficult. For example
the JPA spec doesn't say anything about how persistence providers should handle
quoted identifiers. Also each database has its own way of quoting (quotes on
Oracle and Java DB, brackets on MS SQL and I guess Sybase, backticks on MySQL).
So best to avoid quoted identifiers (in DDL at least).
Comment 13 David Vancouvering 2007-05-31 22:18:27 UTC
I somehow missed your response, Andrei, sorry for not getting back right away.

I think table names with spaces is a real edge case.  Table names with national
characters seems more commonplace.

I like the idea of a heuristic to detect if quotes are needed and do it for the
user then.  I think *in general* quotes should be avoided -- it should be used
for the exceptional case, not the default behavior.

A user also brought up the issue of Grab Structure - that it generates quotes
when it creates the structure.  That may be a separate issue, but something to
be cautious of.  This bit me when I tried to move TRAVEL database from Derby to
PostgreSQL.

David
Comment 14 Andrei Badea 2007-06-01 09:44:03 UTC
The heuristic looks nice, but not trivial. For example you need to handle the
case (on Java DB):

CREATE TABLE "foobar" (...)

View Data can't generate

SELECT * FROM foobar;

for this table, because Java DB would actually execute it as "SELECT * FROM
FOOBAR"), which would fail because the table name is mismatched.
Methods like DatabaseMetaData.storesLowerCaseQuotedIdentifiers() could perhaps
help, not sure. In order to use them you might need to know if the identifier is
quoted or not, and DMD doesn't provide that information. And of course there is
always the risk of drivers with buggy implementation of these methods.
Comment 15 David Vancouvering 2007-06-01 18:59:31 UTC
I don't understand why anyone would quote "foobar" when they create the table,
unless they explicitly asked the tool to use it.  And if they did, wouldn't they
know to quote when they do a query?

What if we had an option "use quotes" or "don't use quotes" that applied to both
DDL and queries?   That way a user doesn't generally have to think about it,
either way.

In those instances when a query fails, the user could enable quotes and try
again.  In the SQL editor, if we get an error saying table, schema, index,
column, etc., doesn't exist, we could suggest in our error message that the user
try again with quotes.

What do you think?
Comment 16 Andrei Badea 2007-06-03 10:27:56 UTC
> wouldn't they know to quote when they do a query?

Yes, they would. But they would expect View Data to quote too, wouldn't they?

> What if we had an option "use quotes" or "don't use quotes" that applied to
> both DDL and queries?

I don't see what users would gain from such a behavior. The IDE should not
generate queries that don't work.

What happened to the argument about other tools not quoting? Have you checked
MS SQL Server and MS Access?
Comment 17 _ gtzabari 2007-06-03 13:06:35 UTC
What about this?

1) SQL command fails due to unknown table, column, etc
2) Netbeans searches for possible matches with different casing
3) "Did you mean <name>? If so please quote the variable name such as <example>."
Comment 18 Andrei Badea 2007-07-10 17:06:34 UTC
I don't see a simple and reliable way to detect that the statement failed due to an unknown object (table, etc.).
Perhaps the exception's SQL code could help here. But anyway, I don't think that this would be the right thing to do.
View Data should just work. The proposed scenario is like NetBeans saying "I tried to display your table, but I'm too
stupid to pick its exact name, so I failed. Could you please pick the right name for me?".
Comment 19 David Vancouvering 2007-07-10 20:11:43 UTC
I agree, I don't think guessing works - view data should just work.

Is this something we can fix as a defect?  Doesn't it require UI changes?
Comment 20 Andrei Badea 2007-07-11 12:49:24 UTC
Yes, it requires UI changes, but we can fix it as a defect.
Comment 21 Andrei Badea 2007-07-11 17:36:49 UTC
This was closed by mistake, so reopening.
Comment 22 John Baker 2007-07-12 09:39:48 UTC
*** Issue 93700 has been marked as a duplicate of this issue. ***
Comment 23 oysteing 2007-07-12 10:22:29 UTC
I might be missing something here, but when generating SQL would it not be straight-forward to use a metadata lookup to
determine whether a name needs quoting?  If a table is registered in the dictionary with all upper case quoting should
not be used, otherwise, it needs to be used.
Comment 24 dwanvik 2007-07-12 13:18:02 UTC
oysteing> I might be missing something here, but when generating SQL would it
oysteing> not be straight-forward to use a metadata lookup to determine whether
oysteing> a name needs quoting?  If a table is registered in the dictionary with
oysteing> all upper case quoting should not be used, otherwise, it needs to be
oysteing> used.

This would work if the database is SQL compliant:

According the the SQL standard (ISO/IEC 9075-2:2003), comparison shall
occur after converting characters in undelimited identifiers from lowercase
and capital case to uppercase, if available, cf. section 5.2 syntax rules
23 onward. Java DB/Derby stores unquoted delimiters in uppercase as is thus
compliant with this requirement.

However, if the database stores unquoted identifiers in lower case
internally, or otherwise uses some other scheme than that mandated by
the standard, it would not necessarily work.

I think the IDE could determine when to quote from the DatabaseMetaData
methods available:
                 
    
supportsMixedCaseIdentifiers() - Retrieves whether this database treats
    mixed case *UNQUOTED* SQL identifiers as case SENSITIVE and as a result
    stores them in mixed case.
storesLowerCaseIdentifiers() - Retrieves whether this database treats mixed
    case *UNQUOTED* SQL identifiers as case INSENSITIVE and stores them in
    lower case.
storesMixedCaseIdentifiers() - Retrieves whether this database treats mixed
    case *UNQUOTED* SQL identifiers as case INSENSITIVE and stores them in
    mixed case.
storesUpperCaseIdentifiers() - Retrieves whether this database treats mixed
    case *UNQUOTED* SQL identifiers as case INSENSITIVE and stores them in
    upper case.

supportsMixedCaseQuotedIdentifiers() - Retrieves whether this database
    treats mixed case *QUOTED* SQL identifiers as case SENSITIVE and as a
    result stores them in mixed case.
storesMixedCaseQuotedIdentifiers() - Retrieves whether this database treats
    mixed case *QUOTED* SQL identifiers as case INSENSITIVE and stores them
    in mixed case.
storesUpperCaseQuotedIdentifiers() - Retrieves whether this database treats
    mixed case *QUOTED* SQL identifiers as case INSENSITIVE and stores them
    in upper case.
storesLowerCaseQuotedIdentifiers() - Retrieves whether this database treats
    mixed case *QUOTED* SQL identifiers as case INSENSITIVE and stores them
    in lower case.


Dag
Comment 25 David Vancouvering 2007-07-12 16:55:54 UTC
Wow, great to have you Derby-ites on board!  Thanks for the very helpful advice.  How did you hear about this issue?

I think Dag's suggestions are very interesting and useful.  Let me think about it.  I'm planning on writing a one-pager
on how I'm going to solve this to make sure we're all on the same (one) page.
Comment 26 David Vancouvering 2007-07-12 20:28:12 UTC
I believe this issue needs to cover not just not quoting when we do DDL, but also
automatically handle existing tables that were created with quoted identifiers. 
Otherwise we're going to have issues with backward compatibility.

Let me try to summarize what I'm thinking:

- No quoting during DDL unless requested by user through checkbox
- By default, don't quote when generating queries
- Support tables/indexes/etc where identifiers are quoted through some attempt to auto-detect when identifiers need to
be quoted
- Useful error message to user to try quoting when we get the error "column/table/schema XXX does not exist"

Regarding the heuristic to decide whether quoting is needed:

I liked Dag/Oystein's suggestion to detect whether the identifier is upper or
lower case, and compare this with what the database says is its
default case.  If it doesn't match, then quote when running the query.
 That covers the *majority* of situations, IMHO.

However, this doesn't handle the less common scenario of spaces or
national characters.  We can  look for spaces and quote if there are
spaces.  National characters are going to take more thought - anyone
know of a way to detect if there are national characters in a string?

Are there any other situations where quoted identifiers may be needed?
 I'm sure there are...

I am thinking for the cases where we don't quote when we should have,
the error message should include the suggestion that
identifiers may need to be quoted, so users aren't completely
mystified when they see "table name/column name 'foo' does not exist'.

This will also be very useful when the user is writing their own hand-coded queries
and are confused by the error messages they are getting (this happens a lot).

I recognize this means detecting this particular error, which is of
course different for each db, and the error codes are generally
undocumented and not a stable interface.  But I still think it would
be worth it for us to check for error codes with the most popular
databases, and deliver a helpful message.  If the vendor changes the
error code (unlikely, but possible), the only negative impact is that
users will no longer get the helpful error message, or will get it for the
wrong error message.  I think it's worth this slight risk to reduce a
significant point of confusion to our users.
Comment 27 David Vancouvering 2007-07-13 17:46:11 UTC
After further thought, I've come to conclude that Andrei has it right: because DatabaseMetadta returns the identifiers
in the proper case, we only need to stop quoting by default when we execute DDL.  This will allow hand-written queries,
without quotes, to work without problems, which has been our biggest issue.  Sorry I didn't get this sooner, Andrei.

I will provide a checkbox in the Create Table and Create Column dialogs to allow users to turn on quotes for the entire
table or for a particular column.  This allows the user to choose quoting if they do odd things like use national
characters, spaces, or want case sensitivity. 

The issue of our own generated queries using quotes I think is a secondary concern, and it may be that what we're doing
now is fine.  I don't like seeing quotes everywhere, but it does guarantee that the generated query runs regardless of
what the user has done when creating the table, and that's a good thing.

gtzabari: is this acceptable to you?  Others?
Comment 28 _ gtzabari 2007-07-13 21:32:32 UTC
I didn't understand the checkbox in Create Table and Create Column. What happens if I check it?

Does an enabled checkbox mean use quoting only for this specific Create Table/Column command? Or does it mean "Use
quoting for this entity even past the create command"? What happens if I check "use quotes" for create table but then
give it a normal name like "FOO"? Will it quote "FOO" from no on in future queries or will it use DatabaseMetadata and
decide not to use quoting because there is no need?
Comment 29 David Vancouvering 2007-07-14 03:57:52 UTC
If you check the "Use quotes" box for the create table dialog, it will quote all identifiers for *that* create table
command.  I can also have it stay checked for future create table commands, as a nice thing to do.

If you check "Use quote" for the create column dialog, it only quotes that column for that create table command.

Meanwhile, whenever we generate queries (e.g. in the "Show Data" command or for automatically bound rowsets in
visualweb, we will quote *every* identifier regardless of what was checked when the table was created.  This works
because the identifier is returned with the right case in DatabaseMetaData, which we use to generate the SQL.

As a second phase, we may remove quotes in "View Data" and other generated commands seen by users, to make it more
readable, but this engenders the risk that the command may fail if the identifier has spaces.  But let's not focus on
this step right now...

David
Comment 30 _ gtzabari 2007-07-15 08:48:52 UTC
Thanks for clarifying David. Sounds good to me :)
Comment 31 Andrei Badea 2007-07-16 20:46:14 UTC
Makes sense to me too.
Comment 32 David Vancouvering 2007-07-18 21:50:27 UTC
I've created a short spec on what we're going to do here.  Please take a look and review

http://wiki.netbeans.org/wiki/view/DatabaseQuotingIdentifiers
Comment 33 David Vancouvering 2007-07-21 00:20:45 UTC
Hi, all.  After doing some experimentation with this, I think the approach I had proposed is not really the best
approach.  This approach was to provide an option in the various dialogs where we run DDL.

The trick is, in some places, there is *no* dialog where DDL is run.  In particular, you can rename a table or a view
just by modifying the name in the explorer, and DDL is run.  Delete table/view/procedure is another place where you just
pick an action and no DDL is run (although in this case for now we could just always quote). (note: why does the Delete
action say "Delete" twice, e.g. "Delete  Delete"?)

Also, the number of places where we would need to put this action are growing.  Besides the ones I've already listed in
the spec, we would also need it for Add Index, Add Column to Index.

So I would like to propose that rather than trying to scatter checkboxes all over, we make this a property/preference on
the connection.  Generally if someone wants to quote, they will be consistent about it, and not change it from command
to command and operation to operation.

If, after implementing this, we get requests to make it optional on a per command basis, we can always add this checkbox
to dialogs later.

Comments most appreciated.



Comment 34 Andrei Badea 2007-07-23 13:07:31 UTC
Another approach could be to never quote identifiers entered by the user in the UI and let the user enter the quotes
explicitly if needed. E.g. the user can enter `foo` (including the brackets) as the table name on MySQL and we will generate

CREATE TABLE `foo` ...

We would still probably need to do something with Recreate Table.
Comment 35 David Vancouvering 2007-07-23 18:38:09 UTC
I like Andrei's suggestion.  I was talking to a database user on Friday, and she said "why would anyone ever want to
quote their identifiers?"  I gave reasons - spaces in the name, international characters, but as I spoke, my general
feeling was that quoting identifiers is by far the exception rather than the rule.

If I can provide a little more detail into what I think Andrei is suggesting:

- When defining a SQL object (table, index, view, column, etc.), we don't quote the identifier.
- Don't provide checkboxes to quote identifiers.  Instead, the user can quote the identifier if they want it quoted.


One concern I have is, I'm not sure how to do this when typing in a table name to rename a table, but I'll experiment

Sound good?

I have some ideas to help with quoting during "View Data" and other commands that refer to existing identifiers, but I
will address that as a separate step.
Comment 36 David Vancouvering 2007-07-23 18:53:09 UTC
One exception: as Andrei mentioned, we'll need to give users an option for quoting when they run "Grab
Structure"/"Recreate Table", as this interface does not provide a place for users to quote identifiers.  
Comment 37 _ jimdavidson 2007-07-23 20:59:40 UTC
So, the idea is that the user quotes the individual table/column names as needed?  I like it.

I like it for two reasons:

* It allows individual identifiers to be quoted, as opposed to all-or-none. 

* It also uses unquoted identifiers by default, which is important for usability IMO.

I'm not familiar with all the places where we specify identifiers; it sounds like there are a couple of cases that need
some extra functionality since the user doesn't have the option of typing quotes.

Now, this would not change the need to quote identifiers when we generate queries (right)?  Even if the schemas that we
generate don't use delimiters, the user may have an existing schema that does.  So, that's the other part of this issue
still to be resolved.
Comment 38 David Vancouvering 2007-07-23 21:34:37 UTC
Yes, you have it right by all counts, Jim.  This addresses quoting SQL objects (tables, indeces, columns) that the
command is *creating* for the first time (e.g. CREATE TABLE, ALTER TABLE, CREATE INDEX, etc.)

When referring to *existing* objects, we will continue quoting identifiers, for now.  As a future step, I may provide an
option to allow users to turn off quoting of existing object identifiers, if they just don't need it in their
environment (which as I understand it is the common case).
Comment 39 _ gtzabari 2007-07-24 00:39:55 UTC
When dealing with existing objects, why can't you just use the DatabaseMetaData API to detect the correct casing and
only quote the name if absolutely necessary?
Comment 40 David Vancouvering 2007-07-24 00:59:39 UTC
gtzabari: As Andrei mentioned, the metadata returns column names in the correct casing already.  If it were just the
issue of casing, we wouldn't need to quote.  The problem is if the column name contains spaces or international
characters.  In those cases, we have to quote.

Granted, these are *very* rare cases.  That's why I'm suggesting if we make it an option (perhaps on the Databases node)
to enable quoting on queries, then by default we can eliminate quoting for queries as well as DDL -- because for most
users quoting is just not needed, but we need to make sure we can quote when we need to.

David
Comment 41 _ gtzabari 2007-07-24 01:12:52 UTC
Okay but I don't understand why you need a checkbox option for that. If the user invokes an operation on a table name
that contains spaces or international characters you can automatically figure out that you must quote in the underlying
operation, can't you? Am I missing your use-case?
Comment 42 David Vancouvering 2007-07-24 01:27:04 UTC
I could determine it for spaces.  But there are also tabs, and other special characters.  How can I guarantee I cover
all the cases?  I don't even know how you can determine that a character "won't work" for a given database.  Then
there's the kooks who use reserved keywords for table names - those would need to be quoted too.

So, I guess I am concerned there may be many cases where quoting is needed, and I'm not going to catch them all. I think
there's a reason so many tools quote when they generate queries...

Remember, what I am proposing is that quoting would be turned off by default, and we just have a property you can set if
you're in the 2% of the user population who needs quotes.

Comment 43 David Vancouvering 2007-07-24 17:43:09 UTC
*** Issue 110093 has been marked as a duplicate of this issue. ***
Comment 44 Andrei Badea 2007-07-30 10:38:11 UTC
*** Issue 111324 has been marked as a duplicate of this issue. ***
Comment 45 David Vancouvering 2007-08-01 19:24:56 UTC
I just found what looks like a very useful DatabaseMetaData method: getExtraNameCharacters()

http://java.sun.com/j2se/1.4.2/docs/api/java/sql/DatabaseMetaData.html#getExtraNameCharacters()

'Retrieves all the "extra" characters that can be used in unquoted identifier names (those beyond a-z, A-Z, 0-9 and _).'

So I am thinking, in phase 2 when we consider not quoting identifiers in user-visible queries, we could use the
following algorithm:

- Try getting extra characters
- If this is supported, then add these characters to a-z, A-Z, 0-9 and _
- Then, if the character is not in the full set of supported nonquoted characters, quote the identifier

This should cover 95% of the cases

Then we can as a final step add a user-configurable option to quote all identifiers.  But perhaps this could be added
only if it is requested by the users.  What do you think?

Comment 46 Andrei Badea 2007-08-02 14:56:59 UTC
Makes sense to me. I'm not against the "always quote" option, on the contrary. But I don't know where to put it. It
really doesn't belong in the popup menu of the Databases node. Probably we could add it later if needed.
Comment 47 David Vancouvering 2007-08-02 18:38:15 UTC
Raising this to a P2, as a lot of folks have complained about this, and there are a number of bugs related to this that
would get resolved or minimized.
Comment 48 David Vancouvering 2007-08-29 20:21:26 UTC
All DDL uses of the libsrc directory appear to have been refactored and unit tested.  Now I will remove quoting from the
DDL and make sure unit tests continue to pass (as well as do some manual testing).
Comment 49 David Vancouvering 2007-08-29 20:30:24 UTC
I closed the wrong issue, I meant to close the dependent bug for the unit test work I was doing.
Comment 50 David Vancouvering 2007-08-31 18:49:46 UTC
We no longer quote when creating database objects.  This *includes* GrabTable/RecreateTable.  

Next step would be to remove quoting of database objects we are referring to for SQL that is editable by users.  I'll
open that under a different issue.

Checking in src/org/netbeans/modules/db/explorer/infos/TableListNodeInfo.java;
/cvs/db/src/org/netbeans/modules/db/explorer/infos/TableListNodeInfo.java,v  <--  TableListNodeInfo.java
new revision: 1.32; previous revision: 1.31
done
Checking in src/org/netbeans/modules/db/explorer/infos/ViewListNodeInfo.java;
/cvs/db/src/org/netbeans/modules/db/explorer/infos/ViewListNodeInfo.java,v  <--  ViewListNodeInfo.java
new revision: 1.29; previous revision: 1.28
done
Checking in libsrc/org/netbeans/lib/ddl/impl/RemoveColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/RemoveColumn.java,v  <--  RemoveColumn.java
new revision: 1.10; previous revision: 1.9
done
Checking in libsrc/org/netbeans/lib/ddl/impl/CreateProcedure.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/CreateProcedure.java,v  <--  CreateProcedure.java
new revision: 1.13; previous revision: 1.12
done
Checking in libsrc/org/netbeans/lib/ddl/impl/AbstractCommand.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/AbstractCommand.java,v  <--  AbstractCommand.java
new revision: 1.36; previous revision: 1.35
done
Checking in libsrc/org/netbeans/lib/ddl/impl/AddColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/AddColumn.java,v  <--  AddColumn.java
new revision: 1.13; previous revision: 1.12
done
Checking in libsrc/org/netbeans/lib/ddl/impl/ColumnCommand.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/ColumnCommand.java,v  <--  ColumnCommand.java
new revision: 1.13; previous revision: 1.12
done
Checking in libsrc/org/netbeans/lib/ddl/impl/CreateTable.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/CreateTable.java,v  <--  CreateTable.java
new revision: 1.14; previous revision: 1.13
done
Checking in libsrc/org/netbeans/lib/ddl/impl/RenameColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/RenameColumn.java,v  <--  RenameColumn.java
new revision: 1.10; previous revision: 1.9
done
Checking in libsrc/org/netbeans/lib/ddl/impl/CreateIndex.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/CreateIndex.java,v  <--  CreateIndex.java
new revision: 1.14; previous revision: 1.13
done
Checking in libsrc/org/netbeans/lib/ddl/impl/CreateView.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/CreateView.java,v  <--  CreateView.java
new revision: 1.12; previous revision: 1.11
done
Checking in libsrc/org/netbeans/lib/ddl/impl/TableColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/TableColumn.java,v  <--  TableColumn.java
new revision: 1.22; previous revision: 1.21
done
Checking in libsrc/org/netbeans/lib/ddl/impl/ColumnListCommand.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/ColumnListCommand.java,v  <--  ColumnListCommand.java
new revision: 1.13; previous revision: 1.12
done
Checking in libsrc/org/netbeans/lib/ddl/impl/Specification.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/Specification.java,v  <--  Specification.java
new revision: 1.23; previous revision: 1.22
done
Checking in libsrc/org/netbeans/lib/ddl/impl/TriggerEvent.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/TriggerEvent.java,v  <--  TriggerEvent.java
new revision: 1.12; previous revision: 1.11
done
Checking in libsrc/org/netbeans/lib/ddl/impl/DriverSpecification.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/DriverSpecification.java,v  <--  DriverSpecification.java
new revision: 1.24; previous revision: 1.23
done
Checking in libsrc/org/netbeans/lib/ddl/impl/ModifyColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/ModifyColumn.java,v  <--  ModifyColumn.java
new revision: 1.10; previous revision: 1.9
done
Checking in libsrc/org/netbeans/lib/ddl/impl/AbstractTableColumn.java;
/cvs/db/libsrc/org/netbeans/lib/ddl/impl/AbstractTableColumn.java,v  <--  AbstractTableColumn.java
new revision: 1.14; previous revision: 1.13
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/dlg/AddIndexDDLTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/dlg/AddIndexDDLTest.java,v  <--  AddIndexDDLTest.java
new revision: 1.3; previous revision: 1.2
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/dlg/AddTableColumnDDLTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/dlg/AddTableColumnDDLTest.java,v  <--  AddTableColumnDDLTest.java
new revision: 1.5; previous revision: 1.4
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/dlg/AddViewDDLTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/dlg/AddViewDDLTest.java,v  <--  AddViewDDLTest.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/org/netbeans/modules/db/explorer/dlg/CreateTableDDL.java;
/cvs/db/src/org/netbeans/modules/db/explorer/dlg/CreateTableDDL.java,v  <--  CreateTableDDL.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/org/netbeans/modules/db/explorer/dlg/AddTableColumnDDL.java;
/cvs/db/src/org/netbeans/modules/db/explorer/dlg/AddTableColumnDDL.java,v  <--  AddTableColumnDDL.java
new revision: 1.3; previous revision: 1.2
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/actions/AddToIndexDDLTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/actions/AddToIndexDDLTest.java,v  <--  AddToIndexDDLTest.java
new revision: 1.3; previous revision: 1.2
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/actions/GrabTableHelperTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/actions/GrabTableHelperTest.java,v  <--  GrabTableHelperTest.java
new revision: 1.3; previous revision: 1.2
done
Checking in test/unit/src/org/netbeans/modules/db/util/DBTestBase.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/util/DBTestBase.java,v  <--  DBTestBase.java
new revision: 1.2; previous revision: 1.1
done
Checking in test/unit/src/org/netbeans/modules/db/explorer/infos/DDLHelperTest.java;
/cvs/db/test/unit/src/org/netbeans/modules/db/explorer/infos/DDLHelperTest.java,v  <--  DDLHelperTest.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/org/netbeans/modules/db/explorer/actions/AddToIndexDDL.java;
/cvs/db/src/org/netbeans/modules/db/explorer/actions/AddToIndexDDL.java,v  <--  AddToIndexDDL.java
new revision: 1.2; previous revision: 1.1
done
Comment 51 Andrei Badea 2007-10-10 11:13:41 UTC
*** Issue 102621 has been marked as a duplicate of this issue. ***