I just tried to duplicate the problem using netbeans 3.4, rc 1, jdk
1.4.0 and the current external editor sources on a unix machine. I was
not able to reproduce the problem. The import tool worked fine for me.
Can you give me the netbeans version you were using and which platform?

Not NetBeans 3.4, the trunk (4.0 dev). There was no "4.0"
release version for the external editor component so I marked
it "current" instead of "3.4".

I've tracked this problem to a change in the editor code. Version 1.8
of the JavaFastImport class constructor had the following line added:

  block = Utilities.getSelectionOrIdentifierBlock (target);

This method casts the returned document to a BaseDocument:

 public static int[] getSelectionOrIdentifierBlock(JTextComponent c,
        int offset) throws BadLocationException {
     BaseDocument doc = (BaseDocument)c.getDocument();
     Caret caret = c.getCaret();
     int[] ret;
     if (caret.isSelectionVisible()) {
        ret = new int[] { c.getSelectionStart(),c.getSelectionEnd() }; 
     } else {
        ret = getIdentifierBlock(doc, caret.getDot());
     }
     return ret;
 }

The external editor is using an ExtEdDocument derived from
DefaultStyledDocument and not a BaseDocument. I'm new to the external
editor so I'm still trying to understand what's going on with it but
the following change to the editor works for the external editor:

 public static int[] getSelectionOrIdentifierBlock(JTextComponent c,
        int offset) throws BadLocationException {
     BaseDocument doc = (BaseDocument)c.getDocument();
     Caret caret = c.getCaret();
     int[] ret;
     if (caret.isSelectionVisible() || !(doc instanceof BaseDocument)) {
        ret = new int[] { c.getSelectionStart(),c.getSelectionEnd() }; 
     } else {
        ret = getIdentifierBlock((BaseDocument)doc, caret.getDot());
     }
     return ret;
 }

Or, something that would not break if the document is not a BaseDocument.

Oops! As I was looking at this again, I see that I failed to modify a
line in the changes I proposed. The line:

 BaseDocument doc = (BaseDocument)c.getDocument();

should have been 

Document doc = c.getDocument();

I applied George's fix (including the final change BaseDocument
to Document) to Utilities.java in editor/libsrc in my CVS copy
and the fix works for the external editor.

fixed in [maintrunk]

/cvs/editor/libsrc/org/netbeans/editor/Utilities.java,v  <-- 
Utilities.java
new revision: 1.58; previous revision: 1.57


George, thanks for the fix. I applied it. I am marking the bug as
3.4.1_CANDIDATE.

The fix is not the ideal substitution, because if selection is not
visible and document is not BaseDocument, the returned int[] will be
caret position and not the identifier block. Perhaps, it should be
solved somehow, but the result will not be as exact as
getIdentifierBlock method.

Martin is right. I think the correct fix for this bug 
involves the case when selection is not
visible and document is not BaseDocument. I am including 
an attachment which shows the modifications/implementation 
needed to establish that. George, please integrate those 
changes into Utilities.java, compile and test it. (btw, I 
haven't compiled it). Once you verify that it's working, 
we can ask Martin to integrate those changes again.

Created attachment 7135 [details]
the functions which need to be modified

Ooops. In 
     public static  WordBlock getWordBlock(...)
I used int[] res and int[] ret interchangeably. George, 
please correct that bug when you try the patch. As I said, 
I haven't compiled it, so you may find some other problems 
during compilation.

Created attachment 7149 [details]
The java source file modified.

Created attachment 7150 [details]
diffs from the changes to Utilities.java

Thanks for the help Melih. I made the changes and tested them. They
work for both BaseDocument and ExtEdDocument. I did run into an xemacs
problem during testing where the import tool would not popup 95% of
the time in xemacs. The problem occurred when selecting text followed
by a parenthesis. For example, if I double-clicked on something like
JTextField(8) and "JTextField" was highlighted, the import tool
wouldn't popup most of the time. We think this is an xemacs problem
and will write another issue to cover it. The import tool would work
if you just placed the cursor within JTextField.

Martin,

Would you look over these changes and implement them if you are
satisfied with them. Thanks.

Melih, George,

unfortunately, there is an API change in your patch. The method public
static String getWord(JTextComponent c, int offset) was removed from
Utilities class. Please consider the patch in the attachement I am
including. If there will be no objections and it will work with
external editor, I will integrate it into the maintrunk.

Thank you,
Martin

Created attachment 7155 [details]
diffs from the changes to Utilities.java

Martin,

I don't think your proposed patch includes all of the possible cases 
as I defined them in getWordBlock, e.g having a dot in the selection. 
Why is it a problem to change the getWord function to getWordBlock? 
(By the way, I was the one who initially wrote getWord() function and 
submitted it to Netbeans a while ago.)

If not, we need to write a new getWordBlock function in addition to 
getWord where both of them do the same calculations, but getWord will 
return the selected string and getWordBlock will return the selected 
strings' start and end position. But I still think that  the previous 
patch I submitted was a better solution.

Melih,

we cannot remove the getWord method because it has public access. The
method was commited into the CVS in 08-Nov-01. Since then the NB3.3
has been released and NB3.4 will have this method also. So if one is
using this method it should break the compatibility. 

Could you please explain the problem of the dot in the selection? When
this problem occur? How to reproduce it? If the selection is made then
the method getWordBlock is not called, because SelectionStart and
SelectionEnd values are used. Then I tried to invoke FastImport after
"String|.", where | is cursor position, and word block was the caret
offset. If I invoked FastImport after "java.la|ng.String" then block
positions of "lang" were returned.

BTW I don't quite understand checking for id.indexOf(".") in the "else
if ((id != null) ..." in Utilities.getWord() around lines 470 because
the BreakIterator in javax.swing.text.Utilities.getWordStart() and
getWordEnd() will chop the "." out from the identifiers already so the
only case could be that the "." would be part of something like
"?.#$%.^". Is that expected behavior or I did overlook something?

Martin,

Assume that you have a string in the document:
   myvar = org.netbeans.foo.Bar.getVar();

and you decided to import org.netbeans.foo.Bar. You set the cursor as 
shown:
   myvar = org.netbeans.foo.B|ar.getVar();

Now: 
 the javax.swing.text.Utilities.getWordStart() returns the 
position of "o" in org and javax.swing.text.Utilities.getWordEnd() 
returns the position of "r" in getVar. Basically the word you get 
is "org.netbeans.foo.Bar.getVar". But you actually wanna get "Bar", 
that's why I am checking for "." in getWord function.(btw, that's 
exactly how the deafult editor acts.)

Of course, your question now is, how come your patch code is also 
working without checking for it. Here is my theory after a little big 
of debugging:
The Import tool ALWAYS calls both the getSelectionOrIdentifierBlock 
and getSelectionOrIdentifier every time it is invoked. 
getSelectionOrIdentifierBlock() calls inside your getIdentifier(), but 
getSelectionOrIdentifier() calls the getWord() function. So, in our 
example, it gets the right word "Bar", but it gets the positions of
"o" in org and "r" in getVar. There is inconsistency. But somehow it 
ignores the positions and thus it does the right thing with the word.

First of all, why do you need to call both 
getSelectionOrIdentifierBlock and getSelectionOrIdentifier? What you 
need is just the string. So, I would assume that you:
*either call getSelectionOrIdentifierBlock and then get subtext of doc 
with the positions returned from that function
* or directly call getSelectionOrIdentifier.

Secondly, even though you are calling getSelectionOrIdentifierBlock , 
it seems like it has no effect on the result. What is it I am 
overlooking?

Anyways, please let me know if this info was useful and/or if I am 
completely missing something here. Also, please look into getWord() 
again. There is also an if statement there which trims the selected 
id. You also ignore that in your getIdentifier(JTextComponent...) 
function. Let me know if you have any questions about that code too.

Melih,

The javax.swing.text.Utilities.getWordStart(c, offs) returns the
position of "B" of the word Bar and
javax.swing.text.Utilities.getWordEnd(c, offs) returns the position of
"r" of the word Bar, provided that the offset is caret position in
your example. Then the word you get is "Bar". It works for me as
described on Win 2000 using JDK 1.3. If it works for you differently,
it could be JDK problem. What
version of JDK and what OS do you use?

I really need to call both getSelectionOrIdentifierBlock and
getSelectionOrIdentifier. Please, look at the JavaFastImport class
that calls these two methods.

line 70: Utilities.getSelectionOrIdentifier(target) gets the
expression string. This string is used as a parameter passed to the
finder. Finder returns the list of all matched classes. Then these
classes are listed in the Fast Import Dialog.

line 71: Utilities.getSelectionOrIdentifierBlock (target) gets the
block occupied by the expression the caret is on. This block is used
if user decides to use Fully Qualified Name instead of importing the
class. In this case the block positions are used when the word is
replacing by the fully qualified name.

Anyway, you are absolutely right as for trimming. The word retrieved
using javax.swing.text.Utilities can really contain dot, whitespace,
parenthesis (see George's problem with JTextField(8)). Why and when?
If the cursor is at the end of the word and is followed by the
mentioned char, then this char is understood as a word token. Example:
"String|." word is ".", "String|(" word is "(" etc. Thus I have to
propose an another diff of fix. It should solve all the problems
mentioned. Except this weird problem with the dot in your example :-(
. If the problem with dot will remain, we will need to use also the
part in the condition else if ((id != null) && (id.length() != 0)  &&
(id.indexOf(".") != -1)){ ... (maybe without id!=null ...)

Melih, I am looking forward to your review and comments. Thanks,
Martin

Created attachment 7195 [details]
diffs from the changes to Utilities.java - second attempt ;-)

Martin,

We are using JDK 1.4.0 on Solaris and  getWordStart an getWordEnd are 
behaving as I described before. So it looks like, there is an 
inconsistency between the platforms. That's why I needed that "weird" 
dot trimming in my example. Otherwise when you call 
Utilities.getSelectionOrIdentifier, it won't work correctly. That's 
why I suggest that you keep the code there. I assume that the same 
problem  will persist with Utilities.getSelectionOrIdentifierBlock.

So, George, can you please try Martin's proposed changes with the 
condition    else if ((id != null) && (id.length() != 0)  && 
(id.indexOf(".") != -1))
 for both importing the package and Fully Qualified Name? 

Are we finally on agreement?

Created attachment 7205 [details]
The modified functions

Martin,

Please see the attached file and let me know if you agree with the 
changes:
 getSelectionOrIdentifierBlock -> modified
 getWord -> modified
 getIdentifierBlock(JText....) ->implemented

George, can you please verify if it works?

Here's an end of the week update: I used the following line of code
from the autoupdate module source ModuleUpdater.java. In it, I
experimented using the following line:

UpdateTracking.Version version = track.addNewModuleVersion( ...);

With Martin's changes and using the built in editor, I placed the
cursor in either UpdateTracking or Version and I would get the correct
response with the import tool. I then selected
"UpdateTracking.Version" with the cursor after the end of the block. I
would then press Fully Qualified Name (fqn) in the import tool. I
would get the following resultant line:

org.netbeans.updater.UpdateTracking$Version version =
track.addNewModuleVersion( ...);

The above wouldn't compile due to the "$". I then tried selecting
"Tracking.Ver" but theimport tool did not popup. 

I switched to the xemacs editor and tried the same selections. Putting
the cursor in UpdateTracking or Version yielded the same results as
with the built in editor. I then tried the same selections. Selecting
"UpdateTracking.Version" with cursor at end and pressing fqn gave me:

UpdateTracking.Versionorg.netbeand.updater.UpdateTracking$Versionversion
= track.addNewModuleVersion( ...);

This wasn't inserted well. Selecting "Tracking.ver" popped up the
import tool. Pressing fqn gave:

sun.misc.Version version = track.addNewModuleVersion(...)

I added Melih's updates then used the built in editor, selected
"UpdateTracking.Version", cursor at the end. Pressed fqn in the import
tool and got:

org.netbeans.updater.UpdateTracking$Version version =
track.addNewModuleVersion( ...);

Because of the dollar sign above, this didn't compile. Selecting
"Tracking.ver" with cursor at the end yielded no import tool popup.
Swithching to xemacs, selected "UpdateTracking.Version", cursor at the
end. Pressed fqn in the import tool and got:

org.netbeans.updater.UpdateTracking$Version version =
track.addNewModuleVersion(...);

When I selected "Tracking.Ver", I got the import tool and qfn yielded:

UpdateTracking.sun.misc.Version version =
track.addNewModuleVersion(...);

I think this is expected since the cursor is in the middle of the word
.Version.

To summarize, Melih's and Martin's updates work the same for the built
in editor. But with Melih's added changes, it handles the
"UpdateTrackin.Version" selection better but not the "Tracking.Ver"
slection when used in conjunction with the "Fully Qualified Name"
button of the import tool. 

Melih, George,

I agree with the changes. Thank you for the great cooperation! As for
'$' in FQN innerclasses, please see the issue #26761 I have created.
As for selected fragment of class, it is not   supported behaviour.
That's why "Tracking.Ver" didn't invoke the popup dialog.

Thanks again,
Martin

P.S: I am very surprised that getWordStart an getWordEnd are
inconsistent between the platforms. I would say it is a bug ;-)

fixed in [maintrunk]

/cvs/editor/libsrc/org/netbeans/editor/Utilities.java,v  <-- 
Utilities.java
new revision: 1.59; previous revision: 1.58

http://editor.netbeans.org/source/browse/editor/libsrc/org/netbeans/editor/Utilities.java.diff?r1=1.58&r2=1.59

[S1S-EVAL]

Hi. This issue is marked as 3.4.1_CANDIDATE. It means that it should be
integrated into release341 one branch. The plan at
http://www.netbeans.org/devhome/docs/releases/34/index.html expected beta1 to be
produced on Dec01. That did not happen due to a lot of outstanding not
integrated candidates like this one. 

Would it be possible to spend few minutes by backporting this fix? Thank you in
advance.

integrated into [release341]

/cvs/editor/libsrc/org/netbeans/editor/Utilities.java,v  <-- 
Utilities.java
new revision: 1.57.50.1; previous revision: 1.57

Tor, could you please verify it. Thanks
(I don't use Emacs.)

Setting as verified to keep things tidy. Tor, feel free to reopen if you find
any issue.