This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 177163 - Support top level directory import (as supported in Eclipse, JDeveloper)
Summary: Support top level directory import (as supported in Eclipse, JDeveloper)
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Project (show other bugs)
Version: 6.x
Hardware: All All
: P1 normal (vote)
Assignee: Tomas Zezula
URL:
Keywords:
: 54165 (view as bug list)
Depends on: 179427 180577 180818
Blocks:
  Show dependency tree
 
Reported: 2009-11-19 01:12 UTC by madhurtanwani
Modified: 2010-04-20 08:16 UTC (History)
4 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Suggesttion for adding "root" folder button for J2SE projects (44.74 KB, image/jpeg)
2010-02-02 05:57 UTC, madhurtanwani
Details
cancel text (no button) in code scan dialog (30.85 KB, image/png)
2010-02-23 00:05 UTC, madhurtanwani
Details
Mismatch in categorization (109.59 KB, image/png)
2010-02-23 00:15 UTC, madhurtanwani
Details

Note You need to log in before you can comment on or make changes to this bug.
Description madhurtanwani 2009-11-19 01:12:26 UTC
This particular request has been the biggest pain point for me to use Netbeans.

My major use case is to Create new netbeans projects with existing sources. Each of these project sources contains about 10-15 directories, each containing separate dirs for source and test code ("something parallel" to how maven projects are organized). And there are multiple such projects.

Everytime, a project changes (adds a new directory, VCS branch change etc..) I have to sit and import all the directories into the netbeans project.

This is a real pain in the neck and takes out a lot of time to get started on a new VCS branch / new project. 

I tried the same using Eclipse and JDeveloper and both support "import source code root directory" feature. This make the process a real fast and easy for the developer to start doing what he is supposed to do.

Eclipse specifically identifies the directory containing sources and imports those. JDev actually works even better - imports all the directories and "interprets" correct paths as test and src paths.

If I try the same thing with Netbeans (add the root dir in the "source directories"), the IDE flags the packages of the java files as incorrect - e.g. if the code was in code/java/com/madhur/X/Y/Z.java, then the IDE expects the package of the Z.java to be "code.java.com.madhur.X.Y" :(
Comment 1 Tomas Zezula 2009-11-19 01:49:53 UTC
Agree. We already had it and it was lost on some branch.
Scheduling for next release
Comment 2 madhurtanwani 2009-11-19 03:19:00 UTC
(In reply to comment #1)
> Agree. We already had it and it was lost on some branch.
> Scheduling for next release

Awesome. Thanks for the quick triage, Tomas.
Please let me know how if I can help it QAed

Looking forward,
Madhur Tanwani
Comment 3 Petr Jiricka 2009-12-08 09:04:52 UTC
I don't understand - what would be the UI? Would this be a new item in the New Project wizard? Or in the File -> Import Project menu? Or just a new capability in the existing New Project with Existing Sources wizard? Thanks.
Comment 4 madhurtanwani 2009-12-10 10:45:11 UTC
(In reply to comment #3)
> I don't understand - what would be the UI? Would this be a new item in the New
> Project wizard? Or in the File -> Import Project menu? Or just a new capability
> in the existing New Project with Existing Sources wizard? Thanks.

In my opinion "just a new capability in the existing New Project with Existing Sources wizard"
Comment 5 Tomas Zezula 2009-12-11 02:33:55 UTC
I agree with just a new capability in the existing New Project with Existing Sources wizard. Probably with no UI, when you select a sources folder the IDE will look if it's directly the source root or it will find all the source roots under this folder.
Comment 6 Petr Jiricka 2009-12-11 02:44:41 UTC
Ok, thanks. Not sure if an analogous change needs to be done for the web project, cc'ing David.
Comment 7 Tomas Zezula 2009-12-11 02:53:46 UTC
Probably yes, but it will be trivial to put it there. I will do it.
Comment 8 David Konecny 2009-12-13 13:59:58 UTC
Thanks Tomas.
Comment 9 Tomas Zezula 2010-01-13 08:34:25 UTC
*** Bug 54165 has been marked as a duplicate of this bug. ***
Comment 10 Tomas Zezula 2010-01-15 07:45:46 UTC
fixed in jet-main bbd0ba61323d
Comment 11 madhurtanwani 2010-01-15 10:39:28 UTC
Hey Tomas - I would like to contribute by testing out this feature. Please let me know how I can get a build with this fix.
Will the nightly / dev build have this : http://bits.netbeans.org/download/trunk/nightly/latest/ ??
Comment 12 Tomas Zezula 2010-01-15 13:11:11 UTC
Unfortunately it's not so fast. I've integrated it into the team repository (jet-main). The build infrastructure will need to create a build, all the tests have to pass and then it's automatically integrated into the silver repository (where all the team repositories are merged). It's built and if all the test passed it's integrated into the golden which is the night build.
The build infrastructure will add a comment into this issue when it will be integrated.
I will notify you when it will be available.
Comment 13 Quality Engineering 2010-01-24 08:39:26 UTC
Integrated into 'main-golden', will be available in build *201001240200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/bbd0ba61323d
User: Tomas Zezula <tzezula@netbeans.org>
Log: #177163:Support top level directory import
Comment 14 Tomas Zezula 2010-01-25 03:46:37 UTC
The fix was integrated, should be available in next night build.
Comment 15 madhurtanwani 2010-02-01 02:01:43 UTC
Tried out the dev build. Thanks for the feature!! My project imports are taking only a few seconds now :).

I have a couple of suggestions for improvements (please let me know if this is not the right place to add them) : 

1.> On first usage I got this exception (after successful scanning of top level src directory) : http://statistics.netbeans.org/analytics/detail.do?id=164775

2.> The scan identified "all" folders with java files - hence the "Source Package Folders" list got populated with "main" AND "test" classes. 

Ditto for "Test Package Folders", which showed an additional error dialog, stating that the "main" sources were duplicates.

I think the scanner should be smart enough to add the "main" sources to the "Source Packages" list and the "test" sources to the "Test Packages" list.

3.> The functionality is present only in the "New Project From Existing Sources Wizard". Here is what I think : 
 a.> The wizard will not have a provision to add ONLY a single top level folder (without the scan). I mean its possible if I select the top level directory, the scan starts and then I press cancel, but that's a long shot.

 b.> The "Sources" section in the "Project Properties" dialog will not be able to use this feature. This might not be a very frequently used feature on this dialog, but is good to have.

Ideally, I think if there were two buttons - "Add Folder" and "Add Top Level Folder" (for both 'test' and 'main' sources) present in the wizard screen as well as project properties, then the intention and expectation of the user would be clear.


Let me know what the Netbeans team thinks. Thanks!
Comment 16 Tomas Zezula 2010-02-01 07:26:18 UTC
1) The exception...
I will fix it. Seems that the runOfEDT has dead line, so it cannot be used for it.
2) It's rather a heuristic to identify source roots by name. It will work by identifying them using the content, but it will be very slow. I will enable drag and drop among the lists if time permits.
3a) Adding a directory which is not a root to source folders is rather invalid. It will start a scan and generate only errors. The user will need to wait until the end of the scan and fix it anyway. There is probably no need to have 2nd button.
3b) I will add it, but this panel is per project type, I will fix it for j2se and create issue for others.
Comment 17 madhurtanwani 2010-02-01 07:44:38 UTC
(In reply to comment #16)
> 2) It's rather a heuristic to identify source roots by name. It will work by
> identifying them using the content, but it will be very slow. I will enable
> drag and drop among the lists if time permits.

Agree. 
However, I think a simpler logic like file path containing "test" may work in most cases. The drag and drop would be a bonus in that case.

(also a recommendation of another user from the DUP of this ticket : https://netbeans.org/bugzilla/show_bug.cgi?id=54165#c0)


> 3a) Adding a directory which is not a root to source folders is rather invalid.
> It will start a scan and generate only errors. The user will need to wait until
> the end of the scan and fix it anyway. There is probably no need to have 2nd
> button.

I'm not sure if adding a "non root src" folder is invalid. 

A use case is to add the "etc" or "conf" directory that contains proprietary/custom XML/prop configuration files. I use this all the time along with the "Go to file" functionality to manage me configurations.

Hence, I think an action of adding a dir, containing no src files cannot be treated as an errors and should be allowed. Adding two buttons (if at all) would simply clear up the implication.
Comment 18 Tomas Zezula 2010-02-01 11:48:45 UTC
>However, I think a simpler logic like file path containing "test" may work in
>most cases. The drag and drop would be a bonus in that case.
Filtering ".*test.*"  out of source roots (not allowing to add them into source roots, only into test roots) seems OK. But the problem is when someone wants to add it into source roots (eg. non junit tests). The only possibility will be to add them to test and move them to sources. Or add the button as you described above. But this button should be single for both source and test roots. The action will put "tests" into test roots and others into source roots. But I have no Idea where to put it in the wizard panel. :-(

>I'm not sure if adding a "non root src" folder is invalid.
>A use case is to add the "etc" or "conf" directory that contains
>proprietary/custom XML/prop configuration files. I use this all the time along with the "Go to file" >functionality to manage me configurations.
If there is no java file the root is added as it is. I've described a bit different case when someone adds a folder containing java files but the folder is not a root (it's a parent of the source root). In this case all the cross references are wrong as the package name are not valid.
Comment 19 madhurtanwani 2010-02-02 05:56:13 UTC
(In reply to comment #18)
> >However, I think a simpler logic like file path containing "test" may work in
> >most cases. The drag and drop would be a bonus in that case.
> Filtering ".*test.*"  out of source roots (not allowing to add them into source
> roots, only into test roots) seems OK. But the problem is when someone wants to
> add it into source roots (eg. non junit tests). The only possibility will be to
> add them to test and move them to sources. Or add the button as you described
> above. But this button should be single for both source and test roots. The
> action will put "tests" into test roots and others into source roots. But I
> have no Idea where to put it in the wizard panel. :-(

A suggestion is attached.


> 
> >I'm not sure if adding a "non root src" folder is invalid.
> >A use case is to add the "etc" or "conf" directory that contains
> >proprietary/custom XML/prop configuration files. I use this all the time along with the "Go to file" >functionality to manage me configurations.
> If there is no java file the root is added as it is. I've described a bit
> different case when someone adds a folder containing java files but the folder
> is not a root (it's a parent of the source root). In this case all the cross
> references are wrong as the package name are not valid.


Makes sense. Sorry I got the wrong intention.
Comment 20 madhurtanwani 2010-02-02 05:57:20 UTC
Created attachment 93759 [details]
Suggesttion for adding "root" folder button for J2SE projects
Comment 21 Tomas Zezula 2010-02-11 08:35:38 UTC
I've added the DND among the lists.
The change of the dialog proposed by madhurtanwani depends on the UI review. If the UI team approves it I will integrate it.
Comment 22 Quality Engineering 2010-02-11 21:20:52 UTC
Integrated into 'main-golden', will be available in build *201002120200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/c9347bdc4089
User: Tomas Zezula <tzezula@netbeans.org>
Log: #177163:Support top level directory import
Comment 23 Ondrej Langr 2010-02-16 03:35:51 UTC
Informal UI review: 

*) Additional button and label as suggested in the screenshot would make the dialog workflow more complex for a "standard" use-case, therefore strongly not recommended.

*) Agreed with Tomas to try to find a way to auto-detect test files and move them to "Test package folder" list automatically. 

*) Agreed to allow drag&drop between the two lists. (only) When one or more folders have been auto-detected as test (and moved to test list), info message in the bottom of the dialog should be displayed saying: "Some folders have been detected to contain tests. Drag & drop them back to sources if the auto-detection went wrong.

Great usability enhancement, thanks to both reporter and Tomas for your time on this!
Comment 24 Ondrej Langr 2010-02-16 05:39:38 UTC
After having discussed this with doc writers (thanks Brad!), we suggest a new info message: "You can drag&drop both source and test packages from one window to the other."
Comment 25 Tomas Zezula 2010-02-16 11:05:40 UTC
Thanks Ondro for review.
Comment 26 Tomas Zezula 2010-02-17 02:27:02 UTC
Fixed for j2se in jet-main: bf5c3f804925 (as Ondra requested).
For web, j2ee I will create an issue.
Comment 27 Quality Engineering 2010-02-17 21:59:06 UTC
Integrated into 'main-golden', will be available in build *201002180200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/bf5c3f804925
User: Tomas Zezula <tzezula@netbeans.org>
Log: #177163:Support top level directory import
Comment 28 madhurtanwani 2010-02-23 00:04:27 UTC
Tested the dev version (netbeans-trunk-nightly-201002220200-linux.sh).
Thanks for implementing the auto categorizing of folders into test and src - its a great help!! The drag and drop works like a champ too. Thanks!

Few observations : 
1.> When the scan for test & src folders is in progress, a "Cancel" text is seen that's clickable but this does not appear like a button. Is this intentional? The functionality works perfectly, I'm just highlighting the UI element.

(screen shot attached)

2.> There is a small issue in the categorization. Attached screen shot explains this point. Basically, there is a folder named "container-test" and had multiple folders under it. While the "container-test" folder it self got added to the "Test Package Folders", all sub-folders in "container-test" got added to the "Source Package Folders".

I think the expectation would be that test folder and its subfolders should be mapped to "Test Package Folders".
Comment 29 madhurtanwani 2010-02-23 00:05:28 UTC
Created attachment 94407 [details]
cancel text (no button) in code scan dialog
Comment 30 madhurtanwani 2010-02-23 00:15:01 UTC
Created attachment 94408 [details]
Mismatch in categorization
Comment 31 Tomas Zezula 2010-03-22 17:48:45 UTC
>1. When the scan for test & src folders is in progress, a "Cancel" text is
>seen that's clickable but this does not appear like a button. Is this
>intentional? The functionality works perfectly, I'm just highlighting the UI
>element.
Probably yes, I am not creating the UI. I am just using progress API to do it. On Mac there is just a little [x] sign.

>2. There is a small issue in the categorization. Attached screen shot explains
>this point. Basically, there is a folder named "container-test" and had
>multiple folders under it. While the "container-test" folder it self got added
>to the "Test Package Folders", all sub-folders in "container-test" got added to
>the "Source Package Folders".
OK, makes sense. I will use all names in the range (selected_folder, detected_root> to determine the test roots. The start from selected_folder is needed to filter out something like C:\test\MyProject or "test" as a name of the project. Most people will select project folder, so I am making it non inclusive.
Comment 32 Tomas Zezula 2010-03-23 11:02:53 UTC
Fixed "test/src" case, jet-main 5cf9c1ef4ecd
Comment 33 Quality Engineering 2010-03-24 05:20:59 UTC
Integrated into 'main-golden', will be available in build *201003240200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/5cf9c1ef4ecd
User: Tomas Zezula <tzezula@netbeans.org>
Log: #177163:Support top level directory import
Comment 34 madhurtanwani 2010-04-20 08:16:05 UTC
Tested this feature on 
Product Version: NetBeans IDE Dev (Build 201003290201)
Java: 1.6.0_06; Java HotSpot(TM) Client VM 10.0-b22
System: Linux version 2.6.9-89.ELsmp running on i386; UTF-8; en_US (nb)

And it works great!

Thanks everyone!!