This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I had the problem, that the IDE doesn´t stop the scan-process. Some infos: -) Debug-Flag (-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE) was enabled. I´ll attach the message-Log. -) Version: Product Version: NetBeans IDE 6.7 (Build 200906241340) Java: 1.6.0_14; Java HotSpot(TM) Client VM 14.0-b16 System: Windows XP version 5.1 running on x86; Cp1252; de_AT (nb) Userdir: G:\.netbeans\6.7 > Does the scanning ever finish? No. > Was it only the first scan issue, or is the problem met whenever you open the project? None of this. I´ve two projects. I opened the main-project yesterday (start of IDE). The second project I opened 16 Hours later (~1 h before the problem occured) > * If scanning didn't finish, did IDE freeze so that it needs to be killed? No. Exit-Button worked fine (in this case). > * If yes, did the main window repaint (was AWT blocked)? AWT wasn´t blocked. Scanning-Progress Task was running, but the strange thing was, that when I click on the task, I didn´t saw any jar/directory. > * What is your project type? Java Application. > * How big is your project (number of classes, disk size, ...)? Local project: Classes: 4000 Files (~30MB) src-Folder: 2600 Java-Files, 5000 files (java+properties+..) (~60MB) Class-Folder (on Server): Classes (as Library): 42000 (~300MB) Source (as Library-Src): 25000 (~450 MB) > * Does your project use some libraries (how many)? One jar-Files (15MB uncompressed) and the Libraries on Server (see question before). > * Is your project public, so that everyone can get the sources and open it? No. > * Do you use any version control system for your project? CVS (and local NB-Version-Control) > * Is your project located on the network drive? No.
Created attachment 84536 [details] Logfile (-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE)
I guess this is more for Vita. If not please re-assign back.
What is in 'J:\develop\' folder? Why do you have it on compile classpath of the other projects? I mean, I can see a problem in the indexing, which I'll fix, but I'm curious about your project configuration. If J:\develop\ is an output (a build folder) of some project, which it looks it is, would not it be better to add directly this project to the compile classpath of those other two projects (E:/Entwicklung65 and E:\EntwKunden\EntwAlpenmilch)? In general, the IDE can work more efficiently if it works with sources rather than binaries of projects. Thanks
J:/develop isn´t an output-folder. It´s a readonly Classpath-folder (classes in Library). My Library looks like: -) Classes-Card: J:/develop this are the compiled files from an server-side ant-build. I need this binaries for compiling! -) Sources-Card: J:/src
Ok, I assume that J:\src contains sources for J:\develop and that from time to time you run the ant script, which recompiles J:\src to J:\develop? Is that right?
Yes it´s right! The complete way: -) I checkin (CVS) a file The CVS-Server makes a copy on J:/src -) The ant-build compiles the time to J:/develop
Created attachment 84765 [details] After add a Run-Configuration (project-properties). The Scan-Duration was about 5 Minutes. I´ll attach the Profiler-Snapshot-File.
Created attachment 84766 [details] The Profiler-Snapshot-File after 5 Minutes Parsing (J:/develop).
Created attachment 84767 [details] After last Attachement, I restarted the IDE. Then the "Scanning-Projects" - Tasks started again. As Attachement the Profiler-Snapshot-File.
Created attachment 84768 [details] And the logfile after restarting.
There are two things. First, there is a defect in the IDE that the IDE reschedules the scanning task for each file change in J:\develop instead of coalescing the changes. I'll fix that. The other thing is that the IDE does not do an up-to-date check on folders that contain binaries like your J:\develop folder. It would be better if you avoided using a folder and changed your ant script to create a jar file (eg. J:\develop.jar) and changed your library to contain the jar file rather then the folder. This way the IDE could determine straightaway if J:\develop.jar file has been modified since it was last scanned and either rescan it again or do nothing. Btw. scanning of J:\develop seems to vary between 160-220 seconds. I assume that the folder contains fairly high number of classes, which however do not change very often.
I added both the coalescing of file change events coming from a binary folder and also timestamps checks on all files in the folder. Please wait for the fixes to propagate to a publicly downloadable build (you'll get notified through this issue) and test them in your scenario. Changing J:\develop to J:\develop.jar should not be neccessary now. Thanks for you help. http://hg.netbeans.org/jet-main/rev/12d7fb9a87b0 http://hg.netbeans.org/jet-main/rev/bcd6d3236deb
I meant to mark this as fixed.
Integrated into 'main-golden', will be available in build *200907160201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/12d7fb9a87b0 User: Vita Stejskal <vstejskal@netbeans.org> Log: #168328: absorb BinaryWork for the same root
I don´t think that the problem is fixed. But maybe it´s another bug. Today morning "Scanning project" - Tasks takes about 30 minutes. I´ll attach the logfile and the NetBeans-Snapshot. An other thing is, that I have now very often the "Scanning projects" - Task. Please have a look at the IDE-Logfile.
Created attachment 84967 [details] the logfile (after the long indexing).
Created attachment 84968 [details] The Snapshot for the long indexing-task
NB-Version: Product Version: NetBeans IDE Dev (Build 200907170201) Java: 1.6.0_14; Java HotSpot(TM) Client VM 14.0-b16 System: Windows XP version 5.1 running on x86; Cp1252; de_AT (nb) Userdir: G:\.netbeans\dev
Created attachment 87283 [details] Today I was waiting 2 hours, then I killed the IDE. NPS-File while Scanning Projects.
Created attachment 87284 [details] The Logfile.
Tested at the current release: Product Version = NetBeans IDE Dev (Build 200909071948) (#d74394f17bf9) Operating System = Windows XP version 5.1 running on x86 Java; VM; Vendor = 1.6.0_15; Java HotSpot(TM) Client VM 14.1-b02; Sun Microsystems Inc. Runtime = Java(TM) SE Runtime Environment 1.6.0_15-b03
If this is a regular behavior for you, please use -J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE when generating log files. That way we will be able to see what was indexed and hopefully will figure out why. Thanks What is G:\.netbeans\dev? Is it a network drive (eg. Windows share mounted by 'net use' or similar)?
G:\.netbeans\dev is a separate partition for NB - User-Data.
Today I installed the new Version und cleared the Cache-Folder and started NetBeans. After Finishing the First Project-Scan (~20 Minutes), I tried to make some snapshots while opening a big java-source-file. After the third big file, the scanning projects task started. After 30 Minutes I stopped the Snapshot and started a new one. I´ll attach the three files.
Created attachment 87354 [details] Logfile; "-J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE" is activated
Created attachment 87355 [details] The Snapshot while opening a big file and then the first ~45 Minutes.
Created attachment 87356 [details] The second Snapshot. Scanning - Projects before I killed the IDE
It´s reproduceable (for my Installation). -) Clear Cache & log -) Start NetBeans and wait while Scanning-Projects has finished (~20 Minutes) -) Open a big source-file (from the server - directory). -> Endless Scanning. I´ll attach the new files (NB killed after 1.75 hours) .
Created attachment 87366 [details] The nps - File
Created attachment 87367 [details] The logfile.
I can see several things in the log file: 1. first, the initial scan ran, which is ok, the times are rather long Complete indexing of 18 binary roots took: 483515 ms Indexing of: file:/J:/develop/ took: 466719 ms Complete indexing of 3 source roots took: 445938 ms Indexing source root file:/E:/Entwicklung65/test/ [only *.java files!] took 2657ms Indexing source root file:/E:/Entwicklung65/src/ [only *.java files!] took 367781ms 2. but then for some reason TaskManagerImpl.clearCache(TaskManagerImpl.java:309) was called twice and caused (IMO) unnecessary rescans -> CCing saubrecht 3. later on when you opened a file from under J:\src this root was scanned - this I think is what you observed and complain about. I'm not exactly sure why this happened, but it was initiated from the incorrect badges processor. Why do you have J:\develop and J:\src defined as a library? Libraries are typically static things, that do not change and their sources are more informational than the real sources. On the contrary to that you seem to actively develop J:\src, which makes me think that it would be better to have a project defined for it. And declare dependency on this project from E:\Entwicklung65 project. If you define J:\src correctly the IDE will be able to index its sources and detect changes that you make and reindex just the affected source files. Also do J:\develop and J:\src point to a local disk or are they network drives?
My project-situation: E:\Entwicklung65 is my local developer - folder. When I checkin a changed Java-Source-file (CVS), the CVS-Server makes a copy to J:/src. After a small a period, an server-side ant-task starts and compiles the changes from J:/src to J:/develop. J:/src: 25.000 Java-Files (52000 files total) J:/develop: 44.000 Class-Files
task list clears its cache for one of the following reasons: - user invoked 'refresh' action in task list ui - user changed settings for one or more file-based scanners - task list window opens and some file(s) were parsed when the task list window was closed
> - user invoked 'refresh' action in task list ui > - task list window opens and some file(s) were parsed when the task list window was closed I always close the Tasks - Window (performance-problems). > - user changed settings for one or more file-based scanners I didn´t changed any settings. My steps: -) I close NB (Tasks - Window is closed) -) I clear the cache-folder (G:\.netbeans\dev\var) -) I start NB Tasks-Window is closed My main project opens -) Waiting ~15 Minutes until Scanning projects is finished -) I press Ctrl-O and Select a file vom J:/src (network drive) Then the Scanning-Projects starts and newer ends. Summary: -) Tasks-Window is always closed -) I did´t change any settings
Re. "My project-situation" - Thanks for the details. Are J:\src and J:\develop network drives or are they on a local disk? Re. Tasklist, settings and your workflow - I think I understand. Could you please answer why you did you set up J:\src and J:\develop as a library versus setting up a project for them? Thanks
> Are J:\src and J:\develop network drives or are they on a local disk? J:\src and J:\develop are network drives (Samba) > Could you please answer why you did you set up J:\src and J:\develop > as a library versus setting up a project for them? I haven´t a jar-file. Is it really better to have a jar-File, witch changes many times at a day versus a folder where only the really changed files changes there timestamps.
> J:\src and J:\develop are network drives (Samba) That's the problem. Network drives are slow and since indexing is file intense operation indexing files stored on network drives is slow. While I think I understand why J:\develop is a network drive, I don't find any reason for J:\src being a network drive. J:\src contains sources pulled from CVS, right? So it can and IMO should be stored on a local disk. Afterall it's your own copy of the sources from the CVS repository. Nobody else should have an access to that copy. I would recommend the following: 1. Move sources from J:\src to a local disk (say C:\src), leave J:\develop as it is 2. Create a freeform project (all you need is Ant's build script, even an empty one should do), add C:\src as a source package folder in that project and J:\develop as a project output folder. You will also have to set up Java Sources Classpath, which should contain all libraries (jars, folders) needed for compiling C:\src. 3. In your other projects that list J:\develop as a library, remove that and add dependency on the freeform project This will result in indexing C:\src files and not J:\develop or J:\src. Because C:\src is local it should be much faster then indexing files on a network disk. Moreover, once indexed the IDE should incrementally reindex only files affected by your changes. You may have problems in setting up the Java Sources Classpath of your freeform project. I assume that the libraries needed to compile J:\src are now stored on the server and may not be shared on the network. If they are shared good, if not please make them accessible through let's say J:\lib. The IDE will have to index them too and since J:\lib is a network drive this indexing is going to be slow, but because they are libraries and never change (I assume) their indexing will run only once and never again. Please let me know if this is doable and if it works. Thanks
Apart from my suggestions posted earlier there seems to be a problem in IncorrectErrorBadges. It schedules scanning of J:\src (library sources) due to the fact that the IDE can't compile the library sources (expected, because they have no classpath), but source files (when opened in the editor) have no error badges. IMO in this case IEB should leave the sources as they are and not attempt to rectify the situation by forcibly reindexing the root. I'll investigate that further.
Thanks for your recommendation, but there´s one thing: Whe are 15 programmers. When I make C:\src for sources and J:\develop for classes, there´s no mechanism to synchronize this two folders. In most cases C:\src (is old, because an other programmer makes a checkin) are different and J:\develop (is newer, because server-side ant makes a build). The only way to resolve this problem is a) A Network-Drive b) Scheduled local copy (rsync, ..)
> When I make C:\src for sources and J:\develop for classes, there´s no mechanism to synchronize this two folders. I'm sorry, I'm a little bit confused. Correct me if I am wrong please, but C:\src is a local copy of sources from a CVS repository. Everybody in your team has his own local copy (his own C:\src). All of you do regular 'cvs commit' and 'cvs update' in order to submit changes to the CVS repository and pull changes made by others. There is a job somehere on the server that automatically (or maybe periodically) pulls changes from the CVS repository and compiles the sources to J:\develop. (In fact in the suggested setup J:\develop is no longer needed for developing in the IDE. You may still need it for other things though.) Or are you trying to say that all 15 developers work on the same shared copy of sources from your CVS repository on J:\src?
The problem is that there are to many source-files (~25000) to build them local (on a pc). The workflow -) I checkout the sources I want to change -) I make the change in the local source-file -) I build the local changes against the local project, J:\develop and some other libs -) After correct local build, I make a checkin -) After a small period (~1 min), the serverside ant-task starts a checkout to J:/src and compiles the files to J:/build/classes. -) After successful ant-build a rsync-job, syncs the classes to J:/develop
One question else: When I have to build local, I can also checkout everything to my project-source-folder (E:\Entwicklung65\src), or is there any reason (performance?) to use a freeform-project?
> When I have to build local, I can also checkout everything to my project-source-folder (E:\Entwicklung65\src), > or is there any reason (performance?) to use a freeform-project? No, from indexing point of view freeform projects are the same as any other java project.
http://hg.netbeans.org/jet-main/rev/5a02c8ff9454 I fixed IncorrectErrorBadges to ignore library sources. The IDE should not try to index J:\src any more if you open a .java file stored in it. We can't do much about indexing speed on network drives. If you want to continue the discussion regarding your project/libraries setup I'd prefer discussing it on dev@editor.netbeans.org. Thanks
Integrated into 'main-golden', will be available in build *200909181401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/5a02c8ff9454 User: Vita Stejskal <vstejskal@netbeans.org> Log: #168328: IncorrectErrorBadges ignores roots that are not controlled by PathRegistry (ie. have not appeared on any classpath known to the IDE)