This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
This is a complicated issue, so I'll start with the source of the problem. I activated moving modified files to the default VCS group and docked the VCS groups window into the bottom of the explorer view. Then I used the internal compiler to build all sources in my mounted VSS directory. After cleaning the old classes, the internal compiler starts compiling. During compilation, suddenly all sources start to appear in the default VCS group (although they are not modified) and VCS commands (LIST_STATUS_READER, REFRESH) start to run, several at a time. After a very long time, when the compiler finally is finished, NetBeans continues to consume 100% CPU time and VCS commands continue to run (very slowly). Some VCS commands you can kill, some not. This situation is covered in the attachment threaddump2.txt and reproducable. It does not happen with fastjavac or the external compiler. Now, although NetBeans consumes all CPU time, you can still exit cleanly. But the next time you start NetBeans, after the splash screen tells "opening main window", immediately VCS commands start to run. One time (not reproducable) during my tests, NetBeans stopped starting at this point with 0% CPU time usage every time I tried to start it. This situation is covered in threaddump1.txt. I could repair (make it start again) NetBeans by deleting all VCS cache directories. Using NetBeans 3.3.1, JDK 1.4, Windows 2000, VSS 6.0c.
Created attachment 4785 [details] Hang at startup.
Created attachment 4786 [details] After compiling.
Created attachment 4803 [details] Reading wrapped thread dumps is a pain (at least for me). The first threaddump unwrapped attached.
ok, I've examined the problem. the internal compile uses Fileobject.outputStream to write the classes. We add the file to the group when writing the fileobject.. One source of slowing down the IDE response is that each file is added as task to the request processor, creating a queue of possibly thousands tasks. (is true for vcsgeneric module only - Martin what were the reasons for that? I didn't experience any probs without it in javacvs) Then the commands seem to be added to get the file's status. This happens on file or directory base, Martin? In eather case this becomes another CPU eater since during the compile and adding to the groups, all directories try to refresh simultaneously. Within the group nodes another thread adds the files to the group. That is done in a bit optimized manner, since the key-refresh reschedules itself when there are many changes at once.. I've tried to play with the fileobject's unimportant flag but it doesn't seem to help. The java module didn't label the newly written class files as unimportant, thus we recognize them as important and need to add them. Curretly I don't see a solution for the problem from our side. :( If the external compiler sets the fileobjects as unimportant, then everything could work.
The first thread dump is IMHO a problem of Java Parser. It's deadlocked with AWT: AWT has locked Mutex, and waits for JavaParser (seems strange to me, that AWT has to wait for a task). Java Parser then waits for Mutex to be released... This probably needs a separate issue for Java module. Thanks Milos for the evaluation. The second thread really is not locked, but consumes a lot of CPU. The RequestProcessor in VcsFileSystem.fileChanged() was added (probably) because of findResource() call caused deadlock under some circumstances (I do not remember with what the collision was). To resolve the problem I would have to remember the modified files in some buffer and then send it to Groups at once. What do you think, Milos? > Then the commands seem to be added to get the file's status. This > happens on file or directory base, Martin? You mean annotateName() ? It returns the status immediately, but runs folder refresh on background if the folder is not in the disk cache.
I have created issue #20800 for the first thread dump.
To resolve the problem I would have to remember the modified files in some buffer and then send it to Groups at once. What do you think, Milos? yes, that could fix the filesystem's part of the slowdown. However I think the performance will still drop.. for example try to open the default group after you've compiled the vcscore module.. it will take pretty much time.. I think that's general node's problem that we can't fix, nodes are not tuned for performance with 100-1000s of subnodes. And the groups are not tuned for automatic actions on such large scale either, *sigh* but currently I don't see what we could do about it..
I just want to emphasize, that this adding of all sources to the default group during compilation should be seen itself as a bug in my opinion. None of the source were modified. Also, this only happens with the internal compiler, not with the external or fastjavac. Sorry about the wrapped threaddumps.
Thomas, I agree that adding the files is wrong. Currently I see it as design flaw of vcs groups. I didn't know about this behaviour and the only way to "fix" the problem within the vcs groups code would be to remove the auto addition property. I don't like that though. As I already mentioned, the only feasible solution I see now is to have the internal compiler set the unimportant flag on the files.. I'll try to negotiate that with the java module people.
Fixed in VcsFileSystem. Also marking as a candidate for 3.3.2. Fixed in dev build Feb 26.
Here is the diff for command-line CVS: http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java.diff?r1=1.162&r2=1.163
fixed in javacvs filesystem as well.. diff is: http://www.netbeans.org/source/browse/javacvs/src/org/netbeans/modules/cvsclient/NbJavaCvsFileSystem.java.diff?r1=text&tr1=1.76&r2=text&tr2=1.77&f=h
post-mortem note: The compiler cannot mark the files "unimportant" by itself. It is only a producer of those files and does not know, or care, what will happen with them. They can be marked as (un)important only by Loaders during the object type recognition - so e.g. Java loader will mark the .class as unimportant, while I'm would bet that Clazz loader will make them important.
thanks Svata. BTW: we do DataObject.find at a later moment in the process, when initially evaluating the bug I've placed the isImportant check before the dataobject creation, the right way is to place it after it.. I completely overlooked that..
Do you think it is okay in latest development builds of NetBeans 3.4, Thomas ?
It's better, but not OK (just tested build 200203110100). I used my 3.3.1 installation as test base. This time, not all classes are being added to the default VCS group, only one local (not in VSS) class and two inner classes (don't know, why only these, because there are more local and inner classes in the project). NetBeans does not consume CPU power anymore, after everything is finally finished. But still, the VCS threads are starting to run LIST_STATUS_READER during internal compilation and they are still running long after compilation has finished. So, fixed is: Not all classes are being added and CPU usage is back to normal at the end. Not fixed: Some classes are still being added and VCS commandos still start to run during internal compilation. I wanted to do a 2nd compilation test, but I can't, because this time the internal compiler stops compiling after a while, because one class file is read only, but that's not true. The class file does not even exist at that moment. As before, the external compiler works perfectly.
Okay, I am glad that the major problem was fixed and therefore verifying this issue. On the other hand if you Thomas feel like the rest is pretty annoying for you, file special bug with appropriate priority and describe accordingly. Thanks.
I am not sure that the main problem is fully fixed because of the fact that the VCS threads again started running during internal compilation. I thought that was a major part of the problem. And it also seems that this fix caused the internal compiler to generate the new error message about the write protected class file. This is not in 3.3.1.
Maybe I found out the source of the remaining problem: In 3.3.1 and in the current dev build, the internal compiler generates not only .class files but also .class~ files and these seem to be responsible for the running of the VCS threads. Should I file a new issue on this? The external compiler does not do this.
Opened a new issue based on the last statement: issue #21876.
Fixed in orion_fcs branch.
Resolved for 3.4.x or earlier, no new info since then -> closing.