This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | Using internal compiler plus VCS groups causes problems | ||
---|---|---|---|
Product: | obsolete | Reporter: | tboerkel <tboerkel> |
Component: | vcscore | Assignee: | issues@obsolete <issues> |
Status: | CLOSED FIXED | ||
Severity: | blocker | CC: | jchalupa |
Priority: | P2 | ||
Version: | 3.x | ||
Hardware: | PC | ||
OS: | Windows ME/2000 | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: |
Hang at startup.
After compiling. Reading wrapped thread dumps is a pain (at least for me). The first threaddump unwrapped attached. |
Description
tboerkel
2002-02-22 09:04:59 UTC
Created attachment 4785 [details]
Hang at startup.
Created attachment 4786 [details]
After compiling.
Created attachment 4803 [details]
Reading wrapped thread dumps is a pain (at least for me). The first threaddump unwrapped attached.
ok, I've examined the problem. the internal compile uses Fileobject.outputStream to write the classes. We add the file to the group when writing the fileobject.. One source of slowing down the IDE response is that each file is added as task to the request processor, creating a queue of possibly thousands tasks. (is true for vcsgeneric module only - Martin what were the reasons for that? I didn't experience any probs without it in javacvs) Then the commands seem to be added to get the file's status. This happens on file or directory base, Martin? In eather case this becomes another CPU eater since during the compile and adding to the groups, all directories try to refresh simultaneously. Within the group nodes another thread adds the files to the group. That is done in a bit optimized manner, since the key-refresh reschedules itself when there are many changes at once.. I've tried to play with the fileobject's unimportant flag but it doesn't seem to help. The java module didn't label the newly written class files as unimportant, thus we recognize them as important and need to add them. Curretly I don't see a solution for the problem from our side. :( If the external compiler sets the fileobjects as unimportant, then everything could work. The first thread dump is IMHO a problem of Java Parser. It's
deadlocked with AWT:
AWT has locked Mutex, and waits for JavaParser (seems strange to me,
that AWT has to wait for a task).
Java Parser then waits for Mutex to be released...
This probably needs a separate issue for Java module.
Thanks Milos for the evaluation. The second thread really is not
locked, but consumes a lot of CPU.
The RequestProcessor in VcsFileSystem.fileChanged() was added
(probably) because of findResource() call caused deadlock under some
circumstances (I do not remember with what the collision was).
To resolve the problem I would have to remember the modified files in
some buffer and then send it to Groups at once. What do you think,
Milos?
> Then the commands seem to be added to get the file's status. This
> happens on file or directory base, Martin?
You mean annotateName() ? It returns the status immediately, but runs
folder refresh on background if the folder is not in the disk cache.
I have created issue #20800 for the first thread dump. To resolve the problem I would have to remember the modified files in some buffer and then send it to Groups at once. What do you think, Milos? yes, that could fix the filesystem's part of the slowdown. However I think the performance will still drop.. for example try to open the default group after you've compiled the vcscore module.. it will take pretty much time.. I think that's general node's problem that we can't fix, nodes are not tuned for performance with 100-1000s of subnodes. And the groups are not tuned for automatic actions on such large scale either, *sigh* but currently I don't see what we could do about it.. I just want to emphasize, that this adding of all sources to the default group during compilation should be seen itself as a bug in my opinion. None of the source were modified. Also, this only happens with the internal compiler, not with the external or fastjavac. Sorry about the wrapped threaddumps. Thomas, I agree that adding the files is wrong. Currently I see it as design flaw of vcs groups. I didn't know about this behaviour and the only way to "fix" the problem within the vcs groups code would be to remove the auto addition property. I don't like that though. As I already mentioned, the only feasible solution I see now is to have the internal compiler set the unimportant flag on the files.. I'll try to negotiate that with the java module people. Fixed in VcsFileSystem. Also marking as a candidate for 3.3.2. Fixed in dev build Feb 26. Here is the diff for command-line CVS: http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java.diff?r1=1.162&r2=1.163 fixed in javacvs filesystem as well.. diff is: http://www.netbeans.org/source/browse/javacvs/src/org/netbeans/modules/cvsclient/NbJavaCvsFileSystem.java.diff?r1=text&tr1=1.76&r2=text&tr2=1.77&f=h post-mortem note: The compiler cannot mark the files "unimportant" by itself. It is only a producer of those files and does not know, or care, what will happen with them. They can be marked as (un)important only by Loaders during the object type recognition - so e.g. Java loader will mark the .class as unimportant, while I'm would bet that Clazz loader will make them important. thanks Svata. BTW: we do DataObject.find at a later moment in the process, when initially evaluating the bug I've placed the isImportant check before the dataobject creation, the right way is to place it after it.. I completely overlooked that.. Do you think it is okay in latest development builds of NetBeans 3.4, Thomas ? It's better, but not OK (just tested build 200203110100). I used my 3.3.1 installation as test base. This time, not all classes are being added to the default VCS group, only one local (not in VSS) class and two inner classes (don't know, why only these, because there are more local and inner classes in the project). NetBeans does not consume CPU power anymore, after everything is finally finished. But still, the VCS threads are starting to run LIST_STATUS_READER during internal compilation and they are still running long after compilation has finished. So, fixed is: Not all classes are being added and CPU usage is back to normal at the end. Not fixed: Some classes are still being added and VCS commandos still start to run during internal compilation. I wanted to do a 2nd compilation test, but I can't, because this time the internal compiler stops compiling after a while, because one class file is read only, but that's not true. The class file does not even exist at that moment. As before, the external compiler works perfectly. Okay, I am glad that the major problem was fixed and therefore verifying this issue. On the other hand if you Thomas feel like the rest is pretty annoying for you, file special bug with appropriate priority and describe accordingly. Thanks. I am not sure that the main problem is fully fixed because of the fact that the VCS threads again started running during internal compilation. I thought that was a major part of the problem. And it also seems that this fix caused the internal compiler to generate the new error message about the write protected class file. This is not in 3.3.1. Maybe I found out the source of the remaining problem: In 3.3.1 and in the current dev build, the internal compiler generates not only .class files but also .class~ files and these seem to be responsible for the running of the VCS threads. Should I file a new issue on this? The external compiler does not do this. Opened a new issue based on the last statement: issue #21876. Fixed in orion_fcs branch. Resolved for 3.4.x or earlier, no new info since then -> closing. |