This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
[ JDK VERSION : 1.6.0_03 ] Let me first say, that this is not an artificial test case, it's a real life problem. We have .class files and .java files in the same directory (like in NB 3.x) and we cannot change that for now. When a package with lots of classes is being opened the first time, NB calls svn.exe very often (I guess to find out about the .class files that are not in the repository, I see PROPGET in the command line with Process Explorer) and this somehow causes the process csrss.exe to need lots of CPU. NetBeans is frozen while doing this and often even the whole computer freezes. This whole process takes *very* long (10 minutes or more) on a very large (500 sources) directory. Sometimes I even get OutOfMemory in the log and I have to kill NetBeans (attaching Stacktrace and ThreadDump of that situation). After that, .class is being added to the ignores property of the directory. That is supposed to be a feature, but I would say it's not needed. Outside of NetBeans, we work with Tortoise and there it's no problem at all to have unversioned .class files together with versioned .java files. Workaround is to add .class to the global ignores list of the Subversion command line client. BUT: What if someone does not know about this (very likely) or forgets it (likely)? This issue is even more important since Subversion support is on by default in NB 6.0.
Created attachment 52091 [details] Stacktrace of OutOfMemory Exception
Created attachment 52092 [details] ThreadDump
This is Beta 2.
could you please attach your whole messages.log file. if possible the one with the OOME. thanks
In the meantime, I restarted NetBeans and the log is gone. The OOME is not in messages.log or messages.log.1 or messages.log.2. Should I try to reproduce the whole issue and then attach the log?
> Should I try to reproduce the whole issue and then attach the log? that would be nice. however, even the messages.log you got would help. I want to now as much as possible about your setup. thanks
tried a simple scenario on linux 1.) versioned project, class set as ignored in system settings 2.) 1000 .java files in a package 3.) externally created 1000 .class files in the package 4.) started nb, opened the package and it worked for me:( so i will give it a try on windows, but there seems to be a bit more to it .... how exactly does your usecase look like? - you have a versioned project with many (~500) .java and .class files in a package - your System Settings/Ignored Files are something like ^(.*\.class|CVS|S ... - how are the .class files created - is it a buildin nb or an external action - ...? > When a package with lots of classes is being opened the first time - this means the first time after a nb start or the first time after the .class files where created? - could you please attach or send over the .svn/dir-props file from the relevant folder
ok, i restarted nb and it took quite a long time to open such a package... i still would like to see the .svn/dir-props file
The usecase is exactly like you describe. The sources are compiled by NB (changed build.classes.dir to the src dir). This is my ignore files system setting: ^(.*\.class|CVS|SCCS|vssver\.scc|#.*#|%.*%|\.(cvsignore|svn|DS_Store)|_svn)$|~$|^\..*$ Before opening the folder in NB, I do not have dir-props at all. I tried again. This time I waited less than 10 minutes before killing NB. The OOME did not occur, nothing interesting in the log. So I guess, the OOME was a side effect. The main problem is that svn.exe seems to cause csrss.exe to use lots of CPU and freezing the machine. I am attaching dir-props after killing NB, so only some of the .class files had been added to the ignore list at this point. In essence, I think NB simply should not add each and every .class file to the ignore list. If it absolutely must do something to the ignores list, then why not add *.class?
>Before opening the folder in NB, I do not have dir-props at all - could you please send the file after it was created? - i still want to see your messages.log file
Created attachment 52101 [details] dir-props after killing NetBeans during scan
Created attachment 52102 [details] log
the only way i was able to reproduce was: 1.) versioned project, many .java files, .class files in src.dir 2.) the files aren't in the svn cache yet - $userdir/var/cache/svncache was removed, or clean/build was invoked for the first time on the project 3.) opening the package > Workaround is to add .class to the global ignores list of the Subversion command line client workaround exists > This issue is even more important since Subversion support is on by default in NB 6.0. this doesn't apply to no svn projects - they aren't scanned by the svn module at all and i also couldn't reproduce. There is no problem just because the svn support is on by default and there is definitely no overall impact caused by this. Again - the only way to reproduce is a svn versioned project with many ignored files in a not ignored folder. To make my point - i don't intend to downgrade the fact that this is very annoying for you, however with all the necessary prerequisites it looks to me like a corner case. And even more - there is a quite simple workaround. We shall see that we fix this. I'm sure there is a way how to optimize the involved logic, unfortunately it's in the modules deeper layers at it will take some time ... > BUT: What if someone does not know about this (very likely) or forgets it (likely)? agree, we should at least add this into our faq thanks
just realized that the OOME was fixed in scope of issue #120659 and the ignored property processing for the use case described above runs now twice as fast... However - it's still to slow, but shouldn't cause huge memory consuming or deadlocks anymore
OK. IMHO, the whole issue under Windows with the csrss.exe process hogging the CPU is caused by calling svn.exe so often. The way I see it, for each and every .class file a PROPGET is called and then each and every .class file is being added to the svn ignore property of the directory. When I already have *.class in the ignorable files system setting of NB, then why add it to svn ignores property at all? And if this can't be avoided, then why not set *.class in the svn ignore property instead of every single .class file? But in my opinion, NetBeans should not touch svn ignores property of the directory at all. I find this too magic.
first of all - i definitely agree that the relevant implementation is suboptimal and we should fix it. > When I already have *.class in the ignorable files system setting of NB, then why add it to svn ignores property > at all? And if this can't be avoided, then why not set *.class in the svn ignore property instead of every single > .class file? unfortunately, there is more to the ignore logic than the mentioned "ignored files" setting... The thing is, that this issue tracks a specific performance problem when files are automatically ignored by netbeans and discussing at this place how exactly and why at all would IMO run a bit over of scope. So, if you feel strongly about this topic, you might write to the interest@subversion.netbeans.org mailing list or contact me directly. There is still some work to be done and we are looking forward to any relevant input from our users. thanks
fixing the horrible performance problem. 69903:ef9e342dc850 With "*.class" set in the global ignores i couldn't reproduce the slow behavior anymore -> P3. However, there are still some issues in the is ignored logic. We should deal with it ...
couldn't find anything bothering in the ignore logic now. fixed.