[ JDK VERSION : 1.6.0_03 ]
Let me first say, that this is not an artificial test case, it's a
real life problem.
We have .class files and .java files in the same directory (like in
NB 3.x) and we cannot change that for now.
When a package with lots of classes is being opened the first time,
NB calls svn.exe very often (I guess to find out about the .class
files that are not in the repository, I see PROPGET in the command
line with Process Explorer) and this somehow causes the process
csrss.exe to need lots of CPU. NetBeans is frozen while doing this
and often even the whole computer freezes.
This whole process takes *very* long (10 minutes or more) on a very
large (500 sources) directory. Sometimes I even get OutOfMemory in
the log and I have to kill NetBeans (attaching Stacktrace and
ThreadDump of that situation).
After that, .class is being added to the ignores property of the
directory. That is supposed to be a feature, but I would say it's not
Outside of NetBeans, we work with Tortoise and there it's no problem
at all to have unversioned .class files together with versioned .java
Workaround is to add .class to the global ignores list of the
Subversion command line client.
BUT: What if someone does not know about this (very likely) or
forgets it (likely)?
This issue is even more important since Subversion support is on by
default in NB 6.0.
Created attachment 52091 [details]
Stacktrace of OutOfMemory Exception
Created attachment 52092 [details]
This is Beta 2.
could you please attach your whole messages.log file. if possible the one with the OOME.
In the meantime, I restarted NetBeans and the log is gone. The OOME is not in messages.log or messages.log.1 or
Should I try to reproduce the whole issue and then attach the log?
> Should I try to reproduce the whole issue and then attach the log?
that would be nice. however, even the messages.log you got would help. I want to now as much as possible about your setup.
tried a simple scenario on linux
1.) versioned project, class set as ignored in system settings
2.) 1000 .java files in a package
3.) externally created 1000 .class files in the package
4.) started nb, opened the package and it worked for me:(
so i will give it a try on windows, but there seems to be a bit more to it ....
how exactly does your usecase look like?
- you have a versioned project with many (~500) .java and .class files in a package
- your System Settings/Ignored Files are something like ^(.*\.class|CVS|S ...
- how are the .class files created - is it a buildin nb or an external action
> When a package with lots of classes is being opened the first time
- this means the first time after a nb start or the first time after the .class files where created?
- could you please attach or send over the .svn/dir-props file from the relevant folder
ok, i restarted nb and it took quite a long time to open such a package...
i still would like to see the .svn/dir-props file
The usecase is exactly like you describe. The sources are compiled by NB (changed build.classes.dir to the src dir).
This is my ignore files system setting:
Before opening the folder in NB, I do not have dir-props at all.
I tried again. This time I waited less than 10 minutes before killing NB. The OOME did not occur, nothing interesting in
the log. So I guess, the OOME was a side effect.
The main problem is that svn.exe seems to cause csrss.exe to use lots of CPU and freezing the machine.
I am attaching dir-props after killing NB, so only some of the .class files had been added to the ignore list at this point.
In essence, I think NB simply should not add each and every .class file to the ignore list. If it absolutely must do
something to the ignores list, then why not add *.class?
>Before opening the folder in NB, I do not have dir-props at all
- could you please send the file after it was created?
- i still want to see your messages.log file
Created attachment 52101 [details]
dir-props after killing NetBeans during scan
Created attachment 52102 [details]
the only way i was able to reproduce was:
1.) versioned project, many .java files, .class files in src.dir
2.) the files aren't in the svn cache yet - $userdir/var/cache/svncache was removed, or clean/build was invoked for the
first time on the project
3.) opening the package
> Workaround is to add .class to the global ignores list of the Subversion command line client
> This issue is even more important since Subversion support is on by default in NB 6.0.
this doesn't apply to no svn projects - they aren't scanned by the svn module at all and i also couldn't reproduce.
There is no problem just because the svn support is on by default and there is definitely no overall impact caused by
this. Again - the only way to reproduce is a svn versioned project with many ignored files in a not ignored folder. To
make my point - i don't intend to downgrade the fact that this is very annoying for you, however with all the necessary
prerequisites it looks to me like a corner case. And even more - there is a quite simple workaround.
We shall see that we fix this. I'm sure there is a way how to optimize the involved logic, unfortunately it's in the
modules deeper layers at it will take some time ...
> BUT: What if someone does not know about this (very likely) or forgets it (likely)?
agree, we should at least add this into our faq
just realized that the OOME was fixed in scope of issue #120659 and the ignored property processing for the use case
described above runs now twice as fast...
However - it's still to slow, but shouldn't cause huge memory consuming or deadlocks anymore
OK. IMHO, the whole issue under Windows with the csrss.exe process hogging the CPU is caused by calling svn.exe so
often. The way I see it, for each and every .class file a PROPGET is called and then each and every .class file is being
added to the svn ignore property of the directory.
When I already have *.class in the ignorable files system setting of NB, then why add it to svn ignores property at all?
And if this can't be avoided, then why not set *.class in the svn ignore property instead of every single .class file?
But in my opinion, NetBeans should not touch svn ignores property of the directory at all. I find this too magic.
first of all - i definitely agree that the relevant implementation is suboptimal and we should fix it.
> When I already have *.class in the ignorable files system setting of NB, then why add it to svn ignores property
> at all? And if this can't be avoided, then why not set *.class in the svn ignore property instead of every single
> .class file?
unfortunately, there is more to the ignore logic than the mentioned "ignored files" setting...
The thing is, that this issue tracks a specific performance problem when files are automatically ignored by netbeans and
discussing at this place how exactly and why at all would IMO run a bit over of scope. So, if you feel strongly about
this topic, you might write to the firstname.lastname@example.org mailing list or contact me directly. There is still
some work to be done and we are looking forward to any relevant input from our users.
fixing the horrible performance problem.
With "*.class" set in the global ignores i couldn't reproduce the slow behavior anymore -> P3.
However, there are still some issues in the is ignored logic. We should deal with it ...
couldn't find anything bothering in the ignore logic now. fixed.