This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
JavaCvsFileSystem caches some data about its fileobjects but refuses to release its cache. If I create a FS over some CVS vorking directory, visit all files and gc(), all AbstractFileObjects allocated will be freed from memory but the cache describing the whole FS will remain. Moreover, the data will be held even if I remove the filesystem. Also, the filesystem instance itself will not be gc()ed. All the cache data is reachable through static org.netbeans.modules.javacvs.caching.CvsFsCache.cacheObject
Created attachment 4033 [details] Test of the filesystem memory usage, tune for your enviroment. To be used by the internal execution.
the cached data about directories will be kept around generally.. the cache is shared among filesystems, if you have another filesystem that has reference to the fileobject, it doesn't get released. I've been testing it in September/October, it was a pain and probably something scr**ed up again.. I'll take a look at it..
I was pointing it to different working dir each time I run the test (inside a single IDE) and every time the memory usage have grown. I was also tracking it using HAT and found (although I had no CVSFS mounted anymore) the static reference to the CacheDir/CacheFile going from static org.netbeans.modules.javacvs.caching.CvsFsCache.cacheObject
Ok, I've fixed to code to suit your test program ;) The static instance is gone, once no mounted fs is present, the cache goes away fully. I was not able to figure where the filesystem is kept. There's just too many references chains and when I tried to check in OptimizeIt, I always ended up with nothing-saying hashmaps and lists. I couldn't figure where thise belong. I figured a strange thing: In CacheHandler where we create WeakReference to fileobjects, I've added a counter of created instances. And in CacheQueue which is the referenceQueue of the WeakReferences I've put a counter of gc-ed instances.. the numbers never matched. :( One explanation might be that the VcsRefreshRequest keeps an enumeration of existing fileobjects that it refreshes. In some circumstances, it could mean that the instances are kept there forever. However the FileObject didn't get gc-ed even when I disabled the refresher.. :( Again I was not able to trace down who holds the fileobjects using OptimizeIt..
OK, I'll try to recheck it using HAT. To the held FileSystems: I've tried a bit different test which created 1000 FSes over the same working dir (no I don't have 1000 different cvs working dirs :-), called addNotify/getRoot/removeNotify on them and keep only WeakRef to them. After a brutal gc() cycle, ~990 was gc()ed, but about 10 remained, even if I respinned the gc() cycle again. I'll test this using HAT as well.
ok, I think I've fixed the memory leak with cache. I started to keep a the WeakReferences around, so that they don't get gc-ed before they can be Reference-queued. That was the actual problem of not-releasing the cache. I've verified only by adding some counts to the referencequeue and a pair of println()s. I've tried to check with OptimizeIt but it doesn't seem to work well with WeakReferences since none or very few were released. Besides it's pretty slow.. Petr, how's HAT? Is it hard to configure on my linuxbox? so in imy eyes the bug's memory leak is gone, please reopen when you disagree. I'll try to work on the not-release fs.. BTW I figured a strange thing once while running optimizeit. 1. I've mounted javacvsfs, unmounted and and restarted. 2. After restart there was a shadow instance of the filesystem in the system. It didn't appear anywhere, but I checked that it was created during deserialization of a mode or topcomponent.. I can't recall if I opened any windows, I have checked my code quickly but didn't find any place where the fs could be serialized..
Using HAT is simple. You'll run the IDE with some options and then inspect the generated file off-line using the HAT as a webserver: $ runide.sh -J-Xhprof:file=nb.hprof,format=b You'll take snapshots using Ctrl-\ $ hat nb.hprof[:optional snapshot number] You'll point your browser to localhost:7000 see http://developer.java.sun.com/developer/onlineTraining/Programming/JDCBook/perf3.html
Ok, I'll try it. Anyway.. THe current trunk code is safe and does reduce the ammount of memory used. It doesn't clear the whole cache though. I have an additional fix handy locally, however I'd prefer to fix only the current state to the release33 branch. (Currently directories and files that don't have an appropriate Reference [fileobject counterpart] don't get removed, however these are minority usually.)
I would liek to ask for approval for this fix. the diffs are linked here: http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/cache/CacheFile.java.diff?r1=text&tr1=1.5&r2=text&tr2=1.4&f=h http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/cache/CacheHandler.java.diff?r1=text&tr1=1.9&r2=text&tr2=1.8&f=h http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/cache/CacheQueue.java.diff?r1=text&tr1=1.5&r2=text&tr2=1.4&f=h http://www.netbeans.org/source/browse/vcscore/src/org/netbeans/modules/vcscore/cache/CacheReference.java.diff?r1=text&tr1=1.6&r2=text&tr2=1.5&f=h
approved
Approved.
adding ian to cc list :)
Approved
merged into release33 branch.
Could you Petr verify the bug ? Thanks.
I've verified it during another HAT memory hunt.
Resolved for 3.4.x or earlier, no new info since then -> closing.