This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I needed to know all Node implementations that override equals() method in the whole IDE (Note: there are _only_20_ such classes in the whole nbbuild/misc) To find this information for the first time, Find usages (not from base class, find overrides, no comments) took about one hour and needed over 512MB of heap to survive the query. Towards the end of one failing attempt I managed to do a partial insane dump of the heap and found that there are >2700 complete java source codes held strongly in the memory alone consuming over 90MB of heap (referenced from ResourceImpl.lastSourceText) The refactoring/mdr should not keep such expensive information in memory when it actually don't need it anymore (after it found out it is not a match). Moreover, I've found out that from those 2700 source files, only about 900 reference Node at all, so even the initial filtering is not good enough and you end up parsing too many sourcess needlessly. (I might be wrong in my assumptions here though). I can provide more informations on demand, but the dump is over 1.6GB in size...
Seems that the resources where an identifier is found are held strongly even after we find out that it was a false positive. Re. 2700 source files instead of 900 containing "Node" - in this case we are looking for files that contain "equals" since you are looking for "all" overrides transitively - not just immediate overrides...
I looked at the source code and it seems I was right in my assumption. I am going to try to fix it...
OK, I have a fix for this. I need to clean it up and will commit it before Monday...
Fixed in main trunk - now the operation can be run with -Xmx128m (however it still takes long because of issue 46516). There were three problems: - all resources where a given identifier was found were hard referenced during the usages collection. To fix this, only the MOFIDs of the resources are held and a single resource is resolved from an ID at a time. Diff: http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/UsageFinder.java?r1=1.27&r2=1.28 - to implement the rollback semantics correctly, ExclusiveMutex remembered all elements that were lazily serialized during a given transaction. So, if you invoked Find Usages right after the initial scan (when no elements are serialized yet), there would be many elements held by ExclusiveMutex during the Find Usages transaction - this is again fixed by holding MOFIDs instead of the elements. Diff: http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/ExclusiveMutex.java?r1=1.43&r2=1.44 - each time a resource is parsed, resourceParsed event is fired. This event takes the resource as a parameter. Since the Find Usages may parse a lot of files and it blocks the MDR mutex, it will fire the resourceParsed event for each parsed resource and this resource will thus be hard-referenced from the firing method - there will be an event for every resource in the event queue, since the event dispatching will be blocked by the MDR mutex (once a listener tries to touch MDR). To fix this, the resources are retrieved by their MOFIDs lazily on the event queue only if the MDR mutex is not blocked. Diff: http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/JMManager.java?r1=1.103&r2=1.104 http://java.netbeans.org/source/browse/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java?r1=1.78&r2=1.79 Thanks in advance for reviewing these changes.
Petre, are you going to review the fix (I mean from QE point of view, we are not equiped to do it)?
I reviewed the diff and it seems to be OK.
> Petre, are you going to review the fix Yes, I'm on it...
My scenario passed at least on 256MB, haven't had enough patience to wait for it at 128MB.
i tested it with 128MB with smaller amount of projects (all required of refactoring). I haven't found any new problem.
Fixed on release41 branch. Checking in src/org/netbeans/modules/javacore/ExclusiveMutex.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/ExclusiveMutex.java,v <-- ExclusiveMutex.java new revision: 1.43.4.1; previous revision: 1.43 done Checking in src/org/netbeans/modules/javacore/JMManager.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/JMManager.java,v <-- JMManager.java new revision: 1.103.2.1; previous revision: 1.103 done Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/ResourceImpl.java,v <-- ResourceImpl.java new revision: 1.78.2.1; previous revision: 1.78 done Checking in src/org/netbeans/modules/javacore/jmiimpl/javamodel/UsageFinder.java; /cvs/java/javacore/src/org/netbeans/modules/javacore/jmiimpl/javamodel/UsageFinder.java,v <-- UsageFinder.java new revision: 1.27.6.1; previous revision: 1.27 done
Reorganization of java component