This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 216896 - evaluate possibility of advanced caching in org.sonatype.aether.RepositoryCache
Summary: evaluate possibility of advanced caching in org.sonatype.aether.RepositoryCache
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Maven (show other bugs)
Version: 7.3
Hardware: PC Mac OS X
: P3 normal (vote)
Assignee: Tomas Stupka
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2012-08-15 06:48 UTC by Milos Kleint
Modified: 2016-07-07 08:37 UTC (History)
1 user (show)

See Also:
Issue Type: TASK
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Milos Kleint 2012-08-15 06:48:44 UTC
currently the org.sonatype.aether.RepositoryCache works per request, is backed by a hard map and is thrown away after the request.

There are multiple things stored in the cache, caution advised with any long term caching, but one interesting bit are the raw models of stuff coming from repository.
Comment 1 Milos Kleint 2012-08-15 07:59:54 UTC
known keys+value pairs: 
String:"org.sonatype.aether.impl.internal.DataPool$Artifact" + 
org.sonatype.aether.impl.internal.ObjectPool

String:"org.sonatype.aether.impl.internal.DataPool$Dependency" + org.sonatype.aether.impl.internal.ObjectPool

String:"org.sonatype.aether.impl.internal.DataPool$Descriptors" + Map

org.apache.maven.repository.internal.DefaultVersionResolver$Key + org.apache.maven.repository.internal.DefaultVersionResolver$Record

org.apache.maven.repository.internal.DefaultModelCache$Key + org.apache.maven.model.building.ModelData
Comment 2 Milos Kleint 2012-08-15 08:01:57 UTC
The artifact and dependency object pools are interesting, potentially we could reduce the number of objects in memory if the pools were shared across projects..

the biggest memory hog is probably the ModelData object which holds the raw parsed o.a.m.m.Model instance.
Comment 3 Milos Kleint 2012-08-15 09:42:23 UTC
http://hg.netbeans.org/core-main/rev/749c43170a4b turns the objectPool caching from request to global. The number of aether's Dependency and Artifact instances should drop significantly, especially for cases where  interrelated projects are opened. tested on maven's own sources, down to about 130 instances from 800+ for both classes. the 130 value is about what a single project uses (one with most dependencies)
Should be fairly safe, as ObjectPool is just a weakHashMap with weak values and both classes are immutable.
Comment 4 Milos Kleint 2012-08-15 12:18:12 UTC
org.apache.maven.model.building.ModelData related items appear to only be used when loading parent poms or dependency import sections.

So the saving ratio is not as big as initially thought, still for many interrelated projects, the parents could be re-read multiple times.

The big question here is how to implement clearing of the cache items once in the cache. The ModelCache$Key class is not really held anywhere outside of the cache map, weak referencing would not work.

Maybe some sort of time based release, given that we are interested in the cached objects staying around during project opening primarily. Unfortunately the sisu-guava lib shipped with maven (a fork of google-guava) contains old guava sources with caching apis without put() method. new guava is over 1 meg size, not worth the effort I suppose.
Comment 5 Quality Engineering 2012-08-16 01:27:31 UTC
Integrated into 'main-golden', will be available in build *201208160001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/749c43170a4b
User: Milos Kleint <mkleint@netbeans.org>
Log: #216896 have just one set of ObjectPool instances in RepositoryCaches in MavenExecutionRequests. The objects in object pools are immutable and referenced by weakhashmap + weakref value.
Comment 6 Martin Balin 2016-07-07 08:37:16 UTC
This old bug may not be relevant anymore. If you can still reproduce it in 8.2 development builds please reopen this issue.

Thanks for your cooperation,
NetBeans IDE 8.2 Release Boss