This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 124767 - Huge caches in /tmp
Summary: Huge caches in /tmp
Status: RESOLVED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: Code Model (show other bugs)
Version: 6.x
Hardware: All Linux
: P2 blocker (vote)
Assignee: Vladimir Kvashin
URL:
Keywords:
: 125662 128280 (view as bug list)
Depends on: 127727
Blocks:
  Show dependency tree
 
Reported: 2008-01-06 10:25 UTC by ziakjan
Modified: 2008-02-26 15:49 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ziakjan 2008-01-06 10:25:19 UTC
The C/C++ NetBeans module puts some kind of data into the directory "/tmp/USERNAME-cnd60-caches-0". After several days
of running NetBeans, the size of this data was several *gigabytes*.
Comment 1 Alexander Pepin 2008-01-09 19:10:18 UTC
Could you please describe what projects you worked with, their size, what operations you did and how long it took to
grow your cache size up to "several" gigabytes. In other words could you provide more exact steps to reproduce the
situation you described.
Comment 2 Thomas Preisler 2008-01-13 17:14:55 UTC
The cache is created by StorageAllocator from the Repository module:

            diskRepositoryPath += File.separator +         //NOI18N
                    System.getProperty("user.name") +  "-cnd60-caches-";  //NOI18N


Comment 3 Sergey Grinev 2008-01-14 11:36:39 UTC
This is code model repository caches for different projects. There are next problems:
- absolute name control for projects which effectively means doubling of the cache size after moving the project
- eternal shelf-life of any cache even for non-existent projects
Vladimir is gonna introduce some aging control mechanism to hunt not used caches.
Comment 4 ziakjan 2008-01-14 12:22:14 UTC
(1) I should have noted that the issue relates to the version of NetBeans available from
[http://bits.netbeans.org/download/trunk/nightly/latest/]. The last version I have is
[netbeans-trunk-nightly-200801121200-linux.sh].

(2) The issue does not seem to be related to "having multiple different projects", but it rather seems the problem is
caused be some kind of "infinite loop which is dispersed over time (a day or so)". The problem may be caused by having
recursive dependencies between symbols in the C++ source code (which is totally common), but the parser/code model/etc
fails to realize there is such a recursion. Or, the Java code building the cache thinks that a certain code element is
not in the cache, while in fact it has been processed already and is in the cache.   Anyway, once the caches in /tmp
start to grow, the CPU usage of NetBeans is quite high.
Comment 5 Vladimir Kvashin 2008-01-14 14:31:57 UTC
So, as I understand, the situation is specific to your project(s).
Could you please attach a listing of cache directory:
ls -lR /tmp/USERNAME-cnd60-caches-0


Comment 6 Vladimir Kvashin 2008-01-16 10:18:40 UTC
Downgrading to P2.
The workaround is: do not leave the IDE for weeks; restart the IDE, say, every day.
I'm not saying we won't fix it - we must!! But it isn't a P1.
Comment 7 ziakjan 2008-01-16 14:42:35 UTC
I attached a debugger to NetBeans when it was writing several megabytes per second to the file
"/tmp/USERNAME-cnd60-caches-0/.../cache-1-data". The file size is currently 654MB, growed from 0 bytes to 654 MB in just
a few minutes. The callstack of the thread "Repository writer 0" (which is writing to that file) looks like this:

java.util.AbstractList$Itr.remove(AbstractList.java:360)
org.netbeans.modules.cnd.repository.sfs.IndexedStorageFile$IndexIterator.remove(IndexedStorageFile.java:399)
org.netbeans.modules.cnd.repository.sfs.DoubleFileStorage._defragment(DoubleFileStorage.java:221)
org.netbeans.modules.cnd.repository.sfs.DoubleFileStorage.maintenance(DoubleFileStorage.java:191)
org.netbeans.modules.cnd.repository.disk.UnitDiskRepository.maintenance(UnitDiskRepository.java:111)
org.netbeans.modules.cnd.repository.disk.DiskRepositoryManager.maintenance(DiskRepositoryManager.java:204)
org.netbeans.modules.cnd.repository.queue.RepositoryWritingThread.waitReady(RepositoryWritingThread.java:79)
org.netbeans.modules.cnd.repository.queue.RepositoryWritingThread.run(RepositoryWritingThread.java:128)
org.netbeans.modules.cnd.repository.queue.RepositoryThreadManager$Wrapper.run(RepositoryThreadManager.java:84)
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:561)
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:986)

The problem seems to be related to defragmentation of the repository. But this is just a "random snapshot" of the
callstack, so I an not stating to have pinpointed the cause of the problem with 100% accuracy.

[Applies to: NetBeans IDE Dev (Build 200801151200)]
Comment 8 Vladimir Voskresensky 2008-01-21 13:50:41 UTC
*** Issue 125662 has been marked as a duplicate of this issue. ***
Comment 9 Vladimir Voskresensky 2008-01-21 13:51:18 UTC
see also http://www.netbeans.org/issues/show_bug.cgi?id=125662
Comment 10 Vladimir Voskresensky 2008-02-25 12:10:27 UTC
*** Issue 128280 has been marked as a duplicate of this issue. ***
Comment 11 Vladimir Voskresensky 2008-02-25 12:12:25 UTC
see also files and size distribution in http://www.netbeans.org/issues/show_bug.cgi?id=128280
Comment 12 Vladimir Kvashin 2008-02-25 15:00:41 UTC
The same in one click:
http://www.netbeans.org/nonav/issues/showattachment.cgi/57170/cnd-log-huge
Comment 13 Vladimir Kvashin 2008-02-26 06:45:14 UTC
I believe I know the reason. 
There is a bug in persistence data partial defragmentation.
If a defragmentation attempt is finished by timeout, then the removed keys aren't deleted from file index. This causes
endless defragmentation attempts that lead to endless file increase.

This bug was introduced in 6.1
Comment 14 Vladimir Kvashin 2008-02-26 07:57:13 UTC
Fixed
http://hg.netbeans.org/main?cmd=changeset;node=a14187871a0e
Comment 15 Vladimir Kvashin 2008-02-26 15:49:42 UTC
There were some doubts whether the error I fixed existed on the moment the bug was filed,
(or the described effect was a consequence of another bug).
I've just checked that the error I fixed did exist.
So I believe this issue is really fixed.