This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
The "automatically check for external changes" implementation is a huge performance hog. From times to times - even without actual external changes, the IDE goes into a heavy I/O caused slowdown because it's scanning the whole project for external changes. Depending on the available ram this also triggers heavy swapping, which takes the slow down to another order of magnitude... and most of the time, this is all for no gain. What I propose is that where supported by the host platform, the actual implementation be based on a hotspot checking. Linux and other unix-based modern OS has many ways to have the kernel monitor a tree for file changes (inotify being the most proeminent one IIRC). Windows also has FindFirstChangeNotification and related APIs. With this and a matching file-type-to-handler queue, the ide can correctly decide whether to take the file change into consideration or to discard it, but the main gain would be to avoid having to do a global reparse due to timeouts or to false positives. If a platform lacks support to such a mechanism, the implementation can fallback to polling mode, but even this polling mode can be revisited: probably every platform netbeans runs has some sort of inode/metadata handling. A cache of file metadata hashes (modification times, size, whatever relevant) can be created and a thread can *slowly* recheck this metadata tree, and when find a change, generate an event to reparse only the relevant files. the reparse engine can update the hash as a side-effect. If the ide detects that a file is modified by any other means (i.e. it modifies itself, or is notified by the external version control), it can generate a change event.
I think this depends on issue 198982, specifically: "create a Notifier impl delegating to WatchService"
Native Filesystem listening is implemented since NetBeans 7.0
To whomever closed as fixed. The bug report is described as this: The "automatically check for external changes" implementation is a huge performance hog. It's still true. Nothing has changed. If the infrastructure I humbly suggested as needed to fix the issue is already present, that doesn't mean it's in use, because the full scan is triggered every single time. There's a difference between detecting a change and reacting to it (only), than detect a change and trigger a global reparse.
Listening on file changes mostly work as you describe. We use native listeners (inotify on Linux, CoreServices on Mac, kernel32 API on Windows, Java NIO2 on others), that trigger refreshing of NetBeans (internal representation of) filesystem. To be exact, not the whole internal filesystem, just parent directory of the modified file (to update list of child files, and fire change events if needed). The problem that you describe is probably related to scanning and indexing of source files. (If some method is deleted from one source file, another source file that calls that method is marked with red error badge, for example.) If some file is modified, files that may be affected by the change are rescanned. If I understand correctly, you are complaining that unnecessary or unrelated scanning is triggered by some file changes. To investigate the problem, we would need to know the environment and project setup. Can you please describe steps to reproduce? If you can reproduce the bug on some open-source project, we could follow your steps and check whether everything works correctly in NetBeans. Thank you.
You can take my report as an attempt to explain this (which I hadn't read at the time): http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints#Detecting_external_changes_in_files And after doing it, I can now summarize my report as this, instead: "Get rid of external change polling". The problem I had was exactly as described: when switching back to netbeans window, it would retrigger the so-called checking for external changes, which judging by the amount of I/O involved, is not simply checking for pontual changes, but doing a lot of work. Which can only means it's not inotify-alike based, but more on the lines of "our implementation is missing some changes, let's re-check every file (or inode metadata) to see if something changed". This is reproducible with any project having quite a few files/subdirectories. In particular C++ freestanding projects seems to be very affected, and it makes netbeans close to unusable if you happen to lack available free RAM, because at least on Linux, having netbeans to repoll everything will also swap a lot). But This is only partially a bug, it's more on the lines of inconvenience: while the behavior itself is a bad implementation indeed, there's a option on the UI for disabling this behavior. The bugs/misbehaviors are: a) having scenarios where this is needed, and: b) having this enabled by default. I can't say anything about "a" for I have no such projects (and I think a right implementation of change watch would catch external changes too; inotify() and friends can pinpoint exactly which file was changed on which subdirectory, so you'd only need to parse that file). I can see that while having "a", "b" is also needed from a correctness point of view. But while there's a workaround (the UI option for disabling external checking), having "b" is annoying and force users to look around on this issue. So it's more like chosing between which users will complain: the ones that need a or the ones that doesn't, like myself. I'm probably on the minority side.
(In reply to Jaroslav Havlin from comment #4) > > The problem that you describe is probably related to scanning and indexing > of source files. (If some method is deleted from one source file, another > source file that calls that method is marked with red error badge, for > example.) > > If some file is modified, files that may be affected by the change are > rescanned. If I understand correctly, you are complaining that unnecessary > or unrelated scanning is triggered by some file changes. > The thing is: there were no external changes. And the internal ones were already parsed by the time I take focus out of netbeans and took it to other window. When netbeans come back, it reparses the whole world, causing chaos. That's it.
(In reply to PolesApart from comment #6) > The thing is: there were no external changes. And the internal ones were > already parsed by the time I take focus out of netbeans and took it to other > window. This is strange. Can you please attach your IDE Log (after the reparsing)?
I'm (fortunately?) currently unable to reproduce this after reenabling the 'check for external changes' and restarting netbeans for the current projects I'm working with. So I went looking for what might have changed since then, and only was able to spot two differences (other than an eventual netbeans update that went through network): 1) Some of the projects where I observed this were backed by a GIT version control. 2) Most of my projects are subversion based instead, and while previously I used javahl, it seems as of svn 1.9 (using native debian version) netbeans is refusing to use javahl and is resorting to it's slow alternative. If there's either a bad javahl or git interaction with external change detection, that could explain why I'm now unable to reproduce it. I'll try with a git-backed project, but feel free to close this report in the mean time.
> I'm (fortunately?) currently unable to reproduce this after reenabling the > 'check for external changes' and restarting netbeans for the current projects > I'm working with. Good news, I would say :-) > 1) Some of the projects where I observed this were backed by a GIT version > control. I think version control should not affect background scanning in this way. > 2) Most of my projects are subversion based instead, and while previously > I used javahl, it seems as of svn 1.9 (using native debian version) > netbeans is refusing to use javahl and is resorting to it's slow > alternative. I don't know whether this can be related to the bug. Also maybe settings could have been broken (e.g. after some import). But it's just surmise. > I'll try with a git-backed project, but feel free to close this report in the > mean time. I'll close the issue as WORKSFORME for now. Please reopen it if you manage to reproduce the bug. Thank you for help.