This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 92883 [details] thread dump
>indexing
*** Bug 177658 has been marked as a duplicate of this bug. ***
*** Bug 167623 has been marked as a duplicate of this bug. ***
Save to parsing API index is cancelable after root, jet-main #da8ba945c77a
Integrated into 'main-golden', will be available in build *201001051209* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/da8ba945c77a User: Tomas Zezula <tzezula@netbeans.org> Log: #178999:Cannot exit from IDE
jet-main e89e396f1de5 There are several fixes related to this issue and duplicates: 1) Already in the 6.8 the CustomIndexer is cancellable 2) Change set da8ba945c77a added ability to cancel the index rebuild. 3) Changeset e89e396f1de5 added cancelability of the BinaryAnalyzer.
Integrated into 'main-golden', will be available in build *201001060200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/e89e396f1de5 User: Tomas Zezula <tzezula@netbeans.org> Log: #178999:Cannot exit from IDE
Not fixed. Step to reproduce: - close project - try to exit from IDE
Created attachment 93192 [details] thread dump
It looks like that the parsing & indexing thread was stuck somewhere in the filesystems...
Yes, it is a different case from the previously reported dumps (the thread blocked in the parsing api). The adding of recursive listener takes a time. The operation is not cancelable. I will ask Jarda about possible canceling.
One of the predominate factors in cancellable tasks is to make them idempotent. That is, no matter how many times you ask for something, you always get the SAME results. With the update of the database from parsing, I think that the SAME results is "an updated database". This is why I have consistently suggested that the right way to manage the database, is to always create a readonly interface to it, and then have updates replace the indicated reference to the database with a new object that has the current state. If things are done this way, and the updates to the database become much more finer grained steps, then there will be 100s of incremental updates happening over time as various things are "discovered" and "parsed" and stopping the "scanning" will never be an issue, because the "updates" that haven't been committed/published yet, can be trivially discarded, because the visible data will still appear out of date and the next restart of the IDE can pick up on updates where the last scanning activity left off. I think that there is probably a larger issue that the scan output is updating a "database" that is not "layered" or "time based" in some way that would make this more trivial. If Lucene is a factor in the continued problems with making the scanning an invisible operation, then perhaps the generation and update of Lucene data needs to be separated from the publish of the "view of the world"? I know that the old mechanisms used to have several separated components and this did in fact get out of sync on occasion and cause problems. Is some type of formal "version" mechanism a potential solution such that if there were multiple separate views/uses of the published "scanning" results tha they could stay in sync without requiring the old "clean out the cache and restart" resolution?
Tomas, should we create a new issue for filesystems and make it block this issue?
OK, I will do it.
Added (exit) check before calling the expensive recursive listener: e6768ea53256
I've added exit check before calling expensive FileUtil.addRecursiveListener. But it does not solve the problem when the cancel comes in time when the indexer is already in it. The addRecursiveListener should be interruptable as it is inherently expensive.
Integrated into 'main-golden', will be available in build *201001240200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/e6768ea53256 User: Tomas Zezula <tzezula@netbeans.org> Log: #178999:Cannot exit from IDE
Created attachment 93992 [details] Proposing fix without changing any API
I can understand that this is not the nicest fix on the earth, but as you are basically the only users of the API, it is simpler to fix the use on your side, then invest in proper API. I can add warning to javadoc, that this method may be quite time consuming...
"Not the nicest fix on the earth" sounds like an euphemism to me. The request processor will continue the scan the folders. The correct solution which can be used even in other use cases is to extend the API or to ensure that the addRecursiveListener is interruptable. So Threead.interrupt or RP.Task.cancel can be used to stop scan. If the method is already interruptable the patch should do RP.Task.cancel when getShuttdownRequest().isRaised().
Re. "The request processor will continue the scan the folders" - not for a long time, the IDE exists and the scan will be over too. At least it was my understanding that this code blocks exit of the IDE.
Not generally, in this case yes. But as far as I remember there was a requirement from the performance team to make the scan interruptable. This will be a blocker for it. Also the creative reschedule into background thread from already background thread degrades performance (non needed context switches).
I still don't know how the API and its implementation shall look like. I've reported bug 180523 to remind me I shall think about it. I am hesitant to make bug 180523 blocker for this one, as there is workaround (https://netbeans.org/bugzilla/attachment.cgi?id=93992) which fixes the problem without bug 180523 being resolved.
local changeset: 5be867a1d8a2
Integrated into 'main-golden', will be available in build *201002190200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/5be867a1d8a2 User: Vita Stejskal <vstejskal@netbeans.org> Log: #178999: workaround for lenghty FileUtil.addRecursiveListener (appying slightly modified Jarda's patch)
Created attachment 94617 [details] thread dump on hanging exiting on chromium project
Does not help at all. The current solution by rescheduling adding of listener to different thread cannot work as the thread doing the addListener is synchronized and blocks the whole IDE until the addListener is done. I still think that adding of listeners should be either fast or cancellable.
I vote again to roll back the usage of the addRecursiveListener in the RepositoryUpdater until the API will be fixed. The current solution with does not helped and introduced possible deadlock is nearly unfixable (except of serialization of add and remove). Trying to remove the synchronized block only opens a race condition window which is even worse.
Created attachment 94637 [details] Workaround
Fixed in jet-main: 973231ff3692 But I still call for real API solution of this problem. Alexander, can you try it with chromium project? Thanks, Tomas
Integrated into 'main-golden', will be available in build *201003030200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/973231ff3692 User: Tomas Zezula <tzezula@netbeans.org> Log: #178999:Cannot exit from IDE: bloked in addRecursiveListener