Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 178999 - Cannot exit from IDE: bloked in addRecursiveListener
Cannot exit from IDE: bloked in addRecursiveListener
Status: RESOLVED FIXED
Product: editor
Classification: Unclassified
Component: Parsing & Indexing
6.x
PC Solaris
: P2 (vote)
: 6.x
Assigned To: Vitezslav Stejskal
issues@editor
:
: 167623 177658 (view as bug list)
Depends on:
Blocks: 174884 178124
  Show dependency treegraph
 
Reported: 2009-12-23 04:42 UTC by Alexander Simon
Modified: 2010-03-02 22:08 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
thread dump (19.14 KB, text/plain)
2009-12-23 04:42 UTC, Alexander Simon
Details
thread dump (22.73 KB, text/plain)
2010-01-11 10:38 UTC, Alexander Simon
Details
Proposing fix without changing any API (3.36 KB, patch)
2010-02-09 05:00 UTC, Jaroslav Tulach
Details | Diff
thread dump on hanging exiting on chromium project (25.90 KB, text/plain)
2010-03-01 03:02 UTC, Alexander Simon
Details
Workaround (5.83 KB, patch)
2010-03-01 10:08 UTC, Tomas Zezula
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Simon 2009-12-23 04:42:26 UTC
Created attachment 92883 [details]
thread dump
Comment 1 Jan Jancura 2010-01-04 05:38:27 UTC
>indexing
Comment 2 Tomas Zezula 2010-01-04 07:18:28 UTC
*** Bug 177658 has been marked as a duplicate of this bug. ***
Comment 3 Tomas Zezula 2010-01-04 07:20:20 UTC
*** Bug 167623 has been marked as a duplicate of this bug. ***
Comment 4 Tomas Zezula 2010-01-04 08:36:59 UTC
Save to parsing API index is cancelable after root, jet-main #da8ba945c77a
Comment 5 Quality Engineering 2010-01-05 10:47:39 UTC
Integrated into 'main-golden', will be available in build *201001051209* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/da8ba945c77a
User: Tomas Zezula <tzezula@netbeans.org>
Log: #178999:Cannot exit from IDE
Comment 6 Tomas Zezula 2010-01-05 10:52:00 UTC
jet-main e89e396f1de5
There are several fixes related to this issue and duplicates:
1) Already in the 6.8 the CustomIndexer is cancellable
2) Change set da8ba945c77a added ability to cancel the index rebuild.
3) Changeset e89e396f1de5 added cancelability of the BinaryAnalyzer.
Comment 7 Quality Engineering 2010-01-06 00:43:58 UTC
Integrated into 'main-golden', will be available in build *201001060200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/e89e396f1de5
User: Tomas Zezula <tzezula@netbeans.org>
Log: #178999:Cannot exit from IDE
Comment 8 Alexander Simon 2010-01-11 10:37:34 UTC
Not fixed. Step to reproduce:
- close project
- try to exit from IDE
Comment 9 Alexander Simon 2010-01-11 10:38:22 UTC
Created attachment 93192 [details]
thread dump
Comment 10 Vitezslav Stejskal 2010-01-12 03:15:38 UTC
It looks like that the parsing & indexing thread was stuck somewhere in the filesystems...
Comment 11 Tomas Zezula 2010-01-12 03:33:08 UTC
Yes, it is a different case from the previously reported dumps (the thread blocked in the parsing api).
The adding of recursive listener takes a time. The operation is not cancelable. I will ask Jarda about possible canceling.
Comment 12 greggwon 2010-01-12 12:56:34 UTC
One of the predominate factors in cancellable tasks is to make them idempotent.  That is, no matter how many times you ask for something, you always get the SAME results.  With the update of the database from parsing, I think that the SAME results is "an updated database".  This is why I have consistently suggested that the right way to manage the database, is to always create a readonly interface to it, and then have updates replace the indicated reference to the database with a new object that has the current state.

If things are done this way, and the updates to the database become much more finer grained steps, then there will be 100s of incremental updates happening over time as various things are "discovered" and "parsed" and stopping the "scanning" will never be an issue, because the "updates" that haven't been committed/published yet, can be trivially discarded, because the visible data will still appear out of date and the next restart of the IDE can pick up on updates where the last scanning activity left off.

I think that there is probably a larger issue that the scan output is updating a "database" that is not "layered" or "time based" in some way that would make this more trivial.  If Lucene is a factor in the continued problems with making the scanning an invisible operation, then perhaps the generation and update of Lucene data needs to be separated from the publish of the "view of the world"?

I know that the old mechanisms used to have several separated components and this did in fact get out of sync on occasion and cause problems.

Is some type of formal "version" mechanism a potential solution such that if there were multiple separate views/uses of the published "scanning" results tha they could stay in sync without requiring the old "clean out the cache and restart" resolution?
Comment 13 David Strupl 2010-01-15 01:31:16 UTC
Tomas, should we create a new issue for filesystems and make it block this issue?
Comment 14 Tomas Zezula 2010-01-15 01:34:46 UTC
OK, I will do it.
Comment 15 Tomas Zezula 2010-01-15 08:45:13 UTC
Added (exit) check before calling the expensive recursive listener: e6768ea53256
Comment 16 Tomas Zezula 2010-01-15 08:50:33 UTC
I've added exit check before calling expensive FileUtil.addRecursiveListener.
But it does not solve the problem when the cancel comes in time when the indexer is already in it.
The addRecursiveListener should be interruptable as it is inherently expensive.
Comment 17 Quality Engineering 2010-01-24 08:45:08 UTC
Integrated into 'main-golden', will be available in build *201001240200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/e6768ea53256
User: Tomas Zezula <tzezula@netbeans.org>
Log: #178999:Cannot exit from IDE
Comment 18 Jaroslav Tulach 2010-02-09 05:00:24 UTC
Created attachment 93992 [details]
Proposing fix without changing any API
Comment 19 Jaroslav Tulach 2010-02-09 05:02:49 UTC
I can understand that this is not the nicest fix on the earth, but as you are basically the only users of the API, it is simpler to fix the use on your side, then invest in proper API. I can add warning to javadoc, that this method may be quite time consuming...
Comment 20 Tomas Zezula 2010-02-09 05:15:10 UTC
"Not the nicest fix on the earth" sounds like an euphemism to me.
The request processor will continue the scan the folders. The correct solution which can be used even in other use cases is to extend the API or to ensure that the addRecursiveListener is interruptable. So Threead.interrupt or RP.Task.cancel can be used to stop scan. If the method is already interruptable the patch should do RP.Task.cancel when getShuttdownRequest().isRaised().
Comment 21 Jaroslav Tulach 2010-02-09 06:08:17 UTC
Re. "The request processor will continue the scan the folders" - not for a long time, the IDE exists and the scan will be over too. At least it was my understanding that this code blocks exit of the IDE.
Comment 22 Tomas Zezula 2010-02-09 10:32:04 UTC
Not generally, in this case yes. But as far as I remember there was a requirement from the performance team to make the scan interruptable. This will be a blocker for it. Also the creative reschedule into background thread from already background thread degrades performance (non needed context switches).
Comment 23 Jaroslav Tulach 2010-02-10 00:59:18 UTC
I still don't know how the API and its implementation shall look like. I've reported bug 180523 to remind me I shall think about it. I am hesitant to make bug 180523 blocker for this one, as there is workaround (https://netbeans.org/bugzilla/attachment.cgi?id=93992) which fixes the problem without bug 180523 being resolved.
Comment 24 Vitezslav Stejskal 2010-02-17 05:02:54 UTC
local changeset: 5be867a1d8a2
Comment 25 Quality Engineering 2010-02-18 21:46:15 UTC
Integrated into 'main-golden', will be available in build *201002190200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/5be867a1d8a2
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #178999: workaround for lenghty FileUtil.addRecursiveListener (appying slightly modified Jarda's patch)
Comment 26 Alexander Simon 2010-03-01 03:02:54 UTC
Created attachment 94617 [details]
thread dump on hanging exiting on chromium project
Comment 27 Tomas Zezula 2010-03-01 08:16:46 UTC
Does not help at all. The current solution by rescheduling adding of listener to different thread cannot work as the thread doing the addListener is synchronized and blocks the whole IDE until the addListener is done. I still think that adding of listeners should be either fast or cancellable.
Comment 28 Tomas Zezula 2010-03-01 09:32:51 UTC
I vote again to roll back the usage of the addRecursiveListener in the RepositoryUpdater until the API will be fixed. The current solution with does not helped and introduced possible deadlock is nearly unfixable (except of serialization of add and remove). Trying to remove the synchronized block only opens a race condition window which is even worse.
Comment 29 Tomas Zezula 2010-03-01 10:08:50 UTC
Created attachment 94637 [details]
Workaround
Comment 30 Tomas Zezula 2010-03-02 03:08:44 UTC
Fixed in jet-main: 973231ff3692
But I still call for real API solution of this problem.
Alexander, can you try it with chromium project?
Thanks,
Tomas
Comment 31 Quality Engineering 2010-03-02 22:08:40 UTC
Integrated into 'main-golden', will be available in build *201003030200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/973231ff3692
User: Tomas Zezula <tzezula@netbeans.org>
Log: #178999:Cannot exit from IDE: bloked in addRecursiveListener


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo