This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 236693 - Find Usages/Refactor fails with error "Can't refactor - scanning in progress" in big source files
Summary: Find Usages/Refactor fails with error "Can't refactor - scanning in progress"...
Status: VERIFIED FIXED
Alias: None
Product: php
Classification: Unclassified
Component: Refactoring (show other bugs)
Version: 7.4
Hardware: All All
: P3 normal (vote)
Assignee: Ondrej Brejla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-10-03 11:19 UTC by victork
Modified: 2013-11-12 13:53 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:

Attachments
Possible fix which needs to be tested. (1.91 KB, patch)
2013-10-11 09:27 UTC, Ondrej Brejla 		Details | Diff
Possible fix which needs to be tested. (1.92 KB, application/octet-stream)
2013-10-11 09:29 UTC, Ondrej Brejla 		Details
Show Obsolete (1) Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description victork 2013-10-03 11:19:48 UTC 
While IDE is focused on some big file, the "Find Usages/Refactoring" menuses are always unavailable(Does not matter how many time a user waits) poping-up an error "Can't refactor - scanning in progress".
In every smaller file it just works and a find UI window pops up.

Details about environment and NetBeans:
RAM: 8GB
CPU: Intel Core-2 E7xxx
Scanning/Index Refresh: Complete and not triggered at all(Pop-up occures then no scanning really goes on).


How-To Reproduce:
Open some very big file from a huge project on non-decent machine(My sample file is about 2800 lines long) while another file of 1400 lines is working - Thats pretty depends on machine vs size vs module dependencies factor/ratio(We have lots of includes).


This bug landed in:
Revision: 262093 (33fcf30ed211)
Info: #234319 - ParserManager.parse called in AWT
Author: Ondrej Brejla

about 5 weeks ago.

The source of the problem is here:

    @NbBundle.Messages("ERR_ScanningInProgress=Can't refactor - scanning in progress.")
    protected void fetchRefactoringUI(Source source, UserTask userTask) {
        Future<?> futureTask = RP.submit(new ParsingTask(source, userTask));
        boolean scanningInProgress = false;
        try {
            futureTask.get(300, TimeUnit.MILLISECONDS);
        } catch (InterruptedException ex) {
            Thread.currentThread().interrupt();
        } catch (ExecutionException ex) {
            LOG.log(Level.WARNING, null, ex);
        } catch (TimeoutException ex) {
            futureTask.cancel(true);
            scanningInProgress = true;
        }

The source of the bug is "Blind" assumption that if futureTask doesn't finish within 300 then a scanning of source code/change is "probably" in progress which is wrong because on older machine+bigger source code combination latencies increase(Without going into discussion of additional slowness added by Java)
this can happen as well.
And my conclusion is not an assumption - I've increased the timeout from 300 to 1500, rebuilt the NB and tested with my custom build - The big file now got Find Usages/Refactoring working.

Previous AWT code which was 'Fixed' by this commit didn't use such a 'Timing' hacks and worked.

Maybe it's possible to add custom exception to ParserTask to notify the Future about real "Scan in progress" and just finish it's own operation thus exiting the thread itself and on outside call Future.get() without timeout(As ParserTask will become non-locking/endless operation).

I hope you will look into this one.
Comment 1 Ondrej Brejla 2013-10-03 11:24:00 UTC 
I'll look at it for next release.
Comment 2 victork 2013-10-03 12:38:13 UTC 
Thanks.
I know it will not be an easy thing to fix.
Comment 3 Ondrej Brejla 2013-10-03 12:43:53 UTC 
In general, it could be...I can just increase the timeout to e.g. 2500 (after 3s slowness detector appears...300ms is really not enough to parse really big files on a slow system). And the message is not too proper either...it is more likely "parsing in progress", because that's the thing what really happen.

But I'll try to look into it deeper ;)
Comment 4 victork 2013-10-03 12:51:35 UTC 
It confused me as well in the beginning("Scanning" while debug console shows no lines typical for scanning events) :)
Comment 5 Ondrej Brejla 2013-10-03 12:58:44 UTC 
Yep, scanning rised from that mentioned bug...because there it really was "infinite" scanning what blocked "parsing", so refactoring module couldn't parse file where refactoring action should be applied :)

But what really happens now, that "necessary parsing", which is needed to assemble refactoring UI, is run in request processor (background thread) and that timeout is there just to ensure that "AWT", which is blocked by "futureTask.get", will be released at least after some "suspicious" time.

So the fix, which will just increase the timeout is absolutely valid (and yes, changing "scanning in progress" to something like "parsing was too long or blah blah ;)").

But I'll think about more proper message, to detect whwther it was "slow parsing of refactored file" or "really scanning was in progress" :)
Comment 6 victork 2013-10-03 13:24:37 UTC 
I remember those times :D
Long "Background scanning of projects" which has been popping on almost every single simple action(in 7.1-7.2) and made the IDE unusable(Worse than "Refreshing Workspace" in Eclipse), then "Refreshing Indices" which took less but blocked the IDE for at least 10 secs on lots of simple actions and during editing(7.3-firstBuilds 7.4) and the great enhancements in latest builds which made the IDE finally usable(And i personally very excited for it's PHP support and constant updates and improvements applied to it - For example Eclipse's PHP support is external and plugin is very buggy and almost not updated).

This bug is relatively small 'side-effect' of otherwise greatly improved system then comparing to previous implementations. I've not used the IDE past 1.5 years, however each 2 months picked fresh sources to check how it's going and finally it became almost rock-solid.

Thank you(All NetBeans team and contributors/volunteers) for all the hard work done on it so 'we'(developers) can enjoy using it and stay productive.
Comment 7 Ondrej Brejla 2013-10-03 13:30:01 UTC 
You are welcome of course :)
Comment 8 Ondrej Brejla 2013-10-11 09:27:12 UTC 
Created attachment 141005 [details]
Possible fix which needs to be tested.
Comment 9 Ondrej Brejla 2013-10-11 09:29:35 UTC 
Created attachment 141006 [details]
Possible fix which needs to be tested.
Comment 10 victork 2013-10-13 08:08:58 UTC 
I'll apply it now and will test it out during daily workflow.
Comment 11 Ondrej Brejla 2013-10-13 08:10:15 UTC 
It was just a note for me, but you can test it too, if you wish of course ;) Thanks.
Comment 12 victork 2013-10-13 09:50:05 UTC 
I've tested on problematic big file and on regular smaller files, played with usages search and auto-complete as well and didn't find any visible regressions meantime(Autocomplete doesn't work for me quite well anyway so it's hard to see if it degraded :)).
IDE responds properly on "Find Usage" command on the big source file.
No deadlocks happened yet.

Thx :)
Comment 13 Ondrej Brejla 2013-10-13 11:25:58 UTC 
Great to read :)
Comment 14 Ondrej Brejla 2013-10-14 11:26:42 UTC 
Fixed in web-main #90e264a638a2
Comment 15 Quality Engineering 2013-10-15 02:09:21 UTC 
Integrated into 'main-silver', will be available in build *201310150001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/90e264a638a2
User: Ondrej Brejla <obrejla@netbeans.org>
Log: #236693 - Find Usages/Refactor fails with error "Can't refactor - scanning in progress" in big source files
Comment 16 Ondrej Brejla 2013-10-31 08:55:18 UTC 
Possibly patch candidate.
Comment 17 mmolda 2013-10-31 14:34:49 UTC 
Verified, thanks. Also, victork thanks for leting us know.

Product Version: NetBeans IDE Dev (Build 201310310001)
Java: 1.7.0_45; Java HotSpot(TM) Client VM 24.45-b08
Runtime: Java(TM) SE Runtime Environment 1.7.0_45-b18
System: Windows 7 version 6.1 running on x86; Cp1250; en_US (nb)
Comment 18 Ondrej Brejla 2013-11-07 14:03:30 UTC 
Transplanted to release74 branch in releases repo. Please, verify, thanks.
Comment 19 mmolda 2013-11-12 13:53:38 UTC 
Verified.

Product Version: NetBeans IDE 7.4 (Build 201311111738)
Java: 1.7.0_45; Java HotSpot(TM) Client VM 24.45-b08
Runtime: Java(TM) SE Runtime Environment 1.7.0_45-b18
System: Windows 7 version 6.1 running on x86; Cp1250; en_US (nb)