Bug 252320 - org.netbeans.modules.groovy.refactoring.RefactoringTask$TextComponentTask.isValid: LowPerformance took 104098 ms.
org.netbeans.modules.groovy.refactoring.RefactoringTask$TextComponentTask.isV...
Status: STARTED
Product: groovy
Classification: Unclassified
Component: Refactoring
8.0
All All
: P2 with 1 vote (vote)
: TBD
Assigned To: bruno.flavio
issues@groovy
EXCEPTIONS_REPORT perf-profileme
: 8.1_WAIVER_APPROVED, PERFORMANCE
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-05-09 17:22 UTC by MarkFlacy
Modified: 2017-07-13 15:40 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
nps snapshot (1.10 MB, application/nps)
2015-05-09 17:22 UTC, MarkFlacy
Details
Groovy-all upgrade to 2.4.5 (2.41 KB, patch)
2015-10-24 23:36 UTC, bruno.flavio
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description MarkFlacy 2015-05-09 17:22:40 UTC
This issue was reported manually by thurka.
It already has 18 duplicates 


Build: NetBeans IDE 8.0.2 (Build 201411181905)
VM: Java HotSpot(TM) 64-Bit Server VM, 24.75-b04, Java(TM) SE Runtime Environment, 1.7.0_75-b13
OS: Mac OS X

User Comments:
GUEST: Scrolled up and right-clicked within a grails project's groovy file.

GUEST: Click in a groovy file that is part of a grails project.

GUEST: Guess

GUEST: Opening a groovy file in a grails project.

GUEST: Right-clicked inside of a groovy file in a grails project.

Netbeans grails support is a horrible joke; I've had pauses that lasted over 20 minutes that required me to kill Netbeans just so that I can try to get some work done.

GUEST: Netbeans was responsive for around 1 second after the previous freeze (which was over 800 seconds long), then it immediately dove into *this* freeze.

Honestly, the grails support is as horrible as circa 6.x Netbeans for java.

GUEST: Switched from one open grails .groovy file to another.

I was absolutely locked out of doing anything in netbeans for over 3 minutes.

GUEST: Clicked within a Config.groovy file within one of 5 open grails projects.

GUEST: Right-click in groovy file.

GUEST: Just more freezing.

GUEST: No idea. 

If I could spare the 30 minutes or so during my work day, I'd report one of those incidents. I have to kill NetBeans externally so that I can attempt to get some work done.

GUEST: Scrolled to the right while working on a grails project.

Over 10 minutes of no work done; I was completely locked out.   Notice that there is an immediately following LONGER freezeout.

My co-workers are really impressed with NetBeans.  Just not in a good way.

GUEST: Browsing multiple grails groovy files.

GUEST: attempting to edit the same grails integration test

GUEST: Grails projects freeze repeatedly and consistently.

MarkFlacy: Changed from one groovy file in a given grails project to another groovy file in a different project.

GUEST: Too slow opening projects

GUEST: Clicked inside a grail integration test case.



Maximum slowness yet reported was 912758 ms, average is 217441
Comment 1 MarkFlacy 2015-05-09 17:22:54 UTC
Created attachment 153585 [details]
nps snapshot
Comment 2 Tomas Hurka 2015-05-12 12:14:42 UTC
Slowness reports show that almost all of the time is spent in converting URLs of class path roots to URIs. This is really strange, since this process should be fast unless there are a really long paths in those URLs. To investigate it please take a heap dump at the time you see NetBeans are frozen. Zip it and upload it at
<http://deadlock.netbeans.org/job/upload/build?delay=0sec>. Thanks.
Comment 3 MarkFlacy 2015-09-08 17:08:14 UTC
I've taken heap dumps (3 in a row) for the slowness reported in id #792739. Those were submitted via project upload build number 1206.
Comment 4 MarkFlacy 2015-09-30 15:58:16 UTC
These long-running lockups are the reason that I cannot and do not recommend Netbeans for Grails development to the other engineers in my organization.  Since my management is willing to purchase IntelliJ licenses for us, I may end up dropping Netbeans at work.

I still get these lockups every day.  I've stopped bothering to report them since I haven't seen any action on this.
Comment 5 Jiri Kovalsky 2015-10-01 08:59:52 UTC
Tomasi, can you please take a look at the heap dumps Mark provided and share your investigations here? Thanks.
Comment 6 Tomas Hurka 2015-10-01 09:55:47 UTC
I took a look at the heap dumps, but I have no idea what is going on here, sorry. Someone with Grails knowledge should investigate it.
Comment 7 Jiri Kovalsky 2015-10-01 13:31:58 UTC
Mark, can we provide your uploaded heap dumps to Bruno Flavio - NetBeans community member who is helping with maintenance of Groovy support in NetBeans?
Comment 8 MarkFlacy 2015-10-01 18:34:43 UTC
(In reply to Jiri Kovalsky from comment #7)
> Mark, can we provide your uploaded heap dumps to Bruno Flavio - NetBeans
> community member who is helping with maintenance of Groovy support in
> NetBeans?

By all means, please do so.

This one is very serious bug, in my opinion.  The IDE is completely unusable during these lockups.  Any other issues that I have with Netbeans grails support are instances of "Netbeans could be doing more to help me do work."  THIS bug is a case of "Netbeans is actively preventing me from doing work."
Comment 9 bruno.flavio 2015-10-02 19:41:50 UTC
I'm requesting a waiver so this bug can be analysed.

Thank you for reporting and providing the heap dump Mark. I'll try to see what was happening and replicate.
Comment 10 Jiri Kovalsky 2015-10-07 12:17:00 UTC
Bug waiver for 8.1 approved.
Comment 11 Jiri Kovalsky 2015-10-07 15:44:11 UTC
Here is the 219 MB ZIP file with 3 heap dumps from Mark:

https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07-11_96ed2da0e986578a82ff95246393b8d4.zip

Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM and start to analyze them:

https://visualvm.java.net/heapdump.html

Good luck!
Comment 12 bruno.flavio 2015-10-07 22:31:23 UTC
(In reply to Jiri Kovalsky from comment #11)
> Here is the 219 MB ZIP file with 3 heap dumps from Mark:
> 
> https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07-
> 11_96ed2da0e986578a82ff95246393b8d4.zip
> 
> Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM
> and start to analyze them:
> 
> https://visualvm.java.net/heapdump.html
> 
> Good luck!

Jirka, Mark, Thank you for providing the heap dumps.

I'm attempting to understand the issue. In the mean time it'd help if I could replicate this.

Mark, your comment says you switched from a project to another when this happened. Also it seems you unfortunately you're getting this hang frequently.. always on similar situations? Is there something more you could tell me about your setup? (which grails/groovy versions are you using, also I wonder: are you using the NetBeans embedded maven, or the the same version that is being used by the grails command?). Anything else that you think might be relevant?

Thank you for reporting this. I'll keep you updated.
Comment 13 MarkFlacy 2015-10-09 02:50:23 UTC
(In reply to bruno.flavio from comment #12)
> (In reply to Jiri Kovalsky from comment #11)
> > Here is the 219 MB ZIP file with 3 heap dumps from Mark:
> > 
> > https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07-
> > 11_96ed2da0e986578a82ff95246393b8d4.zip
> > 
> > Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM
> > and start to analyze them:
> > 
> > https://visualvm.java.net/heapdump.html
> > 
> > Good luck!
> 
> Jirka, Mark, Thank you for providing the heap dumps.
> 
> I'm attempting to understand the issue. In the mean time it'd help if I
> could replicate this.
> 
> Mark, your comment says you switched from a project to another when this
> happened. Also it seems you unfortunately you're getting this hang
> frequently.. always on similar situations? Is there something more you could
> tell me about your setup? (which grails/groovy versions are you using, also
> I wonder: are you using the NetBeans embedded maven, or the the same version
> that is being used by the grails command?). Anything else that you think
> might be relevant?
> 
> Thank you for reporting this. I'll keep you updated.

Well, this is on a Macbook Pro running OS X 10.x. There's 16GB of RAM and the hard drive is an SSD. 

Well, I can be doing almost anything in a groovy file in a grails project to trigger this.  Right-click in a groovy file, type in a few characters, or even scroll to another page in the same file, you name it.

The version of grails we are using at work is 2.4.4; I've also seen it in version 2.5.0.

I've had another horrible lockup today.  In fact, I've connected jvisualvm to the instance and am running the profiler on the locked instance.  The profiler has been running over an hour there and has yet to provide any measurements. (Note: after 2 hours, I've stopped jvisualvm and killed the unresponsive netbeans process.  Nothing at all was recorded by the profiler in all that time.  The Netbeans instance was unresponsive for 30 minutes prior to that.)

I have cloned a copy of that code base to my home Linux workstation (Slackware64 14.1 for those who care) and the user experience in the grails environment is amazingly superior to what I've seen under OS X.  (Netbeans with pure java projects are just fine under OS X; I've been able to do some development that way on that same laptop and have had no issues.)

On the Linux machine, I'm not seeing the random lockups and freeze outs that are the norm on my work Macbook Pro.  I should work for a few hours straight to level the playing field. I'll do that tomorrow and let you know what I see.

(Insert negative rant about Apple products here.)
Comment 14 Jiri Kovalsky 2015-10-09 09:34:58 UTC
Mark, when you say that "Nothing at all was recorded by the profiler in all that time." I would like to ask what you tried to profile? Did you right click the NetBeans node and invoke "Sample" from its popup menu? Did you see any green threads in "Threads" tab? If yes, what did you do next? Do you say that all graphs in "Monitor" tab were empty? Did you try to start CPU or Memory sampling and it did nothing?
Comment 15 MarkFlacy 2015-10-09 13:29:28 UTC
(In reply to Jiri Kovalsky from comment #14)
> Mark, when you say that "Nothing at all was recorded by the profiler in all
> that time." I would like to ask what you tried to profile? Did you right
> click the NetBeans node and invoke "Sample" from its popup menu? Did you see
> any green threads in "Threads" tab? If yes, what did you do next? Do you say
> that all graphs in "Monitor" tab were empty? Did you try to start CPU or
> Memory sampling and it did nothing?

I double-clicked the netbeans node and opened the "profiler" tab.  I then clicked the "cpu" button.  It took around an hour to instrument the ~37K classes (which that panel reported), but after another hour, there were still no profiling results.

The graphs in the "Monitor" tab continually updated (I even could and did force a GC) and I didn't look that closely at the "Threads" tab.
Comment 16 MarkFlacy 2015-10-10 04:02:39 UTC
(In reply to MarkFlacy from comment #13)
>
> On the Linux machine, I'm not seeing the random lockups and freeze outs that
> are the norm on my work Macbook Pro.  I should work for a few hours straight
> to level the playing field. I'll do that tomorrow and let you know what I
> see.

OK, I've done enough work on both machines to determine that:

1) The user experience using Netbeans to develop grails applications on a Linux host is rather pleasant. There *are* some issues where Netbeans isn't very helpful (such as not recognizing Swagger annotations), but I have *NOT* experienced where Netbeans is preventing me from doing work.

2) There is either something horribly wrong with my work Macbook Pro's configuration *or* there is something horribly different about the OS X version of Java that triggers the behavior that I'm seeing. (Or maybe both)

I'm afraid that I still cannot recommend Netbeans to my shop (since they looove their Macbooks) until someone who is using Netbeans on a Macbook to develop Grails applications can tell me that I'm full of something.
Comment 17 bruno.flavio 2015-10-14 18:32:11 UTC
(In reply to MarkFlacy from comment #16)

Mark,

Thank you for reporting back on your experience on the Linux OS. It explains why I couldn't reproduce the issue and narrows down the scope.

Unfortunately I do not have access to the Mac Platform but, if you're up to it, perhaps you could help me:

I've placed a development version of NetBeans on google drive with the following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f  NetBeans-dev-20151014-a577e872ebc1-full.zip

The link for the archive is the following:
https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing

In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did seem to be where the program was most of the time during the snapshots you've provided and I'd like to rule out a problem with this.

Thank you for reporting this bug and conducting further analysis. Let me know if you do find some spare time to try this.

Regards,
Bruno.
Comment 18 MarkFlacy 2015-10-14 21:27:54 UTC
(In reply to bruno.flavio from comment #17)
> (In reply to MarkFlacy from comment #16)
> 
> Mark,
> 
> Thank you for reporting back on your experience on the Linux OS. It explains
> why I couldn't reproduce the issue and narrows down the scope.
> 
> Unfortunately I do not have access to the Mac Platform but, if you're up to
> it, perhaps you could help me:
> 
> I've placed a development version of NetBeans on google drive with the
> following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f 
> NetBeans-dev-20151014-a577e872ebc1-full.zip
> 
> The link for the archive is the following:
> https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing
> 
> In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did
> seem to be where the program was most of the time during the snapshots
> you've provided and I'd like to rule out a problem with this.
> 
> Thank you for reporting this bug and conducting further analysis. Let me
> know if you do find some spare time to try this.
> 
> Regards,
> Bruno.

I will download and test this as soon as possible.

I'm still using the macbook at work and am very interested in getting this fixed in OS X. (Or proving that my machine's configuration is wrong in some fashion.)
Comment 19 MarkFlacy 2015-10-14 23:49:01 UTC
(In reply to bruno.flavio from comment #17)
> (In reply to MarkFlacy from comment #16)
> 
> Mark,
> 
> Thank you for reporting back on your experience on the Linux OS. It explains
> why I couldn't reproduce the issue and narrows down the scope.
> 
> Unfortunately I do not have access to the Mac Platform but, if you're up to
> it, perhaps you could help me:
> 
> I've placed a development version of NetBeans on google drive with the
> following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f 
> NetBeans-dev-20151014-a577e872ebc1-full.zip
> 
> The link for the archive is the following:
> https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing
> 
> In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did
> seem to be where the program was most of the time during the snapshots
> you've provided and I'd like to rule out a problem with this.
> 
> Thank you for reporting this bug and conducting further analysis. Let me
> know if you do find some spare time to try this.
> 
> Regards,
> Bruno.

After launching this version, Netbeans tells me there are 39 updates available.  Should I install the updates or not?
Comment 20 Jiri Kovalsky 2015-10-15 09:58:42 UTC
You can of course update to the latest development modules since Bruno produced the build for you but it's not necessary.
Comment 21 MarkFlacy 2015-10-15 17:38:43 UTC
My immediate feedback is that this version is working much better.  I'd like to soak it for a day or two to make sure.

There is an exception on startup, which I've reported at http://statistics.netbeans.org/analytics/exception.do?id=797738
Comment 22 bruno.flavio 2015-10-15 19:25:55 UTC
(In reply to MarkFlacy from comment #21)

Ok, that'd be good news! Take your time to test it so we can understand if those lock-ups are gone.
Comment 23 MarkFlacy 2015-10-16 15:43:25 UTC
I was able to trigger the freeze via a right-click in the editor window that is displaying a grails service class (seehttp://statistics.netbeans.org/analytics/exception.do?id=797815).

Actually, the *first* time that you right-click inside the editor window that is displaying a grails service class after that panel has keyboard focus triggers that freeze. I've been able to reproduce it that way.  Not all grails service classes have this behavior; the ones that are ~240 lines or more do.


An 832 line groovy utility class in the "Groovy Source Packages" folder did not trigger the freeze when right-clicked.  Controllers, domain classes, and other groovy/java classes in the project appear to be unaffected by the freeze. (Most of those unaffected types have fewer lines than the services, but the 832 line one should have triggered the issue if file length was the problem.)

This build is, nonetheless, much better than what I was using earlier.
Comment 24 bruno.flavio 2015-10-16 21:35:48 UTC
Very well, we may have a partial fix then.

Because we now have something that is repeatable I'd like to ask:

* When you right click the grails service editor window the first time, does it matter *where* you right click? (whitespace, method, imports...)

* If you could be so kind as to test the same thing with your Linux machine... I'm inclined to think that the behaviour should persist there (i.e. the mac specific thing may have been solved with the updated groovy-all and we have uncovered another bug).

* Finally: this lock-up you've reported lasted 12 seconds, when you managed to replicate it did you get similar times?

Thank you for your help!
Comment 25 MarkFlacy 2015-10-17 17:35:59 UTC
(In reply to bruno.flavio from comment #24)
> Very well, we may have a partial fix then.
> 
> Because we now have something that is repeatable I'd like to ask:
> 
> * When you right click the grails service editor window the first time, does
> it matter *where* you right click? (whitespace, method, imports...)
> 
> * If you could be so kind as to test the same thing with your Linux
> machine... I'm inclined to think that the behaviour should persist there
> (i.e. the mac specific thing may have been solved with the updated
> groovy-all and we have uncovered another bug).

Well, Netbeans 8.0.2 doesn't do that under linux (either with JDK 7 or JDK 8). (time passes) Nor does the development version do that under linux.

> 
> * Finally: this lock-up you've reported lasted 12 seconds, when you managed
> to replicate it did you get similar times?

Yes, although repeated invocations of the problem would eventually appear to clear it up for that particular file edit window. (!)

However, I've managed to lock up the OS X version that's running.  I've used jvisualvm to connect to it and have obtained both a heap and a thread dump. I've uploaded a zip archive of those two files in upload 1229 a few minutes ago. I'll leave the OS X version running for a while as I go do something else, but I don't expect it to come back.

I had changed files and right clicked somewhere (sorry) in the new buffer before the navigation panel updated with the new file's class/methods/etc when the lockup happened.

> 
> Thank you for your help!
Comment 26 MarkFlacy 2015-10-19 00:34:09 UTC
(In reply to MarkFlacy from comment #25)
> (In reply to bruno.flavio from comment #24)
> > Very well, we may have a partial fix then.
> > 
> > Because we now have something that is repeatable I'd like to ask:
> > 
> > * When you right click the grails service editor window the first time, does
> > it matter *where* you right click? (whitespace, method, imports...)
> > 
> > * If you could be so kind as to test the same thing with your Linux
> > machine... I'm inclined to think that the behaviour should persist there
> > (i.e. the mac specific thing may have been solved with the updated
> > groovy-all and we have uncovered another bug).
> 
> Well, Netbeans 8.0.2 doesn't do that under linux (either with JDK 7 or JDK
> 8). (time passes) Nor does the development version do that under linux.
> 
> > 
> > * Finally: this lock-up you've reported lasted 12 seconds, when you managed
> > to replicate it did you get similar times?
> 
> Yes, although repeated invocations of the problem would eventually appear to
> clear it up for that particular file edit window. (!)
> 
> However, I've managed to lock up the OS X version that's running.  I've used
> jvisualvm to connect to it and have obtained both a heap and a thread dump.
> I've uploaded a zip archive of those two files in upload 1229 a few minutes
> ago. I'll leave the OS X version running for a while as I go do something
> else, but I don't expect it to come back.
> 
> I had changed files and right clicked somewhere (sorry) in the new buffer
> before the navigation panel updated with the new file's class/methods/etc
> when the lockup happened.
> 
> > 
> > Thank you for your help!

As it so happens, my OS X machine did come back with an NPE.  I've reported that in report 797962.
Comment 27 bruno.flavio 2015-10-24 23:36:10 UTC
Created attachment 156958 [details]
Groovy-all upgrade to 2.4.5

I've attached the partial fix to this performance issue (the upgrade from groovy-all 2.1.7 to 2.4.5).

Jirka: Do I need to do something for the groovy-all file to show up in http://hg.netbeans.org/binaries/ ? Or does the system pick this up from the binaries-list?
Comment 28 bruno.flavio 2015-10-25 00:00:03 UTC
(In reply to MarkFlacy from comment #25)

Mark, Sorry for not replying earlier.

I did manage to confirm, with larger classes, that indeed the AST parser takes longer on services (on Debian), in my case something like 3-5 seconds creating a noticeable pause.

Perhaps this could be alleviated by doing the AST parse when the file is loaded (in the background) or if possible making this refactoring task non-blocking.

Tomorrow I'll investigate these options and try to understand the reason for the NPE you've sent(797962) and report back.
Comment 29 Quality Engineering 2016-01-02 02:42:28 UTC
Integrated into 'main-silver', will be available in build *201601020002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/451798f66932
User: Bruno Fl


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo