This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
| Summary: | org.netbeans.modules.groovy.refactoring.RefactoringTask$TextComponentTask.isValid: LowPerformance took 104098 ms. | ||
|---|---|---|---|
| Product: | groovy | Reporter: | MarkFlacy |
| Component: | Refactoring | Assignee: | bruno.flavio |
| Status: | STARTED --- | ||
| Severity: | normal | CC: | bruno.flavio, gmbuchanan, MarkFlacy, thurka |
| Priority: | P2 | Keywords: | 8.1_WAIVER_APPROVED, PERFORMANCE |
| Version: | 8.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Issue Type: | DEFECT | Exception Reporter: | 215887 |
| Attachments: |
nps snapshot
Groovy-all upgrade to 2.4.5 |
||
|
Description
MarkFlacy
2015-05-09 17:22:40 UTC
Created attachment 153585 [details]
nps snapshot
Slowness reports show that almost all of the time is spent in converting URLs of class path roots to URIs. This is really strange, since this process should be fast unless there are a really long paths in those URLs. To investigate it please take a heap dump at the time you see NetBeans are frozen. Zip it and upload it at <http://deadlock.netbeans.org/job/upload/build?delay=0sec>. Thanks. I've taken heap dumps (3 in a row) for the slowness reported in id #792739. Those were submitted via project upload build number 1206. These long-running lockups are the reason that I cannot and do not recommend Netbeans for Grails development to the other engineers in my organization. Since my management is willing to purchase IntelliJ licenses for us, I may end up dropping Netbeans at work. I still get these lockups every day. I've stopped bothering to report them since I haven't seen any action on this. Tomasi, can you please take a look at the heap dumps Mark provided and share your investigations here? Thanks. I took a look at the heap dumps, but I have no idea what is going on here, sorry. Someone with Grails knowledge should investigate it. Mark, can we provide your uploaded heap dumps to Bruno Flavio - NetBeans community member who is helping with maintenance of Groovy support in NetBeans? (In reply to Jiri Kovalsky from comment #7) > Mark, can we provide your uploaded heap dumps to Bruno Flavio - NetBeans > community member who is helping with maintenance of Groovy support in > NetBeans? By all means, please do so. This one is very serious bug, in my opinion. The IDE is completely unusable during these lockups. Any other issues that I have with Netbeans grails support are instances of "Netbeans could be doing more to help me do work." THIS bug is a case of "Netbeans is actively preventing me from doing work." I'm requesting a waiver so this bug can be analysed. Thank you for reporting and providing the heap dump Mark. I'll try to see what was happening and replicate. Bug waiver for 8.1 approved. Here is the 219 MB ZIP file with 3 heap dumps from Mark: https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07-11_96ed2da0e986578a82ff95246393b8d4.zip Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM and start to analyze them: https://visualvm.java.net/heapdump.html Good luck! (In reply to Jiri Kovalsky from comment #11) > Here is the 219 MB ZIP file with 3 heap dumps from Mark: > > https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07- > 11_96ed2da0e986578a82ff95246393b8d4.zip > > Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM > and start to analyze them: > > https://visualvm.java.net/heapdump.html > > Good luck! Jirka, Mark, Thank you for providing the heap dumps. I'm attempting to understand the issue. In the mean time it'd help if I could replicate this. Mark, your comment says you switched from a project to another when this happened. Also it seems you unfortunately you're getting this hang frequently.. always on similar situations? Is there something more you could tell me about your setup? (which grails/groovy versions are you using, also I wonder: are you using the NetBeans embedded maven, or the the same version that is being used by the grails command?). Anything else that you think might be relevant? Thank you for reporting this. I'll keep you updated. (In reply to bruno.flavio from comment #12) > (In reply to Jiri Kovalsky from comment #11) > > Here is the 219 MB ZIP file with 3 heap dumps from Mark: > > > > https://netbeans.org/projects/qa/downloads/download/252320_2015-09-08_18-07- > > 11_96ed2da0e986578a82ff95246393b8d4.zip > > > > Bruno, you can download the ZIP, extract the *.hprof files, open in VisualVM > > and start to analyze them: > > > > https://visualvm.java.net/heapdump.html > > > > Good luck! > > Jirka, Mark, Thank you for providing the heap dumps. > > I'm attempting to understand the issue. In the mean time it'd help if I > could replicate this. > > Mark, your comment says you switched from a project to another when this > happened. Also it seems you unfortunately you're getting this hang > frequently.. always on similar situations? Is there something more you could > tell me about your setup? (which grails/groovy versions are you using, also > I wonder: are you using the NetBeans embedded maven, or the the same version > that is being used by the grails command?). Anything else that you think > might be relevant? > > Thank you for reporting this. I'll keep you updated. Well, this is on a Macbook Pro running OS X 10.x. There's 16GB of RAM and the hard drive is an SSD. Well, I can be doing almost anything in a groovy file in a grails project to trigger this. Right-click in a groovy file, type in a few characters, or even scroll to another page in the same file, you name it. The version of grails we are using at work is 2.4.4; I've also seen it in version 2.5.0. I've had another horrible lockup today. In fact, I've connected jvisualvm to the instance and am running the profiler on the locked instance. The profiler has been running over an hour there and has yet to provide any measurements. (Note: after 2 hours, I've stopped jvisualvm and killed the unresponsive netbeans process. Nothing at all was recorded by the profiler in all that time. The Netbeans instance was unresponsive for 30 minutes prior to that.) I have cloned a copy of that code base to my home Linux workstation (Slackware64 14.1 for those who care) and the user experience in the grails environment is amazingly superior to what I've seen under OS X. (Netbeans with pure java projects are just fine under OS X; I've been able to do some development that way on that same laptop and have had no issues.) On the Linux machine, I'm not seeing the random lockups and freeze outs that are the norm on my work Macbook Pro. I should work for a few hours straight to level the playing field. I'll do that tomorrow and let you know what I see. (Insert negative rant about Apple products here.) Mark, when you say that "Nothing at all was recorded by the profiler in all that time." I would like to ask what you tried to profile? Did you right click the NetBeans node and invoke "Sample" from its popup menu? Did you see any green threads in "Threads" tab? If yes, what did you do next? Do you say that all graphs in "Monitor" tab were empty? Did you try to start CPU or Memory sampling and it did nothing? (In reply to Jiri Kovalsky from comment #14) > Mark, when you say that "Nothing at all was recorded by the profiler in all > that time." I would like to ask what you tried to profile? Did you right > click the NetBeans node and invoke "Sample" from its popup menu? Did you see > any green threads in "Threads" tab? If yes, what did you do next? Do you say > that all graphs in "Monitor" tab were empty? Did you try to start CPU or > Memory sampling and it did nothing? I double-clicked the netbeans node and opened the "profiler" tab. I then clicked the "cpu" button. It took around an hour to instrument the ~37K classes (which that panel reported), but after another hour, there were still no profiling results. The graphs in the "Monitor" tab continually updated (I even could and did force a GC) and I didn't look that closely at the "Threads" tab. (In reply to MarkFlacy from comment #13) > > On the Linux machine, I'm not seeing the random lockups and freeze outs that > are the norm on my work Macbook Pro. I should work for a few hours straight > to level the playing field. I'll do that tomorrow and let you know what I > see. OK, I've done enough work on both machines to determine that: 1) The user experience using Netbeans to develop grails applications on a Linux host is rather pleasant. There *are* some issues where Netbeans isn't very helpful (such as not recognizing Swagger annotations), but I have *NOT* experienced where Netbeans is preventing me from doing work. 2) There is either something horribly wrong with my work Macbook Pro's configuration *or* there is something horribly different about the OS X version of Java that triggers the behavior that I'm seeing. (Or maybe both) I'm afraid that I still cannot recommend Netbeans to my shop (since they looove their Macbooks) until someone who is using Netbeans on a Macbook to develop Grails applications can tell me that I'm full of something. (In reply to MarkFlacy from comment #16) Mark, Thank you for reporting back on your experience on the Linux OS. It explains why I couldn't reproduce the issue and narrows down the scope. Unfortunately I do not have access to the Mac Platform but, if you're up to it, perhaps you could help me: I've placed a development version of NetBeans on google drive with the following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f NetBeans-dev-20151014-a577e872ebc1-full.zip The link for the archive is the following: https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did seem to be where the program was most of the time during the snapshots you've provided and I'd like to rule out a problem with this. Thank you for reporting this bug and conducting further analysis. Let me know if you do find some spare time to try this. Regards, Bruno. (In reply to bruno.flavio from comment #17) > (In reply to MarkFlacy from comment #16) > > Mark, > > Thank you for reporting back on your experience on the Linux OS. It explains > why I couldn't reproduce the issue and narrows down the scope. > > Unfortunately I do not have access to the Mac Platform but, if you're up to > it, perhaps you could help me: > > I've placed a development version of NetBeans on google drive with the > following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f > NetBeans-dev-20151014-a577e872ebc1-full.zip > > The link for the archive is the following: > https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing > > In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did > seem to be where the program was most of the time during the snapshots > you've provided and I'd like to rule out a problem with this. > > Thank you for reporting this bug and conducting further analysis. Let me > know if you do find some spare time to try this. > > Regards, > Bruno. I will download and test this as soon as possible. I'm still using the macbook at work and am very interested in getting this fixed in OS X. (Or proving that my machine's configuration is wrong in some fashion.) (In reply to bruno.flavio from comment #17) > (In reply to MarkFlacy from comment #16) > > Mark, > > Thank you for reporting back on your experience on the Linux OS. It explains > why I couldn't reproduce the issue and narrows down the scope. > > Unfortunately I do not have access to the Mac Platform but, if you're up to > it, perhaps you could help me: > > I've placed a development version of NetBeans on google drive with the > following sha1 - 23ca17375126eefc1d90bd3e2440457ffbaec98f > NetBeans-dev-20151014-a577e872ebc1-full.zip > > The link for the archive is the following: > https://drive.google.com/file/d/0B3CdZxORsyD5bG1QYWNwcjFyXzQ/view?usp=sharing > > In that version the groovy-all jar was upgraded from 2.1.7 to 2.4.5. It did > seem to be where the program was most of the time during the snapshots > you've provided and I'd like to rule out a problem with this. > > Thank you for reporting this bug and conducting further analysis. Let me > know if you do find some spare time to try this. > > Regards, > Bruno. After launching this version, Netbeans tells me there are 39 updates available. Should I install the updates or not? You can of course update to the latest development modules since Bruno produced the build for you but it's not necessary. My immediate feedback is that this version is working much better. I'd like to soak it for a day or two to make sure. There is an exception on startup, which I've reported at http://statistics.netbeans.org/analytics/exception.do?id=797738 (In reply to MarkFlacy from comment #21) Ok, that'd be good news! Take your time to test it so we can understand if those lock-ups are gone. I was able to trigger the freeze via a right-click in the editor window that is displaying a grails service class (seehttp://statistics.netbeans.org/analytics/exception.do?id=797815). Actually, the *first* time that you right-click inside the editor window that is displaying a grails service class after that panel has keyboard focus triggers that freeze. I've been able to reproduce it that way. Not all grails service classes have this behavior; the ones that are ~240 lines or more do. An 832 line groovy utility class in the "Groovy Source Packages" folder did not trigger the freeze when right-clicked. Controllers, domain classes, and other groovy/java classes in the project appear to be unaffected by the freeze. (Most of those unaffected types have fewer lines than the services, but the 832 line one should have triggered the issue if file length was the problem.) This build is, nonetheless, much better than what I was using earlier. Very well, we may have a partial fix then. Because we now have something that is repeatable I'd like to ask: * When you right click the grails service editor window the first time, does it matter *where* you right click? (whitespace, method, imports...) * If you could be so kind as to test the same thing with your Linux machine... I'm inclined to think that the behaviour should persist there (i.e. the mac specific thing may have been solved with the updated groovy-all and we have uncovered another bug). * Finally: this lock-up you've reported lasted 12 seconds, when you managed to replicate it did you get similar times? Thank you for your help! (In reply to bruno.flavio from comment #24) > Very well, we may have a partial fix then. > > Because we now have something that is repeatable I'd like to ask: > > * When you right click the grails service editor window the first time, does > it matter *where* you right click? (whitespace, method, imports...) > > * If you could be so kind as to test the same thing with your Linux > machine... I'm inclined to think that the behaviour should persist there > (i.e. the mac specific thing may have been solved with the updated > groovy-all and we have uncovered another bug). Well, Netbeans 8.0.2 doesn't do that under linux (either with JDK 7 or JDK 8). (time passes) Nor does the development version do that under linux. > > * Finally: this lock-up you've reported lasted 12 seconds, when you managed > to replicate it did you get similar times? Yes, although repeated invocations of the problem would eventually appear to clear it up for that particular file edit window. (!) However, I've managed to lock up the OS X version that's running. I've used jvisualvm to connect to it and have obtained both a heap and a thread dump. I've uploaded a zip archive of those two files in upload 1229 a few minutes ago. I'll leave the OS X version running for a while as I go do something else, but I don't expect it to come back. I had changed files and right clicked somewhere (sorry) in the new buffer before the navigation panel updated with the new file's class/methods/etc when the lockup happened. > > Thank you for your help! (In reply to MarkFlacy from comment #25) > (In reply to bruno.flavio from comment #24) > > Very well, we may have a partial fix then. > > > > Because we now have something that is repeatable I'd like to ask: > > > > * When you right click the grails service editor window the first time, does > > it matter *where* you right click? (whitespace, method, imports...) > > > > * If you could be so kind as to test the same thing with your Linux > > machine... I'm inclined to think that the behaviour should persist there > > (i.e. the mac specific thing may have been solved with the updated > > groovy-all and we have uncovered another bug). > > Well, Netbeans 8.0.2 doesn't do that under linux (either with JDK 7 or JDK > 8). (time passes) Nor does the development version do that under linux. > > > > > * Finally: this lock-up you've reported lasted 12 seconds, when you managed > > to replicate it did you get similar times? > > Yes, although repeated invocations of the problem would eventually appear to > clear it up for that particular file edit window. (!) > > However, I've managed to lock up the OS X version that's running. I've used > jvisualvm to connect to it and have obtained both a heap and a thread dump. > I've uploaded a zip archive of those two files in upload 1229 a few minutes > ago. I'll leave the OS X version running for a while as I go do something > else, but I don't expect it to come back. > > I had changed files and right clicked somewhere (sorry) in the new buffer > before the navigation panel updated with the new file's class/methods/etc > when the lockup happened. > > > > > Thank you for your help! As it so happens, my OS X machine did come back with an NPE. I've reported that in report 797962. Created attachment 156958 [details] Groovy-all upgrade to 2.4.5 I've attached the partial fix to this performance issue (the upgrade from groovy-all 2.1.7 to 2.4.5). Jirka: Do I need to do something for the groovy-all file to show up in http://hg.netbeans.org/binaries/ ? Or does the system pick this up from the binaries-list? (In reply to MarkFlacy from comment #25) Mark, Sorry for not replying earlier. I did manage to confirm, with larger classes, that indeed the AST parser takes longer on services (on Debian), in my case something like 3-5 seconds creating a noticeable pause. Perhaps this could be alleviated by doing the AST parse when the file is loaded (in the background) or if possible making this refactoring task non-blocking. Tomorrow I'll investigate these options and try to understand the reason for the NPE you've sent(797962) and report back. Integrated into 'main-silver', will be available in build *201601020002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/451798f66932 User: Bruno Fl |