This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
In recent builds I suppose over the last week, the IDEs speed has been negatively impacted. Two things I noticed. 1. PMDing the 1.4 source takes about 4 times as long as it used too. 2. When the IDE is first started the Java Source Parser is running and it is slowing down the whole IDE for the first 3-4 minutes. At first I blamed te JSP, but it occurs to me that it was likely always doing this, but now its taking a lot longer than it used too. Me feeling is that opening files has become upto 4x slower than it was just a few weeks ago.
Possibly related to issue 31422
Please attach thread dumps taken in the period when the parser is running.
Ad (2): The parser is run during the startup sequence because of implemented suggestion from Performance team -- see issue #28596
BTW please check whether the .class files in your rt.jar are newer than the sources you are feed to PMD.
Also please post here list of modules which you have installed/enabled.
"Ad (2): The parser is run during the startup sequence because of implemented suggestion from Performance team -- see issue #28596" Hardly, the warmup parses just templates/classes which can hardly cause 3-4mins of parsing. The problem was ararently with editor's JCUpdater kicking in (because of mounting?)
BTW I noticed one "neat" thing: at least my (stock) installation of JDK-1.4.1_01 contains src.zip where source file timestamps are *more recent* than timestamps of corresponding .class files in rt.jar. Which means the parser will decide to parse the sources at points when it could load and use information from a .class file, just because the .class file *appears* out of date.
Yes, parser db updater is my favourite culprit; still I would like to confirm to myself that the recent changes in java module did not affect the performance.
I hope you all understand I am not suggesting MORE action is taking place. Its my feeling that the existing actions have somehow been drastically slowed down. I run my test several times across a couple of computers so I doubt if any 1 time actions are the issue. What is rt.jar? I get a dump when I get to work. All the dumps I did so far had the JSP running and not much else. Its not so slow on my XP2100+ at home, but on my PIII333 at work, its a dog to start up.
I tested the PMDing on my 2100+ today and say no difference between 3.4.1 release and the dev build from 17th. Ill test it again when I get to work on my laptop where I discovered the issues. I see 2 possibilities. 1. It was fixed. 2. The PMD slowness was because I was running PMD at startup while the IDE was doing something else which has been slowed down, and if I do PMD after a few minutes it works fine. 3. Issue only shows on the slower laptop.
Seems clear on our side -- the parser does only shallow analysis when invoked from completion DB updater, does not search into JDK source dir (if the src.zip is automounted by the Editor).
That suspicion is enough ;-) NB-3.5 is supposed to be faster, not slower, than 3.4.1. By rt.jar I meant the basic runtime package in the JDK (it is located in your JDK/JRE installation in in $jredir/lib).
Let me summarize what we know: 1) The parser is not the culprit here. 2) The issue does not appear on the 2100+ machine with XP. 3) The issue likely appears on the slower PIII333 machine - will be tested. If the issue still occurs could you please watch the processor usage once the IDE start is finished (whether the parser db updating occurs) and possibly take several thread dumps and attach them to this issue? Thank you.
Perhaps IDE startup and project switching slowness is related to Issue #31974. Im still checking my laptop.
I run windows 2000 on an XP2100+ for the record, not XP, which is currently holding my coffee cup ;) Yes this still is happening on my Laptop and I have thread dumps attached. 3, the last one was upto ~5 minutes after project was opened. disk activity was still going. I could use mouse though, but file openning and stuff was slowed.
Created attachment 9443 [details] thread dumps
I modified PMD to spit out times. If I had a profiler I imagine it would be easy to tell where the hold up is!? ASCAD directory - 3/17 dev: 32937mS ASCAD directory - 3.4.1rel: 12257mS I think #2 with the JPS running when the IDE is first started was solved by issue 31974. It does not seem to happen in the 3/17 build. But My thought that it was the cause of the PMD slowdown is unfortunately wrong because PMD is still slowed as you can see by the numbers above.
Actually, whatever is slowing down PMD is likely slowing down the JSP too because they have similar actions. So that is probably exacerbating the situation.
Yes, it could be related to the issue #31974, because after project switching the parser DB files has been deleted and after the return to the project the files were parsed again. This problem has been already fixed in the maintrunk today, so please try to check tomorrow's build. In general the background parsing is started after new filesystem is mounted into the repository. If the parsing was not finished for particular filesystem during the IDE session, then it will be restarted during next session. This can explain the slowness after the startup. You can disable autoparsing after filesystem mounting via the property: Tools/Options/Editing/Editor Settings/Java Editor/Expert/Update Code Completion DB After Mounting If the background parsing is the reason of this performance problem, we can close this issue. Please check the issue #30781, where the similar problem is tracked.
Ops, the versions has been changed accidentally.
No, Martin. Part 1 of this issue would make issue 31974 and issue 30781 and part 2 of this issue worse!!! but its not the same issue. Those issues were fixed in the 3/17 build if I am not mistaken. And I have indicated that part 2. of the initial post for this issue I believe is fixed because of that. Part 1 persists. and In fact part 1 made part 2 worse while part 2 existed. So their was more than 1 bug. 1 of them has been fixed, but the other has not. When I am doing PMD, The parsing from changing projects has already stopped, AFAIK. The current issue is something that is making file operations (such as JavaSourceParser and PMD) 3x slower than before.
OK. Then, please try to make thread dumps during PMDing and attach it again. As you said the parsing has already stopped, there shouldn't be JCUpdater thread in the dumps anymore. (In the thread dumps you attached before it was). Thanks.
Well I ran it several times and did thread dumps. I did catch the JSP running once. Dont know why because it was already finished in the prior 3. seems like PMD is causing it to run? or is it periodic?
Created attachment 9450 [details] thread dump
I did PMD on the NetBeansIDE-dev-200303040100 1st run ASCAD directory - 36000 2nd run ASCAD directory - 12600 3rd run ASCAD directory - 12000 Perhaps some caching has been added / removed? 3.4.1 does ~12000 on its first pass. And I verified that no JPS was running before I started the PMD.
I did PMD on the NetBeansIDE-dev-200303070100 1st run ASCAD directory - 37400 2nd run ASCAD directory - 26100 3rd run ASCAD directory - 26600
I did PMD on the NetBeansIDE-dev-200303170100 1st run ASCAD directory - 12698 2nd run ASCAD directory - 09639 3rd run ASCAD directory - 08762 Of course this was after I removed the -J-Xnoagent -J-Djava.compiler=NONE from the ide.cfg :-D Sorry, my mistake.
OK, so we have sped up everything else in the mean time ;-) Now, I think I have a suggestion for PMD integration speedup: When you're checking whether the file is a Java file, don't use SourceCookie. Asking for SourceCookie will probably force the Java module to parse the source in question, which is useless for PMD as it is doing its own parsing.
Do you know any other way to tell its a java file? Can we check its extension some how?
Yes, you can e.g. check the extension. You can also check if it is a JavaDO if you can depend on java module. so either: do.getPrimaryFile.hasExt("java") or do.getCookie(JavaDataObject.class) The latter will cover forms as well (they have .form as its primary, so the former case will fail)