This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 214624

Summary: [72cat] Big number of not only Slowness regressions occured during developing/debugging Atlassian Maven-based project
Product: projects Reporter: crazyjavahacking
Component: MavenAssignee: Milos Kleint <mkleint>
Status: RESOLVED INVALID    
Severity: normal    
Priority: P3    
Version: 7.2   
Hardware: PC   
OS: Windows 7 x64   
Issue Type: DEFECT Exception Reporter:
Attachments: screensohot with message.

Description crazyjavahacking 2012-06-22 12:13:28 UTC
During developing Atlassian based Maven project I am having big problems with NetBeans usability including not only, but also many Slownesses. The IDE is really almost useless.
I am heading multiple Slowness regressions and also JUnit runner ignores the single selected method to test or debug. I will describe the process how to reproduce all of them:

To setup NetBeans:
====================
1.) Download Atlassian SDK from "https://developer.atlassian.com/display/DOCS/Atlassian+Plugin+SDK+Documentation"
2.) Set it as Maven home (Options->Java->Maven->Maven Home)
  => It should recognize it as Maven 2.x
3.) Put the bin subdirectory of the SDK directory on PATH environment variable

To setup the environment:
====================
4.) Clone the repository: "https://bitbucket.org/atlassian/jira-bitbucket-connector"
5.) Switch to branch jira 4.x
6.) If the NetBeans will not recognize the "atlassian-plugin" packaging, exit NetBeans, locate the project directory with console and execute "atlas-mvn eclipse:eclipse"
  => atlas-mvn will delegate the call into bundled maven 2.x version and download a lot of dependencies
  => now you should be able to correctly open the project in NetBeans

To reproduce the multiple bugs:
====================
7.) Locate GithubRepositoriesTest and select the "addRepoThatDoesNotExist" method
8.) Double click on the method name to select it, then right click on it and select "Run Focused Test Method"
  => problem 1: it will clash because of multiple Ant libraries on the classpath

9.) To solve it, open the pom.xml and click on the Graph tab, right after that back to the Source tab and if you will try to click on project nodes in Projects window, you will get the first Slowness
  ===== !!!!! 1st Slowness !!!!! =====

10.) Go back to the pom.xml and the Graph tab
11.) Search for Ant and exclude it by right clicking on Ant Visual Library node and selecting Exclude
  => Background scanning of projects will run and after that, the IDE will not be responsive once again for few seconds
  ===== !!!!! 2nd Slowness !!!!! =====

12.) Now you are finally able to run the method we initially wanted to execute, repeat steps 7.) and 8.)
  => my Maven command line is as following: "cd C:\Users\Martin Skurla\Desktop\jira-bitbucket-connector; "JAVA_HOME=C:\\Program Files\\Java\\jdk1.6.0_33" M2_HOME=C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven\\bin\\mvn.bat -Dtest=it.com.atlassian.jira.plugins.bitbucket.GithubRepositoriesTest -Dnb.single.run.methodName=addRepoThatDoesNotExist test-compile surefire:test"
  => important part is the -Dnb.single.run.methodName parameter, so just one method should be executed
  => It will took some time, but the result is that all test methods will run, not just the one that was focused 
  ===== !!!!! The JUnit runner will ignore the setting and all tests will run !!!!! =====
  => all the tests should pass

Now lets imagine that there is a bug in the method and you need to run it again or debug it.
====================
13.) Repeat steps 7.) and 8.) and then repeat step 7.) and try to "Debug Focused Test Method"
  => in both cases there is again the same problem with Ant libraries, but how it is even possible? Ant is excluded from Pom in 2 places...
14.) Go to pom.xml and the Source tab
15.) Remove previously added Ant exclusions
16.) Save changes in the pom.xml (e.g. CTRL + SHIFT + S)
  => You will see the background scanning of the project and after that another IDE Slowness
  ===== !!!!! 3rd Slowness !!!!! =====
  => this is probably the duplicate of the 2nd Slowness

17.) Repeat steps 7.) and then "Debug Focus Test Method"
  => my Maven command line is as following: "cd C:\Users\Martin Skurla\Desktop\jira-bitbucket-connector; "JAVA_HOME=C:\\Program Files\\Java\\jdk1.6.0_33" M2_HOME=C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven\\bin\\mvn.bat -Dtest=it.com.atlassian.jira.plugins.bitbucket.GithubRepositoriesTest -DforkMode=once "-Dmaven.surefire.debug=-Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=58785" -Djpda.listen=true -Dnb.single.run.methodName=addRepoThatDoesNotExist -Djpda.address=58785 test-compile surefire:test"
  => again, just one method according to the "-Dnb.single.run.methodName" argument should be executed, but all of the test methods will be executed
  =====!!!!! The JUnit runner will ignore the setting and all tests will be debugged !!!!! =====

Not only all tests are run or debugged all the time, but because of the weird NetBeans behavior, I need to exclude or remove the exclusion before any run or debug of tests methods which includes many Slownesses during the process making the NetBeans IDE almost useless and very painful.


Affects both NetBeans Dev 201206220002 and NetBeans 7.2 RC1

I am using Java 6u33, 64bit and the changes in the netbeans.conf are: "-J-server -J-Xms512m -J-Xmx1024m"

Please let me know if you can or cannot reproduce it or you need any additional settings, or anything else. Otherwise I will have to switch to Eclipse which I don't want to do.
Comment 1 Milos Kleint 2012-06-22 12:58:01 UTC
These are most probably unrelated issues that should have been files separately.

(In reply to comment #0)
> During developing Atlassian based Maven project I am having big problems with
> NetBeans usability including not only, but also many Slownesses. The IDE is
> really almost useless.
> I am heading multiple Slowness regressions and also JUnit runner ignores the
> single selected method to test or debug. I will describe the process how to
> reproduce all of them:
> 
> To setup NetBeans:
> ====================
> 1.) Download Atlassian SDK from
> "https://developer.atlassian.com/display/DOCS/Atlassian+Plugin+SDK+Documentation"
> 2.) Set it as Maven home (Options->Java->Maven->Maven Home)
>   => It should recognize it as Maven 2.x
> 3.) Put the bin subdirectory of the SDK directory on PATH environment variable

I'm extremely reluctant to say we even support that. The wiki page doesn't mention why we need a special Atlassian maven, why it's special and how it's inner workings differ from stock 2.x maven.
It could be just hardcoded remote repositories or quite some more customizations..


> 
> To setup the environment:
> ====================
> 4.) Clone the repository:
> "https://bitbucket.org/atlassian/jira-bitbucket-connector"
> 5.) Switch to branch jira 4.x
> 6.) If the NetBeans will not recognize the "atlassian-plugin" packaging, exit
> NetBeans, locate the project directory with console and execute "atlas-mvn
> eclipse:eclipse"
>   => atlas-mvn will delegate the call into bundled maven 2.x version and
> download a lot of dependencies
>   => now you should be able to correctly open the project in NetBeans
> 

not sure why eclipse:eclipse IMHO could be any maven command like mvn install. If the Netbeans' own priming build or "Build" action doesn't work, then I'd say we don't support that setup.


> To reproduce the multiple bugs:
> ====================
> 7.) Locate GithubRepositoriesTest and select the "addRepoThatDoesNotExist"
> method
> 8.) Double click on the method name to select it, then right click on it and
> select "Run Focused Test Method"
>   => problem 1: it will clash because of multiple Ant libraries on the
> classpath
> 

That's most likely a consequence of having Compile On Save enabled for tests. The internal java runner for tests uses ant as backend to running the "quick" CoS test executions. No solution apart from disabling CoS for tests in the projects that use ant.
Alternatively could be a problem in the project metadata (org.apache.ant groupId vs ant groupId) or in the custom maven distribution setup.



> 9.) To solve it, open the pom.xml and click on the Graph tab, right after that
> back to the Source tab and if you will try to click on project nodes in
> Projects window, you will get the first Slowness
>   ===== !!!!! 1st Slowness !!!!! =====

For every slowness experienced, you should be getting the Slowness detector reports showing up in the status bar of the IDE. That includes a profiling snapshot useful for debugging the problem.

> 
> 10.) Go back to the pom.xml and the Graph tab
> 11.) Search for Ant and exclude it by right clicking on Ant Visual Library node
> and selecting Exclude
>   => Background scanning of projects will run and after that, the IDE will not
> be responsive once again for few seconds
>   ===== !!!!! 2nd Slowness !!!!! =====
> 

same as with point 9.

> 12.) Now you are finally able to run the method we initially wanted to execute,
> repeat steps 7.) and 8.)
>   => my Maven command line is as following: "cd C:\Users\Martin
> Skurla\Desktop\jira-bitbucket-connector; "JAVA_HOME=C:\\Program
> Files\\Java\\jdk1.6.0_33"
> M2_HOME=C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven
> C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven\\bin\\mvn.bat
> -Dtest=it.com.atlassian.jira.plugins.bitbucket.GithubRepositoriesTest
> -Dnb.single.run.methodName=addRepoThatDoesNotExist test-compile surefire:test"
>   => important part is the -Dnb.single.run.methodName parameter, so just one
> method should be executed

The important question is how the property got there in the first place, it's internal to the netbeans codebase and on execution preparation is converted to -Dtest=com.my.project.mavenproject1.AppTest#testApp where #testApp is the focused method.

And it works for me in the sample project of mine, so I will need more details about the problem and how to reproduce it.

>   => It will took some time, but the result is that all test methods will run,
> not just the one that was focused 
>   ===== !!!!! The JUnit runner will ignore the setting and all tests will run
> !!!!! =====
>   => all the tests should pass
> 
> Now lets imagine that there is a bug in the method and you need to run it again
> or debug it.
> ====================
> 13.) Repeat steps 7.) and 8.) and then repeat step 7.) and try to "Debug
> Focused Test Method"
>   => in both cases there is again the same problem with Ant libraries, but how
> it is even possible? Ant is excluded from Pom in 2 places...

As mentioned before, it's probably the problem with Compile on Save. Given that it works in step 16-17, it's more than apparent. Everytime the pom is changed, we disable CoS temporarily because a significant portion of the build could have changed and we need to get a complete maven build again.

> 14.) Go to pom.xml and the Source tab
> 15.) Remove previously added Ant exclusions
> 16.) Save changes in the pom.xml (e.g. CTRL + SHIFT + S)
>   => You will see the background scanning of the project and after that another
> IDE Slowness
>   ===== !!!!! 3rd Slowness !!!!! =====
>   => this is probably the duplicate of the 2nd Slowness
> 
> 17.) Repeat steps 7.) and then "Debug Focus Test Method"
>   => my Maven command line is as following: "cd C:\Users\Martin
> Skurla\Desktop\jira-bitbucket-connector; "JAVA_HOME=C:\\Program
> Files\\Java\\jdk1.6.0_33"
> M2_HOME=C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven
> C:\\devel\\atlassian-plugin-sdk-3.10.4\\apache-maven\\bin\\mvn.bat
> -Dtest=it.com.atlassian.jira.plugins.bitbucket.GithubRepositoriesTest
> -DforkMode=once "-Dmaven.surefire.debug=-Xdebug
> -Xrunjdwp:transport=dt_socket,server=n,address=58785" -Djpda.listen=true
> -Dnb.single.run.methodName=addRepoThatDoesNotExist -Djpda.address=58785
> test-compile surefire:test"
>   => again, just one method according to the "-Dnb.single.run.methodName"
> argument should be executed, but all of the test methods will be executed
>   =====!!!!! The JUnit runner will ignore the setting and all tests will be
> debugged !!!!! =====

Again nb.single.run.methodName has no meaning to maven surefire plugin and is therefore ignored.

> 
> Not only all tests are run or debugged all the time, but because of the weird
> NetBeans behavior, I need to exclude or remove the exclusion before any run or
> debug of tests methods which includes many Slownesses during the process making
> the NetBeans IDE almost useless and very painful.
> 

Disabling Compile on Save for tests or every execution should do the job more reliably.

> 
> Affects both NetBeans Dev 201206220002 and NetBeans 7.2 RC1
> 
> I am using Java 6u33, 64bit and the changes in the netbeans.conf are:
> "-J-server -J-Xms512m -J-Xmx1024m"
> 
> Please let me know if you can or cannot reproduce it or you need any additional
> settings, or anything else. Otherwise I will have to switch to Eclipse which I
> don't want to do.


-> P2, no showstopper I could identify for 7.2
Comment 2 Milos Kleint 2012-06-22 13:11:03 UTC
I haven't tested with the atlassian sdk as it requires me to sign up with atlassian.
Comment 3 Milos Kleint 2012-06-22 13:29:07 UTC
note to self: adding https://maven.atlassian.com/content/repositories/m1 and https://maven.atlassian.com/content/repositories/public seems to get one another mile ahead in building the project, but some artifacts are still unknown
Comment 4 crazyjavahacking 2012-06-22 14:44:05 UTC
Thanks very much for fast response.

This weekend I am doing OS reinstall and then I will have to setup all the infrastructure on my laptop. I will update the notes then or create a VirtualBox image.
Comment 5 crazyjavahacking 2012-06-25 01:28:20 UTC
Here is the packed VirtualBox Image with working environment:
http://crazyjavahacking.org/NetBeans%20Image.rar
Comment 6 crazyjavahacking 2012-06-25 06:47:58 UTC
A small note to execute those test methods:

1.) Switch to branch jira 4.x
2.) Clean install all modules without tests
3.) Run the JIRA server by executing "atlas-debug -Dmaven.test.skip" from the project directory
4.) Follow steps starting from 7.)
Comment 7 Milos Kleint 2012-06-25 08:42:40 UTC
Thank you for your time preparing the virtual machine. It made it way easier to reproduce the problem.

After switching to jira branch, I had problems loading the project. While the build succeeded with the attlasian maven, the project loading done with embedded 3.0.4 failed to find some dependencies/extensions/plugins in local repository.  That's primarily a problem in different artifact resolution in maven 2.x and 3.x. See [1] for details, cannot tell what exactly is responsible in this project.
A workaround I found was to switch temporarily to the Bundled maven in tools/Options and try a build. That downloaded the missing artifacts and project loaded. 
On the IDE side, there's no way of fixing the problem, we have to use the bundled version for project loading. And we have to do it OFFLINE (using just the content of local repository), to be even remotely fast enough for the IDE's purposes. 

To the testing part:
1. When I invoke the Run Single Test method action, it gives me the following text in the status bar and rejects execution. See screenshot attached. The text is not entirely correct. You not only need surefire 2.8+ but you also need junit 4.8+ (or testng). I'll be correcting that message. 
Your project doesn't fulfill any of these conditions. Both your surefire and junit versions are lower.After upgrading I was able to execute the tests using maven. It seems to execute just the test file just right and the -Dtest=class#method parameter.

The status bar suggests you should be using Compile On Save, but in your case I believe it's misleading. When I do so, The "2 versions of ant" error appears once the IDE's ant-based CoS runner attempts to run the test. Additionally I believe I've noticed some extra build steps (downloading a webapp?) that precede the test execution and that might be missing in the CoS execution (or not, I haven't gone that far in trying CoS).

There is no way to recognize in the IDE that CoS execution of tests (or main classes) will actually work. There could be class file postprocessing going on etc. Or just simply a test file is only executed inside a profile (read: integration tests) and hasn't been resolved before. There's no way of knowing that. So we blindly allow CoS  execution and hope it works out fine after the first execution is done by maven. That's what you experienced in some of your steps. After the pom is modified, we cannot tell what changed so we get conservative and do one maven based execution. But the only way to get consistent results is to always use Maven for execution. The price to pay is speed.





[1] https://cwiki.apache.org/MAVEN/maven-3x-compatibility-notes.html
Comment 8 Milos Kleint 2012-06-25 08:43:14 UTC
Created attachment 121285 [details]
screensohot with message.
Comment 9 Milos Kleint 2012-06-25 09:01:59 UTC
I don't see much we can do to get rid of the problem, only in notification of the user when something gets wrong.

1. I've updated the message in status bar to mention junit 4.8+.
http://hg.netbeans.org/core-main/rev/5635b2988a0c

2. I'll try to experiment with showing a warning popup message when CoS build fails, with suggestion to disable CoS entirely.
Comment 10 crazyjavahacking 2012-06-25 10:25:28 UTC
Thank you very much for the time investing in this "bug".

I was doing the VirtualBox image very late in the night so I forget to switch to the jira 4.x branch, sorry for that.

Today I will try to work with the environment you described.

Anyway, do you need VirtualBox with working 4.x branch? I can spend some additional time with switching and solving the dependency problems (so jira 4.x branch could be working out-of-the-box) if it could be helpful.


Thanks again
Comment 11 Milos Kleint 2012-06-25 11:32:45 UTC
(In reply to comment #10)
> Anyway, do you need VirtualBox with working 4.x branch? I can spend some
> additional time with switching and solving the dependency problems (so jira 4.x
> branch could be working out-of-the-box) if it could be helpful.

Not really necessary, I made it work eventually..
Comment 12 Milos Kleint 2012-06-25 11:43:09 UTC
->p3 the problem on netbeans side is only missing or misleading warning ui. the
rest is on the external tool part.
Comment 13 Milos Kleint 2012-06-25 17:10:33 UTC
http://hg.netbeans.org/core-main/rev/024d3f74501a
also marks the output tab with /CoS and shows an information bubble (just like we have one when the test execution is not done through Compile on Save/Ant execution and is using the regular Maven build)
Comment 14 Quality Engineering 2012-06-26 04:57:06 UTC
Integrated into 'main-golden', will be available in build *201206260002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/5635b2988a0c
User: Milos Kleint <mkleint@netbeans.org>
Log: #214624 have the version of junit mentioned in status bar message
Comment 15 crazyjavahacking 2012-06-26 05:39:52 UTC
The important question we almost forget: Do you recognize the slownesses as I described during that setup?
Comment 16 Milos Kleint 2012-10-11 13:29:24 UTC
(In reply to comment #15)
> The important question we almost forget: Do you recognize the slownesses as I
> described during that setup?

if you install a daily build or 7.3 beta and try again, the slownesses should be turned into small bees showing up on your status bar, when you click it you should be able to report the slowness to the netbeans.org servers, along with a profiling snapshot and what not. That's the best way of reporting slownesses. Generally 2 threads are covered: AWT thread slowness and code completion.
Comment 17 crazyjavahacking 2012-10-29 09:10:18 UTC
Turns out that this is not related to Atlassian specific Maven. Reproducible on bundled Maven (3.0.4).

Closing this one and creating 2 new ones http://netbeans.org/bugzilla/show_bug.cgi?id=220995 and http://netbeans.org/bugzilla/show_bug.cgi?id=220996.