This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 107594 [details] Screenshot of failures in IDE test window While working on #197065, I found that at least two tests from the NbJunit module fail when run from the IDE, though they work fine from the command line. I wanted to report this as a separate issue so as not to pollute #197065 with a lot of extraneous detail. I built the IDE from source, using the revision listed below. Here is the output of some commands that Jesse asked me to run: hg version: 1.8.1 java version: 1.6.0_24-b07 ant version: 1.8.2 hg clean --all hg: unknown command 'clean' hg up -c default 0 files updated, 0 files merged, 0 files removed, 0 files unresolved hg st (no output) hg log -r . changeset: 196003:350d60bcb2f8 parent: 196000:b7adfd69579d parent: 196002:7e43ca3e9dbc user: ffjre@netbeans.org date: Tue Apr 05 16:18:14 2011 +0200 summary: Merge of http://hg.netbeans.org/cnd-main/ ant -Dcluster.config=basic build-nozip BUILD SUCCESSFUL ... lots of output ... Total time: 17 minutes 12 seconds ant -f nbjunit/build.xml test ... lots of output ... BUILD SUCCESSFUL Total time: 4 minutes 25 seconds What is odd is that I've configured the IDE to use my own version of Ant rather than the one shipped with NetBeans, so I'd think the result of test execution would be the same in either case. The number of tests that fail vary: in one execution I'll have two tests fail, then when I run it again, I'll find that four tests have failed. I've attached a screenshot of the test window from my IDE showing the test failures. I will also attach another screenshot showing a different set of failures.
Created attachment 107595 [details] Another screenshot showing variation of test failure This screenshot was taken just a few minutes after the first one, during the same IDE session. Instead of two tests failing, this one illustrates that four have failed. There was no known change to the source or environment in the interim.
(In reply to comment #0) > hg clean --all > hg: unknown command 'clean' hg --config extensions.purge= clean --all > What is odd is that I've configured the IDE to use my own version of Ant rather > than the one shipped with NetBeans ...and what happens when you run with the stock Ant (after restarting the IDE to be sure)? Might just be a Windows issue; lots of things fail on Windows and you may well be the only Windows user attempting to run tests for this module.
I re-ran the following commands: hg --config extensions.purge= clean --all hg up -c default hg st hg log -r . ant -Dcluster.config=basic build-nozip ant -f nbjunit/build.xml test then reconfigured my IDE to use the default (bundled) ant and restarted it. Several tests (5 in this case) still failed. It may be a Windows-only problem. Unfortunately the only UNIX machine I have handy right now is an SGI and it doesn't have Java 6. If I get a chance, I will try it under Linux or Solaris at home.
Created attachment 107604 [details] Runs OK on our test farm
Tom, please run one test that fails for you and attach log from the test run. I'll see what I can guess from that, but things look OK as I just verified on our hudson4qe (see the attached picture).
Jesse may be on to something. I ran all the tests several times under Linux last night and they all passed 100% of the time. On MS Windows, several tests fail randomly, but it's consistent that at least some fail. I tried to run just a single test from a single test case, but could not get it to fail. The behavior I've observed only seems to happen when running an entire test case or all the tests in an entire module. I found from the log (which I'll attach momentarily) that several are "can't delete file" errors, most likely caused by Windows holding a file lock from the previous test. But at least one test (testClustersCanBeCumulated) seems to fail for a different (but unknown) reason.
Created attachment 107615 [details] logfile from test execution on MS Windows XP
Created attachment 107616 [details] Screenshot of failures in IDE test window, corresponding to 197065_log.txt attachment