This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 231892 - Please stabilize a little bit xml.axi.SchemaGeneratorTest.* tests
Summary: Please stabilize a little bit xml.axi.SchemaGeneratorTest.* tests
Status: RESOLVED FIXED
Alias: None
Product: xml
Classification: Unclassified
Component: Schema (show other bugs)
Version: 7.4
Hardware: All All
: P3 normal (vote)
Assignee: Svata Dedic
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-06-27 05:02 UTC by Martin Fousek
Modified: 2013-08-01 02:34 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Fousek 2013-06-27 05:02:42 UTC 
We are running XML tests as a part of web-main tests and the last results are up to 16 failures in row in xml.asi.SchemaGeneratorTest.*:
>>> o.n.m.xml.axi.SchemaGeneratorTest.testDeleteExistingLocalElement 	1.866	4
>>> o.n.m.xml.axi.SchemaGeneratorTest.testSimpleToComplexContent 	0.012	4
>>> o.n.m.xml.axi.SchemaGeneratorTest.testCreateSimpleContent 	0.012	16

http://bertram.netbeans.org:8080/job/web-main-tests/709/testReport/

We have more random failures in our build but we are trying to remove at least the most unstable ones. Thanks a lot.
Comment 1 Petr Jiricka 2013-07-30 20:29:01 UTC 
Still failing, currently 11 times in a row. Svata, can you please investigate? Thanks.
Comment 2 Svata Dedic 2013-07-31 08:52:46 UTC 
The test is a bad joke ... the input data contains 4 global elements, but the testcases check for exactly 3 global elements ?? It seems that the test passes only by a chance when some race condition or a _real_ error is hit. Or I am reading to code wrong.
Comment 3 Martin Fousek 2013-07-31 09:18:52 UTC 
(In reply to comment #2)
> The test is a bad joke ... the input data contains 4 global elements, but the
> testcases check for exactly 3 global elements ?? It seems that the test passes
> only by a chance when some race condition or a _real_ error is hit. Or I am
> reading to code wrong.

I think that it's quite good possible - we are facing these failures since switching builds to Java 7. Probably it could contain this race condition on JDK6 regularly before. ;)
Comment 4 Svata Dedic 2013-07-31 09:27:19 UTC 
Thanks for the hint ! I'll check for differencies on both JDKs.
Comment 5 Svata Dedic 2013-07-31 10:01:02 UTC 
So - the outcome of the test relied on order of test methods, as they reused Schema mode in the AbstractModelFactory cache. Since some of the methods created and some deleted global elements, they had to execute in some specific order (testDeleteExistingGlobalElement is teextually first, so it deleted global element and subsequent tests checked the modified value).

Next, since the test was written with the reuse in mind, it would fail randomly even with the correct execution ordering if the WeakReference for the cached model expired for some reason.

The fix was to clear the cache in tearDown(), which hopefully makes the testcases independent.

Fixed in http://hg.netbeans.org/jet-main/rev/6c433e0cfff7
Comment 6 Martin Fousek 2013-07-31 11:30:14 UTC 
Thanks a lot Svato for evaluation and fixing!
Comment 7 Quality Engineering 2013-08-01 02:34:48 UTC 
Integrated into 'main-silver', will be available in build *201307312300* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/6c433e0cfff7
User: Svata Dedic <sdedic@netbeans.org>
Log: #231892: tests clears weak cache between testcases. Fixed actual GE counts