Bug 61014

Summary: Rewrite/enhance current tests
Product: apisupport Reporter: Martin Krauskopf <mkrauskopf>
Component: ProjectAssignee: Martin Kozeny <mkozeny>
Status: NEW --- QA Contact: issues <issues.netbeans.org>
Priority: P1 CC: jglick, pzajac
Version: 5.xKeywords: TEST
Target Milestone: TBD   
Hardware: All   
OS: All   
Whiteboard:
Issue Type: TASK Exception Report:
Bug Depends on: 87802, 94821, 103078, 110844    
Bug Blocks:    

Description Martin Krauskopf 2005-07-15 09:50:58 UTC
Currently a lot of tests are written in the manner that they copy
example modules (example-external-projects directory in test data)
and subseqently use them for testing. This may be enough for simple
tests but is not enough for special cases like subpackages. Since
this would require additional editing of those modules which
originally served only as examples for new module developers (as it
is stated in the README there).
So either:
1) generate a new data for test cases as they individually needed
2) or create rich artificial projects in some test folder which
   would contains modules with everything possible that can be
   tested

Both ways have theirs pros and cons. Hints/experience from others
are welcome :)
Comment 1 Jesse Glick 2005-07-15 15:37:55 UTC
Style #1 is more likely to be maintable, I think.

The toughest part is testing netbeans.org-specific functionality. Ideally would
create a complete fake netbeans.org CVS tree - i.e. just a handful of dummy
files in nbbuild/ plus whatever fake modules are needed to make the test run.
Would be better than the current situation where a lot of tests have to be
modified every time the deps list of some netbeans.org module changes, for example.
Comment 2 Martin Krauskopf 2005-07-22 20:51:40 UTC
Let's make this started. I think it is the right time to begin write something
more inteligent than copying examples. Probably current tests will leave as they
are for a while (may be not) but new ones shouldn't be written in such a dirty
manner.
Comment 3 Martin Krauskopf 2005-08-31 15:16:10 UTC
Basic task is done. We have good enough infractructure for easily generate
arbitrary modules, suites and suite components. All new tests should use it and
probably all "touched" already existing tests should be always rewritten to use
the generating methods.

But there is still netbeans.org module phase. I don't think it is fixable soon.
Closing.
Comment 4 Jesse Glick 2005-08-31 16:32:54 UTC
Don't consider this fixed yet, even allowing that nb.org tests cannot be easily
rewritten at this point.

1. apisupport/project/test/unit/data/example-external-projects still exists and
could probably be removed entirely.

2. apisupport/project/test/cfg-unit.xml ought to at least have an additional
testbag which includes every test that does not assume a nb.org CVS checkout, so
we can have it be run automatically on tester machines.
Comment 5 Martin Krauskopf 2005-09-12 17:15:47 UTC
Continuing... "anybody" feel free to rewrite tests when you are enhancing,
correcting, ... it ;)

Checking in project/ProjectXMLManagerTest.java; 1.22 --> 1.23

TODO (to get rid of example-external-projects):

  queries/SourceForBinaryImplTest.java
  queries/UpdateTrackingFileOwnerQueryTest.java
  queries/ClassPathProviderImplTest.java
  queries/JavadocForBinaryImplTest.java
  queries/SubprojectProviderImplTest.java
  NbModuleProjectTest.java
  universe/ModuleListTest.java


Comment 6 Jesse Glick 2005-10-07 22:22:28 UTC
*** Issue 66126 has been marked as a duplicate of this issue. ***
Comment 7 Martin Krauskopf 2006-08-18 11:56:49 UTC
I've just did a quick try to cache generated projects - just
generateStandaloneModuleDirectory(); and time spent in the method for all tests
was about 25-30 seconds better then without caching. Probably if we have some
sophisticated caching for suites and suites-components as well we could be about
1-2 minutes faster. But probably does not worth the effort (with current 10-12
minutes for whole apisupport-xtest run).
Comment 8 Jesse Glick 2006-08-28 20:21:30 UTC
When fixed, can

s/stable-cvs/stable/g
Comment 9 Martin Krauskopf 2008-02-07 17:50:40 UTC
I'm not working on APISupport anymore. Reassigning to owner of the component, so
the issue is not 'forgotten' forever.
Comment 10 Jesse Glick 2008-06-20 03:13:26 UTC
nb.org modules probably cannot be safely tested. For everything else, test projects should be generated programmatically
as part of the test; this is generally easier to maintain than samples checked into version control.
Comment 11 Quality Engineering 2008-07-10 04:11:07 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #310 build
Changeset: http://hg.netbeans.org/main/rev/548a4f10fa24
User: Richard Michalsky <rmichalsky@netbeans.org>
Log: #61014: Fixing apisupport tests...
Comment 12 Quality Engineering 2008-07-17 04:36:22 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #324 build
Changeset: http://hg.netbeans.org/main/rev/f2a1ee3aaf0b
User: Richard Michalsky <rmichalsky@netbeans.org>
Log: #61014: ClassPathProviderImplTest, not yet passing
Comment 13 Quality Engineering 2008-09-30 06:04:53 UTC
Integrated into 'main-golden', will be available in build *200809300201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/8716056fd119
User: Richard Michalsky <rmichalsky@netbeans.org>
Log: Commented out failing tests so that they can be run regularily, part of #61014 task.
Comment 14 rmichalsky 2009-03-25 15:00:06 UTC
*** Issue 161144 has been marked as a duplicate of this issue. ***
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo