This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 19443 - Separate openide into libraries
Summary: Separate openide into libraries
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 3.x
Hardware: All All
: P2 blocker (vote)
Assignee: Jaroslav Tulach
URL: http://openide.netbeans.org/proposals...
Keywords: API, ARCH, UMBRELLA
Depends on: 28227 14722 16854 17431 20538 20539 20898 20942 22758 26126 27911 28048 28158 28259 28422 29125 29130 29458 29460 29461 29914 30998 31042 32143 34758 57941 58258
Blocks: 28325 58807
  Show dependency tree
 
Reported: 2002-01-15 09:25 UTC by Jaroslav Tulach
Modified: 2008-12-22 19:49 UTC (History)
10 users (show)

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
Revised patch (204.64 KB, patch)
2002-07-29 20:30 UTC, Jesse Glick
Details | Diff
Patch to nb_all - missing just a couple of manifest spec vers changes and addition of CurrentProjectNode to projects module (1.00 MB, patch)
2002-10-16 07:03 UTC, Jesse Glick
Details | Diff
Commit log (204.83 KB, text/plain)
2002-10-16 07:19 UTC, Jesse Glick
Details
Patch to NB sources, gzipped (note: addn'l last-minutes changes also to core/ui/manifest.mf and nbbuild/build.properties to make platform build work) (117.53 KB, application/octet-stream)
2002-11-02 21:28 UTC, Jesse Glick
Details
Commit log (gzipped) (10.98 KB, application/octet-stream)
2002-11-02 21:28 UTC, Jesse Glick
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jaroslav Tulach 2002-01-15 09:25:07 UTC
The openide was designed around TopManager. That was not bad decision then, but
as seen from currrent point of view the TopManager connects too much
functionality together and makes separation of openide into smaller pieces hard.

In present days it is obsolete because of Lookup and some parts has already been
modified to use it, but still there is a bunch of other libraries that need to
be changed not to depend on TopManager.
Comment 1 Jaroslav Tulach 2002-01-15 09:30:04 UTC
ErrorManager.getDefault () should simplify life of all module writers:
issue 16854
Comment 2 Jaroslav Tulach 2002-01-15 10:45:03 UTC
Lookup should be enhanced to allow "standard" registration of entries
- issue 14722
Comment 3 Jesse Glick 2002-02-05 13:08:36 UTC
Probably all libraries which are not yet fixed should be listed here,
and separate tasks filed.

Also we need a task for someone to write some kind of test suite that
can be automated and which checks that library separation is not
violated: perhaps that things are separately compilable (will not work
for e.g. TMUtil classes); or that all classes reachable (at compile
time) from a given JAR's entry points can be loaded with no
NoClassDefFoundError's. The latter might be possible using a bytecode
analyzer, say. Such a test will keep us honest. See issue #17431.
Comment 4 Jesse Glick 2002-05-17 19:41:39 UTC
So is this task FIXED or what?
Comment 5 Jaroslav Tulach 2002-05-20 12:04:18 UTC
Well, I am not sure. We can continue to work on additional stuff -
move of org.openide.src.* to java modules, etc. In next versions. I'll
change the target milestone and assign it to myself to specify the
subtasks.
Comment 6 Jaroslav Tulach 2002-07-15 07:40:06 UTC
A branch sepration_19443 has been created for modules nbbuild, core 
and openide. The goal is to remove compat.jar and cleanup 
interdependencies between different parts of openide while still 
maintaining compatibility. 
Comment 7 Jaroslav Tulach 2002-07-15 07:40:29 UTC
A branch separation_19443 has been created for modules nbbuild, 
core and openide. The goal is to remove compat.jar and cleanup 
interdependencies between different parts of openide while still 
maintaining compatibility. 
Comment 8 Jaroslav Tulach 2002-07-19 08:24:08 UTC
Work finished, updated with main trunk and available in xtest_19433
branch. The xtest has yet to be updated. Use:

cvs co -r xtest_19433 nbbuild core openide xtest

to get that branch.
Comment 9 Jaroslav Tulach 2002-07-19 08:36:32 UTC
The issue is not blocked by issue 25807, the xtest framework does not
work. Has to be solved before we can integrate it.
Comment 10 Jaroslav Tulach 2002-07-29 12:55:51 UTC
Jesse, I have merged the xtest_19433 into the trunk and then remove
the changes. Trung made me recognize that you are the person that
should check the changes in because it most of all affects module
system and you are the responsible owner of it.

So please do the review, do changes which you want, and then apply the
xtest_19433 branch. Please keep in mind that these changes require a
work by release engineering (sigtest) and xtest (use of lib instead of
lib/ext) which should be done in synch in one day.

I'd approciate if the changes were done soon in release cycle, because
of possible hidden problems and also because this is necessary for
clean up of the APIs.
Comment 11 Jesse Glick 2002-07-29 20:30:21 UTC
Created attachment 6933 [details]
Revised patch
Comment 12 Jesse Glick 2002-07-29 20:43:15 UTC
I have attached a complete patch (nb_all, nbextra) against the current
trunk. It compiles and passes unit tests OK.

Unfortunately the goal of maintaining backward compatibility does not
work. I tried it with AbstractNode: I made a class:

package org;
import org.openide.nodes.*;
public class Foo {
    public static void main(String[] x) {
        AbstractNode n = new AbstractNode(Children.LEAF);
        System.out.println(n.getCookieSet());
    }
}

and compiled against a copy of AbstractNode.java that had getCookieSet
marked as public. I confirmed that the compiled class would run
against the patched AbstractNode.class but not regular openide.jar.

I then tried to run this class (precompiled) using internal execution
in an IDE built with this patch. It threw a VerifyError, indicating
that to the VM, getCookieSet is still protected.

Which is what I would expect: essentially the VM now sees:

public class AbstractNodePatch extends Node {
    // ...
    public abstract CookieSet getCookieSet();
}
public class AbstractNode extends AbstractNodePatch {
    // ...
    protected CookieSet getCookieSet() {...}
}

so classes compiled against AbstractNode see the method as protected,
not public.

So I have a number of problems with this patch.

1. Backward compatibility does not, in fact, work.

2. Some of the progress we previously made using openide-compat.jar
for patching is apparently gone in this system. For example,
CompilerGroup's add/removeCompilerListener is no longer final. And
Toolbar's bogus fields are now once again available for modification
to someone compiling from trunk source. So we actually have a
regression in the list of things which we successfully made clean
while keeping compatibility: even assuming we fix #1, we will have
reverted some of the APIs to their older, pre-cleanup state.

3. The stated goals of this issue report are not covered by the
current patch: it says "The goal is to remove compat.jar and cleanup
interdependencies between different parts of openide while still
maintaining compatibility.", of which we did not do the first (it is
only moved and changed format, not removed) and did not do the second.
So really this patch should be a sub-issue and keep #19443 as a
tracking issue.

4. There is not a clear explanation anywhere here of what is solved by
using the superclass patching that did not work using simple bytecode
overriding and the compatkit. If we are going to change how we handle
backwards compatibility, to a more complicated system, there must be
some justification of what we will get from it. Maybe you told me at
some point, but I cannot remember now, and it is not written down here.

5. For Martin specifically: I know I have asked before, but again
please consider removing Module.java from xtest ASAP; the patching
required to make a org.netbeans.core.modules.Module is getting really
ugly, and I have lost track of what changes there are in core.jar not
in xtest-ext.jar. I tried to apply some changes here, but I'm not at
all sure I got it right. The tests do seem to run as expected.
Comment 13 Martin Brehovsky 2002-07-30 09:19:29 UTC
Concerning XTest, Module.java patch is not available in XTest anymore
(xtest_19433 branch), all the required functinality was moved directly
to Module.java in core (see lines 803 to 826).
Comment 14 Jaroslav Tulach 2002-07-30 09:54:41 UTC
Well, if "1. Backward compatibility does not, in fact, work." then I
am pretty stupid. I tried that using scripting console and it worked
well, but if that does not work in Java the whole thing is
meaningless. I try to check it out.
Comment 15 Jesse Glick 2002-07-30 15:26:07 UTC
"Module.java patch is not available in XTest anymore (xtest_19433
branch)": here is another very good reason why a branch base tag is
nice! Module.java has no xtest_19433 tag at all, but it also has no
build tags, so there is no indication whatsoever in this file that it
does not exist on the branch (such as a dead rev). If it is supposed
to be deleted on a branch, you actually have to delete it... I spent
some time trying to patch it. Is this file
(xtest/src/org/netbeans/core/modules/Module.java) in fact supposed to
be deleted as part of the patch? (Don't worry, I'm not going to commit
anything in nb_all/xtest/ nor nbextra/xtest/, just asking so I can
test the patch in full first.)
Comment 16 Jesse Glick 2002-07-30 16:39:55 UTC
Please switch to issue #26126 for any further comments re. the
superclass replacement patch.
Comment 17 Martin Brehovsky 2002-07-30 16:43:19 UTC
Sorry Jesse, I forgot to create branch tag, I just created a simple
tag, so Module.java and org.netbeans.Main.java are supposed to be
deleted. Now the probem should be corrected (I created xtest_19433 branch)
Comment 18 Jesse Glick 2002-07-30 17:18:34 UTC
OK, well I am creating a new branch as part of #26126 anyway. I will
make sure Main.java and Module.java are deleted in this branch in
xtest sources.
Comment 19 Jesse Glick 2002-10-08 17:22:10 UTC
Things which are proposed:

- do #20898, i.e. move all of org.openide.src plus SourceCookie to
java.netbeans.org

- make a new autoload module for org.openide.execution plus
ExecCookie; some kind of interaction with core/execution, i.e.
core/execution serves as an impl companion module

- make a new deprecated autoload module for org.openide.compiler plus
CompilerCookie plus CompilerSupport; core/compiler serves as (again
deprecated) impl companion module

- make a new deprecated autoload module for general unwanted stuff
from openide; initially: all of org.openide.debugger and
DebuggerCookie; ExecSupport; TopManager, perhaps, if its remaining
useful methods can be placed elsewhere; maybe other stuff?

I will create a branch off the CVS trunk to hold such changes. Petr Z.
and I will work on it. Changes will be merged back to trunk if and
when they are "compatible", i.e. existing binary modules run
unchanged. Note that modules might still not run *usefully* due to
project changes - e.g. an old module with a CompilerType will not do
anything useful if 4.0 projects no longer uses CompilerType at all -
but at least (1) they should not throw linkage errors, (2) until 4.0
projects are merged, the trunk will continue to run with old-style
execution, compilation, and debugging using the 3.4 UI.
Comment 20 Jesse Glick 2002-10-09 03:58:43 UTC
Created branch on a few modules:

cvs rtag -b -r separation_19443_a separation_19443_a_branch
core_nowww_1 openide_nowww_1 java_nowww_1
Comment 21 Jesse Glick 2002-10-09 14:57:19 UTC
Interesting summary from Yarda:

http://java.netbeans.org/doc/projects/issues-compilation-execution.html
Comment 22 Jesse Glick 2002-10-09 15:22:49 UTC
Adding nbbuild and classfile to the set of branched module under
separation_19443_a_branch.
Comment 23 Peter Zavadsky 2002-10-09 16:01:08 UTC
Jesse, I have the branch now.
Please devide the separation to subtasks and assign some of them to me.

Comment 24 Jesse Glick 2002-10-09 16:15:07 UTC
Well, I am starting to break stuff up. The problem is that at this
early stage, I am still not entirely sure what kinds of problems we
will run into. So it is hard to assign subtasks: I don't yet know what
will need to be done. I will try to commit a snapshot of what I am
working on rather soon. So far I am creating submodules
openide/deprecated and java/srcmodel.
Comment 25 Peter Zavadsky 2002-10-09 17:02:08 UTC
OK.
Do you want to move the <stuff> to openide/deprecated first and just
then try to make those core/<compiler stuff>, core/<execution stuff>
and <rests stuff> submodules or you have changed the plan?
Comment 26 Jesse Glick 2002-10-09 17:28:05 UTC
Well, for now I'm moving a lot of classes to openide/deprecated and
java/srcmodel. If I have to create openide/compiler now, I will, but
it might be easier to wait on that. Creating proper core submodules is
a lower priority, I guess. For now I'm still trying to figure out
which classes are not going to compile and which should be moved around.
Comment 27 Jesse Glick 2002-10-09 17:47:59 UTC
First commit on the branch, take a look. First priority right now is
to get rid of usage of TopManager. Two categories here:

1. TopManager methods that have a proper replacement already. E.g.
systemClassLoader -> Lookup.lookup(ClassLoader).

2. Methods that have no equivalent yet. E.g. getIO, createDialog,
setStatusText, exit. Need new APIs somewhere. I am thinking about
using WindowManager for some of these, as it is logical enough... not
yet sure whether it is acceptable to have a bunch of other APIs
depending on WindowManager, though.

#1 you could start working on right away if you wanted - just try
compiling openide/build.xml and look at the breakage. For #2, I will
try to start writing such APIs.
Comment 28 Peter Zavadsky 2002-10-09 18:00:33 UTC
I see. If it wait till tommorow I start to work on #1 morning (I'm
leaving right now :-( ).
Comment 29 Jesse Glick 2002-10-10 03:47:13 UTC
Current status: org.openide.src is moved to java/srcmodel (except for
documentation etc.), and lib/openide-deprecated.jar contains dead
classes such as TopManager. IDE w/ just java module seems to run fine.

The main task now is to make core*.jar not depend on
openide-deprecated.jar, permitting that JAR to be moved to the
autoload area for eventual retirement. This is important: if core.jar
depends on TopManager, then it depends on Debugger, which depends on
ConstructorElement, which is in
netbeans/modules/autoload/java-src-model.jar. That means that if you
ran the debugger and tried to set breakpoints it is possible there
would be NoClassDefFoundError's. This needs to get cleaned up.
Comment 30 Peter Zavadsky 2002-10-10 09:15:25 UTC
I'm working on the core*.jar to not depend on openide/deprecated.
Comment 31 Peter Zavadsky 2002-10-10 11:37:13 UTC
Removing the dependencies on TopManager.
Checked just first part, will continue.

But there seems to be some problems, especially with Project dependencies.
Comment 32 Jaroslav Tulach 2002-10-10 12:37:46 UTC
During the moving it would be nice to clean *.util.* package. Issue
27910 and issue 27911 describe the task. I've added them here, but if
you think they should not be here, remove them.
Comment 33 Jesse Glick 2002-10-10 13:11:01 UTC
#27911 maybe. #27910 we could do any time, I don't think it's
important now.
Comment 34 Jesse Glick 2002-10-10 14:31:04 UTC
BTW comments requested on newly introduced interfaces (found via
lookup) which replace some TopManager functionality:

void org.openide.awt.HtmlBrowser.URLDisplayer.showURL(URL)

[Radim said maybe this is unnecessary, you can make a new
BrowserComponent and call setURL, and that handles internal vs.
external browsers and browser selection choice correctly - pending]

Object org.openide.DialogDisplayer.notify(NotifyDescriptor)
Dialog org.openide.DialogDisplayer.createDialog(DialogDescriptor)

void org.openide.LifecycleManager.saveAll()
void org.openide.LifecycleManager.exit()

OutputWriter org.openide.windows.IOProvider.getStdout()
InputOutput org.openide.windows.IOProvider.getIO(String, boolean)

Basically these are just the result of splitting a few methods off
into their own interfaces. It should now be easier to e.g. write unit
tests exercising some but not all former TM functionality - for
example, dump a trivial impl of DialogDisplayer into
META-INF/services/ and you can test code that involves displaying
dialogs (as a lot of openide code does, I am afraid). No need to have
core.jar and start Plain, etc.
Comment 35 rbalada 2002-10-10 14:36:35 UTC
Add us back to Cc: when necessary.
Comment 36 Peter Zavadsky 2002-10-10 16:56:05 UTC
I've put there second part, so the core should be now dependent on
TopManager, but I didn't took these sources into account (assuming
they are going to be moved to core/deprecated):
beaninfo/editors/CompilerTypeEditor
                 DebuggerTypeEditor
        /DebuggerType
        /DebuggerTypeBeanInfo
core/actions/ProjectsOpenAction
             ProjectsCloseAction
             ProjectsSetActiveAction
             SettingsAction
    /ui/WorkplaceNode
    /ClassLoaderSupport
    /ControlPanelNode
    /ModuleInstaller
    /NbControlPanel
    /NbProject
    /NbProjectOperation
    /UpdateSupport

I was trying also SessionManager, but there was is too many dependencies.

Anyway, before that, in that above shape, there are still a couple of
compilation errors when trying to build the core. So I'll try to coupe
with them now.

Please let me know, if some of the above classes assumed to be moved
to core/deprecated shouldn't be there.
Comment 37 Peter Zavadsky 2002-10-10 16:57:24 UTC
Sorry the mistake, core should not be now dependent on TopManager :-)
Comment 38 Jesse Glick 2002-10-10 23:48:47 UTC
OK, I made core-deprecated.jar as stated earlier. Still some problems,
but definitely getting there. NB seems to run at least.
Comment 39 Jesse Glick 2002-10-11 22:25:46 UTC
A bunch more work done. The two deprecated JARs are now actual
autoloads - meaning that if you are not using them, they get turned
off. :-) They are enabled automatically if any old modules are present
that do not declare a dependency on the new API version.
Comment 40 Jesse Glick 2002-10-12 18:08:38 UTC
I branched xtest too - it refers to TopManager in a few places.
Comment 41 Martin Brehovsky 2002-10-14 14:18:52 UTC
I branched also closed source code of XTest (compiled to xtest-ext.jar
binary on cvs.netbeans.org) in nbextra repository, since there are
several TopManager calls as well.
Comment 42 Jesse Glick 2002-10-14 20:10:53 UTC
[breh somehow changed subcomp + targ milestone + vers]

OK. I have already committed a few changes in xtest.netbeans.org in
the branch. For some reason ide-mode tests are still not shutting down
when they finish - not sure why yet.
Comment 43 Jesse Glick 2002-10-14 20:43:48 UTC
WARNING: the nbextra xtest branch is misspelled:
separation_19433_a_branch.

I will try to work on it in that branch. I see something that could
well be causing problems.
Comment 44 Jesse Glick 2002-10-14 21:00:18 UTC
Need to branch junit too, I guess. Adding that to the list of branched
modules.
Comment 45 Jesse Glick 2002-10-16 07:03:16 UTC
Created attachment 7666 [details]
Patch to nb_all - missing just a couple of manifest spec vers changes and addition of CurrentProjectNode to projects module
Comment 46 Jesse Glick 2002-10-16 07:19:22 UTC
Created attachment 7667 [details]
Commit log
Comment 47 Jesse Glick 2002-10-16 07:21:53 UTC
OK, phase I committed. Still some lesser cleanup tasks to do, but
things basically seem to work... Only apparent problems I know of
involve the projects module: (1) sometimes Workplace appears; (2)
Project tab is gone pending root node installation improvement (this
is long since filed); (3) execute/compile/build project don't do
anything, which I think I can fix easily.
Comment 48 Martin Brehovsky 2002-10-16 17:36:56 UTC
Houston, we have a problem. After the commit to trunk, the new XTest
is not able to run tests in IDE mode (IDE checked out and built this
afternoon, run on Solaris 8). Basically everything looks ok, but it
seems IDE is not started in GUI mode and there are several exceptions
thrown by IDE. The test are then runned, though.

Anyway, you can easily reproduce the problem by running test examples
from XTest. Just cd to nb_all/xtest/examples/MyModule/, type 'ant' to
build the fake example module, cd to test/ subdirectory and run the
tests by 'ant -Dxtest.attribs=all,ide'.

I'm wondering what's wrong with XTest's
org.netbeans.xtest.ide.Main.java, since when IDE is started manually,
it works ok. Any help ?
Comment 49 Jesse Glick 2002-10-16 21:36:17 UTC
Martin - I suggest you open a separate issue to track any problems
with XTest. I tried what you said, and indeed it shows a few
exceptions, but then runs the tests anyway.

"XTest: Lookup.getDefault().lookup(OutputSettings.class) is NULL !!!"
- not sure what is going on here. I can easily fix it though.

"FNFE: SFS/xml/memory/": mystery to me. Will look at.

"Cannot save window system": also a mystery.
Comment 50 Jesse Glick 2002-10-17 01:04:49 UTC
XTest ought to be fixed now, please confirm. Problem in a nutshell was
that NbTopManager.get() could return a half-started NbTM while another
thread was still installing modules and doing other pieces of startup.
So XTest was being started much too early.
Comment 51 Martin Brehovsky 2002-10-17 10:05:45 UTC
Great, things look good now for XTest. Thanks Jesse.
Comment 52 Jesse Glick 2002-10-18 06:44:45 UTC
Tagging for phase II:

modules="nbbuild_nowww_1 openide_nowww_1 core_nowww_1 openidex_nowww_1
ant_nowww_1 apisupport_nowww_1 applet_nowww_1 autoupdate_nowww_1
beans_nowww_1 classfile_nowww_1 clazz_nowww_1 debuggercore_nowww_1
debuggerjpda_nowww_1 diff_nowww_1 editor_nowww_1 extbrowser_nowww_1
form_nowww_1 html_nowww_1 httpserver_nowww_1 i18n_nowww_1
image_nowww_1 j2eeserver_nowww_1 jarpackager_nowww_1 java_nowww_1
javacvs_nowww_1 javadoc_nowww_1 jndi_nowww_1 libs_nowww_1
projects_nowww_1 properties_nowww_1 rmi_nowww_1 schema2beans_nowww_1
scripting_nowww_1 text_nowww_1 tomcatint_nowww_1 ui_nowww_1
usersguide_nowww_1 utilities_nowww_1 vcscore_nowww_1 vcscvs_nowww_1
vcsgeneric_nowww_1 web_nowww_1 xml_nowww_1 xtest_nowww_1 junit_nowww_1
classclosure_nowww_1 corba_nowww_1 cpp_nowww_1 db_nowww_1
debuggertools_nowww_1 externaleditor_nowww_1 filecopy logger_nowww_1
monitor"; cvs rtag separation_19443_b_base $modules; cvs rtag -b -r
separation_19443_b_base separation_19443_b $modules
Comment 53 Jesse Glick 2002-10-18 22:17:34 UTC
Compiler API is separated in the new branch. More to come.
Comment 54 Jesse Glick 2002-10-28 21:09:53 UTC
Branch progress: have successfully separated

openide/compiler
openide/io
openide/execution
core/compiler
core/execution
core/output
core/term

core + openide build OK now. Not sure how it runs, yet.
Comment 55 Jesse Glick 2002-10-29 22:19:24 UTC
moduleconfig=superslim now builds, and runs fairly well (still looking
for little errors).
Comment 56 Jesse Glick 2002-11-02 21:28:11 UTC
Created attachment 7836 [details]
Patch to NB sources, gzipped (note: addn'l last-minutes changes also to core/ui/manifest.mf and nbbuild/build.properties to make platform build work)
Comment 57 Jesse Glick 2002-11-02 21:28:43 UTC
Created attachment 7837 [details]
Commit log (gzipped)
Comment 58 Jesse Glick 2002-11-02 21:29:13 UTC
OK, finally merged phase II.
Comment 59 Jesse Glick 2002-11-07 21:28:11 UTC
Remaining tasks cannot all be solved in the 4.0 timeframe.
Comment 60 Jaroslav Tulach 2002-12-11 14:18:39 UTC
Request to separate util, nodes and explorer.
Comment 61 Marian Mirilovic 2003-12-22 13:27:23 UTC
As described in
http://www.netbeans.org/servlets/ReadMsg?msgId=619519&listName=nbdiscuss
the
current work on projects prototype has been stopped.

removed PROJECTS keyword.
Comment 62 Jesse Glick 2005-04-06 22:23:34 UTC
Planned for 4.2?
Comment 63 Jaroslav Tulach 2005-06-04 07:40:40 UTC
And implemented in 4.2. Finally.