This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 37994

Summary: DOMParser init error parsing Ant scripts
Product: projects Reporter: Jesse Glick <jglick>
Component: AntAssignee: Jesse Glick <jglick>
Status: RESOLVED FIXED    
Severity: blocker CC: andrew, l950637, maddy, sandipchitale
Priority: P4 Keywords: RANDOM
Version: 3.x   
Hardware: PC   
OS: Linux   
Issue Type: DEFECT Exception Reporter:
Bug Depends on: 35762    
Bug Blocks:    
Attachments: Diagnostic patch to Xerces to see where the CCE is coming from
ide.log with Jesse's patch applied
Revised patch - should show location of offending JAR
ide.log.3 as a result of Jesse's revised patch

Description Jesse Glick 2003-12-10 18:28:02 UTC
Observed a few times in recent dev builds:

Annotation: Strange parse error in merge-try.xml
java.lang.ClassCastException
	at
org.apache.xerces.parsers.DOMParser.<init>(Unknown
Source)
	at
org.apache.xerces.parsers.DOMParser.<init>(Unknown
Source)
[catch] at
org.apache.tools.ant.module.xml.AntProjectSupport.parseDocument(AntProjectSupport.java:246)
	at
org.apache.tools.ant.module.xml.AntProjectSupport.getDocument(AntProjectSupport.java:224)
	at
org.apache.tools.ant.module.xml.AntProjectSupport.getProjectElement(AntProjectSupport.java:320)
	at
org.apache.tools.ant.module.loader.AntActionInstance.getValue(AntActionInstance.java:123)
	at
org.apache.tools.ant.module.loader.AntActionInstance.stateChanged(AntActionInstance.java:185)
	at
org.openide.util.WeakListener$Change.stateChanged(WeakListener.java:618)
	at
org.apache.tools.ant.module.xml.AntProjectSupport$ChangeFirer.run(AntProjectSupport.java:433)
	at org.openide.util.Task.run(Task.java:136)
	at
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:330)
	at
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:678)

No idea what that could mean. Affected code is
probably

(XMLParserConfiguration)ObjectFactory.createObject(
"org.apache.xerces.xni.parser.XMLParserConfiguration",
"org.apache.xerces.parsers.XML11Configuration")

So presumably it is returning something else than
an XMLParserConfiguration. Not sure what.

Conflict between versions of Xerces, maybe? Only
starting happening recently. What changed?
Comment 1 Jesse Glick 2003-12-10 19:03:16 UTC
Of course nothing has changed in CVS recently, and it is
unreproducible with a fresh user dir...

Running with a diagnostic Xerces patch to see what is going on.
Comment 2 Jesse Glick 2003-12-10 19:03:52 UTC
Created attachment 12515 [details]
Diagnostic patch to Xerces to see where the CCE is coming from
Comment 3 Jesse Glick 2003-12-12 16:08:35 UTC
*** Issue 38059 has been marked as a duplicate of this issue. ***
Comment 4 Jesse Glick 2003-12-12 16:09:27 UTC
See issue #38059 for more. Still unable to reproduce.
Comment 5 andrew 2003-12-12 17:14:24 UTC
Jesse,

I'll attach new ide.log with your xerces patch
(I jave applied it to 2.5, but it seems to work :-).
The ide.log.2 contains RefactorIT traces (it
is distributed with own xerces now). Is it the
issue reason?
Comment 6 andrew 2003-12-12 17:15:35 UTC
Created attachment 12555 [details]
ide.log with Jesse's patch applied
Comment 7 Jesse Glick 2003-12-12 17:20:36 UTC
I can't find a copy of Xerces in refactorit.nbm. I have that module
installed but disabled. (Currently running 1.9.6 "2.0 beta".) Where
did you find that please?
Comment 8 Jesse Glick 2003-12-12 17:22:55 UTC
Ah, I see it now in your log file. Thanks, this is invaluable. You
have com.xpdev.refactory 1.9.6.1, so I will try to get that and see if
that makes things reproducible, and what to do about it.
Comment 9 Jesse Glick 2003-12-12 17:34:02 UTC
Created attachment 12556 [details]
Revised patch - should show location of offending JAR
Comment 10 Jesse Glick 2003-12-12 18:06:38 UTC
Andrew reports xerces.jar 2.6.0 exists in
http://aqris.com/~anton/EAP/jarsonly/refactorit.tgz, which I can
confirm. However the instructions for NB installation do *not* say to
copy xercesImpl.jar. Did you do this installation manually rather than
from NBM? If you remove xercesImpl.jar from modules/refactorit/lib/
(or wherever you put it) does the problem persist?
Comment 11 andrew 2003-12-12 18:34:07 UTC
Jesse,

Without xercesImpl.jar in modules/refactorit/lib
audit filter saving doesn't work. I'm not sure
it is NB or RIT problem. If it is RIT issue, it
means, any module can not be distributed with
it's own libs with preferable for _this_ module
version. Is it so? Well, I don't insist on anything.
I'm ready to forward the issue to RIT team :-)
Comment 12 Jesse Glick 2003-12-12 18:53:52 UTC
Re. audit filter not saving without special Xerces: probably a result
of them requiring Xerces 2.6.0 features and we currently shipping
2.4.0. See issue #35762.

Re. "any module can not be distributed with it's own libs with
preferable for _this_ module version" - this is not true; the module
system supports parallel libraries, and it works fine for normal Java
class usage. However I suspect that some code in Xerces uses
Thread.getContextClassLoader to initialize itself dynamically; the
context class loader in NB is taken from all modules, so if you try to
look up a class or resource loadable from several modules, the result
is arbitrary. If this is the problem, it is ultimately a Xerces bug
that it (1) uses T.gCCL (2) does not verify that the class it finds
comes from the correct loader.

The diagnostic patch I have attached here actually solves one symptom
of that presumed Xerces bug (except for printing the information to
stderr). A complete patch for Xerces would enhance ObjectFactory to
pay attention to what it is loading; I suspect its logic w.r.t.
multiple class loaders is wrong. Similar problems could in principle
arise in application servers, etc. NB is hit by T.gCCL frequently
because it has to run multiple modules in the same threads since it is
a GUI app.

Please run with the revised diagnostic patch so I can confirm this
suspicion. It would also explain the difficulty in reproducing - it
may depend on which modules you have installed, or other runtime
factors, which module gets to push its Xerces classes into other parsers.
Comment 13 andrew 2003-12-13 00:38:19 UTC
Created attachment 12559 [details]
ide.log.3 as a result of Jesse's revised patch
Comment 14 andrew 2003-12-13 00:40:19 UTC
Jesse,

New ide.log is uploaded. Again, xerces-2.4.0-1.jar
is actually build from (patched) 2.5 version.
Comment 15 Jesse Glick 2003-12-13 01:16:28 UTC
So my suspicion is correct, you wll need to remove
/wrk/usr/nb/projects/nb40/modules/refactory/lib/xercesImpl.jar and
deal with whatever problems that causes for RefactorIt. Sounds like
they may be using some features of DOM3, which is not yet released, so
that is partially their problem. (NB modules can use
org.openide.xml.XMLUtil to serialize DOM documents.)

Now as to a workaround that would preserve RI's functionality... I
don't know. If I have a moment I will see if I can create a patch for
Xerces to be safer in the presence of multiple copies of the lib in
different class loaders. But at best that would let Xerces 2.7.0 (or
2.6.1 or whatever) not have this problem; that could be a long way
away. (And it would be necessary to patch both NB's copy and RI's.)

One workaround would be to replace
modules/autoload/ext/xerces-2.4.0-1.jar in your NB installation with
Xerces 2.6.0 (keeping the now incorrect JAR name, or editing
modules/autoload/nb-xerces-wrapper.jar's manifest to match). I haven't
tried but probably it would not cause any problems.
Comment 16 Jesse Glick 2003-12-13 01:17:07 UTC
BTW what makes you say that xerces-2.4.0-1.jar is actually Xerces 2.5?
Comment 17 andrew 2003-12-13 01:34:19 UTC
1. "xerces-2.4.0-1.jar is actually Xerces 2.5":
I have used 2.5 src, and renamed resulted jar to
xerces-2.4.0-1.jar - exactly what you say as
about "keeping the now incorrect JAR name".

2. Is it correct to send to RIT team the issue ref?
Comment 18 Jesse Glick 2003-12-13 03:00:26 UTC
Yes, please file a bug report for RefactorIt (to see if they can avoid
relying on a private xerces.jar when running in NB) and supply the URL
to this bug. Also when you do so please include the RI bug URL here
for reference.

Although I don't consider this to be ultimately a NB bug, I will leave
it open until either issue #35762 is done and we ship with Xerces
2.6.0 anyway, or a proper bug report has been submitted to xml.apache.org.
Comment 19 andrew 2003-12-13 05:42:05 UTC
1. I have tried to:

- replace NB's xerces with (renamed) xerces 2.6.0
  without succsess,
- replace NB's xerces with (renamed) xerces from
  RIT's lib directory, and this case seems to be
  a workaround of the issue. I have not noticed any
  side effects during 3-4 hours of work.

2. I have notified Anton (RIT) about the issue.
Comment 20 Maddy 2003-12-15 08:06:59 UTC
The reason we have a private xerces jar is that we need to have RIT
working in a wide range of IDEs currently supported (NB3.4-3.5, S1S,
JB5-X, JDev) and some of them doesn't have xml api at all. There is no
special reason we are bundling the latest one, just "why not?".
Not sure yet how we are going to solve the problem.
Comment 21 l950637 2003-12-15 11:21:08 UTC
> Re. "any module can not be distributed with it's own libs
> with preferable for _this_ module version" - this is not
> true; the module system supports parallel libraries, and
> it works fine for normal Java class usage.

it is true if you replace "any module" with "any module that uses
context classloader"

> However I suspect that some code in Xerces uses
> Thread.getContextClassLoader to initialize itself
> dynamically; the context class loader in NB is taken
> from all modules, so if you try to look up a class
> or resource loadable from several modules, the result
> is arbitrary.

Yes, Xerces uses context classloaders extensively. The same with any
other XML parser or XSLT transformer. This behavior is even specified
in JAXP spec! So, using context classloader is normal Java class usage
in this case!

> If this is the problem, it is ultimately a Xerces bug
> that it (1) uses T.gCCL (2) does not verify that the
> class it finds comes from the correct loader.

It is not a Xerces bug, it is a flaw in NetBeans context classloader
loading classes from all modules! Even if you have single Xerces
version in all places but loaded by different classloaders you could
get ClassCastException if current classloader differs from context
classloader. And there is no way to fix it in Xerces!
Comment 22 Jesse Glick 2003-12-15 18:22:15 UTC
Re. bundling private xerces.jar in other IDEs - fine, but if you are
producing an NBM then that is NB-specific and there is no need to
include xerces.jar in it (unless there is some problem with the NB
version of xerces.jar of course).

Re. use of T.gCCL - there is (AFAIK) no other sensible ClassLoader to
set as the context loader; as I mentioned, there is no way to
determine "which module is active" at a given time because many
modules can be doing tiny jobs in various threads at once, as is
typical in a GUI app. Trying to give each module its own CCL is
impractical. Not setting T.gCCL is also impractical because many libs
will just never work at all. Of course a particular module can
temporarily set the current thread's CCL to a desired value while
performing some operation (try - finally) if it needs to. T.gCCL seems
a poor API to me (does not permit container to supply a dynamic
default), but not much to be done except work around it. Suggestions
welcome.

Re. it not being a Xerces bug - it *is* a (robustness) bug that Xerces
loads a named class from a potentially unknown class loader, then
casts it to a locally-loaded interface with no further error checking.
It should at least handle the case that the loaded class was not
assignable to the local interface, and deal with this gracefully, e.g.
using the default fallback impl of that interface, after logging a
warning. Otherwise it is impossible to use the library in situations
where the CCL might contain a different copy of the same lib.
Comment 23 l950637 2003-12-16 08:09:44 UTC
> but if you are producing an NBM then that is NB-specific
> and there is no need to include xerces.jar in it

The problem is that old version of NetBeans needs to be supported too
(3.4, SunOne Studio) and it doesn't have xerces module! In RIT we
found a way how to workaround the problem - we'll use XMLUtils.write()
for serialization (not very-well formatted, but at least it works),
but if another Xerces functionality would be required then there is no
way to have it in NetBeans (XNI API has changed frequently in Xerces
and become more or less stable not long ago, but still might be changed).

> Trying to give each module its own CCL is impractical.

Agree

> Not setting T.gCCL is also impractical because many libs
> will just never work at all. Of course a particular module
> can temporarily set the current thread's CCL to a desired
> value while performing some operation (try - finally) if
> it needs to. T.gCCL seems a poor API to me (does not
> permit container to supply a dynamic default), but not
> much to be done except work around it.

It is not impractical as you described a workaround. Setting CCL in
try-finally looks ugly, but it does its job - classloading is
controllable and predictable. But if CCL is a sum of all modules then
classloading is really impractical because it is not controllable and
not predictable. I understand that NB CCL cannot be changed in 3.x
versions, but for 4.0 IMHO CCL should be changed to default (system
classloader).

> it not being a Xerces bug - it *is* a (robustness) bug that
> Xerces loads a named class from a potentially unknown class
> loader, then casts it to a locally-loaded interface with no
> further error checking.

ClassCastException is the appropriate error-chacking. Fallback to some
default would be not what the user wants...

> Otherwise it is impossible to use the library in
> situations where the CCL might contain a different
> copy of the same lib.

Exactly. It is impossible to use library that calls CCL in NetBeans
because CCL might contain a different copy of the same lib and CCL is
not controllable!
Comment 24 Jesse Glick 2003-12-16 11:36:00 UTC
Re. supporting NB 3.4 or older - I am not sure what to suggest, other
than producing a different NBM for this version.

Probably the discussion re. T.gCCL should be moved to nbdev, but some
quick comments:

Re. disabling the special CCL by default for a future version of NB -
I am not so sure that this would be an improvement. For most purposes
in most modules, the current CCL (loaded from all modules) works fine.
JAXP normally works, JNDI works, etc. For those cases where it
doesn't, due to this class of problem, you can set a temporary CCL to
whatever you prefer, and that will let you control it precisely.
Removing the call to T.sCCL from the NB core would mean that *every*
module using a CCL-sensitive lib would need to wrap every call to this
lib that might check T.gCCL with a T.sCCL call. Not necessarily bad -
at least it is explicit - but that's quite a lot of code to add that
isn't currently necessary. Any module code that needs the workaround
now would not be helped by such a change; it would only cause an
additional burden for modules that currently work. The only benefit
would be better robustness in *some* existing modules by forcing them
to declare the CCL they need at every point; not all cases are subject
to this class of bug, however.

Re. fallback to a default (in xerces.jar) not being what the user
wants - maybe not always, but it's certainly what the user wants in
this particular bug. Some internal Xerces impl class (DOMParser) is
choking during a perfectly regular parse because it can't load a class
(XML11Configuration) which is defined nearby in the Xerces impl, which
is the default impl of its (also Xerces-private) interface
(XMLParserConfiguration), and which no one is attempting to override.
If you haven't set any property etc. to specify a *special* impl of
this interface, there is no reason to call T.gCCL at all, since the
normal impl is right there in the same JAR. I.e. Xerces is just
gratuitously sensitive to the CCL even when the CCL API is not needed
for what it is doing.

I should also point out that this bug is *not* a result of JAXP
mandating CCL - that should be safe, since JAXP interfaces are defined
in a non-overridable lower layer (the JRE), and you can just pick a
conforming impl from any loader which makes one available. (Whether
you get the impl you "wanted" is open to question, but JAXP doesn't
define any way to filter the results except by loading the desired
impl explicitly.)

The root of the problem is that Xerces decides to use CCL *internally*
to configure its own private interfaces with suitable implementations.
This is fine only if there can be just one copy of the interfaces
(regardless of how many copies of the implementations there are,
assuming they link against the interfaces). However both interfaces
and implementations are really more or less private to Xerces and
bundled together in xerces.jar, which is bundled as a whole by anyone
who wants a copy.

(In principle if the Xerces internal contracts were stable, there
could be one xerces-config.jar defining just interfaces, shared by the
whole system, and possibly multiple versions of xerces-impl.jar doing
the actual parsing. This setup is probably not realistic for Xerces
but it is useful to compare it to the current design to understand the
problem.)

Xerces' ObjectFactory seems to have some support for a "fallback" flag
which loads a class from the xerces.jar loader in case it cannot be
found in CCL. This is useful, but needs to be extended so that
createObject checks the expected type (Class.isInstance) of the
returned object, and handles CCE's. It might handle it by trying to
load the impl class from the xerces.jar loader, for example. Or by
using only the xerces.jar loader (never CCL) when loading default impl
classes. Etc.
Comment 25 Jesse Glick 2004-01-28 16:26:21 UTC
Note: NB 3.6 dev builds now include Xerces 2.6.0 so I am not aware of
any reason to include it in refactorit/lib. That should at least
reduce the practical impact of this bug. Patch for
xml.apache.org/xerces still pending.
Comment 26 Marian Mirilovic 2004-06-14 14:11:42 UTC
re-evaluate please ...
Comment 27 Jesse Glick 2008-11-21 23:40:53 UTC
The upstream bug: http://issues.apache.org/jira/browse/XERCESJ-814