This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | DOMParser init error parsing Ant scripts | ||
---|---|---|---|
Product: | projects | Reporter: | Jesse Glick <jglick> |
Component: | Ant | Assignee: | Jesse Glick <jglick> |
Status: | RESOLVED FIXED | ||
Severity: | blocker | CC: | andrew, l950637, maddy, sandipchitale |
Priority: | P4 | Keywords: | RANDOM |
Version: | 3.x | ||
Hardware: | PC | ||
OS: | Linux | ||
Issue Type: | DEFECT | Exception Reporter: | |
Bug Depends on: | 35762 | ||
Bug Blocks: | |||
Attachments: |
Diagnostic patch to Xerces to see where the CCE is coming from
ide.log with Jesse's patch applied Revised patch - should show location of offending JAR ide.log.3 as a result of Jesse's revised patch |
Description
Jesse Glick
2003-12-10 18:28:02 UTC
Of course nothing has changed in CVS recently, and it is unreproducible with a fresh user dir... Running with a diagnostic Xerces patch to see what is going on. Created attachment 12515 [details]
Diagnostic patch to Xerces to see where the CCE is coming from
*** Issue 38059 has been marked as a duplicate of this issue. *** See issue #38059 for more. Still unable to reproduce. Jesse, I'll attach new ide.log with your xerces patch (I jave applied it to 2.5, but it seems to work :-). The ide.log.2 contains RefactorIT traces (it is distributed with own xerces now). Is it the issue reason? Created attachment 12555 [details]
ide.log with Jesse's patch applied
I can't find a copy of Xerces in refactorit.nbm. I have that module installed but disabled. (Currently running 1.9.6 "2.0 beta".) Where did you find that please? Ah, I see it now in your log file. Thanks, this is invaluable. You have com.xpdev.refactory 1.9.6.1, so I will try to get that and see if that makes things reproducible, and what to do about it. Created attachment 12556 [details]
Revised patch - should show location of offending JAR
Andrew reports xerces.jar 2.6.0 exists in http://aqris.com/~anton/EAP/jarsonly/refactorit.tgz, which I can confirm. However the instructions for NB installation do *not* say to copy xercesImpl.jar. Did you do this installation manually rather than from NBM? If you remove xercesImpl.jar from modules/refactorit/lib/ (or wherever you put it) does the problem persist? Jesse, Without xercesImpl.jar in modules/refactorit/lib audit filter saving doesn't work. I'm not sure it is NB or RIT problem. If it is RIT issue, it means, any module can not be distributed with it's own libs with preferable for _this_ module version. Is it so? Well, I don't insist on anything. I'm ready to forward the issue to RIT team :-) Re. audit filter not saving without special Xerces: probably a result of them requiring Xerces 2.6.0 features and we currently shipping 2.4.0. See issue #35762. Re. "any module can not be distributed with it's own libs with preferable for _this_ module version" - this is not true; the module system supports parallel libraries, and it works fine for normal Java class usage. However I suspect that some code in Xerces uses Thread.getContextClassLoader to initialize itself dynamically; the context class loader in NB is taken from all modules, so if you try to look up a class or resource loadable from several modules, the result is arbitrary. If this is the problem, it is ultimately a Xerces bug that it (1) uses T.gCCL (2) does not verify that the class it finds comes from the correct loader. The diagnostic patch I have attached here actually solves one symptom of that presumed Xerces bug (except for printing the information to stderr). A complete patch for Xerces would enhance ObjectFactory to pay attention to what it is loading; I suspect its logic w.r.t. multiple class loaders is wrong. Similar problems could in principle arise in application servers, etc. NB is hit by T.gCCL frequently because it has to run multiple modules in the same threads since it is a GUI app. Please run with the revised diagnostic patch so I can confirm this suspicion. It would also explain the difficulty in reproducing - it may depend on which modules you have installed, or other runtime factors, which module gets to push its Xerces classes into other parsers. Created attachment 12559 [details]
ide.log.3 as a result of Jesse's revised patch
Jesse, New ide.log is uploaded. Again, xerces-2.4.0-1.jar is actually build from (patched) 2.5 version. So my suspicion is correct, you wll need to remove /wrk/usr/nb/projects/nb40/modules/refactory/lib/xercesImpl.jar and deal with whatever problems that causes for RefactorIt. Sounds like they may be using some features of DOM3, which is not yet released, so that is partially their problem. (NB modules can use org.openide.xml.XMLUtil to serialize DOM documents.) Now as to a workaround that would preserve RI's functionality... I don't know. If I have a moment I will see if I can create a patch for Xerces to be safer in the presence of multiple copies of the lib in different class loaders. But at best that would let Xerces 2.7.0 (or 2.6.1 or whatever) not have this problem; that could be a long way away. (And it would be necessary to patch both NB's copy and RI's.) One workaround would be to replace modules/autoload/ext/xerces-2.4.0-1.jar in your NB installation with Xerces 2.6.0 (keeping the now incorrect JAR name, or editing modules/autoload/nb-xerces-wrapper.jar's manifest to match). I haven't tried but probably it would not cause any problems. BTW what makes you say that xerces-2.4.0-1.jar is actually Xerces 2.5? 1. "xerces-2.4.0-1.jar is actually Xerces 2.5": I have used 2.5 src, and renamed resulted jar to xerces-2.4.0-1.jar - exactly what you say as about "keeping the now incorrect JAR name". 2. Is it correct to send to RIT team the issue ref? Yes, please file a bug report for RefactorIt (to see if they can avoid relying on a private xerces.jar when running in NB) and supply the URL to this bug. Also when you do so please include the RI bug URL here for reference. Although I don't consider this to be ultimately a NB bug, I will leave it open until either issue #35762 is done and we ship with Xerces 2.6.0 anyway, or a proper bug report has been submitted to xml.apache.org. 1. I have tried to: - replace NB's xerces with (renamed) xerces 2.6.0 without succsess, - replace NB's xerces with (renamed) xerces from RIT's lib directory, and this case seems to be a workaround of the issue. I have not noticed any side effects during 3-4 hours of work. 2. I have notified Anton (RIT) about the issue. The reason we have a private xerces jar is that we need to have RIT working in a wide range of IDEs currently supported (NB3.4-3.5, S1S, JB5-X, JDev) and some of them doesn't have xml api at all. There is no special reason we are bundling the latest one, just "why not?". Not sure yet how we are going to solve the problem. > Re. "any module can not be distributed with it's own libs > with preferable for _this_ module version" - this is not > true; the module system supports parallel libraries, and > it works fine for normal Java class usage. it is true if you replace "any module" with "any module that uses context classloader" > However I suspect that some code in Xerces uses > Thread.getContextClassLoader to initialize itself > dynamically; the context class loader in NB is taken > from all modules, so if you try to look up a class > or resource loadable from several modules, the result > is arbitrary. Yes, Xerces uses context classloaders extensively. The same with any other XML parser or XSLT transformer. This behavior is even specified in JAXP spec! So, using context classloader is normal Java class usage in this case! > If this is the problem, it is ultimately a Xerces bug > that it (1) uses T.gCCL (2) does not verify that the > class it finds comes from the correct loader. It is not a Xerces bug, it is a flaw in NetBeans context classloader loading classes from all modules! Even if you have single Xerces version in all places but loaded by different classloaders you could get ClassCastException if current classloader differs from context classloader. And there is no way to fix it in Xerces! Re. bundling private xerces.jar in other IDEs - fine, but if you are producing an NBM then that is NB-specific and there is no need to include xerces.jar in it (unless there is some problem with the NB version of xerces.jar of course). Re. use of T.gCCL - there is (AFAIK) no other sensible ClassLoader to set as the context loader; as I mentioned, there is no way to determine "which module is active" at a given time because many modules can be doing tiny jobs in various threads at once, as is typical in a GUI app. Trying to give each module its own CCL is impractical. Not setting T.gCCL is also impractical because many libs will just never work at all. Of course a particular module can temporarily set the current thread's CCL to a desired value while performing some operation (try - finally) if it needs to. T.gCCL seems a poor API to me (does not permit container to supply a dynamic default), but not much to be done except work around it. Suggestions welcome. Re. it not being a Xerces bug - it *is* a (robustness) bug that Xerces loads a named class from a potentially unknown class loader, then casts it to a locally-loaded interface with no further error checking. It should at least handle the case that the loaded class was not assignable to the local interface, and deal with this gracefully, e.g. using the default fallback impl of that interface, after logging a warning. Otherwise it is impossible to use the library in situations where the CCL might contain a different copy of the same lib. > but if you are producing an NBM then that is NB-specific > and there is no need to include xerces.jar in it The problem is that old version of NetBeans needs to be supported too (3.4, SunOne Studio) and it doesn't have xerces module! In RIT we found a way how to workaround the problem - we'll use XMLUtils.write() for serialization (not very-well formatted, but at least it works), but if another Xerces functionality would be required then there is no way to have it in NetBeans (XNI API has changed frequently in Xerces and become more or less stable not long ago, but still might be changed). > Trying to give each module its own CCL is impractical. Agree > Not setting T.gCCL is also impractical because many libs > will just never work at all. Of course a particular module > can temporarily set the current thread's CCL to a desired > value while performing some operation (try - finally) if > it needs to. T.gCCL seems a poor API to me (does not > permit container to supply a dynamic default), but not > much to be done except work around it. It is not impractical as you described a workaround. Setting CCL in try-finally looks ugly, but it does its job - classloading is controllable and predictable. But if CCL is a sum of all modules then classloading is really impractical because it is not controllable and not predictable. I understand that NB CCL cannot be changed in 3.x versions, but for 4.0 IMHO CCL should be changed to default (system classloader). > it not being a Xerces bug - it *is* a (robustness) bug that > Xerces loads a named class from a potentially unknown class > loader, then casts it to a locally-loaded interface with no > further error checking. ClassCastException is the appropriate error-chacking. Fallback to some default would be not what the user wants... > Otherwise it is impossible to use the library in > situations where the CCL might contain a different > copy of the same lib. Exactly. It is impossible to use library that calls CCL in NetBeans because CCL might contain a different copy of the same lib and CCL is not controllable! Re. supporting NB 3.4 or older - I am not sure what to suggest, other than producing a different NBM for this version. Probably the discussion re. T.gCCL should be moved to nbdev, but some quick comments: Re. disabling the special CCL by default for a future version of NB - I am not so sure that this would be an improvement. For most purposes in most modules, the current CCL (loaded from all modules) works fine. JAXP normally works, JNDI works, etc. For those cases where it doesn't, due to this class of problem, you can set a temporary CCL to whatever you prefer, and that will let you control it precisely. Removing the call to T.sCCL from the NB core would mean that *every* module using a CCL-sensitive lib would need to wrap every call to this lib that might check T.gCCL with a T.sCCL call. Not necessarily bad - at least it is explicit - but that's quite a lot of code to add that isn't currently necessary. Any module code that needs the workaround now would not be helped by such a change; it would only cause an additional burden for modules that currently work. The only benefit would be better robustness in *some* existing modules by forcing them to declare the CCL they need at every point; not all cases are subject to this class of bug, however. Re. fallback to a default (in xerces.jar) not being what the user wants - maybe not always, but it's certainly what the user wants in this particular bug. Some internal Xerces impl class (DOMParser) is choking during a perfectly regular parse because it can't load a class (XML11Configuration) which is defined nearby in the Xerces impl, which is the default impl of its (also Xerces-private) interface (XMLParserConfiguration), and which no one is attempting to override. If you haven't set any property etc. to specify a *special* impl of this interface, there is no reason to call T.gCCL at all, since the normal impl is right there in the same JAR. I.e. Xerces is just gratuitously sensitive to the CCL even when the CCL API is not needed for what it is doing. I should also point out that this bug is *not* a result of JAXP mandating CCL - that should be safe, since JAXP interfaces are defined in a non-overridable lower layer (the JRE), and you can just pick a conforming impl from any loader which makes one available. (Whether you get the impl you "wanted" is open to question, but JAXP doesn't define any way to filter the results except by loading the desired impl explicitly.) The root of the problem is that Xerces decides to use CCL *internally* to configure its own private interfaces with suitable implementations. This is fine only if there can be just one copy of the interfaces (regardless of how many copies of the implementations there are, assuming they link against the interfaces). However both interfaces and implementations are really more or less private to Xerces and bundled together in xerces.jar, which is bundled as a whole by anyone who wants a copy. (In principle if the Xerces internal contracts were stable, there could be one xerces-config.jar defining just interfaces, shared by the whole system, and possibly multiple versions of xerces-impl.jar doing the actual parsing. This setup is probably not realistic for Xerces but it is useful to compare it to the current design to understand the problem.) Xerces' ObjectFactory seems to have some support for a "fallback" flag which loads a class from the xerces.jar loader in case it cannot be found in CCL. This is useful, but needs to be extended so that createObject checks the expected type (Class.isInstance) of the returned object, and handles CCE's. It might handle it by trying to load the impl class from the xerces.jar loader, for example. Or by using only the xerces.jar loader (never CCL) when loading default impl classes. Etc. Note: NB 3.6 dev builds now include Xerces 2.6.0 so I am not aware of any reason to include it in refactorit/lib. That should at least reduce the practical impact of this bug. Patch for xml.apache.org/xerces still pending. re-evaluate please ... The upstream bug: http://issues.apache.org/jira/browse/XERCESJ-814 |