This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
the shared libraries effort for 6.1 created a situation that we didn't have in previous incarnations of the IDE. the libraries provided by the IDE's modules are identified by the string identifier only. when the user updates the IDE, the libraries in user's projects are automatically used from the new ide installation. For shared libraries since 6.1 this "autoupdate" feature will not longer work. This issue is an umbrella to collect potencial and real problems with users having old library binaries in their projects and to discuss and come up with a solution on how and when to upgrade users to new binaries.
I keep wondering if we actually need any sort of APIs and UI. Maybe writing up some guidelines how to tackle the problem (when it arises) should be enough. Let me explain in more detail. The owner of the library knows the SHA1 checksums of the old version, new version and knows how severe the problem is. Computing the SHA1 checksum of the actual user's bits is a trivial task. (The only one who's interested in the checksum of a particular library is the library owner most probably so setting up infrastructure to cache the checksums somehow is not adding value) These days any module can submit a ProjectOpenedHook (via LookupProvider) so it should be easy get the foot in the door to start the checksum calculations. So maybe this task boils down to providing a utility method that calculates a SHA1 checksum? Comments?
I have a tendency to agree with you Milos. The only thing I afraid is that module owners realize they need to do something two days before NB6.5 RC2. How would this work for 3rd party modules: eg. form module when generating JPanel to a project checks (apart form other things) presence of a class from swing-layout-1.0.3.jar on project's classpath. If such a well known class is not there ts adds the jar to project classpath via classpath modifier. If in NB6.5 form comes with swing-layout-2.0.jar what should they do? Are you saying they should impl ProjectOpenHook?
depends. The solution is different for jars and libraries. for jars that get added on classpath, the proper process is: 1. at the time you would add the binary to the project, check for class from the binary. 2. that way you figure the jar it resides in, calculate it's SHA1. 3. if not good, remove the old one via the ClassPathModifier api and add the new one. for libraries 1. at the time you would add the library, check for class from the binary. Not sure if it's better to check for the class or got library name. AFAIK 3rd party modules cannot figure out the correct LibraryManager instance as the library path for the project is accessible by AntProjectHelper which isn't exposed outside of the project (correct me if I'm wrong) 2. again after you figured the actual binary(ies), check the SHA1 token. 3. if not good, either replace the definition of library in librarymanager or add the new library under different name. None is currently possible I think. removing a library from the project will just nuke the reference and adding the library later with new content one will only delegate to the old one (with old binaries). So I guess we actually need to do something about libraries upgrade from 3rd party modules. Any ideas?
a possible solution for libraries is to create a new library named string2.6 for example, remove the old library reference from project (spring) and add the new library (spring26). That way the new library gets added to shared location and project references the new library.
> remove the old library reference from project (spring) and add the new library (spring26) If this approach is chosen, we need to communicate this in advance (before FF of 6.5) to all developers. An FAQ should be created in the Developer FAQs area (http://wiki.netbeans.org/NetBeansDeveloperFAQ) and announced loudly.
> remove the old library reference from project (spring) and add the new library (spring26) If this approach is chosen, we need to communicate this well in advance (before FF of 6.5) to all developers. An FAQ should be created in the Developer FAQs area (http://wiki.netbeans.org/NetBeansDeveloperFAQ) and announced loudly.
Milos, David, this issue should be resolved now, with some conclusion and a follow-up for the parts of IDE which provide libraries for projects...
*** Issue 140959 has been marked as a duplicate of this issue. ***
Four months ago I said "The only thing I afraid is that module owners realize they need to do something two days before NB6.5 RC2" and funnily enough I just realized I need to fix <copylibs> tasks. See separate issue 146736 for more details.
We have a similar problem with upgrading JAX-WS 2.1 library from 2.1.2 to 2.1.3 (implementation version) Renaming library name for this purpose doesn't seem to be a conceptual solution. Looking at library-declaration DTD I found there is a library:version attribute that should be used to identify the version of that library: http://www.netbeans.org/dtds/library-declaration-1_0.dtd As a module developer, I'd like to have ability to detect the version of library used in project regardless the library is in shared libraries or not. Another option is update the library (in shared libraries) automatically when this version attribute is changed, or - better - notify user that the library was upgraded, in a confirmation dialog, asking if jars in shared libraries should be upgraded or not.
mkuchtiak: unfortunately the version field is meant to denote the version of the dtd document, rather than the version of the actual library. We don't have a version of the library encoded anywhere. I agree that it is useful, I can't think of why we don't have it. Having just the name of the library as identifier seems wrong to me, but I might just be too maven centric here. "As a module developer, I'd like to have ability to detect the version of library used in project regardless the library is in shared libraries or not." Unfortunately you will never have the capability, unless we explicitly start declaring versions in the libraries. And eventually we will need to leak the "version" concept to the user as the user is allowed to change the content of the library. So if the IDE ships with 1.2.1, the user changes to 1.2.3 manually and we ship the next version of netbeans with 1.2.2 for some reason, how will we know what is the old version in user's space and if we shall upgrade? It's impossible without user interaction either at time when the library is changed or when the user upgrades the library. Always silently upgrading the user's libraries is a big non-no for me, no matter that we did it for a few releases anyway. The asking for upgrade can also be tricky, as the user neither the libraries infrastructure knows what are the consequences of upgrading/not upgrading. The only one who knows is the module providing the library.
Perhaps we should just fix libraries versioning. If a library does not have a version it would be considered being "0.0"; any user defined/altered library is special "USER" version. That would let module providers to version their libraries and upgrade them but not override libraries customized by user. Change like this is backward compatible, should be relatively simple to implement and does not force users to deal with versioning in the UI. Advanced users could tweak versions directly in definition files if they really want to do that. Do you think a solution like this could work? Milos?
so far we had only issues with libraries used in build script. The usecase I'm worried with the silent upgrade is this: people use 6.1 on their production projects, they beta test 65, we upgrade the libs and build script, they go down to 6.1, we regenerate the old build script (do we?) but the new library stays. Actually we regenerate the build script in some situations only and therefore potentially a shared library location can serve both old and new build scripts. That's a reoccuring problem and IMHO one of the reasons some people pick freeform over the IDE maintained scripts. There basically only 2 usecases when we ever need to upgrade the user forcefully anyway (IMHO). 1. it's needed by the buildscript (our 2 usecases we already had) 2. it's needed by some of IDE's code generation features. I would argue that in this situation the IDE should bend, not the user and we should generate different code based on the classpath of the project. (We do in case of api support file wizards for example) On the general level I'd rather go a way of allowing the user to *manually* synchronize/upgrade their shared lib content with the content from the IDE, rather than doing a silent/forced upgrade of everything based on the criteria we pick. However the IDE will never be a full scale software library repository, so not sure how complex the UI shall actually be..
We discussed our use cases with Milos this week. In our case, we use JAX-WS 2.1 library for purposes : 1. it's needed for build script only (ant task puts the library on classpath) 2. it's needed for project (project's classpath) only 3. needed for both At least we(plugin) should be able to detect that library located in shared libraries is different from IDE version Also user should be able to upgrade the library easily. See the picture. I noticed, for now even you remove the library from shared libraries, you only remove the reference. Then, if you import the library again, another folder (Library_Name-2) is created.
Created attachment 70952 [details] Library Manager - obsolite library detection
Re. "people use 6.1 on their production projects, they beta test 65, ... they go down to 6.1" - I wonder if in practice this really is an issue. Your project and sharable libraries are in VCS. If you try out newer version of IDE you can see all the changes IDE did (VCS diff) and you can always revert/ignore them and commit them once whole your team migrated to newer version of IDE. Re. "they go down to 6.1, we regenerate the old build script (do we?)" - yes, we do. genfiles.properties keeps CRC of stylesheet which will be different in older version Previously I was talking only about fixing libraries infrastructure by adding a version to library definition. How such an infrastructure would be used by IDE does depend on usage scenario. In three scenarios Milan described: 1. it's needed for build script only - in this instance upgrade should happen automatically otherwise IDE will not work; it is internal detail/behaviour and user should not be involved; it is in our interest to make change like this backward and forward compatible (two versions should be enough) that is if a jar gets upgraded eg. in NB6.5 and user later runs NB6.1 it should still work; if that's not achievable and newer version is significantly incompatible than library name should be changed so that two versions of the same library could coexist 2. it's needed for project - in this case user should be in charge; an "Upgrade" button could facilitate the process in UI based on knowledge of a version of current sharable library and presence of a newer version in global libraries catalogue; in some situations when a module knows that old version of a library is buggy it can suggest user upgrade from project open hook(?) 3. needed for both - this is a variation of #2; user should be in charge but module can request/suggest/insist on an upgrade Side question: if we enhance library infrastructure with a version should we allow module to provide one library in multiple versions? Any usecase in which that would make sense? Does it make sense? Did I miss something? I'm just brainstorming a solution.
I think, the plugin should be able to detect the version of library used in project, for individual use cases, and notify user that the version is old. User can upgrade or not. Note:In case of build script, we are able to customize Ant tasks regarding to the library version.
I compared libraries from NB6.5Beta (all clusters build)(more changed possible since then) with libraries in NB6.1. I compared that number of jars in a library has not changed and that file size of jars in a library has not changed. Later I excluded J2ME libraries as they do not libraries sharability. Following is list of changed libraries which we should properly upgrade in NB6.5 (I tried to group libs logically or by cluster): - IdentityAntTasks - restapi - jaxws21 - restlib - jsfsupport-runtime - jsfsupport-designtime - jsf12-support - exceptionhandler-runtime - jdbcsupport-designtime - jdbcsupport-runtime - spring-framework - CopyLibs - this one is already handled The list is alarmingly long!
dkonecny: no, it's not long. And you included all that are changed. I'd like to reiterate that I don't think we should automatically upgrade all libraries (eg. not sure the old version of spring framework will cause any trouble to the IDE integration) "Side question: if we enhance library infrastructure with a version should we allow module to provide one library in multiple versions? Any usecase in which that would make sense?" hardly ever going to happen, it would just increase the download size without any clear benefit.
Re. "I'd like to reiterate that I don't think we should automatically upgrade all libraries" - agreed. What needs to be done though is to talk to owners of changed libraries and made them aware of potential issues in sharable libraries scenario and help them solve problem if them have one. And we should do it today and early next week so that they can still fix it before code freeze. CopyLibs library problem I fixed is good example; IdentityAntTasks could be the same case. Perhaps modules should just detect that older version is presented on project's classpath and tell user to upgrade it if necessary.
< Perhaps modules should just detect that older version is presented on project's classpath and tell user to upgrade it if necessary. It's only possible when library is on project's classpath. In case of ant script - it doesn't need to be. BTW. What are the best steps, for user, to upgrade library in shared libraries ? I guess first remove then import, right ? Another option is to replace library manually in shared library folder.
Re. "What are the best steps" - see http://hg.netbeans.org/main/diff/3b86a600922a/j2ee.common/src/org/netbeans/modules/j2ee/common/project/classpath/ClassPathSupport.java for how I detect version in CopyLibs case and upgrade it. If there would be more time I would add to AntProjectHelper a new method removeLibrary(Library). dongmeic, potingwu - could you look at all other libraries (jsfsupport-runtime, jsfsupport-designtime, jsf12-support, exceptionhandler-runtime, jdbcsupport-designtime, jdbcsupport-runtime, spring-framework) and check if upgrade is necessary or desired? We have last week before feature freeze so it needs to be fixed this week.
well, since there already is a removeLibrary() method in refHelper.getProjectLibraryManager().removeLibrary(lib) I'd suggest to use a different name, eg. upgradeLibrary(Library) where the parameter will be library with the same name but from global library manager. anyone who copies David's code, please point to your changelog here, so that we can replace it with the api method created for the netbeans.next release. thanks.
Reassigning to default owner.