This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
As described in the proposal mentioned in the URL. I have already updated the demo (nbbuild/www/proposals/scrambler/) to demonstrate inclusion of LFI #s in XML comments in the build script; and <unscramble> now requires a logfile parameter, to which it will write out entries such as: On August 23, 2002 6:53:09 PM EDT the user jglick accepted the license /space/src/nb_all/nbbuild/www/proposals/scrambler/license1.txt with SHA-1 hash C222A3256D068DD740B8E58F939EB16647BA2342 I plan to make a working prototype in a CVS branch and then hand off to RE (probably Ruda) to merge. RE needs to take control of the final steps, since there are probably various scripts and tools that could be affected by this change which will need to be updated in parallel.
I have created the branch scrambler_26758 with base tag scrambler_26758_base in: nb_all: standard, plus: corba jellytools jemmy jini junit mdr objectbrowser qa xtest nbextra: everything (".") That should cover everything that deals with extra binaries, I hope...
Current list of files to be unscrambled (all were listed in nbbinaries in nbextra/CVSROOT/modules, so should be considered already approved): ant/external/ant-docs-1.4.1.zip ant/external/ant-1.4.1.jar ant/external/ant-optional-1.4.1.jar corba/external/openorb-1.0.2.jar corba/external/openorb.xml core/external/xerces-2.0.2.jar core/external/crimson-1.1.3.jar core/external/xml-apis-1.0b2.jar core/external/jh-1.1.2_02.jar core/external/regexp-1.2.jar httpserver/external/webserver.jar httpserver/external/servlet-2.2.jar java/external/javamake-1.2.12.jar java/external/javac.jar java/external/java-gj.jar java/external/javac_ja.jar java/external/java-gj_ja.jar java/external/fastjavac.exe java/external/fastjavac.linux java/external/fastjavac.messages java/external/fastjavac.sun java/external/fastjavac.sun.intel java/external/fastjavac.unix.ja_JP.eucJP.messages java/external/fastjavac.unix.ja_JP.messages java/external/fastjavac.unix.ja_JP.PCK.messages java/external/fastjavac.unix.ja_JP.ujis.messages java/external/fastjavac.unix.ja_JP.UTF-8.messages java/external/fastjavac.unix.ja.messages java/external/fastjavac.unix.japanese.messages java/external/fastjavac.win32.ja_JP.messages libs/external/xalan-2.3.1.jar libs/external/xerces-2.0.2-1.jar mdr/external/jmi.jar mdr/external/mof.jar nbbuild/external/jhall-1.1.2_02.jar objectbrowser/external/infobus-1.2.jar scripting/external/bsh-1.2b5.jar scripting/external/djava.jar scripting/external/jython.jar tomcatint/external/jakarta-tomcat-4.0.4-nojavadoc.zip web/external/jasper-compiler-4.0.4.jar web/external/jasper-runtime-4.0.4.jar web/external/servlet-2.3.jar xml/external/resolver-1_1_nb.jar xml/external/flute.jar xml/external/sac.jar xml/external/xerces2.jar Notes: 1. I have added version numbers to the names of a number of files - wherever I was able to determine what version a file really was. In some cases I was unable to do so. Module owners should correct these cases in the future. In the case of javac.jar and java-gj.jar this is not necessary since these JARs are built from sources in nbextra. 2. I included Japanese localizations of the Java extrabin JARs (javac.jar, java-gj.jar) as well as the fastjavac message files. All of these were missing from the nbbinaries alias, but clearly seemed part and parcel of the main files. Probably Japanese builds of S1S were being made directly out of nbextra and so were not affected.
Forgot about: nbbuild/external/ant.jar (bootstrap Ant lib)
*** Issue 17120 has been marked as a duplicate of this issue. ***
Builds standard config now. To-do: - support modules not in standard config with extrabins, e.g. corba - deal with modules with extrabins not in nbextra, e.g. junit; maybe nothing to be done for now (i.e. you need to have an nbextra checkout to use them - extrabins download does not have them) - for user convenience, if moduleconfig=stable, pre-unscramble all binaries in modules in that config, so user does not have to accept licenses at two-minute intervals (annoyance) - permit bundled Ant to be unpacked from scrambled version. Currently you need a copy of Ant to start the build, or have nbextra checked out. Would be nice to make a standalone JAR that would just unpack Ant, nothing else, so you could make a complete build from sources using only a JDK.
Michael - 1. Is Lenka now handling legal stuff re. extrabins? If so, please feel free to add her as a CC. I wasn't sure who would be responsible for this kind of thing currently. 2. To date I have only one LFI # (for djava.jar in scripting). It would be nice to get as many LFI #s added to the */external/build.xml files as soon as possible, so we can more easily track where binaries came from.
corba, mdr, and objectbrowser now build on the branch without nbextra. I am CCing JUnit and XTest module lists. These two modules have some JARs in nbextra which are apparently required for them to run. However these JARs are not listed in nbextra/CVSROOT/modules:nbbinaries, the master alias which gives files "permission" to be included in the extra binaries download. Therefore I have not tried to scramble them and move them into cvs.netbeans.org. So currently on the branch, you cannot run XTest (or build JUnit). ==> Michael and/or module owners, please indicate which is true: 1. You do not have legal approval to distribute these JARs. Anyone who wants to run XTest must have access to the nbextra repository, i.e. be a Sun employee, or find the stuff somewhere else. 2. It is fine to distribute these JARs under the proper licenses. They were only excluded from the extra binaries download to save download size. They should be included in cvs.netbeans.org in scrambled form. Affected licenses: IBM Public License for junit.jar; Apache for XTest's xerces & xalan; Apache plus DOM & SAX licenses for xml-apis; Apache for xtest-ext.jar (modified Apache sources? does it matter?); *unknown* for kill.exe and lib.jnikill.* (are these license-controlled at all, or just SPL?). On http://junit.netbeans.org/JUnit_build.html for building from sources it suggests to download junit.jar yourself from junit.org. However, our NBM file on AU includes junit.jar, hopefully marked with the IPL. On http://xtest.netbeans.org/Download/license-xtest-extra-binaries.html you are offered to get the XTest binaries under a BCL which includes the Apache and IPL licenses. (Why a BCL?? All the code is SPL, Apache, or IPL; Sun has nothing to "grant".) So my guess is that #2 is correct.
All modules except junit & xtest should now be converted to build on the branch, I think.
I forgot to ask about Jini: this also has bins in nbextra (jini*.jar) which are not listed for inclusion in the extra binaries download. However AFAIK these libraries have never been present in jini.nbm, i.e. the user is expected to download them separately, so I have no idea if they might be approved for scrambling. I will assume not, and leave them in nbextra - in order to build this module you must have an nbextra checkout, or create an nbextra dir and place downloaded jini*.jar in the correct places.
Concerning XTest: number 2 is correct. Concerning the BCL vs SPL: in xtest-ext.jar there is code, which is based on Apache's license and some code, which was developed at Sun (and is not based on Apache's code). Because the sources of the Sun's code are not available, we decided to use BCL instead of SPL. Personally I would like to get rid of xtest-ext.jar at all, but this will not definitely happen until the next major version of XTest.
OK, so junit & xtest will be supported in scrambled form as follows: junit.jar - IPL xerces, xalan, xml-apis - as for in core & libs modules kill.exe, lib.jnikill* - found SPL sources for these, so do not need to be scrambled at all, just move directly into xtest/lib/ for example xtest-ext.jar - will use BCL modified by Apache license
Have now hopefully fixed junit & xtest in the branch. Still to-do: 1. For convenience of the user, unscramble everything likely to be needed right at the beginning of the build, so the user does not have to click OK at three-minute intervals. 2. Have a bootstrap JAR which would unpack a runnable version of Ant 1.4.1 (and run it too on nbbuild/build.xml). 3. Fix up any remaining references to nbextra in build scripts of other modules on netbeans.org - TBD which.
Forgot: 4. Implement console license acceptance mode - possibly later. 5. Test on Windows, and as run from within NB using the Ant module.
#1 done. #3 to include modules: cpp db externaleditor icebrowser java3d lexer monitor sim tasklist #5: works OK from within NB.
#3 done.
Passing off to Ruda for evaluation & merging. I need someone to check that this all builds OK on Windows - I was about to do it myself but my XP registry just died a bloody death and Windoze will no longer even boot. Oh well. :-) Michael I would appreciate a final check of the contents of the */external/ directories - can send file list or contents upon request, or ask Ruda to send to you. Ruda: the following commands should suffice to merge changes into a complete trunk checkout: cd nb_all cvs up -j scrambler_26758_base -j scrambler_26758 ant apisupport applet autoupdate beans classfile clazz corba core cpp db debuggercore debuggerjpda diff editor extbrowser externaleditor form html httpserver i18n icebrowser image j2eeserver jarpackager java java3d javacvs javadoc jellytools jemmy jini jndi junit lexer libs mdr monitor nbbuild objectbrowser openide openidex projects properties qa rmi schema2beans scripting sim tasklist text tomcatint ui usersguide utilities vcscore vcscvs vcsgeneric web xml xtest cd ../nbextra cvs up -j scrambler_26758_base -j scrambler_26758 Then all should build as usual: cd ../nb_all ant -f nbbuild/build.xml ant -f xtest/instance/build.xml buildtests runtests With these changes, nbextra would only be required for: - working with non-open sources, such as the Java parser - building modules with unapproved extra binaries, e.g. jini - running QA functional tests - I left jemmy.jar and jelly2-nb.jar where they are, though QA may wish to move them into nb_all somewhere, I don't know - looking at patches made to standard JARs, e.g. Xerces patches in libs/ - working on scrambler.jar sources - nbbuild/ still has a minimal copy of Ant 1.4.1 for convenience (but see issue #26902) - qa/ has some stuff apparently needed for testing - not sure what it is exactly - web/core/release/docs/jsp-1_2-fcs-docs.zip and web/core/release/docs/servlet-2_3-fcs-docs.zip are probably no longer used for anything? Summary: most Sun developers working only on NetBeans will not need an nbextra checkout at all, or only occasionally. Possible impact on build systems and site: - no longer any need to create "extra binaries" archives for dev sources - if you try, they will be empty anyway - BCL license acceptance page needed only for downloading builds, never sources - NB continuous and daily builds in the trunk should not keep an nbextra checkout, as it will not be required - changelogs need not examine the nbextra repository - automated build machines must pass to Ant in order to bypass the need for GUI license acceptance: -Dscrambler=<<<MAGIC-TOKEN>>> (where the magic token is as given in nbextra/nbbuild/antsrc/org/netbeans/nbbuild/Unscramble.java) Let me know if there are other considerations I am not thinking of. Merge when you want, but remember that you longer you wait, the higher the chance of merge conflicts that will be time-consuming to solve.
Added LFI #s for scrambled extra binaries. These are stored in */external/build.xml files in CVS, right next to the lines that actually unscramble them.
I just copied the new 1.19 trunk nbextra version of xtest-ext.jar into the scrambler branch (in scrambled form of course). That is the only relevant trunk nbextra change I have seen since making the branch.
Will be merge conflicts in libs/ project: during merge please delete libs/apache-license.txt (all licenses must be named */external/*-license.*, and apache-license.txt is already present in nbbuild/external/), and be prepared for conflicts in libs/*/build.xml.
#5 OK on windows. done
I just merged content of scrambler_26758 branch to trunk. So there is no more need to download extra binaries from web.
Webmasters should update the download pages to link to extrabins no longer.
Webmasters, FYI. For branch/release builds, we still need to link to extra bins; for dev builds, we do not. Michal/Ruda - please merge nbextra too, not just nb_all. The entire nbextra repository was branched. Mark this FIXED when that is done, unless we are still waiting for web pages to be updated.
I know about nbextra, I'll merge it tomorow or at Monday. This was reason I didn't mark it as FIXED yet.
nbextra was merged too. Please verify.
Looks merged to me, thanks!
Web updates done.
Adjusting page loc'n to reflect implementation. Don't worry, there is a redirect.