This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 32497 - xml-apis-license.txt is not UTF-8 compliant
Summary: xml-apis-license.txt is not UTF-8 compliant
Status: VERIFIED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 3.x
Hardware: All All
: P1 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords:
Depends on: 31962
Blocks:
  Show dependency tree
 
Reported: 2003-03-31 09:53 UTC by rbalada
Modified: 2008-12-22 22:52 UTC (History)
7 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Patch for xml-apis-license.txt (682 bytes, patch)
2003-03-31 10:00 UTC, rbalada
Details | Diff
Scrambled xml-apis-1.0b2.jar binary (105.95 KB, application/octet-stream)
2003-03-31 10:02 UTC, rbalada
Details
Proposed patch for this one and #31962 (3.23 KB, patch)
2003-04-02 10:25 UTC, Michal Zlamal
Details | Diff
Scrambled xml-apis-1.0b2.jar (105.95 KB, application/octet-stream)
2003-04-02 12:24 UTC, Michal Zlamal
Details
removed a hidden Latin-1 char (5.51 KB, text/plain)
2003-04-02 13:39 UTC, _ ttran
Details

Note You need to log in before you can comment on or make changes to this bug.
Description rbalada 2003-03-31 09:53:45 UTC
The license file
core/external/xml-apis-license.txt is not UTF-8
compliant. The scrambled binary
core/external/xml-apis-1.0b2.jar.scrambled should
be changed in the same commit.
Comment 1 rbalada 2003-03-31 10:00:59 UTC
Created attachment 9598 [details]
Patch for xml-apis-license.txt
Comment 2 rbalada 2003-03-31 10:02:07 UTC
Created attachment 9599 [details]
Scrambled xml-apis-1.0b2.jar binary
Comment 3 _ ttran 2003-03-31 10:24:43 UTC
be careful.  I don't think we can convert this file to UTF-8 encoding
yet.  If all license files must use UTF-8 encoding this must be
documented somewhere and even better change the file extensions from
.txt to something like .utf8.  People usually don't expect this, incl
the current version of the unscrambler task.

I also suspect our webmaster just copy/paste all those license files
together and present the blurb on the download page.  Converting the
file to utf-8 will break this.

I personally would change the c in a circle in xml-api-license.txt to
'(c)', but don't know what the lawyers would say :-(
Comment 4 Jesse Glick 2003-03-31 14:53:43 UTC
Partially agreed with Trung for 3.5 - please just use (c) and (r),
this is the safest quick fix.

For the trunk, do not agree with Trung. All text files should be
assumed to be UTF-8 unless there is a good reason to treat them
differently, IMHO.

1. All license files should use UTF-8 encoding. Fine, we can document
it. Not all licenses are in ASCII, some are in Japanese.

2. Issue #31962 should be fixed again to read all license files in
this encoding.

3. Of course scrambler should read them in this encoding when
displaying license text in a text pane. Anyway the current system is
not correct since the scrambler does not know the encoding to use and
is probably just guessing at ISO-8859-1 for most platforms.

4. If webmasters copy/paste these license files together for the
download page - which I do *not* think is the case, I think they get
the master merged BCL straight from lawyers - it is their problem to
set the encoding correctly on the resulting page.

Ruda next time please do a complete clean build (JDK 1.3.1, Ant 1.4.1)
before committing any change (esp. in release35 branch); we should
have caught this problem *before* the patch for issue #31962 was
committed.
Comment 5 Martin Brehovsky 2003-04-01 10:22:37 UTC
When updating the license and the scrabled file, please don't forget
to update xml-apis-1.0b2.jar.scrambled binary in module
xtest/external. We use the license from core to unscramble the binary.
Comment 6 Michal Zlamal 2003-04-02 10:25:47 UTC
Created attachment 9649 [details]
Proposed patch for this one and #31962
Comment 7 _ ttran 2003-04-02 11:21:41 UTC
Michal, please attach the xml-apis-1.0b2.jar.scrambled using the
sanitized license file.
Comment 8 Michal Zlamal 2003-04-02 12:24:20 UTC
Created attachment 9657 [details]
Scrambled xml-apis-1.0b2.jar
Comment 9 _ ttran 2003-04-02 13:38:05 UTC
Man, you didn't rebuild the whole thing w/ JDK 1.3.1!!!
Your modified xml-apis-license.txt still contains one hidden non-UTF8
char.  I'll attach the correct one.  Please re-scramble the jar
Comment 10 _ ttran 2003-04-02 13:39:49 UTC
Created attachment 9660 [details]
removed a hidden Latin-1 char
Comment 11 Jesse Glick 2003-04-02 18:18:02 UTC
Michal's patch looks OK to me with Trung's corrected text file.

There was an 0xa0 used as a space for no good reason, which Trung
removed. I checked that there are no more non-ASCII chars in it:

od -tx1 -w1 -v /tmp/xml-apis-license.txt | cut -c9 | sort | uniq
Comment 12 Jesse Glick 2003-04-02 21:01:37 UTC
I will take care of this in trunk & r35.
Comment 13 Jesse Glick 2003-04-03 00:13:44 UTC
Fixed in r35:

committed   * Up-To-Date  1.2.72.1   
core/external/xml-apis-1.0b2.jar.scrambled
committed   * Up-To-Date  1.2.72.1    core/external/xml-apis-license.txt
committed   * Up-To-Date  1.1.14.1    libs/external/w3c-license.html
committed   * Up-To-Date  1.1.14.1   
libs/external/xhtml-basic.zip.scrambled
committed   * Up-To-Date  1.1.14.1   
libs/external/xhtml-modularization.zip.scrambled
committed   * Up-To-Date  1.1.14.1    libs/external/xhtml1.zip.scrambled
committed   * Up-To-Date  1.1.14.1    libs/external/xhtml11.zip.scrambled
committed   * Up-To-Date  1.2.48.1    mdr/external/mof-license.txt
committed   * Up-To-Date  1.2.48.1    mdr/external/mof.jar.scrambled
committed   * Up-To-Date  1.1.26.5   
nbbuild/antsrc/org/netbeans/nbbuild/MakeLNBM.java
committed   * Up-To-Date  1.36.2.6   
nbbuild/antsrc/org/netbeans/nbbuild/MakeNBM.java
committed   * Up-To-Date  1.2.54.1    xml/external/flute-sac-license.html
committed   * Up-To-Date  1.2.54.1    xml/external/flute.jar.scrambled
committed   * Up-To-Date  1.2.54.1    xml/external/sac.jar.scrambled
committed   * Up-To-Date  1.2.32.1   
xtest/external/xml-apis-1.0b2.jar.scrambled
Comment 14 Jesse Glick 2003-04-03 01:49:27 UTC
And fixed better in the trunk:

committed     Up-To-Date  1.3        
core/external/xml-apis-1.0b2.jar.scrambled
committed     Up-To-Date  1.3         core/external/xml-apis-license.txt
committed     Up-To-Date  1.2         libs/external/w3c-license.html
committed     Up-To-Date  1.2        
libs/external/xhtml-basic.zip.scrambled
committed     Up-To-Date  1.2        
libs/external/xhtml-modularization.zip.scrambled
committed     Up-To-Date  1.2         libs/external/xhtml1.zip.scrambled
committed     Up-To-Date  1.2         libs/external/xhtml11.zip.scrambled
committed     Up-To-Date  1.3         mdr/external/mof-license.txt
committed     Up-To-Date  1.3         mdr/external/mof.jar.scrambled
committed     Up-To-Date  1.6        
nbbuild/antsrc/org/netbeans/nbbuild/MakeLNBM.java
committed     Up-To-Date  1.42       
nbbuild/antsrc/org/netbeans/nbbuild/MakeNBM.java
committed     Up-To-Date  1.3         nbbuild/external/scrambler.jar
committed     Up-To-Date  1.3         nbbuild/www/scrambler.html
committed     Up-To-Date  1.3         xml/external/flute-sac-license.html
committed     Up-To-Date  1.3         xml/external/flute.jar.scrambled
committed     Up-To-Date  1.3         xml/external/sac.jar.scrambled
committed     Up-To-Date  1.3        
xtest/external/xml-apis-1.0b2.jar.scrambled
Comment 15 _ ttran 2003-04-03 09:05:05 UTC
I already gave my approval to Jesse to get this fix in r35 via phone.
 Adding a note here for tracking purpose.
Comment 16 Marian Mirilovic 2003-04-07 09:56:20 UTC
Ruda, can you verify this one, thanks in advance :)
Comment 17 Marian Mirilovic 2004-03-15 15:09:55 UTC
verified