Bug 99344 - Provide FileEncodingQueryImplementation in project's lookup
Provide FileEncodingQueryImplementation in project's lookup
Status: VERIFIED FIXED
Product: ruby
Classification: Unclassified
Component: Code
6.x
All All
: P2 (vote)
: 6.x
Assigned To: Torbjorn Norbye
issues@ruby
:
Depends on:
Blocks: 97848
  Show dependency treegraph
 
Reported: 2007-03-29 08:57 UTC by Tomas Zezula
Modified: 2007-07-17 11:47 UTC (History)
3 users (show)

See Also:
Issue Type: TASK
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Zezula 2007-03-29 08:57:46 UTC
The new SPI FileEncodingQueryImplementation allows IDE components like editor,
search, etc to handle correctly encoding of files. The implementation of
FileEncodingQueryImplementation should be provided in the project's lookup.
The typical implementation of this query uses a project property with encoding
and returns a Charset.forName (evaluator.getProperty(name_of_encoding_property).
The project should also provide a control in the project's customizer to allow
user to change the encoding of the project.
Comment 1 Masaki Katakai 2007-04-17 10:38:42 UTC
It is very important to support this file encoding in Ruby Pack. I understand
many Japanese developers are interested in NetBeans Ruby Pack :-) One big issue
is that we cat not use UTF-8 in it. Most users want to use UTF-8 encoding on
Windows (Shift_JIS is native encoding in Japanese Windows) but currently it's
not possible. I hope it will be fixed soon and more users can eveluate and use
Ruby Pack using UTF-8 encoding.
Comment 2 Torbjorn Norbye 2007-04-17 18:28:01 UTC
Yes, I will do this. As a temporary workaround, you may want to try adding 
-J-Dfile. encoding=UTF-8 
to your netbeans.conf.
Comment 3 Masaki Katakai 2007-04-18 03:09:13 UTC
It's good news! Thank you Tor for updating bug report.
Comment 4 Torbjorn Norbye 2007-05-26 00:21:13 UTC
I have implemented this. I haven't integrated it yet because I know very little
about encoding, or how to test it.

Can you (masaki) take the bits for a spin and see if they seem to work? If so, I
will integrate the fixes. For now, I have only implemented this for the Ruby On
Rails project type, not the Ruby Application project type. To try it, download
the IDE pointed to in this page:
http://mediacast.sun.com/share/tor/netbeans-rubyide-hudson-2.zip

Unzip, then run nbrubyide/bin/nbrubyide and construct a Rails project. Hopefully
it will work as expected :)
Comment 5 Masaki Katakai 2007-05-26 00:33:20 UTC
Thank you Tor. Sure, I'll try and let you know the results :-)
Comment 6 simon_nicholls 2007-05-27 13:15:37 UTC
I've tried the utf-8 support in netbeans-rubyide-hudson-2.zip (& build #3), but
can't create new utf-8 files from within the IDE - they are always created with
MacRoman encoding (U.S non extended system default), regardless of the project
encoding settings. I checked this using the file properties context menu in the IDE.

Editing utf-8 files created from vim & textmate seems to be fine though, as did
editing a more complicated example found at
http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt - though they show no
encoding property in the IDE.
Comment 7 Masaki Katakai 2007-05-27 16:15:56 UTC
I just started the evaluation. I could verify
   - encoding property is now in the project property dialog.
   - default encoding is UTF-8.

It's great! And I tried some file type
   - Start NetBeans in ja_JP.EUC locale
   - Create a ruby on rails project
   - Make sure project encoding is UTF-8 on property dialog of project
   - Create some type of files, ruby class, text file and javascript files
     I could verify these files are stored as UTF-8 encoding
     However, when I tried a html file, it's EUC encoding.

Simon, which file type are you using? At least I could verify that ruby files,
text and javascript files are properly encoded in UTF-8.

I'll continue testing.
Comment 8 simon_nicholls 2007-05-27 16:25:34 UTC
Aha!! I was testing unfortunately with html files, since the view is my driver
for initial utf-8 use.

I can now also verify that rhtml, rb, txt, and other files are created just
fine. Great!
Comment 9 simon_nicholls 2007-05-27 16:26:09 UTC
Aha!! I was testing unfortunately with html files, since the view is my driver
for initial utf-8 use.

I can now also verify that rhtml, rb, txt, and other files are created just
fine. Great!
Comment 10 Masaki Katakai 2007-05-28 02:05:59 UTC
I think UTF-8 should be used by the default setting even for html file.

Who (which component) should take care html file type ?
Comment 11 simon_nicholls 2007-05-28 14:07:08 UTC
Would be nice to see this integrated, with a new issue raised for html encoding
support.

Another issue for me is that I think my default project encoding was
automatically set to UTF-8, rather than my system default, which wasn't what I
expected.
Comment 12 simon_nicholls 2007-05-28 14:08:26 UTC
Would be nice to see this integrated, with a new issue raised for html encoding
support.

Another issue for me is that I think my default project encoding was
automatically set to UTF-8, rather than my system default, which wasn't what I
expected.
Comment 13 Tomas Zezula 2007-05-28 14:16:56 UTC
There is already in issue for HTML module:
http://www.netbeans.org/issues/show_bug.cgi?id=97873. But I sugest to use the
project encoding as a default, for most users UTF-8 but if someone decides to
have the project in some other encoding (s)he may want to have also the html
file in this encoding.
Comment 14 simon_nicholls 2007-05-28 15:39:35 UTC
Sorry for these double comments btw. There's something funky going on between my BonEcho Mac widget 
build, its plugins, and this issue tracker. I will try using Safari instead.

I also think UTF-8 should be the default, and I use it throughout the stack for projects to avoid trouble in 
the long run, but it automatically changed an existing Rails project too (I think). Not a problem for me, but 
I wonder how that might affect users on platforms and charsets more exotic than my Mac/MacRoman 
setup.

Having said that, I'd much rather have an errant UTF-8 file in my project than something less universal, 
and UTF-8 is the default as of Rails 1.2 after all.
Comment 15 simon_nicholls 2007-05-28 18:28:07 UTC
For the benefit of people looking in on this and wanting to use UTF-8 in their html pages, you can just add 
the usual content-type meta to your document head, but be sure to do this before a file save containing 
"extended" charset content. Works for regular builds as well as the special build being discussed here.

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
Comment 16 Torbjorn Norbye 2007-05-28 19:14:36 UTC
This is now checked in (and I've done it for plain Ruby projects as well). I
will mark this bug as fixed; there is already a different issue tracking the
HTML problem. Regarding default encoding of UTF-8 - I'm basically doing the same
thing that is done for other project types in NetBeans (such as Java); if we
change it here we should probably do it globally. I'm not sure exactly what you
advocated; do you think UTF-8 is the best default, or the current VM encoding?

Checking in projects/nbproject/project.properties;
/cvs/scripting/ruby/projects/nbproject/project.properties,v  <--  project.properties
new revision: 1.65; previous revision: 1.64
done
Checking in
railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/CustomizerRun.form;
/cvs/scripting/ruby/railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/CustomizerRun.form,v
 <--  CustomizerRun.form
new revision: 1.3; previous revision: 1.2
done
Checking in
railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/Bundle.properties;
/cvs/scripting/ruby/railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/Bundle.properties,v
 <--  Bundle.properties
new revision: 1.3; previous revision: 1.2
done
Checking in
railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/RailsProjectProperties.java;
/cvs/scripting/ruby/railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/RailsProjectProperties.java,v
 <--  RailsProjectProperties.java
new revision: 1.3; previous revision: 1.2
done
Checking in
railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/CustomizerRun.java;
/cvs/scripting/ruby/railsprojects/src/org/netbeans/modules/ruby/railsprojects/ui/customizer/CustomizerRun.java,v
 <--  CustomizerRun.java
new revision: 1.3; previous revision: 1.2
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/CustomizerSources.form;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/CustomizerSources.form,v
 <--  CustomizerSources.form
new revision: 1.2; previous revision: 1.1
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/Bundle.properties;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/Bundle.properties,v
 <--  Bundle.properties
new revision: 1.3; previous revision: 1.2
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/RubyProjectProperties.java;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/RubyProjectProperties.java,v
 <--  RubyProjectProperties.java
new revision: 1.2; previous revision: 1.1
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/CustomizerSources.java;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/ui/customizer/CustomizerSources.java,v
 <--  CustomizerSources.java
new revision: 1.2; previous revision: 1.1
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/RubyProjectGenerator.java;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/RubyProjectGenerator.java,v
 <--  RubyProjectGenerator.java
new revision: 1.3; previous revision: 1.2
done
Checking in projects/src/org/netbeans/modules/ruby/rubyproject/RubyProject.java;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/RubyProject.java,v
 <--  RubyProject.java
new revision: 1.3; previous revision: 1.2
done
RCS file:
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/queries/RubyProjectEncodingQueryImpl.java,v
done
Checking in
projects/src/org/netbeans/modules/ruby/rubyproject/queries/RubyProjectEncodingQueryImpl.java;
/cvs/scripting/ruby/projects/src/org/netbeans/modules/ruby/rubyproject/queries/RubyProjectEncodingQueryImpl.java,v
 <--  RubyProjectEncodingQueryImpl.java
initial revision: 1.1
done
IDE: [5/28/07 11:11 AM] Committing finished
Comment 17 simon_nicholls 2007-05-28 20:06:29 UTC
UTF-8 is great for me, and would be a great default for pretty much all users I think. However, I was 
thinking of users in say Japan, where UTF-8 does not always handle all requirements, and it's possible a 
user's default encoding would be preferrable (assuming the JVM supports relevant alternative character 
sets).

A web page regarding some of the issues can be found here: http://www.jbrowse.com/text/unij.html

I don't want to overstate it as a being a problem though, and just wanted to say something about it so that 
people who are closer to the issues involved might respond. If UTF-8 is the default for other project types, 
I think it's best to go with it. Thanks for the checkin!
Comment 18 Tomas Zezula 2007-05-29 07:22:39 UTC
I think that the most correct behaviour is:
1) When project is created it uses FileEncodingQuery.getDefaultEncoding() The
project is created in encoding that the user used last time, for most users
UTF-8, but if someone wants to use different encoding (s)he won't need to change
the encoding everytime (s)he creates project from UTF-8 to her/his encoding.

2) When new file is created in the project it uses project's encoding.
In most cases all files under the project have the same encoding.

3) If the user changes project's encoding, the project's customizer updates
the default encoding using FileEncodingQuery.setDefaultEncoding(). Newly created
project will use this encoding.
Comment 19 simon_nicholls 2007-05-29 11:13:53 UTC
I'd prefer a UI accessible IDE level preference for encoding, that defaults to UTF-8, so that the encoding 
type that will be used for new projects is more explicit. This would be then be picked up when creating 
new projects, and could also be overridden at project creation time.

Whilst changing a project's default encoding after creation is good for new files, the actual creation of 
projects should also be encoding aware, for when new projects generate their initial (encoded) file 
structure.

I'd like to request this as an enhancement, as I think this issue/ticket is fixed as is, but I realize this is 
outside of the Ruby scope somewhat.
Comment 20 Masaki Katakai 2007-05-30 15:38:30 UTC
I was thinking that UTF-8 would be the best for the default encoding,
but when I started to use this Ruby project on NetBeans, I think
it's better to use the platform encoding because Ruby is a script language.

The ruby runtime will be running in the platform encoding, e.g.
MS932 in Japanese Windows, so it's natural for users to use MS932
in .rb files. Currently UTF-8 is the default, so the following
simple scenario is not working.

1) Start NetBeans
2) Create a Ruby project
3) Change the code to
    puts "...<japanese characters>..."
    <japanese characters> are encoded in UTF-8 by default
4) Run
   but runtime is running in MS932, but the codes are UTF-8, so Japanese
   characters become garbled on outout window.

There is no problem in Java codes, which will be compileted by compiler with
the proper encoding option before the execution. The problem may happen
for scripting language. Btw, Eclipse is using the default encoding of
platform.

What do you think?
Comment 21 simon_nicholls 2007-05-30 16:24:30 UTC
Hi Masaki,

You've confirmed something I had some concerns about. For me, UTF-8 is great,
but then it's a superset of the ASCII encoding I'd be using by default for
English. For environments where UTF-8 is not the obvious extended character set
choice, it can be problematic, especially considering the BOM free nature of the
encodes.
Comment 22 Ken Frank 2007-06-03 20:21:28 UTC
IMO we need to get a unified model and experience for all parts of nb6 rather than
different per project type

such as for what the default encoding assumption might be.

Also, remember that even if default encoding is utf-8, that is for internal
handling -
it will or should still handle encoding of the files you create if you start ide
in another
locale (until you might change the project encoding).

Tomas, please tell me if this is not a correct assumption.

ken.frank@sun.com
Comment 23 Masaki Katakai 2007-06-07 15:33:15 UTC
If we need one default solution, platform encoding would be reasonable as the
default value for most users.
As I mentioned before, platform encoding should be used in script projects. Most
users will use the scripts in the default encoding. Also there is no problem if
we use the default encoding in Java, Web and other projects. Eclipse is using
the platform encoding by default.

I'll file an RFE for this.

Comment 24 Tomas Zezula 2007-06-07 20:01:27 UTC
The default encoding should be an UTF-8 until user changes it to something else,
the UTF-8 is the most portable encoding. Most OS have UTF-8 encoding by default.
The java and web project will definitely use UTF-8 as default, the apisupport
project supports only UTF-8 because of shareability.
Comment 25 Jiri Kovalsky 2007-07-03 14:14:24 UTC
Reassigning this issue to newly created 'ruby' component.
Comment 26 Jiri Kovalsky 2007-07-03 14:44:54 UTC
Changing target milestone of all resolved Ruby issues from TBD to 6.0 Beta 1 build.
Comment 27 Ken Frank 2007-07-07 19:09:08 UTC
verifying this issue only as to what is related to basic project feq of reading and writing
global project encoding

- ruby or rails project uses global project encoding value

- when changing ruby or rails project encoding value, that value is used in new ruby or rails
or other project like j2se.


for compilation and running, and how ruby modules handle encoding of the actual file - I think there are
other issues like 106089, 99058, 106560 that might cover these and perhaps the other topics discussed below.

if those issues don't cover that, please file new issues in ruby category and please add word I18N to summary line
and add kfrank to cc list.

ken.frank@sun.com
Comment 28 Masaki Katakai 2007-07-17 11:47:43 UTC
 > I think that the most correct behaviour is:
 > 1) When project is created it uses FileEncodingQuery.getDefaultEncoding() The
 > project is created in encoding that the user used last time, for most users
 > UTF-8, but if someone wants to use different encoding (s)he won't need to change
 > the encoding everytime (s)he creates project from UTF-8 to her/his encoding.
 > 
 > 2) When new file is created in the project it uses project's encoding.
 > In most cases all files under the project have the same encoding.
 > 
 > 3) If the user changes project's encoding, the project's customizer updates
 > the default encoding using FileEncodingQuery.setDefaultEncoding(). Newly created
 > project will use this encoding.

I have been using NetBeans 6 M10 for creating several new
projects and loading from 5.x projects, I'm now thinking
this scenario would not be good.

In general, settings for particular project should not affect
other projects. So it would not be easy for users to understand
the behavior around the encoding.

I'll file an RFE to provide the global setting on option dialog
and change current behavior.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo