This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 183724 - output window parses utf-8 incorrectly
Summary: output window parses utf-8 incorrectly
Status: RESOLVED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 6.x
Hardware: PC Windows Vista
: P3 normal (vote)
Assignee: Alexander Simon
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2010-04-08 16:13 UTC by xym777
Modified: 2011-09-17 03:50 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
screenshot of netbeans output window (338.97 KB, image/jpeg)
2010-04-08 16:15 UTC, xym777
Details
output file from gcc (760 bytes, text/plain)
2010-04-08 16:17 UTC, xym777
Details
output file from gcc in pic to show Chinese in utf-8 (231.03 KB, image/jpeg)
2010-04-08 16:19 UTC, xym777
Details
output file from gcc converted into utf-8 with BOM in case you need it (763 bytes, text/plain)
2010-04-08 16:21 UTC, xym777
Details
source code to cause error purposely (1.76 KB, text/plain)
2010-04-08 16:23 UTC, xym777
Details
incorrect displayed chars (228.37 KB, image/jpeg)
2010-12-23 09:50 UTC, xym777
Details
env (1.92 KB, text/plain)
2011-01-17 07:55 UTC, xym777
Details
output with build-pro cat output none cmake (184.59 KB, image/jpeg)
2011-01-20 08:35 UTC, xym777
Details
about of my netbeans (54.41 KB, image/jpeg)
2011-01-20 08:44 UTC, xym777
Details
msysinfo in reply to comment#26 (1.19 KB, text/plain)
2011-01-20 08:49 UTC, xym777
Details
encoding of welcome_1 without cmake (in reply to comment #26) (50.12 KB, image/jpeg)
2011-01-20 08:57 UTC, xym777
Details
encoding of welcome_2 cmake (48.86 KB, image/jpeg)
2011-01-20 09:00 UTC, xym777
Details
output incorrect new from cmake (287.58 KB, image/jpeg)
2011-01-20 09:33 UTC, xym777
Details
output correct by click run (296.71 KB, image/jpeg)
2011-01-20 09:36 UTC, xym777
Details
set eoncoding utf-8 in cmake not solve the problem (117.26 KB, image/jpeg)
2011-01-21 05:31 UTC, xym777
Details
msys make screen with cmake encoding utf-8 (78.05 KB, image/jpeg)
2011-01-21 08:05 UTC, xym777
Details
msys make output error message (340 bytes, text/plain)
2011-01-21 08:09 UTC, xym777
Details

Note You need to log in before you can comment on or make changes to this bug.
Description xym777 2010-04-08 16:13:29 UTC
Gcc's output message is in utf-8-no-bom format.
Netbeans' output windows dispaly gcc's output message incorrectly if there are multi-byte chars, e.g. Chinese chars inside those gcc output message.

Each Chinese char occupies 3 bytes in utf-8 format.
It seems the output window parses each 2 none ascii bytes into Chinese char.

Please refer to attachment for detail.
Comment 1 xym777 2010-04-08 16:15:39 UTC
Created attachment 96912 [details]
screenshot of netbeans output window
Comment 2 xym777 2010-04-08 16:17:18 UTC
Created attachment 96913 [details]
output file from gcc
Comment 3 xym777 2010-04-08 16:19:45 UTC
Created attachment 96914 [details]
output file from gcc in pic to show Chinese in utf-8
Comment 4 xym777 2010-04-08 16:21:02 UTC
Created attachment 96915 [details]
output file from gcc converted into utf-8 with BOM in case you need it
Comment 5 xym777 2010-04-08 16:23:05 UTC
Created attachment 96916 [details]
source code to cause error purposely
Comment 6 xym777 2010-04-24 19:04:33 UTC
This bug is not solved in 6.9
Comment 7 xym777 2010-08-12 04:15:12 UTC
OK, this is solved.
Thanks.
Comment 8 xym777 2010-08-12 07:43:48 UTC
(In reply to comment #7)
> OK, this is solved.
> Thanks.

This bug is fixed in 6.9.1
Comment 9 xym777 2010-12-23 09:50:32 UTC
Created attachment 104435 [details]
 incorrect displayed chars
Comment 10 xym777 2010-12-23 09:52:20 UTC
it seems that this issue is not solved completely.
Please refer to the attachement "incorrect displayed chars".
Comment 11 Martin Entlicher 2011-01-11 22:58:49 UTC
Since only the output from gcc is corrupted, it looks like the problem is in the transfer of gcc output to the Output Window. Please evaluate in CND...
Comment 12 Vladimir Voskresensky 2011-01-11 23:35:07 UTC
xym777, have you tried to set up tool chain Encoding property in Tools->Options->C/C++->Build Tools panel?

Thanks,
Vladimir.
Comment 13 xym777 2011-01-12 03:35:36 UTC
(In reply to comment #12)
> xym777, have you tried to set up tool chain Encoding property in
> Tools->Options->C/C++->Build Tools panel?
> 
> Thanks,
> Vladimir.

Encoding property in 
Tools->Options->C/C++->Build Tools panel 
is utf-8
Comment 14 Vladimir Voskresensky 2011-01-14 21:44:50 UTC
Sasha, do you have ideas?
Comment 15 Alexander Simon 2011-01-14 23:00:12 UTC
(In reply to comment #14)
> Sasha, do you have ideas?
I need more information:
- What tool collections are available?
- Is cygwin installed?
- Is msys installed?
- output of env and set command.

My idea is: commands and tool collection has different encoding.
Comment 16 xym777 2011-01-15 02:57:05 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Sasha, do you have ideas?
> I need more information:
> - What tool collections are available?
> - Is cygwin installed?
> - Is msys installed?
> - output of env and set command.
> 
> My idea is: commands and tool collection has different encoding.

Tool collection:
MinGW

Cygwin is not installed.
MSYS is installed.
CMake is installed also.
Comment 17 Alexander Simon 2011-01-16 04:03:12 UTC
(In reply to comment #16)
> Tool collection:
> MinGW
> 
> Cygwin is not installed.
> MSYS is installed.
> CMake is installed also.
What is about output of env and set command?
Comment 18 xym777 2011-01-16 05:39:18 UTC
(In reply to comment #17)
> (In reply to comment #16)
> > Tool collection:
> > MinGW
> > 
> > Cygwin is not installed.
> > MSYS is installed.
> > CMake is installed also.
> What is about output of env and set command?

What does "output of env and set command" refer to?
What do you want to know by asking "output of env and set command"
Comment 19 Alexander Simon 2011-01-16 09:50:11 UTC
(In reply to comment #18)
> What do you want to know by asking "output of env and set command"
- Start console (for example msys\msys.bat)
- type in console:
env
- attach output of env command in issue.
Comment 20 xym777 2011-01-17 07:55:23 UTC
Created attachment 105058 [details]
env

This is the output of env.
Comment 21 Alexander Simon 2011-01-18 14:06:42 UTC
(In reply to comment #3)
> Created an attachment (id=96914) [details]
> output file from gcc in pic to show Chinese in utf-8
Could you explain how did you catch this file?
Comment 22 xym777 2011-01-19 05:22:45 UTC
(In reply to comment #21)
> (In reply to comment #3)
> > Created an attachment (id=96914) [details] [details]
> > output file from gcc in pic to show Chinese in utf-8
> Could you explain how did you catch this file?

To catch this file, you can also refer to the attachment "screenshot of netbeans output window" for detail. It's url is http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=96912

Steps of catch this file is as below:
Step 0. Install netbeans on to a Chinese Windows (my one is vista) system.


Step 1. create a netbeans project, eg. named "Welcome_1"


Step 2. in "welcome.cc", input:
#include <iostream>
#include <winsock2.h>
int main(int argc, char**argv) {
    WSADATA info;
    if (WSAStartup(MAKEWORD(2,0), &info)) {
      throw "Could not start WSA";
    }
    return 0;
}

This source code is uploaded as "source code to cause error purposely". The url is: http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=96916


Step 3. Add include path to the project where "winsock2.h" is.


Step 4. Make sure NOT add libgcc_s_dw2-1.dll into the project to cause error.


Step 5. Compile the project, you should get error message. If your system is in Chinese, those message is displayed incorrectly. Please refer to right-bottom side of "screenshot of netbeans output window" for detail. It's url is http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=96912


Step 6. If you want to save the output message into a file, you would not get it. You will get another bug which I reported at url: http://netbeans.org/bugzilla/show_bug.cgi?id=183725


Step 7. In order to get what those error message actually is, open MSYS interface and cd the project directory( my one is /d/_work/Welcome_1)


Step 8. run "make > make.output 2>&1". This command will save error message of make into "make.output". Please refer to left side of the attachement "screenshot of netbeans output window" for detail. It's url is http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=96912


Step 9. Open make.output in netbeans. You can find netbeans can display it correctly.
Please refer to right-top side of the attachement "screenshot of netbeans output window" for detail. It's url is http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=96912


Step 10. Comparing right side top and bottom, those are different display of the same message.
Comment 23 xym777 2011-01-19 05:33:27 UTC
Step 11. run a binary editor to show make.output. My one is PSPad. PSPad can not display Chinese characters also. It displays "?" instead.
Comment 24 Alexander Simon 2011-01-19 08:25:10 UTC
xym777,
thank you for detailed steps.
Could you make following experiment:
- open Makefile in editor
- change pre-build target:
.build-pre:
	@cat <path to output file from gcc>
- attach screen shot with whole IDE output
Comment 25 Alexander Simon 2011-01-19 10:30:56 UTC
Could you also attach content of IDE "Help->About" dialog (content of text area with Product Version, Java, and System information)?
Comment 26 Andrew Krasny 2011-01-19 17:00:14 UTC
xym777, thanks for your help if looking for a root of the problem.. 
I would like to ask you for the following: (in addition to what Alexander have asked you for)

- output of msysinfo command;
- what Encoding is set in your project's properties? (right-click on a project->Properties->General->Encoding)


Thanks!
Comment 27 xym777 2011-01-20 08:35:39 UTC
Created attachment 105187 [details]
output with build-pro cat output none cmake

This is output when set "@cat make.output" to build-pre of makefile of project welcome_1 which is not a cmake project.
Comment 28 xym777 2011-01-20 08:41:34 UTC
(In reply to comment #24)
> xym777,
> thank you for detailed steps.
> Could you make following experiment:
> - open Makefile in editor
> - change pre-build target:
> .build-pre:
>     @cat <path to output file from gcc>
> - attach screen shot with whole IDE output

Please refer to the attachment "output with build-pro cat output none cmake" url: http://bugzilla-attachments-183724.netbeans.org/bugzilla/attachment.cgi?id=105187
Comment 29 xym777 2011-01-20 08:44:53 UTC
Created attachment 105188 [details]
about of my netbeans

about of my netbeans
Comment 30 xym777 2011-01-20 08:49:22 UTC
Created attachment 105189 [details]
msysinfo in reply to comment#26

msysinfo in reply to comment#26
Comment 31 xym777 2011-01-20 08:57:30 UTC
Created attachment 105190 [details]
encoding of welcome_1 without cmake (in reply to comment #26)

encoding of welcome_1 without cmake (in reply to comment #26)

encoding of this project is utf-8
Comment 32 xym777 2011-01-20 09:00:32 UTC
Created attachment 105191 [details]
encoding of welcome_2 cmake

(in reply to comment#26)

This project is with cmake.
It's encoding is utf-8 but I found that output of make is in ANSI format.
Comment 33 Alexander Simon 2011-01-20 09:20:28 UTC
Could you clarify problem
- make output is correct
- cmake output icorrect
Yes?
If it is right the problem is in cmake.
All part of tool collection should have same encoding.
Comment 34 xym777 2011-01-20 09:33:19 UTC
Created attachment 105192 [details]
output incorrect new from cmake

case 1: the project is newly created from cmake;
case 2: the project is "re-configured"

both abovementioned cases will cause auto "make clean" and auto "make".
In both cases output window is display incorrectly.
Comment 35 xym777 2011-01-20 09:36:23 UTC
Created attachment 105193 [details]
output correct by click run

In the same welcome_2 project created from cmake;
if "run" is clicked to cause make,
this time output window is displayed correctly.
Comment 36 xym777 2011-01-20 09:52:20 UTC
In order to catch the problem, I created a nother project:

1. copy welcome_2 from welcome_1;

2. delete everything other than welcome.cc and welcome.h;

3. create CMakeLists.txt in directory of welcome_2;

4. insert 3 lines into CMakeLists.txt:
project(welcome2)
set(src_list welcome.cc)
add_executable(welcome2 ${src_list})

5. create a new project basing on current src of welcome_2, auto configure using cmake

6. The project will auto create from cmake, auto make clean, then auto make.
This time the output window displays incorrectly.
or
If I right-click project "welcome_2", and select "sourcecode help" - "reconfigure", it will cause auto make clean and auto make also. This time the output window displays incorrectly also.
Please refer to attachment "output incorrect new from cmake", url: http://netbeans.org/bugzilla/attachment.cgi?id=105192

7. Then, if I click the "run" button, it will cause make also, this time the output is displayed correctly.
Please refer to attachment "output correct by click run", url: http://netbeans.org/bugzilla/attachment.cgi?id=105193
Comment 37 xym777 2011-01-20 10:01:33 UTC
(In reply to comment #0)
> Gcc's output message is in utf-8-no-bom format.
> Netbeans' output windows dispaly gcc's output message incorrectly if there are
> multi-byte chars, e.g. Chinese chars inside those gcc output message.
> 
> Each Chinese char occupies 3 bytes in utf-8 format.
> It seems the output window parses each 2 none ascii bytes into Chinese char.
> 
> Please refer to attachment for detail.

In the comment#0, I said that gcc's output message is in utf-8. This is not accurate. This time output is in ANSI format.

My OS has re-installed several times.
Comment 38 Alexander Simon 2011-01-20 10:21:25 UTC
Could you try to set encoding in CMakeLists.txt:
SET(ENCODING utf-8)
?
Comment 39 xym777 2011-01-21 05:18:04 UTC
(In reply to comment #37)
> (In reply to comment #0)
> > Gcc's output message is in utf-8-no-bom format.
> > Netbeans' output windows dispaly gcc's output message incorrectly if there are
> > multi-byte chars, e.g. Chinese chars inside those gcc output message.
> > 
> > Each Chinese char occupies 3 bytes in utf-8 format.
> > It seems the output window parses each 2 none ascii bytes into Chinese char.
> > 
> > Please refer to attachment for detail.
> 
> In the comment#0, I said that gcc's output message is in utf-8. This is not
> accurate. This time output is in ANSI format.
> 
> My OS has re-installed several times.

Sorry, comment #37 is wrong. 
I'd like to delete comment #37. Can I?
Comment 40 xym777 2011-01-21 05:31:32 UTC
Created attachment 105225 [details]
set eoncoding utf-8 in cmake not solve the problem

(In reply to comment #38)
Comment 41 Alexander Simon 2011-01-21 07:21:42 UTC
(In reply to comment #40)
> Created an attachment (id=105225) [details]
> set eoncoding utf-8 in cmake not solve the problem
What does make command print in msys console?
Comment 42 xym777 2011-01-21 08:05:36 UTC
Created attachment 105227 [details]
msys make screen with cmake encoding utf-8

In reply to comment #41

MSYS displays incorrectly.
Comment 43 xym777 2011-01-21 08:09:18 UTC
Created attachment 105228 [details]
msys make output error message

In reply to comment #41

msys make output error message which is in uft-8 format.
Comment 44 xym777 2011-09-17 03:47:58 UTC
The second part of this bug, which is described from comment #9 to comment #41 is solved in 7.0
Comment 45 xym777 2011-09-17 03:50:12 UTC
(In reply to comment #44)
> The second part of this bug, which is described from comment #9 to comment #41
> is solved in 7.0

The second part of this bug, which is described from comment #9 to comment #43
is solved in 7.0