[Encoding]How to do proper encoding output redirection on cmd.exe?

Problem

If you are doing automation on windows and you are redirecting the output of different commands (internal cmd.exe or external, you’ll discover that your log files contains combined Unicode and ANSI output (meaning that they are invalid and will not load well in viewers/editors).

Is it is possible to make cmd.exe work with UTF-8? This question is not about display, s about stdin/stdout/stderr redirection and Unicode.

I am looking for a solution that would allow you to:

redirect the output of the internal commands to a file using UTF-8
redirect output of external commands supporting Unicode to the files but encoded as UTF-8.
If it is impossible to obtain this kind of consistence using batch files, is there another way of solving this problem, like using python scripting for this? In this case, I would like to know if it is possible to do the Unicode detection alone (user using the scripting should not remember if the called tools will output Unicode or not, it will just expect to convert the output to UTF-8.

For simplicity we’ll assume that if the tool output is not-Unicode it will be considered as UTF-8 (no codepage conversion).

Solving

You can use chcp to change the active code page. This will be used for redirecting text as well:

chcp 65001

Keep in mind, though, that this will have no effect if cmd was started with the /u switch which forces Unicode (UTF-16 in this case) redirection output. If that switch is active then all output will be in UTF-16LE, regardless of the codepage set with chcp.

Also note that the console will be unusable for interactive output when set to Raster Fonts. I’m getting fun error messages in that case:

C:\Users\Johannes Rössel\Documents>x
Active code page: 65001

The system cannot write to the specified device.

The system cannot write to the specified device.
So either use a sane setup (TrueType font for the console) or don’t pull this stunt when using the console interactively and having a path that contains non-ASCII characters.

Code Page Supported

The following table lists each code page supported and its country/region or language:
Code page Country/region or language
437 United States
850 Multilingual (Latin I)
852 Slavic (Latin II)
855 Cyrillic (Russian)
857 Turkish
860 Portuguese
861 Icelandic
863 Canadian-French
865 Nordic
866 Russian
869 Modern Greek

Refer

https://technet.microsoft.com/en-us/library/bb490874.aspx
http://www.it1352.com/516804.html#