UTF-8 support for JSON-RPC #2740

No description provided.

I'm all for supporting UTF-8, but raw utf8 is non-standard JSON and will likely break compatibility with some parsers. Escaping using \uNNNN is the norm. I'm not convinced we should enable it:

raw_utf8 = 0x02, // This prevents non-printable characters from being escapted using "\uNNNN" notation.
                 // Note, this is an extension to the JSON standard. It disables the escaping of
                 // non-printable characters allowing UTF-8 sequences held in 8 bit char strings
                 // to pass through unaltered.

@laanwj How is it non-standard? The specification seems to say raw UTF-8 is standard and parsers must handle it...

JSON Spirit without raw_utf8 escapes every byte as UTF-8 character. For example, U+0432 will be escaped as \u00D0\u00B2. raw_utf8 is solution.

Well the help text said it is non-standard, and I was going on that, but according to http://www.ietf.org/rfc/rfc4627.txt?number=4627

All Unicode characters may be placed within the
quotation marks except for the characters that must be escaped:
quotation mark, reverse solidus, and the control characters (U+0000
through U+001F).

So, as long as it still escapes those characters it is OK.

@jgarzik commented on 370df18 "now builds on unix" tells us nothing about the commit: why it was needed -- because it builds on "unix" here, what needed changing etc. Suggested commit message:

makefile.unix: link with boost_chrono to fix build on [your platform]

Sorry, it was my own issue. Solved by upgrading boost libs from v1.52 to v1.53.

Changelog for 4.04 to 4.06

Version 4.04, 8 January 2011 Added the raw_utf8 feature to the write functions Added a << std::dec before writing to a stream to ensure that integer variables appear in base 10 notation Added additional .h files to CMakeLists.txt Output doubles using precision of 17 instead of 16 Added option to remove trailing zeros when outputting doubles Ensures that write functions return the state of a given IO stream to its original state on completion

Version 4.05, 12 September 2011 Added a new output option for single line arrays Fixed non-standard zero digits following decimal point with "remove trailing zeros" output option Reduced build times with new JSON_SPIRIT_VALUE_ENABLED type #defines Added a constructor for variant types Added a constructor for container iterators

Version 4.06, 14 May 2013 Added support for Javascript type comments, i.e. // and /* */ Added json_spirit_writer_options.h to CMake install Improved error message on attempt to extract the wrong type of data from a value Correct bug that reduced write performance Added always_escape_nonascii writer flag

http://www.codeproject.com/Articles/20027/JSON-Spirit-A-C-JSON-Parser-Generator-Implemented#hist

Any reason not to update?

I'd like to see json_spirit update separate from any UTF8 fixes. i.e. two commits: update json spirit, then switch raw_utf8

@jgarzik Seems like a good thing to do.

Also I hope the new json_spirit version fixes a compiler warning I get:

<pre> \src\json\json_spirit_writer_template.h:31: Warnung:typedef 'Char_type' locally defined but not used [-Wunused-local-typedefs] typedef typename String_type::value_type Char_type; ^ </pre>

@jgarzik Done. @Diapolo Can't reproduce this warning on master branch. gcc-4.7.3, GNU/Linux.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/2ecb7555a9df1e843fd25f588819e4ca1d94b266 for binaries and test log. This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/ Contact BlueMatt on freenode if something looks broken.

@constantined I also did not have that error with MinGW < 4.8.1, but perhaps latest json_spirit fixes this anyway :).

Looks pretty good. Made one inline comment.

Do Qt guys agree that RPC console should pretty print?

Did a quick web search, but perhaps you can tell me what pretty-print means for RPC console or perhaps give a formated example?

I tried this out and didn't see any change to the JSON as printed in the console before/after this patch. So it's ACK by me.

If this happens, please be sure that #2980 is fixed.

Is there anything blocking this?

Works for me.

Had to revert this -- it broke JSON-RPC values. See #3126