qt: prevent console hang on large RPC responses #942

Problem

Fixes #932.

Running getrawtransaction on a very large non-standard transaction in the bitcoin-qt console causes the GUI to become completely unresponsive.

Reproduction:

getrawtransaction 30794005e228089a56cca58b6ad85e8cc0667cdc06b0ee6f9fe72aa19f50acd2

The transaction is ~3.98 MB and sits in block 938523. Entering the command above leaves the node window frozen until the process is killed. Reported in #932.

Root cause

The hang is not in the node or the RPC execution path. It occurs in the Qt UI thread after the result is delivered to the console widget.

Two operations in RPCConsole::message() scale badly with payload size:

1. GUIUtil::HtmlEscape() processes the result string character-by-character. A 3.98 MB transaction encodes to ~7.96 MB of hex, all of which this function walks before returning.

2. QTextEdit::append() is then called with an HTML string of the same magnitude. Qt re-lays-out the entire text document on each call; for a payload this size the layout pass does not complete in any practical amount of time, stalling the event loop indefinitely.

Fix

Add a 1 MiB size guard in RPCExecutor::request(), immediately after RPCExecuteCommandLine() returns and before QString::fromStdString() is called. When the result exceeds the limit the widget receives a short notice instead of the raw output:

Response too large to display (7962624 bytes). Use bitcoin-cli to retrieve the full result.

Because the guard fires before QString conversion, the oversized string is never converted to UTF-16, never HTML-escaped, and never handed to the text widget.

The 1 MiB threshold:

• reuses the _MiB literal already present in this file (ui->lineEdit->setMaxLength(16_MiB))
• is well above any realistic interactive debugging output
• is well below the ~7.96 MiB that triggers the hang

Tests

A mock RPC command rpcLargeOutput is registered in src/qt/test/rpcnestedtests.cpp. It returns a string of 1_MiB + 1 bytes. The test asserts that RPCExecuteCommandLine returns the complete result unmodified, confirming the size guard in RPCExecutor::request() does not affect the underlying RPC call, only what is rendered in the console widget.

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--021abf342d371248e50ceaed478a90ca-->

Reviews

See the guideline for information on the review process. A summary of reviews will appear here.

<!--5faf32d7da4f0f540f40219e4f7537a3-->

Hi @hebasto @fanquake , gentle ping on this one!

This PR fixes a hard hang in the bitcoin-qt console when running RPC commands that return large payloads (e.g. getrawtransaction on a ~3.98 MB transaction).

Root cause: two operations in RPCConsole::message() scale badly with payload size:

• GUIUtil::HtmlEscape() walks the entire result string character-by-character
• QTextEdit::append() re-lays-out the full text document on each call, for a ~7.96 MB hex payload this stalls the Qt event loop indefinitely

Fix: a 1 MiB size guard in RPCExecutor::request() fires before QString::fromStdString() is called, so the oversized string is never converted, never escaped, and never handed to the widget. The user sees a short notice instead:

Response too large to display (7962624 bytes). Use bitcoin-cli to retrieve the full result.

Test coverage: a mock RPC command rpcLargeOutput is registered in src/qt/test/rpcnestedtests.cpp, returning 1_MiB + 1 bytes, asserting the guard does not affect the underlying RPC result, only the console rendering.

No behaviour change for normal-sized responses. Happy to adjust the threshold or message wording if needed.

Fixes #932.