INSERT payloads, execution telemetry, and error signals. It is the protocol behind the command-line client and the C++ and most third-party native drivers.
This page covers the protocol itself: packet framing, the connection state machine, version negotiation, and the body of every non-Block message. The bytes inside Data-family packets (the Block, its columns, and the per-type encodings) are a separate concern, documented in the Native Format specification.
Companion specificationThis page is one half of a pair and is published together with the companion Native Format specification. The two specs split the work cleanly: this page owns the packet and transport layer; the Native Format spec owns the bytes inside
Data-family packets.BlockInfo, so a single misplaced byte desynchronizes everything that follows. It is stateful, and each TCP connection processes one query at a time — there is no multiplexing. Fixed-width integers are little-endian.
Overview
Every message on the wire starts with a
VarUInt packet type code, followed by a body whose shape depends on that code and on the negotiated protocol version.
A connection runs through three phases — a one-time handshake, then any number of Ping or Query exchanges, then close:
The native TCP protocol always carries tabular data in the Native format, regardless of any FORMAT clause in the SQL. Re-formatting into RowBinary, CSV, JSON, and so on is the client’s job, done after it decodes the Native blocks. (The HTTP interface is a different code path that does honour the FORMAT clause; HTTP is out of scope here.)
Security
Transport security (TLS)
TLS lives at the transport layer, below the protocol. When it is enabled the entire TCP stream is encrypted, and the protocol messages are byte-for-byte identical whether TLS is in use or not.Authentication
Authentication happens during the handshake, in theClientHello message. The user and password fields travel as plaintext strings, so transport-level encryption (TLS) is what protects credentials in transit.
SSH challenge-response authentication is available from protocol version 54466 onward — see SSH challenge-response authentication.
Inter-server secret
For distributed query execution, servers authenticate to one another by proving knowledge of a shared secret — without putting the secret on the wire. Each Query carries a 32-byte SHA-256auth_hash in Query field 4, computed over a salt, nonce, the configured secret, and the query, which the receiving server recomputes and compares. This is gated by the INTERSERVER_SECRET feature (v54441). External clients always send an empty string here. See Inter-server authentication.
Versioning and feature gates
Version negotiation
Both client and server declare their maximum supported protocol version during the handshake. The negotiated version is the smaller of the two:Feature gates
A feature is identified by the protocol version that introduced it, and is active when the negotiated version is greater than or equal to that number.Feature table
Packet envelope
Every message on the wire shares the same outer structure, in both directions:VarUInt, not a fixed-width byte. For values below 128 a VarUInt produces the same single byte, but implementations must use VarUInt encoding so they stay compatible should future packet types reach 128 or above.
The message reference documents only the body of each packet — the bytes after the packet type code. Field numbering starts at 1 with the first body field.
Chunked framing (v54470+)
When theCHUNKED_PROTOCOL feature is negotiated (see the handshake), every packet on the wire is wrapped in chunked framing. The wrapping is per-direction: client→server and server→client are negotiated separately and may end up in different modes (chunked versus unframed).
Wire layout per packet:
VarUInt is inside the chunked stream: it is the first byte of the packet payload (the first byte of the first chunk), not a separate byte sent ahead of the framing. Each packet’s chunk payload is the full [VarUInt packet_type_code][message body] from the packet envelope. A client that leaves the packet type outside the chunked stream makes the peer read that type byte as the first byte of the u32 chunk size, desynchronizing the connection.
A single packet may be split across several chunks if the writer’s buffer fills mid-packet; a split can fall anywhere, including inside the packet type’s VarUInt. The reader concatenates chunk payloads and treats the trailing 4-byte zero as a transparent packet boundary — it consumes it but does not surface it to whatever is reading packet bodies.
No-body packets are still wrapped: a single-byte packet such as Ping or Pong becomes [u32 size = 1][0x04][u32 0] once chunking is negotiated. Any “single byte on the wire” description elsewhere on this page is the pre-chunking form.
Negotiation. ServerHello and Addendum each carry two String fields, one per direction, with values drawn from {"chunked", "notchunked", "chunked_optional", "notchunked_optional"}:
chunked/notchunkedare strict: that side requires exactly that mode.- The
_optionalvariants are flexible: they accept whichever mode the other side picks.
On the client side, the client’s SEND preference negotiates against the server’s RECV preference, and vice versa.
Timing. The negotiation strings travel on the unframed wire: ClientHello → ServerHello (server prefs) → Addendum (client’s negotiated values). The framing flip applies to every byte sent after the Addendum is flushed. The Addendum itself, the ClientHello, and the ServerHello are always unframed.
Connection lifecycle
At any moment a connection is in exactly one of four states:HANDSHAKE, READY, READING_RESPONSE, or terminated. Because the protocol does not multiplex, a client that sends a new request before draining the previous response interleaves bytes on the wire and corrupts the stream.
States
The happy path runs straight down —HANDSHAKE → READY → READING_RESPONSE → READY — with the Ping/Pong self-loop and every failure edge funnelling into the single Terminated sink.
Handshake phase
Authenticates and negotiates the protocol version. Happens exactly once per connection, before anything else. The TCP connection has just opened and no messages have been exchanged. The flow:-
The client sends
ClientHellowith its maximum supported protocol version. -
The client reads the response and dispatches by packet type:
-
If
negotiated_version ≥ 54458(theADDENDUMfeature), the client sends anAddendum. This decision is based on the negotiated version, not the client’s declared version.
READY; on any error it terminates.
Ping phase
An application-level liveness check, independent of TCP keepalive. A successful Ping/Pong round-trip confirms the TCP connection is alive in both directions and the server is responsive. Ping is stateless and uncorrelated with any query, so multiple sequential Pings are independent. Starting fromREADY, the flow is:
-
The client sends
Ping. -
The client reads the response:
Query phase
The client submits a SQL statement; the server streams back result blocks and execution telemetry. The response is a sequence of packets terminated by exactly oneEndOfStream or Exception.
Starting from READY, the flow is:
On error at any point the server sends an Exception instead of EndOfStream, which terminates the query.
-
The client sends
Querywith a uniquequery_id(typically a UUID). -
The client sends any external tables, then the empty Data marker. The empty Data packet has
table_name = "",num_columns = 0,num_rows = 0. The server does not begin executing the query until it receives this marker. -
The client moves to
READING_RESPONSEand flushes its write buffer. -
The client reads response packets in a loop, dispatching by type:
EndOfStream or a handled Exception the connection returns to READY. A protocol violation or I/O error terminates it.
The
num_rows == 0 case trips up new implementations. A zero-row block is a boundary marker or schema header, not an end-of-stream signal. Only EndOfStream or Exception ends the response.INSERT phase
The INSERT phase is the Query phase with two extra exchanges. The client submits anINSERT statement; the server replies with a schema block describing the target table; the client streams Data packets with the rows, then the empty Data marker; the server finishes with EndOfStream or Exception.
Starting from READY, the SQL is an INSERT of the form INSERT INTO <table> [(<cols>)] VALUES — with no inline VALUES (...) literal, since the row data flows through Data packets. The flow:
- The client sends
Querywithbodyset to the INSERT SQL. - The client sends any external tables (rare for INSERT). Unlike the Query phase, it does not send an empty Data marker here. The
INSERTQuerypacket is sent with pending data, so the empty end-of-data block is deferred to step 5; sending it before the schema block would make the server read it as the end of the row stream, finish the INSERT with no rows, and then parse the first real row packet as a stray top-level packet. - The client drains metadata packets (TableColumns, Progress, ProfileInfo, Log, ProfileEvents) until it reads the schema Data packet — a Block with 0 rows but full column structure (names and types). The schema block is the contract: the rows the client sends next must match these column shapes.
- The client sends data block(s). For each block it writes
VarUInt(ClientPacket::Data = 2), thenString("")for the empty external-table name, then the Block. Column types must align with the schema block’s columns by position. - The client sends the end-of-input terminator: a Data packet with an empty Block (0 columns, 0 rows).
- The client drains the response stream until
EndOfStream(success) orException(failure).
async_insert = 1, the server queues the rows and flushes them as part of a batch. At negotiated version ≥ 54484 (PROGRESS_IN_ASYNC_INSERT), once the flush completes the server emits an extra Progress packet, immediately followed by the insert’s ProfileEvents, then EndOfStream. Below 54484 the server skips that trailing Progress. The packet is an ordinary Progress; because the server resets the query pipeline before folding in the write counts, the increment in practice carries only the elapsed time, and the written-row and byte stats reach the client via the accompanying ProfileEvents. A client that already drains interleaved Progress in step 6 needs only to accept one more packet.
The connection returns to READY on EndOfStream or a handled Exception. Protocol violations and I/O errors terminate it.
Message reference
Fields are listed in wire order. TheType column uses:
VarUInt— variable-length unsigned integer (see VarUInt).String— VarUInt-prefixed bytes (see String).UInt8,Int32, and so on — fixed-width little-endian integers.Bool— a single byte,0x00or0x01.
Role column says who uses each field:
- client — set by external clients.
- inter-server — meaningful only for server-to-server communication; external clients write a default value.
- universal — used by both.
ClientHello (packet type 0)
Client → Server. The first message after the TCP connection opens.ServerHello (packet type 0)
Server → Client. The reply to ClientHello on successful authentication.
Rule — an element of
password_complexity_rules:
The list reflects the server operator’s password-policy configuration and is purely advisory — the server does not enforce these rules during the handshake. A client that exposes password change/set functionality may use the rules to flag errors before round-tripping a non-compliant password to the server.
To bound resource use against a hostile or misconfigured server, cap the decoded
count at 256 entries and each pattern and message String at 4096 bytes. A count of 0 (no following pairs) is the common case for servers with no password policy configured.Addendum (no packet type)
Client → Server, gated byADDENDUM (v54458). Sent immediately after the handshake exchange completes. It is not a distinct packet type — the fields go on the wire raw, with no packet type byte prefix.
The chunked-framing flip applies after this Addendum is flushed — the Addendum itself is unframed.
Ping (packet type 4)
Client → Server. No body — the packet is a single byte0x04 before chunked framing; when chunking is negotiated the byte becomes the one-byte payload of a chunk (see chunked framing).
Pong (packet type 4)
Server → Client. No body — the packet is a single byte0x04 before chunked framing; when chunking is negotiated the byte becomes the one-byte payload of a chunk (see chunked framing).
Exception (packet type 2)
Server → Client. Sent when the server hits an error during any phase.Query (packet type 1)
Client → Server.ClientInfo (embedded in Query)
Client → Server, embedded in the Query body (field 2). Gated byCLIENT_INFO (v54032). (Some fields inside ClientInfo are gated by later versions, as noted per-field below.)
Interface-dependent layout (fields 7–12)Fields 7–12 above are the TCP branch. When
query_interface (field 6) is not TCP, these fields are replaced by a different wire layout — they are not merely optional omissions, so a decoder must branch on field 6.query_interface = 2(HTTP): the server-forwarded HTTP request info is written instead —http_method(UInt8),http_user_agent(String), thenforwarded_for(String, gated byX_FORWARDED_FOR_IN_CLIENT_INFOv54443) andhttp_referer(String, gated byREFERER_IN_CLIENT_INFOv54447). Noos_user/client_hostname/client_name/version_*/protocol_versionfields are present.- Any other interface: none of the TCP fields (7–12) and none of the HTTP fields are written; the stream continues directly with
quota_key.
quota_key (field 13) and distributed_depth (field 14) follow for all interfaces, then version_patch (field 15) is written only for TCP.This branch matters mainly for inter-server traffic, where the initiating server forwards a query that originally arrived over HTTP. A decoder that always reads the TCP fields will misread such packets — treating http_method or http_user_agent as quota_key.Inter-server authentication
The Query field 4 (auth_hash) is not the shared cluster secret on the wire. Sending the raw secret would both fail authentication and leak it. Instead, a server acting as an inter-server client proves it knows the secret with a salted SHA-256 hash:
- Enter inter-server mode. The connecting server signals it inside
ClientHello: theuserfield is the inter-server marker andpasswordis empty. It then appends two more strings — the cluster name and a freshly-generated 32-bytesalt(encodeSHA256of a random value) — immediately after theuser/passwordfields, as part of the sameClientHellopacket. The server reads these two strings before it sendsServerHello, so a client must write them up front; waiting forServerHellofirst deadlocks, because the server is blocked reading them. - Obtain the nonce.
ServerHellocarries an 8-byteUInt64nonce whenINTERSERVER_SECRET_V2(v54462) is negotiated. - Compute the hash. For every non-
InitialQueryQuery packet, the client writesencodeSHA256(salt + nonce + cluster_secret + query + query_id + initial_user + external_roles)into field 4 — a 32-byte digest. (nonceis its decimal string form, present only when negotiated ≥ v54462;external_rolesis appended only whenINTERSERVER_EXTERNALLY_GRANTED_ROLES(v54472) is negotiated.) For anInitialQuery, or when no cluster secret is configured, the client writes an empty string instead. - Verify. The server reads field 4 with a 32-byte cap and recomputes the same concatenation using its own copy of the cluster secret; the connection is rejected if the digests differ.
auth_hash.
Setting
Encoded inline in the Query body’s settings list (the Query packet, field 3). The list is always present, regardless of negotiated version, and is terminated by a Setting with an empty key — a singleVarUInt 0, with no flags or value following. Only the per-setting encoding depends on the negotiated version, gated by SETTINGS_SERIALIZED_AS_STRINGS (v54429).
v54429+ (STRINGS_WITH_FLAGS) — each setting is the triple shown here:
Fields 2 and 3 are absent when
key is empty.
Pre-54429 (BINARY) — each setting is [String key][type-specific binary value]: the flags field is not written, and the value is encoded in the setting’s native binary form (for example a fixed-width integer or a length-prefixed string) rather than as a decimal/text string. The list is still terminated by an empty key. A client that targets a negotiated version below 54429 must read and write this binary form, not the triple above. (Custom user-defined settings are the exception: they always carry flags and a string value, in both encodings.)
The flags field packs:
0x01— Important: the setting affects query results and must not be silently ignored by older peers.0x02— Custom: a user-defined custom setting.0x0c— a 2-bit tier field, not an independent flag:0x00= Production,0x04= Obsolete,0x08= Experimental,0x0c= Beta. Read all 2 bits (flags & 0x0c) — a naiveflags & 0x04test would misclassify Beta (0x0c) as Obsolete.0x80— HotReload (config reload without restart; defined in the flags enum, encountered mainly for coordination settings).
Parameter
Query parameters, for parameterized queries likeSELECT {x:UInt64}. Encoded identically to a Setting with the Custom flag (0x02) set, and terminated by an empty key in the same way.
The parameter value is the SQL representation of the value, not a raw literal. String-typed parameters must be passed already single-quoted (for example, the value for
{name:String} is 'Alice', not Alice); otherwise the server’s value parser rejects them.Data (packet type 1 server→client, packet type 2 client→server)
Both directions. Carries result blocks, INSERT data, external tables, and end-of-data markers. The wire format is symmetric — both directions include atable_name prefix before the Block. Only the packet type byte differs.
The end-of-data marker is a packet whose Block is empty —
0 columns and 0 rows — regardless of table_name. The server treats a client Data packet as the terminator only when the decoded block is empty (block.empty()); a packet with table_name = "" and a non-empty block is an ordinary row packet, not a terminator. So an INSERT row stream is a sequence of non-empty Data blocks followed by one empty Data block that ends it.
The block variants and what they mean are documented under Block variants.
Progress (packet type 3)
Server → Client. Sent periodically during query execution. All fields are VarUInt, and each packet carries increments since the previousProgress packet, not cumulative totals. Before sending, the server reads its counters and atomically resets them to zero, and computes elapsed_ns as the time delta since the last send. A client therefore must accumulate successive packets locally to obtain running totals — treating a packet as an absolute value makes the progress display jump backwards or undercount once more than one packet arrives.
ProfileInfo (packet type 6)
Server → Client. Sent once per query, near the end of execution.Totals (packet type 7)
Server → Client. Sent for queries withWITH TOTALS. Wire format is identical to Data: a table_name string (always empty) followed by a Block. Only the packet type byte differs.
Extremes (packet type 8)
Server → Client. Sent when theextremes setting is enabled. Wire format is identical to Data. The block has exactly 2 rows: row 0 holds the minimum of each column, row 1 holds the maximum.
Log (packet type 10)
Server → Client. Sent when the query has an active logs queue (thesend_logs_level setting; see log streaming).
Same envelope and body format as Data. The block has a fixed num_columns = 8 and a predefined schema. Each log line is one row across all 8 columns, and a single Log packet may carry many rows.
ProfileEvents (packet type 14)
Server → Client. Carries per-query performance counters. Same envelope and body format as Data. The block has a fixednum_columns = 6 and a predefined schema. Each event is one row.
The
value column’s element type is not fixed across packets — older servers emit UInt64, newer ones Int64. Read the column’s type string from the block header rather than assuming one width.TableColumns (packet type 11)
Server → Client, gated byCOLUMN_DEFAULTS_METADATA (v54410). The server sends it before the INSERT schema block to carry column-default metadata, but only when the negotiated version is ≥ 54410 and the input_format_defaults_for_omitted_fields setting is enabled. Below 54410 the packet is never sent, so an older client must not wait for it — the schema Data block comes directly. A v54410+ client should be ready for either order: an optional TableColumns, then the schema block.
Compressed body at v54481+At negotiated version ≥ 54481 (
COMPRESSED_LOGS_PROFILE_EVENTS_COLUMNS) the server writes both fields through the same optionally-compressed output path, so when the query has compression = true the whole TableColumns body (external_table + columns_description) is inside the compression frame; the client reads it through the matching decompressed stream. When the query has no compression, the body is on the wire uncompressed exactly as the table above shows. This matters for INSERT schema responses: a client that switches compression handling for Log and ProfileEvents but not TableColumns will misread the response when query compression is enabled.TimezoneUpdate (packet type 17)
Server → Client, gated byTIMEZONE_UPDATES (v54464). Sent in exactly one place: the initializer for the input table function (a query of the form INSERT INTO <table> SELECT ... FROM input('<structure>'), which streams rows from the client). Right after the server sends the input-schema Data block (see the INSERT phase), it emits TimezoneUpdate carrying the query context’s current session_timezone so the client parses the rows it is about to send with the same timezone. The server does not emit this packet for arbitrary mid-query SET session_timezone changes, nor to tell the client how to format later result blocks.
The packet arrives once, immediately after the input-schema block and before the client starts sending row blocks. A decoder that ignores
TimezoneUpdate MUST still consume the trailing String to keep the wire aligned.
SSH challenge-response authentication (packet types 11, 12, 18)
Gated bySSH_AUTHENTICATION (v54466), and opt-in only. A connection enters the SSH flow when ClientHello sends user = " SSH KEY AUTHENTICATION " + <real_user> (with the leading and trailing spaces) and password = "". The server reads the prefix, strips it to recover the real user, and switches to challenge-response.
The flow runs in place of password authentication, and the challenge-response exchange happens before ServerHello — the server defers its Hello reply until authentication succeeds:
- The client sends ClientHello with the SSH marker prefix and an empty password.
-
The client sends
SSHChallengeRequest(packet 11). The server has not sent ServerHello yet — it processes authentication first and blocks here waiting for this packet. -
The server replies with
SSHChallengecarrying random bytes (packet 18). -
The client builds the string to sign and signs that, not the raw challenge, then sends
SSHChallengeResponse(packet 12) with the signature. The signed message is the byte-wise concatenation, with no separators, of four parts in this exact order: -
The server verifies the signature against the user’s registered public key, reconstructing the same
decimal(protocol_version) + default_database + user + challengestring. On success it sendsServerHello— the same reply as in the password flow — and the handshake continues normally (Addendum, etc.); on failure it returns anExceptionand terminates the connection. A client that signs only the raw challenge bytes will fail authentication.
This is the reverse of the password handshake, where ServerHello immediately follows ClientHello. Under SSH auth, ServerHello is withheld until after the signature is verified, so the SSH challenge-response is interleaved into the handshake before any ServerHello is seen.
Packet type reference
Client → Server
Server → Client
Configuration
This section covers the tunables that shape native protocol connections:- Transport-layer settings — TCP socket options and timeouts, which affect how the TCP connection itself behaves.
- Application-layer settings — per-query tunables carried in the Query packet’s settings list, which affect what the server sends on the wire or how it is framed.
- Settings out of scope — settings often confused with protocol settings but which actually control SQL execution or storage.
Transport-layer settings
Socket options
Timeouts
The timeouts nest like this:
Connection limits
A connection that issues queries regularly can live indefinitely. Only idle connections are reaped after an hour, and there is no default maximum lifetime.
Application-layer settings
These settings travel per-query in the Query packet’s settings list. They change what the server sends on the wire, or how it is framed.Compression
The
compression flag in the Query packet (field 6) toggles compression on and off; these settings select which codec is used when it is on.
Log streaming
Setting
send_logs_level to anything other than "none" makes the server emit Log packets during query execution.
Progress reporting
This is a target minimum, not a strict maximum: the server may send Progress packets less often when the query is not producing work fast enough.
Result envelope
Async INSERT
Distributed tracing
Settings out of scope
These settings are sometimes mistaken for protocol-level settings, but they control SQL execution, storage, or CPU use rather than wire behaviour. A protocol implementation does not need to handle them specially.max_threads— parallelism within query execution.max_memory_usage— per-query memory cap.max_block_size,preferred_block_size_bytes— internal block sizing during query processing; wire blocks are independent of these.compile_expressions— JIT compilation; CPU only.async_insert_max_data_size— server-side queue buffer.- All
input_format_*andoutput_format_*settings except theinput_format_native_*/output_format_native_*family — the non-nativeones select or tune other formats (for example over HTTP) and do not change the native protocol’sDatablocks.
*_native_* settings are the exception: they change the bytes inside native TCP Data blocks, so a protocol implementation must account for them. output_format_native_encode_types_in_binary_format switches the column type field from a textual string to a binary type encoding, output_format_native_write_json_as_string emits JSON columns as a String, and output_format_native_use_flattened_dynamic_and_json_serialization selects the FLATTENED Dynamic/JSON layout. Because these affect the block body rather than the packet envelope, they are specified in the Native Format spec — see column wire layout and versioned types.
Glossary
Cancel — a client-initiated packet (type 3) that aborts a running query. Not specified in detail on this page. End-of-client-data marker — an empty Data packet (0 columns, 0 rows) the client sends to close an input stream. Its position differs by query kind:- Normal query (
SELECT, etc.): sent after the Query packet and any external-table Data packets to signal “no more external data”. The server then begins executing. INSERT: the client does not send a pre-schema marker. The server sends the schema block first, the client streams its row Data blocks, and only then sends the empty Data packet to terminate the row stream. Sending an empty marker before the schema block would be read as an immediate end-of-rows and lose the data.
min(client_version, server_version), computed during the handshake. Determines which features are active for the lifetime of the connection.
Packet — a wire message: a VarUInt packet type code followed by a body whose format depends on the type. See packet envelope.
Packet type code — the leading VarUInt of a packet that identifies its format. Values 0–18 are currently assigned. See the packet type reference.
Response stream — the sequence of packets the server emits during a query. Open-ended in length, terminated by exactly one EndOfStream (success) or Exception (failure). See the query phase.
Schema block — the header block (a Block with columns but 0 rows) that the server sends during the INSERT phase to announce the expected column shapes before the client sends data.
Settings list — a sequence of (key, flags, value) tuples in the Query body, terminated by an empty key. Carries per-query application-layer configuration. See Setting.
Stage — a VarUInt field in the Query packet (field 5) controlling how far the server executes the query. External clients typically send 2 (Complete); distributed queries and serialized query plans use the higher values. See Query field 5 for the full set of wire values.
Terminator — a packet that ends a stream. The Query response ends on EndOfStream (success) or Exception (failure). The client’s input stream ends on the empty Data marker.