QUIC Server Guide

This guide covers setting up and configuring QUIC servers in Erlang applications.

Quick Start

%% Start the QUIC application
application:ensure_all_started(quic).

%% Start a server with TLS certificates
{ok, _Pid} = quic:start_server(my_server, 4433, #{
    cert => CertDer,
    key => PrivateKey,
    alpn => [<<"h3">>, <<"myproto">>]
}).

%% Get the listening port (useful when using port 0)
{ok, Port} = quic:get_server_port(my_server).

%% Get the full bound address (resolves inet6 / {ip, _} / {ifaddr, _})
{ok, {IP, Port}} = quic:get_server_sockname(my_server).

Server Configuration Options

Required Options

TLS Options

The signature scheme is derived from the server's key type by default; signature_algs only needs setting to restrict or reorder the advertised set. A server pinned to groups => [secp256r1] exercises the HelloRetryRequest path for x25519-only clients that also advertise secp256r1.

Connection Options

Socket Options

Pass inet6 (or an IPv6 {ip, Addr}) in extra_socket_opts to listen on IPv6. The address family is inferred from these options; the default is IPv4.

%% Listen on the IPv6 wildcard
quic:start_server(my_server, 4433, Opts#{extra_socket_opts => [inet6]}).

%% Bind to a specific IPv6 address
quic:start_server(my_server, 4433,
    Opts#{extra_socket_opts => [{ip, {16#2001, 16#db8, 0, 0, 0, 0, 0, 1}}]}).

Use quic:get_server_sockname/1 to read the resolved {IP, Port} the listener actually bound to, which is the reliable way to learn the address when it depends on inet6, {ip, _} or {ifaddr, _}.

Server Pool Options

A refused client gets CONNECTION_CLOSE in an Initial packet and stops at once instead of waiting for its handshake timeout (RFC 9000 Section 5.2.2). Each listener sends at most 100 of these per second and drops the rest without a reply.

Advanced Options

Observability

Server connections expose batching counters via quic_connection:get_stats/1:

• batch_flushes — number of times the per-connection batch was flushed
• packets_coalesced — packets that left the socket in a multi-packet flush

On socket_backend => socket + Linux with UDP_SEGMENT support, packets_coalesced / batch_flushes is the effective coalesce ratio (GSO super-datagrams). quic_socket:info/1 exposes the same counters plus backend, gso_supported, gso_size, gro_enabled, and max_batch_packets.

Crash reports

Crash reports and sys:get_status/1 of erlang_quic processes show no private key, PSK, traffic secret or session ticket. Status and the process state go through format_status/1. The stack trace of a crash can still carry the failing call's arguments, so when the quic application starts (or QUIC distribution starts before it), erlang_quic adds a primary logger filter, quic_redact. It drops the arguments of erlang_quic stack frames in crash, terminate and supervisor reports, and in reports from its own processes drops every frame's arguments and the secrets in the reason and mailbox.

The filter stays out of other log events. Removing it brings back the arguments, and with them the secrets they hold:

logger:remove_primary_filter(quic_redact).

Loading Certificates

From PEM Files

load_cert_and_key(CertFile, KeyFile) ->
    {ok, CertPem} = file:read_file(CertFile),
    {ok, KeyPem} = file:read_file(KeyFile),

    %% Decode certificate
    [{'Certificate', CertDer, _}] = public_key:pem_decode(CertPem),

    %% Decode private key
    KeyDer = case public_key:pem_decode(KeyPem) of
        [{'RSAPrivateKey', Der, not_encrypted}] ->
            public_key:der_decode('RSAPrivateKey', Der);
        [{'ECPrivateKey', Der, not_encrypted}] ->
            public_key:der_decode('ECPrivateKey', Der);
        [{'PrivateKeyInfo', Der, not_encrypted}] ->
            public_key:der_decode('PrivateKeyInfo', Der)
    end,

    {CertDer, KeyDer}.

Generating Test Certificates

# Generate self-signed certificate for testing
openssl req -x509 -newkey rsa:2048 \
    -keyout key.pem -out cert.pem \
    -days 365 -nodes \
    -subj '/CN=localhost'

What a certificate may be used for

A client that verifies your server checks what its certificate allows: an Extended Key Usage must list serverAuth, and a Key Usage must set digitalSignature. A certificate with neither extension is fine. When you issue one from your own CA, set both:

openssl x509 -req -in server.csr -CA ca.pem -CAkey ca.key \
    -out cert.pem -days 365 \
    -extfile <(printf 'extendedKeyUsage=serverAuth\nkeyUsage=digitalSignature\nsubjectAltName=DNS:localhost')

A server with require_client_cert (or verify => true) checks client certificates the same way, with clientAuth in place of serverAuth.

Connection Handling

Using connection_handler Callback

%% Define a connection handler
handle_connection(ConnPid, _DCID) ->
    %% The second argument is the destination connection id.
    {ok, Peer} = quic:peername(ConnPid),
    io:format("New connection from ~p~n", [Peer]),

    %% Spawn a process to handle this connection. It becomes the owner
    %% of the connection. Return {error, Reason} instead to decline the
    %% connection: the listener stops it and answers the client with
    %% CONNECTION_REFUSED.
    {ok, spawn(fun() -> connection_loop(ConnPid) end)}.

%% Start server with handler
quic:start_server(my_server, 4433, #{
    cert => Cert,
    key => Key,
    connection_handler => fun handle_connection/2
}).

Choosing the owner by ALPN

Use alpn_handler when one listener serves several protocols on one port. It is called in the connection process once the ClientHello has negotiated an ALPN, before any stream data can arrive, and the pid it returns becomes the connection's owner.

quic:start_server(my_server, 4433, #{
    cert => Cert,
    key => Key,
    alpn => [<<"h3">>, <<"myproto">>],
    alpn_handler => fun
        (Conn, <<"h3">>) ->
            quic_h3:accept(Conn, #{handler => fun my_http:handle/5, owner => Router});
        (Conn, <<"myproto">>) ->
            {ok, spawn(fun() -> myproto:loop(Conn) end)}
    end
}).

• Return {ok, Pid}. Anything else, or an exception, fails the handshake with handshake_failure.
• Return quickly and do not call the connection synchronously: the hook runs inside the handshake.
• A ClientHello with no ALPN in common is refused with no_application_protocol before the hook runs.
• For HTTP/3, quic_h3:accept/2 starts the HTTP/3 process; see HTTP3.md for the listener options it needs.

When a connection_handler already started the owner, use the 3-argument form: it also gets the current owner, the one connection_handler returned (or the listener without one), so you can tell it the ALPN and keep it.

alpn_handler => fun(Conn, Alpn, Owner) ->
    Owner ! {alpn, Conn, Alpn},
    {ok, Owner}
end

Manual Connection Handling

%% Get all active connections
{ok, Connections} = quic:get_server_connections(my_server).

%% Each connection is a pid that can be used with quic API
[ConnPid | _] = Connections,
{ok, StreamId} = quic:open_stream(ConnPid),
ok = quic:send_data(ConnPid, StreamId, <<"Hello">>, true).

Message Handling

The connection owner process receives these messages:

receive
    %% Connection established
    {quic, ConnRef, {connected, Info}} ->
        handle_connected(ConnRef, Info);

    %% New stream opened by peer
    {quic, ConnRef, {stream_opened, StreamId}} ->
        handle_stream_opened(ConnRef, StreamId);

    %% Data received on stream
    {quic, ConnRef, {stream_data, StreamId, Data, Fin}} ->
        handle_data(ConnRef, StreamId, Data, Fin);

    %% Stream reset by peer
    {quic, ConnRef, {stream_reset, StreamId, ErrorCode}} ->
        handle_reset(ConnRef, StreamId, ErrorCode);

    %% Datagram received (RFC 9221)
    {quic, ConnRef, {datagram, Data}} ->
        handle_datagram(ConnRef, Data);

    %% Connection closed
    {quic, ConnRef, {closed, Reason}} ->
        handle_closed(ConnRef, Reason)
end.

Server Pool for High Concurrency

%% Start a server pool with multiple listener processes
{ok, _} = quic:start_server(high_perf_server, 4433, #{
    cert => Cert,
    key => Key,
    pool_size => erlang:system_info(schedulers),  % One per scheduler
    alpn => [<<"h3">>]
}).

Load Balancer Integration (draft-ietf-quic-load-balancers-22)

%% Configure QUIC-LB for load balancer routing
LBConfig = #{
    server_id => <<1, 2, 3, 4>>,        % Unique server identifier
    algorithm => encrypted,              % plaintext | encrypted
    key => crypto:strong_rand_bytes(16), % Encryption key (not for plaintext)
    config_rotation => 0                 % Config rotation bits (0-6)
},

{ok, _} = quic:start_server(lb_server, 4433, #{
    cert => Cert,
    key => Key,
    lb_config => LBConfig
}).

Notes:

• plaintext puts the server ID in the CID as is. encrypted needs a 16-byte key and produces the CIDs a draft-22 load balancer decodes with the same key.
• server_id and nonce_len (default 4, at least 4) add up to 19 bytes or less. The CID is one byte longer than their sum.
• stream_cipher and block_cipher, from earlier drafts, are refused: the server does not start and returns {error, {invalid_lb_config, {unsupported_lb_algorithm, Algorithm}}}.
• quic_lb:decode_server_id/2 takes the same configuration and returns the server ID of a CID, for a load balancer written in Erlang.

Connection Migration (RFC 9000 Section 9)

The server automatically handles connection migration when clients change their network addresses (e.g., WiFi to cellular, NAT rebinding).

Key concept: The server address stays the same.

Migration means the client's address changed, not the server's. The server receives packets from a new source address but continues listening on the same port:

BEFORE: Client {192.168.1.10:54321} ───> Server {203.0.113.50:4433}

AFTER:  Client {10.0.0.5:62000} ───────> Server {203.0.113.50:4433}
               ▲                                (same server!)
               └── Client's address changed (NAT rebind or network switch)

How It Works

When the server receives a packet from an address different from the current remote_addr, it:

1. Detects the change type:

   • NAT rebinding: Same IP, different port (common with NAT timeouts)
   • Active migration: Different IP address (network change)

2. Validates the new path by sending PATH_CHALLENGE:

Server state machine:

  idle ──────────────────────────────────────────────────────┐
    │                                                        │
    │ packet from new address                                │
    ▼                                                        │
  validating_peer ───────────────────────────────────────────┤
    │                                                        │
    │ PATH_RESPONSE received          │ timeout (3*PTO)     │
    │ (matching challenge)            │ retry up to 3x      │
    ▼                                 ▼                      │
  complete_migration()            stay on current path ──────┘
    │
    │ - Reset congestion control
    │ - Reset loss detection
    │ - Reset PMTU to 1200
    │ - Switch to fresh CID
    ▼
  idle (new path active)

3. Completes migration if PATH_RESPONSE matches:
   • Updates remote_addr to the new address
   • Resets congestion control (new path may have different RTT/bandwidth)
   • Resets PMTU discovery (new path may have different MTU)
   • Switches to a fresh Connection ID (prevents path linkability)

Preferred Address

Servers can advertise a preferred address for clients to migrate to:

%% Server advertises preferred address in transport params
{ok, _} = quic:start_server(my_server, 4433, #{
    cert => Cert,
    key => Key,
    %% Client will validate and migrate to this address
    preferred_ipv4 => {{203, 0, 113, 10}, 4433},
    preferred_ipv6 => {{16#2001, 16#db8, 0, 0, 0, 0, 0, 1}, 4433}
}).

%% Client automatically validates and migrates to preferred address
%% after receiving server's transport parameters

Disabling Migration

To disable active migration (e.g., for load balancer compatibility):

%% Advertise disable_active_migration in transport params
{ok, _} = quic:start_server(my_server, 4433, #{
    cert => Cert,
    key => Key,
    disable_active_migration => true
}).

%% Clients will receive {error, migration_disabled} if they call migrate/1
%% Server will still handle NAT rebinding (port-only changes)

State Tracking

The server tracks migration state in the connection record:

Best Practices

1. Certificate Management

• Use proper CA-signed certificates in production
• Implement certificate rotation before expiry
• Store private keys securely (consider HSM for production)

2. Resource Limits

%% Set appropriate limits to prevent resource exhaustion
#{
    max_streams_bidi => 100,       % Limit concurrent streams
    max_streams_uni => 100,
    max_data => 10 * 1024 * 1024,                % 10 MB connection limit
    max_stream_data_bidi_local => 1024 * 1024,   % 1 MB per stream we open
    max_stream_data_bidi_remote => 1024 * 1024,  % 1 MB per peer-opened stream
    max_stream_data_uni => 1024 * 1024,          % 1 MB per uni stream
    idle_timeout => 30000           % Close idle connections
}

3. Connection Supervision

%% Embed server in your supervision tree
init([]) ->
    ServerSpec = quic:server_spec(my_server, 4433, #{
        cert => get_cert(),
        key => get_key(),
        alpn => [<<"myproto">>]
    }),

    {ok, {{one_for_one, 10, 60}, [ServerSpec]}}.

4. Graceful Shutdown

%% Stop server gracefully (allows draining)
ok = quic:stop_server(my_server).

%% Close individual connections
ok = quic:close(ConnRef, normal).

5. Monitoring

%% Get server information
{ok, Info} = quic:get_server_info(my_server).
%% Info = #{pid => Pid, port => Port, opts => Opts}

%% List all active servers
Servers = quic:which_servers().

%% Get connection statistics
{ok, Stats} = quic:get_stats(ConnRef).
%% Stats = #{packets_sent => N, packets_received => N, ...}

6. Enable QLOG for Debugging

%% Enable QLOG tracing for debugging
#{
    qlog => #{
        enabled => true,
        dir => "/var/log/quic/qlog",
        events => all  % or specific: [packet_sent, packet_received]
    }
}

Example: Echo Server

-module(echo_server).
-export([start/1, stop/0]).

start(Port) ->
    {ok, CertPem} = file:read_file("cert.pem"),
    {ok, KeyPem} = file:read_file("key.pem"),
    [{'Certificate', Cert, _}] = public_key:pem_decode(CertPem),
    [{'RSAPrivateKey', KeyDer, _}] = public_key:pem_decode(KeyPem),
    Key = public_key:der_decode('RSAPrivateKey', KeyDer),

    quic:start_server(echo, Port, #{
        cert => Cert,
        key => Key,
        alpn => [<<"echo">>],
        connection_handler => fun handle_connection/2
    }).

stop() ->
    quic:stop_server(echo).

handle_connection(ConnPid, _DCID) ->
    {ok, spawn(fun() -> echo_loop(ConnPid) end)}.

echo_loop(ConnPid) ->
    receive
        {quic, _, {stream_data, StreamId, Data, Fin}} ->
            %% Echo data back
            quic:send_data(ConnPid, StreamId, Data, Fin),
            echo_loop(ConnPid);
        {quic, _, {closed, _}} ->
            ok
    end.

Troubleshooting

Server Won't Start

1. Check certificate/key format (must be DER-encoded or properly decoded)

2. Verify port is available: netstat -an | grep <port>

3. Check for proper permissions on low ports (<1024)

Connections Dropping

1. Check idle_timeout setting
2. Enable keep-alive: keep_alive_interval => 15000
3. Review flow control limits

Performance Issues

1. Increase pool_size for high connection counts
2. Tune max_streams_* limits
3. Consider enabling BBR congestion control (if available)
4. Use QLOG to identify bottlenecks
5. Check UDP buffer sizes (see Performance Tuning below)

Performance Tuning

UDP Buffer Sizing

QUIC performance depends heavily on UDP socket buffer sizes. By default, erlang_quic requests 7MB buffers (matching quic-go, quiche, lsquic). Undersized buffers can reduce throughput by 40%+.

Check actual buffer sizes:

%% After starting a server
{ok, Socket} = gen_udp:open(0, [{recbuf, 7340032}, {sndbuf, 7340032}]),
{ok, Opts} = inet:getopts(Socket, [recbuf, sndbuf]),
io:format("Actual buffers: ~p~n", [Opts]).

Linux: Increase system limits

# Check current limits
sysctl net.core.rmem_max
sysctl net.core.wmem_max

# Increase to 7MB (requires root)
sudo sysctl -w net.core.rmem_max=7340032
sudo sysctl -w net.core.wmem_max=7340032

# Make persistent in /etc/sysctl.conf
echo "net.core.rmem_max=7340032" | sudo tee -a /etc/sysctl.conf
echo "net.core.wmem_max=7340032" | sudo tee -a /etc/sysctl.conf

macOS: System limits macOS typically caps UDP buffers at 2-4MB. While lower than Linux, this is still much better than the ~128KB defaults.

Custom buffer sizes:

%% Server with custom buffers
quic:start_server(my_server, 4433, #{
    cert => Cert,
    key => Key,
    recbuf => 4194304,  % 4MB
    sndbuf => 4194304
}).

%% Client with custom buffers
quic:connect("example.com", 4433, #{
    recbuf => 4194304,
    sndbuf => 4194304
}).

Benchmark buffer impact:

%% Compare different buffer sizes
quic_throughput_bench:compare_buffer_sizes().