Example¶

If certificates have been set up correctly then you should be able to start a TLS server by doing:

 nbdkit --tls=require memory 1G

and connect to it by doing:

 nbdinfo nbds://localhost

If certificates are in a non-standard directory and you have libnbd ≥ 1.10:

 nbdkit --tls=require --tls-certificates=/certs memory 1G
 nbdinfo nbds://localhost?tls-certificates=/certs

TLS with X.509 certificates¶

When nbdkit starts up, it loads TLS certificates from some built-in paths, or from the directory specified by the --tls-certificates option.

In this directory, nbdkit expects to find several files:

ca-cert.pem

The Certificate Authority certificate.

server-cert.pem

The server certificate.

server-key.pem

The server private key.

ca-crl.pem

(Optional) The certificate revocation list.

Setting up the Certificate Authority

This step only needs to be done once per organization. It may be that your organization already has a CA.

 $ certtool --generate-privkey > ca-key.pem
 $ chmod 0600 ca-key.pem

The ca-key.pem file is the CA private key and is extremely sensitive data. With possession of this key, anyone can create certificates pretending to be your organization!

To create the CA certificate file:

 $ cat > ca.info <<EOF
 cn = Name of your organization
 ca
 cert_signing_key
 EOF
 $ certtool --generate-self-signed \
            --load-privkey ca-key.pem \
            --template ca.info \
            --outfile ca-cert.pem

Issuing a server certificate for the nbdkit server

Each nbdkit server (or host) needs a secret key and certificate.

 $ certtool --generate-privkey > server-key.pem
 $ chmod 0600 server-key.pem

The server key file is sensitive. Setting the mode to 0600 helps to prevent other users on the same machine from reading it.

The common name ("cn" below) field must be the fully qualified hostname that the client connects to. However most clients and servers including nbdkit support the Subject Alternative Name extension (RFC 2818) which uses the "dns_name" and "ip_address" fields and deprecates "cn".

 $ cat > server.info <<EOF
 organization = Name of your organization
 cn = nbd-server.example.com
 dns_name = nbd-server
 dns_name = nbd-server.example.com
 ip_address = 10.0.1.2
 ip_address = 2001:24::92
 tls_www_server
 encryption_key
 signing_key
 EOF
 $ certtool --generate-certificate \
            --load-ca-certificate ca-cert.pem \
            --load-ca-privkey ca-key.pem \
            --load-privkey server-key.pem \
            --template server.info \
            --outfile server-cert.pem

Issuing and checking client certificates

Note: You don't need to create client certificates unless you want to check and limit which clients can connect to nbdkit. nbdkit does not check client certificates unless you specify the --tls-verify-peer option on the command line. There are other methods for limiting access to nbdkit including nbdkit-ip-filter(1).

For each client you should generate a private key and a client certificate:

 $ certtool --generate-privkey > client-key.pem
 $ chmod 0600 client-key.pem

The client key file is sensitive.

The client DNS name ("cn" below) is the client's name for information only.

 $ cat > client.info <<EOF
 country = US
 state = New York
 locality = New York
 organization = Name of your organization
 cn = client.example.com
 tls_www_client
 encryption_key
 signing_key
 EOF
 $ certtool --generate-certificate \
            --load-ca-certificate ca-cert.pem \
            --load-ca-privkey ca-key.pem \
            --load-privkey client-key.pem \
            --template client.info \
            --outfile client-cert.pem

Client certificates do not need to be present anywhere on the nbdkit host. You don't need to copy them into nbdkit's TLS certificates directory. The security comes from the fact that the client must present a client certificate signed by the Certificate Authority, and nbdkit can check this because it has the ca-cert.pem file.

To enable checking of client certificates, specify the --tls-verify-peer option on the command line. Clients which don't present a valid certificate (eg. not signed, incorrect signature) are denied. Also denied are clients which present a valid certificate signed by another CA. Also denied are clients with certificates added to the certificate revocation list (ca-crl.pem).

Connecting nbd-client to nbdkit with TLS certificates¶

With the TLS certificates files generated above in the current directory (".") you can use:

 nbdkit --tls=require --tls-certificates=. --tls-verify-peer memory 1G
 nbd-client /dev/nbd0 \
           -cacertfile ca-cert.pem \
           -certfile client-cert.pem \
           -keyfile client-key.pem

--tls-verify-peer is only required if you want to check the client certificate. If you want to allow any client to connect then you can omit it.

TLS with Pre-Shared Keys (PSK)¶

As a simpler alternative to TLS certificates, you may use pre-shared keys to authenticate clients.

Create a PSK file containing one or more "username:key" pairs. It is easiest to use psktool(1) for this:

 mkdir -m 0700 /tmp/keys
 psktool -u alice -p /tmp/keys/keys.psk

The PSK file contains the hex-encoded random keys in plaintext. Any client which can read this file will be able to connect to the server.

Use the nbdkit --tls-psk option to start the server:

 nbdkit --tls=require --tls-psk=/tmp/keys/keys.psk file disk.img

This option overrides X.509 certificate authentication.

Clients must supply one of the usernames in the PSK file and the corresponding key in order to connect.

An example of connecting using nbdinfo(1) using an NBD URI is:

 nbdinfo 'nbds://alice@localhost?tls-psk-file=/tmp/keys/keys.psk'

An example of connecting using qemu-img(1) is:

 qemu-img info \
   --object tls-creds-psk,id=tls0,dir=/tmp/keys,username=alice,endpoint=client \
   --image-opts \
   file.driver=nbd,file.host=localhost,file.port=10809,file.tls-creds=tls0,file.export=/

Default TLS behaviour¶

If nbdkit was compiled without GnuTLS support, then TLS is disabled and TLS connections will be rejected (as if --tls=off was specified on the command line). Also it is impossible to turn on TLS in this scenario. You can tell if nbdkit was compiled without GnuTLS support because "nbdkit --dump-config" will contain "tls=no".

If TLS certificates cannot be loaded either from the built-in path or from the directory specified by --tls-certificates, then TLS defaults to disabled. Turning TLS on will give a warning (--tls=on) or error (--tls=require) about the missing certificates.

If TLS certificates can be loaded from the built-in path or from the --tls-certificates directory, then TLS will by default be enabled (like --tls=on), but it is not required. Clients can choose whether or not to use TLS and whether or not to present certificates.

TLS client certificates are not checked by default unless you specify --tls-verify-peer.

If the --tls-psk option is used then TLS is enabled (but not required). To ensure that all clients are authorized you must use --tls=require.

Each of these defaults is insecure to some extent (including --tls=on which could be subject to a downgrade attack). If you expect TLS then it is best to specify --tls=require, and if you want to check client certificates, additionally use the --tls-verify-peer option.

Controlling TLS fallback to plaintext¶

When --tls=on is used, the connection can fall back to plaintext. You can use nbdkit-tls-fallback-filter(1) to provide safe fallback content to plaintext connections. With this filter the underlying plugin content is only served on secure connections.

Alternatively a plugin may wish to serve different content depending on whether the client is using TLS. The function nbdkit_is_tls(3) can be used during the ".open" callback for that purpose.

NBD URIs for TLS¶

Tools such nbdcopy(1), nbdinfo(1) and nbdsh(1) (from libnbd(3)) allow you to use "nbds://" or "nbds+unix://" URIs to connect to nbdkit servers using TLS.

The syntax is fully documented in the NBD URI specification: https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md. This section contains an outline. You can also find further examples in nbd_connect_uri(3).

Use the nbdkit --print-uri option to print the URI when nbdkit starts up.

nbds://example.com

Connect over TCP with TLS, to "example.com" port 10809. If the server does not support TLS then this will fail.

nbds+unix:///?socket=SOCKET

As above, but connect over a Unix domain socket called SOCKET.

nbds+unix:///?socket=SOCKET&tls-certificates=DIR

As above, but specify the directory DIR containing TLS certificates (used by the client to verify the server, and to present client authentication to the server). Note this requires libnbd ≥ 1.10.

nbds+unix:///?socket=SOCKET&tls-psk-file=FILENAME

As above, but use TLS with Pre-Shared Keys (PSK), stored in the secrets file FILENAME.

nbds+unix://alice@/?socket=SOCKET&tls-psk-file=FILENAME

As above, but use "alice" as the username.

Default location of certificates¶

Without --tls-certificates nbdkit and libnbd look in several locations for certificates.

If nbdkit is started as a non-root user (note this does not include use of the -u or -g options), nbdkit looks in each of these paths in turn:

 $HOME/.pki/nbdkit/
 $HOME/.config/pki/nbdkit/

If nbdkit is started as root:

 $sysconfdir/pki/nbdkit/

where $sysconfdir is set when nbdkit is compiled, usually /etc. (Use "nbdkit --dump-config" and look at the "root_tls_certificates_dir" setting to get the actual directory built into the binary.)

In libnbd the paths are different. For non-root:

 $HOME/.pki/libnbd/
 $HOME/.config/pki/libnbd/

For root:

 $sysconfdir/pki/libnbd/

In nbdkit you can override these directories by using --tls-certificates=/path/to/certificates.

In libnbd you can use nbd_set_tls_certificates(3). In libnbd ≥ 1.10 you can append "&tls-certificates=/path/to/certificates" to URIs.

Choice of TLS algorithms¶

TLS has a bewildering choice of algorithms that can be used. To enable you to choose a default set of algorithms, there is a configure setting --with-tls-priority. This defaults to "NORMAL" which, to quote the GnuTLS documentation:

You could also set the TLS priority so that it can be configured from a file at runtime:

 ./configure --with-tls-priority=@SYSTEM

means use the policy from /etc/crypto-policies/config.

 ./configure --with-tls-priority=@NBDKIT,SYSTEM

means use the policy from /etc/crypto-policies/local.d/nbdkit.config and fall back to /etc/crypto-policies/config if the first file does not exist.

More information can be found in gnutls_priority_init(3).

Debugging TLS connections¶

Encrypted connections makes snooping on network traffic with Wireshark impossible, by design.

The TLS library used by nbdkit, called gnutls, supports the "SSLKEYLOGFILE" standard: https://web.archive.org/web/20200118013150/https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

Before running nbdkit set the environment variable to point to a log file (note it will append to the file if it already exists):

 SSLKEYLOGFILE=/tmp/keylog nbdkit [...]

In Wireshark go to Edit → Preferences → Protocols → TLS and set (Pre)-Master-Secret log filename to the log file name. Wireshark should be able to see the unencrypted traffic. For further information read: https://wiki.wireshark.org/TLS#using-the-pre-master-secret