Bug 658

Summary: unbound using TLS in a forwarding configuration does not verify the server's certificate
Product: unbound Reporter: Daniel Kahn Gillmor <dkg>
Component: serverAssignee: unbound team <unbound-team>
Status: RESOLVED FIXED    
Severity: enhancement CC: cathya, ilf+nlnetlabs.nl, jimp, martin.monperrus, spam, wouter
Priority: P5    
Version: unspecified   
Hardware: x86_64   
OS: Linux   

Description Daniel Kahn Gillmor 2015-03-25 01:16:34 CET
(this is for unbound 1.5.3, but i don't see it in the list above)

I have the following unbound.conf to act as forwarder to another TLS-over-TCP unbound implementation:


--------
forward-zone:
 name: "."
 forward-addr: 192.0.2.3@443
server:
 ssl-upstream: yes
 tcp-upstream: yes
 do-daemonize: no
 logfile: ""
 verbosity: 10
--------

However, there appears to be no verification of the server's certificate -- the server i'm speaking to has a self-signed certificate, and unbound has no way to know about it initially, so i don't see how it could possibly be verifying the cert.

how to verify the cert itself is an interesting question, though:  you can't identify the remote host by name (because we don't want to have to have DNS to do the lookup), and we generally discourage X.509 certs with an IP address in the subjectAltNames.

So i think the configuration mechanism should be one of the following options (maybe we can offer :

A)
 * forward-fingerprint: an sha256 sum of the Subject Public Key Info of the server's certificate

B)
 * forward-name: the name of upstream server (this is like forward-host, except that it is only used for cert validation, and not looked-up before connection)
 * forward-cafile: a file containing the certificates of the root certification authorities allowed to issue certs for the forwarded server; (the verification mechanism looks for a valid cert for forward-name (if present) or forward-host).
Comment 1 Wouter Wijngaards 2015-03-25 08:57:50 CET
Hi Daniel,

Yes the ssl-upstream feature wraps traffic in tls, but does not verify certificates.  It's documented.  For authentication, perhaps we could see what the dprive IETF WG comes up with in their standardisation?  They are currently working on dns over tls.  I would not want to pre-empt their solution.

I would recommend to use DNSSEC to authenticate the data across this connection.

Best regards,
   Wouter
Comment 2 Daniel Kahn Gillmor 2015-04-06 22:34:04 CEST
(In reply to Wouter Wijngaards from comment #1)

> Yes the ssl-upstream feature wraps traffic in tls, but does not verify
> certificates.  It's documented.

Hm, the documentation of this gap wasn't clear to me.  I did not see any mention of lack of verification in unbound.conf (i inferred it from the behavior of the tool)

> For authentication, perhaps we could see
> what the dprive IETF WG comes up with in their standardisation?  They are
> currently working on dns over tls.  I would not want to pre-empt their
> solution.

Please do not wait on output from dprive; You will not pre-empt anything.  If anything, dprive needs input from real-world deployments (which unbound can provide) to drive its decision-making process.

> I would recommend to use DNSSEC to authenticate the data across this
> connection.

That would be a great step as well, though it is perhaps a chicken-and-egg solution (how do we get the DNSSEC info if all we have is a TLS connection to the upstream server?).

Getting started with something simple like (A) or (B) proposed above would be much more straightforward, and could be overridden by a DNSSEC-verification mechanism (maybe via a TLS extension?) in the future.

Would you be interested in a patch for proposal (A)?
Comment 3 Matt 2017-11-04 12:23:34 CET
There is some additional information in authenticating dns over tls servers here:
https://tools.ietf.org/html/draft-ietf-dprive-dtls-and-tls-profiles-01#section-5

In particular, they propose using SRV records, as long as DNSSEC is verified, or spki's to, if I am understanding correctly, verify the certificate's name.
Comment 4 Matt 2017-11-04 12:28:30 CET
There is some additional info on authentication here:https://tools.ietf.org/html/draft-ietf-dprive-dtls-and-tls-profiles-01#section-5

In particular, they propose using srv records to authenticate the hosts on the certificate or spki
Example service records (for TLS and DTLS respectively):

      _domain-s._tcp.dns.example.com.  SRV 0 1 853 dns1.example.com.
      _domain-s._tcp.dns.example.com.  SRV 0 1 853 dns2.example.com.

      _domain-s._udp.dns.example.com.  SRV 0 1 853 dns3.example.com.
Comment 5 Wouter Wijngaards 2017-11-06 09:30:26 CET
Hi Matt, Daniel,

The issue is that there is no good proposal in that list in that document.  All of them require some sort of pre-distributed keys (hard-coded keys are a recipe for failure), or in fact pre-coded something else (what you use to look up that DNSSEC stuff).  All of that only works for hardcoded 'use this public resolver' and not actually for the normal resolving cases.

I would like to have code that implements authentication.  Preferably in a good way (eg. won't cause further problems, I mean keys can change, you know, like X509 certificates allow key changes).  It is just that all the current proposals are bad, and a lot a similar, they simply fail to be usable.  Because they are all similarly bad, eg. only for public resolvers, not enabled by default, not for 10.0.0.0/8 resolvers, and so on.  I don't want to move ahead of the vetted 'this is the correct solution', because I cannot choose and I do not want to choose a bad security solution.

So, anyway, that stuff is in progress.  And in the document the actual problem changed to 'public resolver', 'fixed destination' and 'pre-configured parameters'.  Which is vastly different from the workgroup mission, DNS privacy, which starts somewhere when you log on.  And this has mostly 10.0.0.0/8 (LAN) DNS, with no authenticated components.

So, in summary, I want good security too, but the proposals are bad, and I don't want to implement 'too early' a bad solution.

Best regards, Wouter
Comment 6 Martin 2018-03-24 22:07:19 CET
For the record, here is how stubby does. It provides the following configuration for SPKI pinsets:

tls_pubkey_pinset:
  - digest: "sha256"
  - value: 62lKu9HsDVbyiPenApnc4sfmSYTHOVfFgL3pyB+cBL4=

This is used in strict mode:  "In 'Strict' mode authentication information (e.g. an authentication name or a SPKI pinset) MUST be provided for each nameserver and DNS queries will only be sent if Stubby can authenticate the namerserver using this information."

If you decide to go for this solution, I guess that a similar configuration option could be added in unbound, eg "forward-ssl-tls-pubkey-pinset"

Best regards, --Martin
Comment 7 Daniel Kahn Gillmor 2018-04-10 16:35:54 CEST
i recommend starting by offering the name of the upstream resolver as an authentication mechanism, and only afterward implementing spki pinsets if you think they offer some sort of concrete advantage.

https://tools.ietf.org/html/rfc8310 recommends authentication based on name in several forms.

note that the two largest DNS-over-TLS deployments today offer valid X.509 certificates for their names:

  9.9.9.9 → dns.quad9.net
  1.1.1.1 → cloudflare-dns.com
Comment 8 Martin 2018-04-11 21:26:44 CEST
On this topic, I've just written a post that may be of interest for you: https://www.monperrus.net/martin/randomization-encryption-dns-requests
Comment 9 Wouter Wijngaards 2018-04-19 14:42:58 CEST
Hi Daniel,

The current code repository implements TLS authentication for forwarders.

The syntax is forward-addr: <IP-address>[@port][#tls-authentication-name]
And the ca bundle can be set with: tls-cert-bundle: "ca-bundle.pem"
(or the ca-bundle.crt file).

Example
server:
  tls-cert-bundle: "/etc/pki/tls/certs/ca-bundle.crt"
forward-zone:
  name: "."
  forward-tls-upstream: yes
  forward-addr: 9.9.9.9@853#dns.quad9.net
  forward-addr: 1.1.1.1@853#cloudflare-dns.com

The hashtag name trick makes it so that the tls authentication name can also be set for eg. stub-zones and with unbound-control forward control commands.  It was also easier in the code.  There should be no spaces around the '@' and '#'.

The port number is really optional, but defaults to port 53 right now (like for ordinary DNS).

When I tested, the servers for dns.quad9 and cloudflare-dns work.

The code looks easy to change for having tls authentication for authority servers (it probably already works using configuration with stub-addr).

Best regards, Wouter
Comment 10 Wouter Wijngaards 2018-04-19 16:24:28 CEST
Hi Daniel, Martin,

Fixed the default, which is now port 853 when you specify a tls authname.  (And still 53 for others).

So the examples can be simplified like this
  forward-addr: 9.9.9.9#dns.quad9.net
  forward-addr: 1.1.1.1#cloudflare-dns.com

Best regards, Wouter
Comment 11 Martin 2018-04-20 08:42:07 CEST
Thanks a lot Wouter.

Do you know a reference list of DNS-over-TLS servers that we can trust?

--Martin
Comment 12 Wouter Wijngaards 2018-04-20 08:55:06 CEST
Hi Martin,

I don't know how to get a list of trusted servers.  You could try asking the unbound-users mail list to see if other users know them.  Two public resolvers are listed in above entries that support DNS-over-TLS.

Best regards, Wouter
Comment 13 ilf+nlnetlabs.nl 2018-04-20 11:09:20 CEST
There is a list of servers at https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Test+Servers, but that doesn't say anything about "trust".
Comment 14 Wouter Wijngaards 2018-04-23 16:20:47 CEST
Hi,

I guess this resolves the bugreport, thanks for the persistence.

Best regards, Wouter
Comment 15 Jim P. 2018-07-16 18:34:53 CEST
On Unbound 1.7.3, with a configuration matching either comment 9 or comment 10, I can get replies but I cannot trigger a verification failure.

--------
# TLS Configuration
tls-cert-bundle: "/etc/ssl/cert.pem"
[...]
# Forwarding
forward-zone:
	name: "."
	forward-tls-upstream: yes
	forward-addr: 9.9.9.9@853#blah.example.com
	forward-addr: 1.0.0.1@853#doesnotexist.com
	forward-addr: 149.112.112.112@853#somethingwrong.com
--------

At verbosity 5, the log shows the hostname expected from the configuration, and the certificate, but always claims that the SSL connection authenticated.

--------
: clog /var/log/resolver.log | egrep -i '(cert|auth|doesnot)' | tail -4
Jul 16 10:31:12 jack unbound: [29373:0] debug:   [doesnotexist.com] ip4 1.0.0.1 port 853 (len 16)
Jul 16 10:31:12 jack unbound: [29373:0] debug: peer certificate:          Issuer: C=US, O=DigiCert Inc, CN=DigiCert ECC Secure Server CA         Validity             Not Before: Mar 30 00:00:00 2018 GMT             Not After : Mar 25 12:00:00 2020 GMT         Subject: C=US, ST=CA, L=San Francisco, O=Cloudflare, Inc., CN=*.cloudflare-dns.com         X509v3 extensions:             X509v3 Authority Key Identifier:                  keyid:A3:9D:E6:1F:F9:DA:39:4F:C0:6E:E8:91:CB:95:A5:DA:31:E2:0A:9F              X509v3 Subject Key Identifier:                  DF:97:4D:E5:43:B3:B0:41:A7:42:F2:90:CF:89:7F:AE:12:57:84:E1             X509v3 Subject Alternative Name:                  DNS:*.cloudflare-dns.com, IP Address:1.1.1.1, IP Address:1.0.0.1, DNS:cloudflare-dns.com, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:1001             X509v3 Key Usage: critical                 Digital Signature             X509v3 Extended Key Usage:                  TLS Web Server Authentication, TLS Web Client 
Jul 16 10:31:12 jack unbound: [29373:0] debug: SSL connection authenticated ip4 1.0.0.1 port 853 (len 16)
Jul 16 10:31:12 jack unbound: [29373:0] info: incoming scrubbed packet: ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0  ;; QUESTION SECTION: 0.pfsense.pool.ntp.org.	IN	AAAA  ;; ANSWER SECTION:  ;; AUTHORITY SECTION: pool.ntp.org.	1500	IN	SOA	a.ntpns.org. hostmaster.pool.ntp.org. 1531751408 5400 5400 1209600 3600  ;; ADDITIONAL SECTION: ;; MSG SIZE  rcvd: 95
--------

At first I thought this was because the server IP address was also listed in the certificate, but I tried it against another Unbound instance with a Let's Encrypt certificate which only contains a hostname, and it still succeeds there when it should fail the hostname check.
Comment 16 Wouter Wijngaards 2018-07-17 09:08:34 CEST
Hi Jim,

Do you have openssl 1.1.x?  Without that the SSL_set1_host call is not available, and unbound does not do verification.  This is detected at configure time, though.  But I did not want to stop unbound's compilation in its entirety for that.

You can see what openssl is linked with unbound with unbound -h.  You must have an older openssl, for the facts you describe match it.

Best regards, Wouter
Comment 17 Jim P. 2018-07-17 14:19:47 CEST
That is correct, this is on FreeBSD 11.2 with base OpenSSL which is a patched OpenSSL 1.0.2o. Is there no possible way to verify the host that doesn't require 1.1.x? Or at least a way to log the issue at run-time?

It seems like a bad idea to fail open when configured this way and the user expects the verification to work.

Ideally it should fail to run and print an error that the configured verification is not possible because of the missing function, but at a minimum logging the error rather than silently disabling the check would help the user figure out what is happening.
Comment 18 Wouter Wijngaards 2018-07-17 14:20:49 CEST
Hi Jim,

Yes logging is certainly possible.  Adding that.

Best regards, Wouter
Comment 19 Wouter Wijngaards 2018-07-17 14:32:11 CEST
Hi Jim,

Code added to printout that name verification won't work if not supported by the ssl library.  It is in the code repository.  Thanks for the report.

It prints for stubs and forwards and for delegations added via remote-control. 

Index: daemon/remote.c
===================================================================
--- daemon/remote.c	(revision 4783)
+++ daemon/remote.c	(revision 4785)
@@ -1950,6 +1950,11 @@
 				return NULL;
 			}
 		} else {
+#ifndef HAVE_SSL_SET1_HOST
+			if(auth_name)
+			  log_err("no name verification functionality in "
+				"ssl library, ignored name for %s", todo);
+#endif
 			/* add address */
 			if(!delegpt_add_addr_mlc(dp, &addr, addrlen, 0, 0,
 				auth_name)) {
Index: iterator/iter_fwd.c
===================================================================
--- iterator/iter_fwd.c	(revision 4783)
+++ iterator/iter_fwd.c	(revision 4785)
@@ -239,6 +239,11 @@
 				s->name, p->str);
 			return 0;
 		}
+#ifndef HAVE_SSL_SET1_HOST
+		if(tls_auth_name)
+			log_err("no name verification functionality in "
+				"ssl library, ignored name for %s", p->str);
+#endif
 		if(!delegpt_add_addr_mlc(dp, &addr, addrlen, 0, 0,
 			tls_auth_name)) {
 			log_err("out of memory");
Index: iterator/iter_hints.c
===================================================================
--- iterator/iter_hints.c	(revision 4783)
+++ iterator/iter_hints.c	(revision 4785)
@@ -252,6 +252,11 @@
 				s->name, p->str);
 			return 0;
 		}
+#ifndef HAVE_SSL_SET1_HOST
+		if(auth_name)
+			log_err("no name verification functionality in "
+				"ssl library, ignored name for %s", p->str);
+#endif
 		if(!delegpt_add_addr_mlc(dp, &addr, addrlen, 0, 0,
 			auth_name)) {
 			log_err("out of memory");


Best regards, Wouter
Comment 20 Jim P. 2018-07-17 14:34:22 CEST
Thanks for adding that, it should at least prevent some head scratching like I was doing :-)