Bug 1449 - Spurious SERVFAIL, may be triggered by DS hashed with SHA-384
Spurious SERVFAIL, may be triggered by DS hashed with SHA-384
Status: RESOLVED WORKSFORME
Product: unbound
Classification: Unclassified
Component: server
1.6.0
x86_64 Linux
: P5 minor
Assigned To: unbound team
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-09-22 14:01 CEST by Stéphane Bortzmeyer
Modified: 2017-09-22 14:45 CEST (History)
3 users (show)

See Also:


Attachments
Test with unbound-host (312.06 KB, text/plain)
2017-09-22 14:22 CEST, Stéphane Bortzmeyer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stéphane Bortzmeyer 2017-09-22 14:01:24 CEST
Unbound SERVFAILs on ada.eu.org, while Knot and BIND have no problems, and DNSviz or Zonemaster find nothing wrong with this zone.

You can see that for instance with OARC's ODVR but I also can reproduce it with a local Unbound.

The only thing that is "special" on this zone is the DS hashed with SHA-384.

Local Unbound 1.6.0 on Debian:

%  dig +dnssec DNSKEY ada.eu.org

; <<>> DiG 9.10.3-P4-Debian <<>> +dnssec DNSKEY ada.eu.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 32702
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DNSKEY

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Sep 22 13:59:44 CEST 2017
;; MSG SIZE  rcvd: 39

  dig +cd +dnssec DNSKEY ada.eu.org

; <<>> DiG 9.10.3-P4-Debian <<>> +cd +dnssec DNSKEY ada.eu.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24404
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DNSKEY

;; ANSWER SECTION:
ada.eu.org.		84519 IN DNSKEY	257 3 8 (
				AwEAAajrW/DAD+SsLFFIyo48kihgoJYTIg7/iM3AK7Vb
				RI31RkTE33YCxj4Sxt0GApXcToX8W+wzCcts5HbEysgB
				17v4vXf73kj9bDUciM+jN2hOY+YaB7RwTqaIz5y4zwz+
				z5cOW/SmT4D7vYAAz9mDlzcVVtn6HoNqAkRVhyKzcn2Z
				+91pukvDwlUHLTFLYDNv4rVkY/Y9jWa9UpUYFhTnKDt+
				uxrsSBjsp5KOBkmcd3/UVRKIJ/d3AlAPV1hZQ7Mk8G6R
				WCHqzvylehM6eyJ4mUrVh7yDOADBSnjUXDRPAzl3yzMQ
				9GanTZoet+41OoikOOzs/XKClWFHh8kEQowM7gc=
				) ; KSK; alg = RSASHA256; key id = 11501
ada.eu.org.		84519 IN DNSKEY	256 3 8 (
				AwEAAcw+tWkHgC1iRjdG5gfRyvvtd3YvszerTnnxMrdA
				kzZCD/ssDRfCAbh9fOSZGUUKyEvJdeX8+9ojKaacnzfB
				a24jSjD1ZVYn2Hk9kF3oDCvZM2k4vffXGNcZVuoxNhuf
				gZXN9w7aOeUpD9JLnq1qFoLfG2/3BvsZXkNrWUR+muoJ
				) ; ZSK; alg = RSASHA256; key id = 5724
ada.eu.org.		84519 IN RRSIG DNSKEY 8 3 86400 (
				20170929162051 20170915162051 11501 ada.eu.org.
				JVeQPh/KvGNNqJD/cKr5DAtKK++3jeVEKfRCrnoyuaz9
				9SXFy2AQ9+xFHFUVPECVVnY1UkFZgNBc4qNovNaepoCF
				m2pA1Vn02rezClQ1WnVGanCQEPXnF+KMJcaZARY3ZWZ9
				sPOkjT1+rHm2EqVsNBDDKb016SUuU6uVtw2yAY5RiX7K
				gtTCGQQ3akf+VNvIzJRVLxZ0tU52FgaF6SZr89oJWPhX
				fLMZM96TnZIU/BELbcSYjo3M9QD8BfEgFBs0lSWehC4d
				42spzE/m6zvRe6laMjLUKoJtB6b8EshFY7ncvRRQiDvb
				e7d50/61N4HJJz3jqEBFbaCkfhtGRrYP4Q== )

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Sep 22 13:59:54 CEST 2017
;; MSG SIZE  rcvd: 761


The DS :
%  dig +dnssec DS ada.eu.org   

; <<>> DiG 9.10.3-P4-Debian <<>> +dnssec DS ada.eu.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10158
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DS

;; ANSWER SECTION:
ada.eu.org.		83913 IN DS 1954 8 4 (
				68CED336A28C6C8D60BA2D7AAA880907C9CFC267A9DE
				DB0B3524E3308598ECA6152CCECE420AC7DB7A938A24
				A79DD8FF )
ada.eu.org.		83913 IN DS 11501 8 1 (
				331C24087805DA5ED9AE4951761E4AA10F848C99 )
ada.eu.org.		83913 IN DS 11501 8 2 (
				AD305063A21884E0829390E16FAE1A14239655EA3E11
				5E1E7C0519FD45106660 )
ada.eu.org.		83913 IN DS 1954 8 2 (
				9DD9A8CDA9CF184FAC693F3B7511E4F7DA2338598526
				920B30453E1DEAB49869 )
ada.eu.org.		83913 IN RRSIG DS 8 3 86400 (
				20171006200333 20170911092008 11664 eu.org.
				jqgzm2XVu/sWxcUencAoLs9QuLvQmD/XnmGf8hROlaiT
				BG8g972INgsbIj4GhEHlHFddHc4gtz1jWo0t4ndy87j0
				aUnyRGnf7ODq1sQEMO4JxkBMXYmadA7MShqsGIEdzoU9
				2IyIbXyibjv8tuS+jEIHKsxBqFhhQlKqSm/nXT8= )

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Sep 22 14:00:09 CEST 2017
;; MSG SIZE  rcvd: 401


Could it be a problem similar to those triggered by section 3 of RFC 4509? Unbound ignoring the DS?
Comment 1 Stéphane Bortzmeyer 2017-09-22 14:05:11 CEST
For the record, the DNSviz test http://dnsviz.net/d/ada.eu.org/WcTxzg/dnssec/ and the Zonemaster one https://zonemaster.net/test/a5093b4f1ed7449b
Comment 2 Wouter Wijngaards 2017-09-22 14:12:42 CEST
Hi Stephane,

I don't have any issues, I get secure.

Can you run perhaps unbound-host -vdddd -f /etc/root.key -t A ada.eu.org
This prints verbosely to the console, and also prints (with the -v even without -d) the dnssec error that unbound encountered.

You can also enable val-log-level: 2 on the unbound server instance.  That logs one line per dnssec failure with more details.

Best regards, Wouter
Comment 3 Stéphane Bortzmeyer 2017-09-22 14:18:16 CEST
And the tests with DNS-OARC's ODVR, to show that it is not just me :

 % dig +dnssec @bind.odvr.dns-oarc.net DNSKEY ada.eu.org

; <<>> DiG 9.10.3-P4-Debian <<>> +dnssec @bind.odvr.dns-oarc.net DNSKEY ada.eu.org
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2078
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DNSKEY

;; ANSWER SECTION:
ADA.eu.org.		84153 IN DNSKEY	257 3 8 (
				AwEAAajrW/DAD+SsLFFIyo48kihgoJYTIg7/iM3AK7Vb
				RI31RkTE33YCxj4Sxt0GApXcToX8W+wzCcts5HbEysgB
				17v4vXf73kj9bDUciM+jN2hOY+YaB7RwTqaIz5y4zwz+
				z5cOW/SmT4D7vYAAz9mDlzcVVtn6HoNqAkRVhyKzcn2Z
				+91pukvDwlUHLTFLYDNv4rVkY/Y9jWa9UpUYFhTnKDt+
				uxrsSBjsp5KOBkmcd3/UVRKIJ/d3AlAPV1hZQ7Mk8G6R
				WCHqzvylehM6eyJ4mUrVh7yDOADBSnjUXDRPAzl3yzMQ
				9GanTZoet+41OoikOOzs/XKClWFHh8kEQowM7gc=
				) ; KSK; alg = RSASHA256; key id = 11501
ADA.eu.org.		84153 IN DNSKEY	256 3 8 (
				AwEAAcw+tWkHgC1iRjdG5gfRyvvtd3YvszerTnnxMrdA
				kzZCD/ssDRfCAbh9fOSZGUUKyEvJdeX8+9ojKaacnzfB
				a24jSjD1ZVYn2Hk9kF3oDCvZM2k4vffXGNcZVuoxNhuf
				gZXN9w7aOeUpD9JLnq1qFoLfG2/3BvsZXkNrWUR+muoJ
				) ; ZSK; alg = RSASHA256; key id = 5724
ADA.eu.org.		84153 IN RRSIG DNSKEY 8 3 86400 (
				20170929162051 20170915162051 11501 ada.eu.org.
				JVeQPh/KvGNNqJD/cKr5DAtKK++3jeVEKfRCrnoyuaz9
				9SXFy2AQ9+xFHFUVPECVVnY1UkFZgNBc4qNovNaepoCF
				m2pA1Vn02rezClQ1WnVGanCQEPXnF+KMJcaZARY3ZWZ9
				sPOkjT1+rHm2EqVsNBDDKb016SUuU6uVtw2yAY5RiX7K
				gtTCGQQ3akf+VNvIzJRVLxZ0tU52FgaF6SZr89oJWPhX
				fLMZM96TnZIU/BELbcSYjo3M9QD8BfEgFBs0lSWehC4d
				42spzE/m6zvRe6laMjLUKoJtB6b8EshFY7ncvRRQiDvb
				e7d50/61N4HJJz3jqEBFbaCkfhtGRrYP4Q== )

;; Query time: 138 msec
;; SERVER: 2620:ff:c000:0:1:0:64:20#53(2620:ff:c000:0:1:0:64:20)
;; WHEN: Fri Sep 22 14:17:29 CEST 2017
;; MSG SIZE  rcvd: 765

% dig +dnssec @unbound.odvr.dns-oarc.net DNSKEY ada.eu.org

; <<>> DiG 9.10.3-P4-Debian <<>> +dnssec @unbound.odvr.dns-oarc.net DNSKEY ada.eu.org
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 2016
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DNSKEY

;; Query time: 852 msec
;; SERVER: 2620:ff:c000:0:1:0:64:21#53(2620:ff:c000:0:1:0:64:21)
;; WHEN: Fri Sep 22 14:17:36 CEST 2017
;; MSG SIZE  rcvd: 39
Comment 4 Stéphane Bortzmeyer 2017-09-22 14:21:28 CEST
And the full resolution, by unbound-host -vdddd -f /var/lib/unbound/root.key -t A ada.eu.org, attached.
Comment 5 Stéphane Bortzmeyer 2017-09-22 14:22:19 CEST
Created attachment 459 [details]
Test with unbound-host
Comment 6 Wouter Wijngaards 2017-09-22 14:23:57 CEST
Hi Stephane,

Yes this error looks like failure in algorithm rollover to me.

validation failure <ada.eu.org. A IN>: no keys have a DS with algorithm RSASHA256 from 195.154.227.159 for key ada.eu.org. while building chain of trust

I guess you see that but I don't.  Perhaps geo-balance on the servers?
Also I haven't looked further yet.

Best regards, Wouter
Comment 7 Wouter Wijngaards 2017-09-22 14:30:51 CEST
Hi Stephane,

You were right, it is the hash algorithm.

ada.eu.org.     86400   IN      DS      1954 8 2 9DD9A8CDA9CF184FAC693F3B7511E4F7DA2338598526920B30453E1DEAB49869
ada.eu.org.     86400   IN      DS      1954 8 4 68CED336A28C6C8D60BA2D7AAA880907C9CFC267A9DEDB0B3524E3308598ECA6152CCECE420AC7DB7A938A24A79DD8FF
ada.eu.org.     86400   IN      DS      11501 8 1 331C24087805DA5ED9AE4951761E4AA10F848C99
ada.eu.org.     86400   IN      DS      11501 8 2 AD305063A21884E0829390E16FAE1A14239655EA3E115E1E7C0519FD45106660

These are the DS records.  The key we need is 11501. Now 11501 does not have a hash with the strongest hash algorithm SHA384 but only with SHA256.  The spec says you have to use the strongest hash algorithm present, and unbound does that, ignoring the weaker hash algorithm altogether.

This is why unbound fails to resolve.  Update the delegation information to include a SHA384 for 11501, or remove the SHA384 for 1954.

(perhaps this is just a typo, and the SHA1 for 11501 is meant to be SHA384, and then it would work).

Best regards, Wouter
Comment 8 Stéphane Bortzmeyer 2017-09-22 14:31:16 CEST
It's a personal domain, I doubt there is anycast or load-balancing on the servers. Besides, I see all the keys, from the same machine:

% dig @195.154.227.159 DNSKEY ada.eu.org

; <<>> DiG 9.10.3-P4-Debian <<>> @195.154.227.159 DNSKEY ada.eu.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56380
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;ada.eu.org.		IN DNSKEY

;; ANSWER SECTION:
ada.eu.org.		86400 IN DNSKEY	256 3 8 (
				AwEAAcw+tWkHgC1iRjdG5gfRyvvtd3YvszerTnnxMrdA
				kzZCD/ssDRfCAbh9fOSZGUUKyEvJdeX8+9ojKaacnzfB
				a24jSjD1ZVYn2Hk9kF3oDCvZM2k4vffXGNcZVuoxNhuf
				gZXN9w7aOeUpD9JLnq1qFoLfG2/3BvsZXkNrWUR+muoJ
				) ; ZSK; alg = RSASHA256; key id = 5724
ada.eu.org.		86400 IN DNSKEY	257 3 8 (
				AwEAAajrW/DAD+SsLFFIyo48kihgoJYTIg7/iM3AK7Vb
				RI31RkTE33YCxj4Sxt0GApXcToX8W+wzCcts5HbEysgB
				17v4vXf73kj9bDUciM+jN2hOY+YaB7RwTqaIz5y4zwz+
				z5cOW/SmT4D7vYAAz9mDlzcVVtn6HoNqAkRVhyKzcn2Z
				+91pukvDwlUHLTFLYDNv4rVkY/Y9jWa9UpUYFhTnKDt+
				uxrsSBjsp5KOBkmcd3/UVRKIJ/d3AlAPV1hZQ7Mk8G6R
				WCHqzvylehM6eyJ4mUrVh7yDOADBSnjUXDRPAzl3yzMQ
				9GanTZoet+41OoikOOzs/XKClWFHh8kEQowM7gc=
				) ; KSK; alg = RSASHA256; key id = 11501
ada.eu.org.		86400 IN RRSIG DNSKEY 8 3 86400 (
				20170929162051 20170915162051 11501 ada.eu.org.
				JVeQPh/KvGNNqJD/cKr5DAtKK++3jeVEKfRCrnoyuaz9
				9SXFy2AQ9+xFHFUVPECVVnY1UkFZgNBc4qNovNaepoCF
				m2pA1Vn02rezClQ1WnVGanCQEPXnF+KMJcaZARY3ZWZ9
				sPOkjT1+rHm2EqVsNBDDKb016SUuU6uVtw2yAY5RiX7K
				gtTCGQQ3akf+VNvIzJRVLxZ0tU52FgaF6SZr89oJWPhX
				fLMZM96TnZIU/BELbcSYjo3M9QD8BfEgFBs0lSWehC4d
				42spzE/m6zvRe6laMjLUKoJtB6b8EshFY7ncvRRQiDvb
				e7d50/61N4HJJz3jqEBFbaCkfhtGRrYP4Q== )

;; Query time: 5 msec
;; SERVER: 195.154.227.159#53(195.154.227.159)
;; WHEN: Fri Sep 22 14:29:43 CEST 2017
;; MSG SIZE  rcvd: 761
Comment 9 Stéphane Bortzmeyer 2017-09-22 14:32:03 CEST
So, your analysis is that Unbound is right, and Knot, BIND, DNSviz and Zonemaster are too lax?
Comment 10 Wouter Wijngaards 2017-09-22 14:34:42 CEST
Hi Stephane,

Ralph asks, do you have harden-algo-downgrade turned on?

Because, for me I get secure, and the default for that option is turned off.  And then Unbound gets the same as Knot and Bind do.

Or perhaps you have an old version, Ralph notes that the ODVR has 1.5.4 which is old and has the wrong algorithm strictness default still.

Best regards, Wouter
Comment 11 Stéphane Bortzmeyer 2017-09-22 14:34:58 CEST
Testing with the RIPE Atlas probes seem to indicate that "harsh" resolvers are a minority but do exist:

Without cd, 10 % of SERVFAIL:

% atlas-resolve --requested=500 --type=A ada.eu.org
[ERROR: FORMERR] : 3 occurrences 
[213.186.37.73] : 438 occurrences 
[ERROR: SERVFAIL] : 49 occurrences 
[TIMEOUT(S)] : 4 occurrences 
Test #9325755 done at 2017-09-22T12:22:55Z

With +cd, the same probes, no SERVFAIL :

% atlas-resolve --requested=500 --checkingdisabled --old_measurement 9325755 --type=A ada.eu.org
[213.186.37.73] : 491 occurrences 
[TIMEOUT(S)] : 3 occurrences 
Test #9325758 done at 2017-09-22T12:28:51Z
Comment 12 Stéphane Bortzmeyer 2017-09-22 14:36:40 CEST
I have "qname-minimisation: yes" but no "harden*" option.

Also, the tests with the Atlas probes seem to indicate the issue is common.
Comment 13 Wouter Wijngaards 2017-09-22 14:37:22 CEST
Hi Stephane,

That is unfortunate, but the fix is from a time ago (not as long ago for the DS as for the DNSKEY algorithms, but still out there for a while, and that is what you see with atlas).

For surety, the output I see when I try is the same as in your unbound-host logs.  With the SHA384.  But my version does not servfail it, with defaults for the harden-downgrade option.

Best regards, Wouter
Comment 14 Stéphane Bortzmeyer 2017-09-22 14:41:23 CEST
So, 1.6.0 is too old (this is the version in Debian stable)?
Comment 15 Wouter Wijngaards 2017-09-22 14:41:47 CEST
Hi Stephane,

The issue you detect is fixed in Unbound 1.6.2 (24 Apr, 2017):
- harden-algo-downgrade: no also makes unbound more lenient about digest algorithms in DS records.

It is perhaps fairly good deployment for the bugfix from April (90% works after 5 months?) ?  Oh wait, I guess the non-DNSSEC users are also in the 'it works' category.

Best regards, Wouter
Comment 16 Wouter Wijngaards 2017-09-22 14:45:33 CEST
Hi Stephane,

I guess you are right that 1.6.0 is too old, and so are the two testing resolvers you used.  But I cannot really fix that, so I'll mark this bug as 'already fixed' (in 1.6.2).

Thanks for the report!  If we had not fixed it in April, we certainly should fix it now.

Best regards, Wouter