Bug 1223 - unbound-1.6.1 fails to link
unbound-1.6.1 fails to link
Product: unbound
Classification: Unclassified
Component: server
x86_64 Linux
: P5 normal
Assigned To: unbound team
Depends on:
  Show dependency treegraph
Reported: 2017-02-21 20:47 CET by Gaetan Bisson
Modified: 2017-02-27 20:21 CET (History)
3 users (show)

See Also:

log (53.29 KB, application/octet-stream)
2017-02-21 20:47 CET, Gaetan Bisson
configlexer.c generated by flex-2.6.1 (157.96 KB, application/octet-stream)
2017-02-27 14:36 CET, Bruce Dubbs
configlexer.c generated by flex-2.6.3 (159.26 KB, application/octet-stream)
2017-02-27 14:38 CET, Bruce Dubbs

Note You need to log in before you can comment on or make changes to this bug.
Description Gaetan Bisson 2017-02-21 20:47:45 CET
Created attachment 373 [details]

On an up-to-date Arch Linux system, unbound-1.6.1 fails to build. Simply running `./configure && make` (no build flags) ends with the following error:

/tmp/ccnSG7rs.ltrans0.ltrans.o: In function `ub_c_parse':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/util/configparser.c:2094: undefined reference to `ub_c_lex'
/tmp/ccnSG7rs.ltrans30.ltrans.o: In function `config_read':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/util/config_file.c:933: undefined reference to `ub_c_in'
/tmp/ccnSG7rs.ltrans1.ltrans.o: In function `daemon_delete':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/daemon/daemon.c:688: undefined reference to `ub_c_lex_destroy'

I'm attaching the build logs.
Comment 1 Wouter Wijngaards 2017-02-22 08:26:48 CET
Hi Gaetan,

This should work fine, and the routines should be there in util/configlexer.c.

The ..destroy() call is even gated with an ifdef that detected whether that function exists ... looks like something is going wrong.

Can you try the build flag --disable-flto?

Best regards, Wouter
Comment 2 Wouter Wijngaards 2017-02-22 08:27:58 CET
(adding version number to bugzilla)
Comment 3 Bruce Dubbs 2017-02-22 18:55:26 CET
I get exactly the same results, with or without -flto

gcc -I. -g -O2 -flto -pthread -o unbound .libs/acl_list.o .libs/cachedump.o .libs/daemon.o .libs/remote.o .libs/stats.o .libs/unbound.o .libs/worker.o .libs/dns.o .libs/infra.o .libs/rrset.o .libs/dname.o .libs/msgencode.o .libs/as112.o .libs/msgparse.o .libs/msgreply.o .libs/packed_rrset.o .libs/iterator.o .libs/iter_delegpt.o .libs/iter_donotq.o .libs/iter_fwd.o .libs/iter_hints.o .libs/iter_priv.o .libs/iter_resptype.o .libs/iter_scrub.o .libs/iter_utils.o .libs/localzone.o .libs/mesh.o .libs/modstack.o .libs/view.o .libs/outbound_list.o .libs/alloc.o .libs/config_file.o .libs/configlexer.o .libs/configparser.o .libs/fptr_wlist.o .libs/locks.o .libs/log.o .libs/mini_event.o .libs/module.o .libs/net_help.o .libs/random.o .libs/rbtree.o .libs/regional.o .libs/rtt.o .libs/dnstree.o .libs/lookup3.o .libs/lruhash.o .libs/slabhash.o .libs/timehist.o .libs/tube.o .libs/winsock_event.o .libs/autotrust.o .libs/val_anchor.o .libs/validator.o .libs/val_kcache.o .libs/val_kentry.o .libs/val_neg.o .libs/val_nsec3.o .libs/val_nsec.o .libs/val_secalgo.o .libs/val_sigcrypt.o .libs/val_utils.o .libs/dns64.o .libs/cachedb.o .libs/netevent.o .libs/listen_dnsport.o .libs/outside_network.o .libs/ub_event.o .libs/keyraw.o .libs/sbuffer.o .libs/wire2str.o .libs/parse.o .libs/parseutil.o .libs/rrdef.o .libs/str2wire.o .libs/strlcat.o .libs/strlcpy.o .libs/reallocarray.o .libs/arc4random.o .libs/arc4random_uniform.o .libs/explicit_bzero.o .libs/arc4_lock.o  -lssl -lcrypto -pthread 

  -- Bruce Dubbs
Comment 4 Gaetan Bisson 2017-02-22 20:40:56 CET
Wouter: Setting --disable-flto as build flag ./configure says:

configure: error: in `/opt/arch/svn-community/unbound/trunk/unbound-1.6.1':
configure: error: C compiler cannot create executables

I'm using GCC 6.3 and ld from GNU Binutils 2.27.
Comment 5 Wouter Wijngaards 2017-02-23 09:34:48 CET
Hi Gaetan,

Your compiler seems to be broken?  It works for Bruce and it works for me.  The disable-flto flag makes configure simply not check and not add the -flto flag to compilation.  So that flag should be completely harmless and omit an optimisation option.

May be a fresh install/compile environment.  Look in config.log at the 'C compile cannot create executables' part, where configure keeps extensive logs about what is going on.

Best regards, Wouter
Comment 6 Gaetan Bisson 2017-02-23 09:49:35 CET
My mistake: I was passing --disable-flto as a flag to gcc, not ./configure.

So running `./configure --disable-flto && make` yields the same results, like Bruce reported.

Compiling object files is fine but linking fails with:

libtool: link: gcc -I. -g -O2 -pthread -o unbound .libs/acl_list.o .libs/cachedump.o .libs/daemon.o .libs/remote.o .libs/stats.o .libs/unbound.o .libs/worker.o .libs/dns.o .libs/infra.o .libs/rrset.o .libs/dname.o .libs/msgencode.o .libs/as112.o .libs/msgparse.o .libs/msgreply.o .libs/packed_rrset.o .libs/iterator.o .libs/iter_delegpt.o .libs/iter_donotq.o .libs/iter_fwd.o .libs/iter_hints.o .libs/iter_priv.o .libs/iter_resptype.o .libs/iter_scrub.o .libs/iter_utils.o .libs/localzone.o .libs/mesh.o .libs/modstack.o .libs/view.o .libs/outbound_list.o .libs/alloc.o .libs/config_file.o .libs/configlexer.o .libs/configparser.o .libs/fptr_wlist.o .libs/locks.o .libs/log.o .libs/mini_event.o .libs/module.o .libs/net_help.o .libs/random.o .libs/rbtree.o .libs/regional.o .libs/rtt.o .libs/dnstree.o .libs/lookup3.o .libs/lruhash.o .libs/slabhash.o .libs/timehist.o .libs/tube.o .libs/winsock_event.o .libs/autotrust.o .libs/val_anchor.o .libs/validator.o .libs/val_kcache.o .libs/val_kentry.o .libs/val_neg.o .libs/val_nsec3.o .libs/val_nsec.o .libs/val_secalgo.o .libs/val_sigcrypt.o .libs/val_utils.o .libs/dns64.o .libs/cachedb.o .libs/netevent.o .libs/listen_dnsport.o .libs/outside_network.o .libs/ub_event.o .libs/keyraw.o .libs/sbuffer.o .libs/wire2str.o .libs/parse.o .libs/parseutil.o .libs/rrdef.o .libs/str2wire.o .libs/strlcat.o .libs/strlcpy.o .libs/reallocarray.o .libs/arc4random.o .libs/arc4random_uniform.o .libs/explicit_bzero.o .libs/arc4_lock.o .libs/getentropy_linux.o  -lssl -lcrypto -pthread
.libs/configlexer.o: In function `yylex':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/<stdout>:3716: undefined reference to `yywrap'
.libs/daemon.o: In function `daemon_delete':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/daemon/daemon.c:688: undefined reference to `ub_c_lex_destroy'
.libs/config_file.o: In function `config_read':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/util/config_file.c:933: undefined reference to `ub_c_in'
.libs/configparser.o: In function `ub_c_parse':
/opt/arch/svn-community/unbound/trunk/unbound-1.6.1/util/configparser.c:2094: undefined reference to `ub_c_lex'
collect2: error: ld returned 1 exit status
Comment 7 Wouter Wijngaards 2017-02-23 10:24:06 CET
Hi Gaetan,

Unbound uses flex (lex) and bison (yacc) to make the file that the link error is about.  It could be that your version of lex and yacc are broken somehow and produce lex or yacc results that do not contain the output needed?

Or, do you have them installed even?  They are detected by configure (there are different versions on different systems).

Unbound ships with flex and yacc output that is fine, created over here (on Fedora 25).  But they are overwritten during the compile process when you have the tools installed.

Best regards, Wouter
Comment 8 Bruce Dubbs 2017-02-23 16:50:06 CET
I am using flex 2.6.3.  We updated to that on January 1st.

  -- Bruce
Comment 9 Bruce Dubbs 2017-02-23 16:52:14 CET
I am using flex 2.6.3.  We updated to that on January 1st.

And let me note that unbound 1.6.0 builds fine in my environment.

  -- Bruce
Comment 10 Wouter Wijngaards 2017-02-23 16:52:29 CET
Hi Gaetan,

This is what the code that ships is generated with:

$ flex --version
flex 2.6.0
$ bison --version
bison (GNU Bison) 3.0.4

Best regards, Wouter
Comment 11 Gaetan Bisson 2017-02-23 18:26:38 CET
I've got flex-2.6.3 and bison-3.0.4. Uninstalling flex and/or bison (so unbound uses the internal copy) does not help.
Comment 12 Gaetan Bisson 2017-02-23 18:31:01 CET
(In reply to Gaetan Bisson from comment #11)
> I've got flex-2.6.3 and bison-3.0.4. Uninstalling flex and/or bison (so
> unbound uses the internal copy) does not help.

Actually, yes, uninstalling both fixes the problem...
Comment 13 Gaetan Bisson 2017-02-23 18:34:00 CET
So the new flex seems to be the culprit. I'll report back if I can pin this down further.
Comment 14 Gaetan Bisson 2017-02-24 18:14:01 CET
Can either of you replicate this problem with flex-2.6.3? (Or is it just me?)
Comment 15 Wouter Wijngaards 2017-02-27 10:15:54 CET

There are no obvious differences that could cause this between 1.6.0 and 1.6.1.  The util/configlexer.lex file is processed by flex.  It is prepended with util/configyyrename.h to rename a lot of yy content so that it does not name-collide with others.

     echo "#include \"config.h\"" > util/configlexer.c
     echo "#include \"util/configyyrename.h\"" >> util/configlexer.c
     $(LEX) -t $(srcdir)/util/configlexer.lex >> util/configlexer.c

The -t flag should make flex output the result to stdout.

Actually unbound does not use flex multiple times; so the rename part is not strictly necessary (combined with library symbol export features in the dynamic linker; because the user of libunbound might also want to use flex).

Best regards, Wouter
Comment 16 Bruce Dubbs 2017-02-27 14:22:42 CET
Just to confirm.

I reverted my system to flex-2.6.1 and unbound-1.6.1 built without issues.

I then restored flex-2.6.3 and tried to rebuild unbound-1.6.1 and it failed.

config.h and configyyrename.h are identical in both builds.

Doing a simple diff, configlexer.c is quite different between the two builds, however the string 'ub_c' does not show up in either file.

I do not understand the output of flex, configlexer.c, but I will attach the two files.
Comment 17 Bruce Dubbs 2017-02-27 14:36:56 CET
Created attachment 377 [details]
configlexer.c generated by flex-2.6.1

Builds unbound without issues.
Comment 18 Bruce Dubbs 2017-02-27 14:38:20 CET
Created attachment 378 [details]
configlexer.c generated by flex-2.6.3

The unbound executable fails to build with this file.
Comment 19 Wouter Wijngaards 2017-02-27 14:59:14 CET
Hi Bruce,

File with flex 2.6.3 that you uploaded contains these lines in it.  These cause the problem:
    #define yy_create_buffer yy_create_buffer

    #define yy_delete_buffer yy_delete_buffer


    #define yylex yylex

    #define yylex_destroy yylex_destroy

During compilation my compiler warns of double #defines for them.  I try to #define them to a different value, but flex #defines them to themselves?  Why is flex outputting this list of #define yy_ <the same>?

Best regards, Wouter
Comment 20 Wouter Wijngaards 2017-02-27 15:04:44 CET
Hi Bruce, Gaetan,

This is a bug in flex 2.6.3: https://github.com/westes/flex/issues/162
and fixed in flex 2.6.4.

Best regards, Wouter
Comment 21 Bruce Dubbs 2017-02-27 20:21:37 CET
I can confirm that unbound-1.6.1 builds find with the flex git master.

Unfortunately, the flex maintainers have not released flex-2.6.4.