Bug 4131 - error: ‘YY_CURRENT_BUFFER’ undeclared
error: ‘YY_CURRENT_BUFFER’ undeclared
Status: RESOLVED FIXED
Product: unbound
Classification: Unclassified
Component: server
1.7.3
x86_64 other
: P5 normal
Assigned To: unbound team
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-07-20 12:24 CEST by Jeffrey Walton
Modified: 2018-07-20 14:59 CEST (History)
2 users (show)

See Also:


Attachments
config.log (24.94 KB, application/octet-stream)
2018-07-20 12:24 CEST, Jeffrey Walton
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jeffrey Walton 2018-07-20 12:24:37 CEST
Created attachment 515 [details]
config.log

I'm building libunbound 1.7.3 from sources on Solaris 11.3 x86_64. I encountered the following error during make:

./util/configlexer.lex:168:19: error: ‘YY_CURRENT_BUFFER’ undeclared (first use in this function)
  yy_switch_to_buffer(s->buffer);


It does not look like YY_CURRENT_BUFFER is defined:

unbound-1.7.3$ /usr/gnu/bin/grep -IR YY_CURRENT_BUFFER
util/configlexer.lex:   s->buffer = YY_CURRENT_BUFFER;
util/configlexer.lex:   yy_delete_buffer(YY_CURRENT_BUFFER);
util/configlexer.c:     s->buffer = YY_CURRENT_BUFFER;
util/configlexer.c:     yy_delete_buffer(YY_CURRENT_BUFFER);
unbound-1.7.3$

========================================

unbound-1.7.3$ gmake
./libtool --tag=CC --mode=compile gcc -I. -I/usr/local/include -DNDEBUG -I/usr/local/include -I/usr/local/include -g2 -O2 -m64 -march=native -fPIC -std=c99 -D_REENTRANT -pthreads  -o configlexer.lo -c util/configlexer.c
libtool: compile:  gcc -I. -I/usr/local/include -DNDEBUG -I/usr/local/include -I/usr/local/include -g2 -O2 -m64 -march=native -fPIC -std=c99 -D_REENTRANT -pthreads -c util/configlexer.c  -fPIC -DPIC -o .libs/configlexer.o
In file included from util/configlexer.c:1:0:
./config.h:934:0: warning: "NDEBUG" redefined [enabled by default]
 #  define NDEBUG
 ^
<command-line>:0:0: note: this is the location of the previous definition
./util/configlexer.lex:34:0: warning: "ECHO" redefined [enabled by default]

 ^
util/configlexer.c:70:0: note: this is the location of the previous definition
 #define ECHO fprintf(yyout, "%s",yytext)
 ^
./util/configlexer.lex:47:2: error: unknown type name ‘YY_BUFFER_STATE’
  struct inc_state* next;
  ^
./util/configlexer.lex: In function ‘config_start_include’:
./util/configlexer.lex:102:14: error: ‘YY_CURRENT_BUFFER’ undeclared (first use in this function)
  s->next = config_include_stack;
              ^
./util/configlexer.lex:102:14: note: each undeclared identifier is reported only once for each function it appears in
./util/configlexer.lex:107:2: warning: implicit declaration of function ‘ub_c_switch_to_buffer’ [-Wimplicit-function-declaration]
 }
  ^
./util/configlexer.lex:107:2: warning: implicit declaration of function ‘ub_c_create_buffer’ [-Wimplicit-function-declaration]
./util/configlexer.lex:107:46: error: ‘YY_BUF_SIZE’ undeclared (first use in this function)
 }
                                              ^
./util/configlexer.lex: In function ‘config_end_include’:
./util/configlexer.lex:168:2: warning: implicit declaration of function ‘ub_c_delete_buffer’ [-Wimplicit-function-declaration]
  yy_switch_to_buffer(s->buffer);
  ^
./util/configlexer.lex:168:19: error: ‘YY_CURRENT_BUFFER’ undeclared (first use in this function)
  yy_switch_to_buffer(s->buffer);
                   ^
gmake: *** [configlexer.lo] Error 1
Comment 1 Wouter Wijngaards 2018-07-20 12:48:36 CEST
Hi Jeffrey,

From the log I see configure is using lex and there is no yacc (gnu bison) or gnu flex installed.  I think this may be causing the issue; eg. have bison and flex.  Flex then gets used and overrides the other lex.  If you cannot do that, before compilation touch util/configlexer.c and touch util/configlexer.h so that lex does not get invoked.

Because of tarballs and unpacking, those files can end up older than the .lex input and then it reruns the lex command, even though the source distribution ships with working .c and .h files without bison and flex necessary.

Best regards, Wouter
Comment 2 Jeffrey Walton 2018-07-20 12:58:17 CEST
(In reply to Wouter Wijngaards from comment #1)
> Hi Jeffrey,
> 
> From the log I see configure is using lex and there is no yacc (gnu bison)
> or gnu flex installed.  I think this may be causing the issue; eg. have
> bison and flex.  Flex then gets used and overrides the other lex.  If you
> cannot do that, before compilation touch util/configlexer.c and touch
> util/configlexer.h so that lex does not get invoked.
> 
> Because of tarballs and unpacking, those files can end up older than the
> .lex input and then it reruns the lex command, even though the source
> distribution ships with working .c and .h files without bison and flex
> necessary.
> 
> Best regards, Wouter

Thanks Wouter,

I tested back to 1.6.8 and issues were present. So it is not a new issue.

I noticed this while testing:

libtool: compile:  gcc -I. -I/usr/local/include -DNDEBUG -I/usr/local/include -I/usr/local/include -g2 -O2 -m64 -march=native -fPIC -std=c99 -D_REENTRANT -pthreads -c util/alloc.c -o alloc.o >/dev/null 2>&1
if test "lex" != ":"; then \
        echo "#include \"config.h\"" > util/configlexer.c ;\
        echo "#include \"util/configyyrename.h\"" >> util/configlexer.c ;\
        lex -t ./util/configlexer.lex >> util/configlexer.c ;\
fi
"./util/configlexer.lex":line 184: Error: missing translation value
gmake: *** [util/configlexer.c] Error 1
gmake: *** Waiting for unfinished jobs....

This may be useful, too:

$ lex -V
lex: Software Generation Utilities (SGU) Solaris-ELF (4.0)

Let me try to touch the files and set their file time to 1970.

Thanks.
Comment 3 Jeffrey Walton 2018-07-20 13:20:13 CEST
From https://www.am-utils.org/docs/am-utils/BUGS.txt :

(14) Solaris

The line "%option" in *.l files may cause Solaris /usr/ccs/bin/lex to abort
with the error "missing translation value."  This is a bug in Solaris lex.

Moreover, both Solaris yacc and lex produce code that does not pass strict
compilation such as "gcc -Wall -Werror".

Use GNU Flex and Bison instead.  You can download ready-made binaries from
www.sunfreeware.com.  Note, however, that sometimes the binaries on
sunfreeware.com don't seem to work, often because they are built against an
older revision of Solaris or build tools.  In that case, build a fresh
version of GNU flex and/or bison from the latest stable sources.  See
http://www.gnu.org/software/flex/ and http://www.gnu.org/software/bison/.
Comment 4 Wouter Wijngaards 2018-07-20 13:54:25 CEST
Hi Jeffrey,

Perhaps this patch helps?  It checks for %option support and ignores the lex otherwise, that should set LEX="" for you and then the lex is not run and then you use the distributed source.  For flex, for me, it allows flex to run.

You can also download the patched configure file instead of applying this patch:
www.nlnetlabs.nl/svn/unbound/trunk/configure

I think that fixes the problem, Best regards, Wouter


Index: configure
===================================================================
--- configure	(revision 4782)
+++ configure	(working copy)
@@ -6274,6 +6274,8 @@
 
 
 
+
+
 for ac_prog in flex lex
 do
   # Extract the first word of "$ac_prog", so it can be a program name with args.
@@ -6433,6 +6435,7 @@
 rm -f conftest.l $LEX_OUTPUT_ROOT.c
 
 fi
+if test "$LEX" != ""; then
 
 	{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for yylex_destroy" >&5
 $as_echo_n "checking for yylex_destroy... " >&6; }
@@ -6445,6 +6448,21 @@
 	else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }; fi
 
+
+	{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for lex %option" >&5
+$as_echo_n "checking for lex %option... " >&6; }
+	if cat <<EOF | $LEX -t 2>&1 | grep yy_delete_buffer >/dev/null 2>&1; then
+%option nounput
+%%
+EOF
+		{ $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+	else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; };
+		LEX=""
+	fi
+
+fi
 for ac_prog in 'bison -y' byacc
 do
   # Extract the first word of "$ac_prog", so it can be a program name with args.
Index: configure.ac
===================================================================
--- configure.ac	(revision 4782)
+++ configure.ac	(working copy)
@@ -327,8 +327,23 @@
 	else AC_MSG_RESULT(no); fi
 ])
 
+AC_DEFUN([ACX_YYLEX_OPTION], [
+	AC_MSG_CHECKING([for lex %option])
+	if cat <<EOF | $LEX -t 2>&1 | grep yy_delete_buffer >/dev/null 2>&1; then
+%option nounput
+%%
+EOF
+		AC_MSG_RESULT(yes)
+	else AC_MSG_RESULT(no);
+		LEX=""
+	fi
+])
+
 AC_PROG_LEX
+if test "$LEX" != ""; then
 ACX_YYLEX_DESTROY
+ACX_YYLEX_OPTION
+fi
 AC_PROG_YACC
 AC_CHECK_PROG(doxygen, doxygen, doxygen)
 AC_CHECK_TOOL(STRIP, strip)
Comment 5 Jeffrey Walton 2018-07-20 14:50:50 CEST
(In reply to Wouter Wijngaards from comment #4)
> Hi Jeffrey,
> 
> Perhaps this patch helps?  It checks for %option support and ignores the lex
> otherwise, that should set LEX="" for you and then the lex is not run and
> then you use the distributed source.  For flex, for me, it allows flex to
> run.

Thanks Wouter.  I think this is a step in the right direction. I think you are very close to figuring out the work around.

Now I see:

libtool: compile:  gcc -I. -I/usr/local/include -DNDEBUG -I/usr/local/include -I/usr/local/include -g2 -O2 -m64 -march=native -fPIC -std=c99 -D_REENTRANT -pthreads -c util/alloc.c -o alloc.o >/dev/null 2>&1
if test "" != ":"; then \
        echo "#include \"config.h\"" > util/configlexer.c ;\
        echo "#include \"util/configyyrename.h\"" >> util/configlexer.c ;\
         -t ./util/configlexer.lex >> util/configlexer.c ;\
fi
/bin/sh: line 3: -t: not found
gmake: *** [util/configlexer.c] Error 127
Comment 6 Wouter Wijngaards 2018-07-20 14:52:58 CEST
Hi Jeffrey,

Yes I used "" instead of ":", fixup patch, or download again.  Thanks for testing and letting me know!


Best regards, Wouter

Index: configure
===================================================================
--- configure	(revision 4795)
+++ configure	(working copy)
@@ -6435,7 +6435,7 @@
 rm -f conftest.l $LEX_OUTPUT_ROOT.c
 
 fi
-if test "$LEX" != ""; then
+if test "$LEX" != "" -a "$LEX" != ":"; then
 
 	{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for yylex_destroy" >&5
 $as_echo_n "checking for yylex_destroy... " >&6; }
@@ -6459,7 +6459,7 @@
 $as_echo "yes" >&6; }
 	else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; };
-		LEX=""
+		LEX=":"
 	fi
 
 fi
Index: configure.ac
===================================================================
--- configure.ac	(revision 4795)
+++ configure.ac	(working copy)
@@ -335,12 +335,12 @@
 EOF
 		AC_MSG_RESULT(yes)
 	else AC_MSG_RESULT(no);
-		LEX=""
+		LEX=":"
 	fi
 ])
 
 AC_PROG_LEX
-if test "$LEX" != ""; then
+if test "$LEX" != "" -a "$LEX" != ":"; then
 ACX_YYLEX_DESTROY
 ACX_YYLEX_OPTION
 fi
Comment 7 Jeffrey Walton 2018-07-20 14:59:38 CEST
(In reply to Wouter Wijngaards from comment #6)
> Hi Jeffrey,
> 
> Yes I used "" instead of ":", fixup patch, or download again.  Thanks for
> testing and letting me know!

Yep, that was it. The updated configure tested good.

Thank you for the fast help. I owe you a cold beer.

Jeff