secondexpansion interference

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

secondexpansion interference

Martin d'Anjou-5
Hi,

I concocted a contrived example where the presence of a prerequisite alters
the execution of GNU Make depending on its presence. I also use second
expansion. This is the makefile:

.SECONDEXPANSION:
file_x  := f.q
build_x := touch

%.tcl: $(interference) $$(file_$$*)
        $(build_$*) \
  $@

%.q: %.tcl
        touch $@

To run, first touch a few files so they exist:
$ touch f.q file.txt

Next, run make (this works):
$ make x.tcl
touch \
  x.tcl

Ok, clean up:
$ rm x.tcl

Now interfere with the process:
$ make x.tcl interference=file.txt
make x.tcl
\
  f.tcl
make: f.tcl: Command not found
make: *** [f.tcl] Error 127

This problem feels familiar. Can someone explain this behaviour?

Thanks,
Martin
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Philip Guenther-2
On Fri, Apr 25, 2014 at 1:08 PM, Martin d'Anjou
<[hidden email]> wrote:
> I concocted a contrived example where the presence of a prerequisite alters
> the execution of GNU Make depending on its presence. I also use second
> expansion. This is the makefile:

I'm unable to reproduce this following your instructions with GNU make
4.0 (nor with 3.82) on OpenBSD.


Philip Guenther

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Martin d'Anjou-5
On 14-04-25 04:40 PM, Philip Guenther wrote:

> I'm unable to reproduce this following your instructions with GNU make
> 4.0 (nor with 3.82) on OpenBSD.
>
>
> Philip Guenther

Bad news here too. I cannot reproduce it on Linux Mint 13 (make
3.82.90), but I can see it on RHEL 5.5 and RHEL 6.4 with both make 3.82
and 4.0. Should I run it with strace or some other debug tool?

Martin

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Paul Smith-20
On Sat, 2014-04-26 at 15:14 -0400, Martin d'Anjou wrote:
> On 14-04-25 04:40 PM, Philip Guenther wrote:
>
> > I'm unable to reproduce this following your instructions with GNU make
> > 4.0 (nor with 3.82) on OpenBSD.
>
> Bad news here too. I cannot reproduce it on Linux Mint 13 (make
> 3.82.90), but I can see it on RHEL 5.5 and RHEL 6.4 with both make 3.82
> and 4.0. Should I run it with strace or some other debug tool?

strace tracks system calls; that's unlikely to be useful here.  Most
likely it's a memory issue; try installing valgrind and running it using
that ("valgrind .../make ...").

There have been a few fixes since 4.0 was released that may be relevant.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Martin d'Anjou-5
On 14-04-26 04:41 PM, Paul Smith wrote:

> On Sat, 2014-04-26 at 15:14 -0400, Martin d'Anjou wrote:
>> On 14-04-25 04:40 PM, Philip Guenther wrote:
>>
>>> I'm unable to reproduce this following your instructions with GNU make
>>> 4.0 (nor with 3.82) on OpenBSD.
>> Bad news here too. I cannot reproduce it on Linux Mint 13 (make
>> 3.82.90), but I can see it on RHEL 5.5 and RHEL 6.4 with both make 3.82
>> and 4.0. Should I run it with strace or some other debug tool?
> strace tracks system calls; that's unlikely to be useful here.  Most
> likely it's a memory issue; try installing valgrind and running it using
> that ("valgrind .../make ...").


valgrind gives me 8240 lines of output... I am not sure what to do with
it...


> There have been a few fixes since 4.0 was released that may be relevant.
>

When I compile from the git source, I get this error:
autopoint: *** The AM_GNU_GETTEXT_VERSION declaration in your
configure.ac <http://configure.ac> file requires the infrastructure from
gettext-0.18.1 but this version is older. Please upgrade to
gettext-0.18.1 or newer.

I do have gettext 0.18.1, but it is installed locally in my home
directory. How do I specify an alternative location for gettext to
autoreconf -i?

Thanks,
Martin
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Paul Smith-20
On Sun, 2014-04-27 at 09:04 -0400, Martin d'Anjou wrote:
> valgrind gives me 8240 lines of output... I am not sure what to do
> with it...

Compress it and email it directly to me.

> I do have gettext 0.18.1, but it is installed locally in my home
> directory. How do I specify an alternative location for gettext to
> autoreconf -i?

I think you need to use the -B (--prepend-dir) flag to autoreconf and
add the aclocal directory for your local gettext.

However, it might be simpler to just edit the configure.ac file in your
make directory and change the AM_GNU_GETTEXT_VERSION value to the
version you have installed by default.  That probably will work well
enough for your purposes.  There's an issue with older versions of
GETTEXT on _some_ platforms (for some reason I'm thinking Solaris) but
it shouldn't be a problem on Linux.



_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Martin d'Anjou-5
On 14-04-27 09:11 AM, Paul Smith wrote:

>
>> I do have gettext 0.18.1, but it is installed locally in my home
>> directory. How do I specify an alternative location for gettext to
>> autoreconf -i?
> I think you need to use the -B (--prepend-dir) flag to autoreconf and
> add the aclocal directory for your local gettext.
>
> However, it might be simpler to just edit the configure.ac file in your
> make directory and change the AM_GNU_GETTEXT_VERSION value to the
> version you have installed by default.  That probably will work well
> enough for your purposes.  There's an issue with older versions of
> GETTEXT on _some_ platforms (for some reason I'm thinking Solaris) but
> it shouldn't be a problem on Linux.
>
>
To get passed the gettext problem, I changed configure.ac like so:

AM_GNU_GETTEXT_VERSION([0.17])
#AM_PROG_AR
m4_pattern_allow([AM_PROG_AR], [AM_PROG_AR])

Next, when I run ./configure --prefix=..., I get:

./configure: line 8834: PKG_PROG_PKG_CONFIG: command not found
./configure: line 8846: syntax error near unexpected token `GUILE,'
./configure: line 8846: `   PKG_CHECK_MODULES(GUILE, guile-2.0,
have_guile=yes,'

I also tried --with-guile=no but that did not work. I have pkg-config
0.28 in my path.

Martin

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Martin d'Anjou-5
On 14-04-27 12:16 PM, Martin d'Anjou wrote:
> On 14-04-27 09:11 AM, Paul Smith wrote:
>>
>>> I do have gettext 0.18.1, but it is installed locally in my home
>>> directory. How do I specify an alternative location for gettext to
>>> autoreconf -i?
>> I think you need to use the -B (--prepend-dir) flag to autoreconf and
>> add the aclocal directory for your local gettext.

Sorry about the multiple postings. I was able to compile the latest make
by doing the following:

Change configure.ac:
#AM_PROG_AR
m4_pattern_allow([AM_PROG_AR], [AM_PROG_AR])

#AM_GNU_GETTEXT_VERSION([0.18.1])
AM_GNU_GETTEXT_VERSION([0.17])

Next type these (providing gettext 0.17 in on the PATH):

touch ABOUT-NLS config/config.rpath po/Makefile.in.in
aclocal -I /usr/share/aclocal -I config
autoheader
automake --add-missing
autoconf
./configure
make make

The version I compiled is:
git log -n 1
commit 8411528fdd21e978076365df2f49e2b934becb84

I still see the problem I posted at first. I will send Paul the valgrind
on this latest make version.

Martin

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Paul Smith-20
On Sun, 2014-04-27 at 12:45 -0400, Martin d'Anjou wrote:
> I still see the problem I posted at first. I will send Paul the
> valgrind on this latest make version.

Hm, the valgrind results are totally clean: no heap issues at all.  Of
course there could still be stack issues.  Let me look at the original
problem again.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Paul Smith-20
In reply to this post by Martin d'Anjou-5
On Fri, 2014-04-25 at 16:08 -0400, Martin d'Anjou wrote:

> .SECONDEXPANSION:
> file_x  := f.q
> build_x := touch
>
> %.tcl: $(interference) $$(file_$$*)
>         $(build_$*) \
>   $@
>
> %.q: %.tcl
>         touch $@
>
> To run, first touch a few files so they exist:
> $ touch f.q file.txt
>
> Next, run make (this works):
> $ make x.tcl
> touch \
>   x.tcl
>
> Ok, clean up:
> $ rm x.tcl
>
> Now interfere with the process:
> $ make x.tcl interference=file.txt
> make x.tcl
> \
>   f.tcl

Yeah, sorry for the delay I got sidetracked.

I think there is a bug here but it's somewhat subtle, and not related to
memory issues at all.  I'm not sure why Philip is not seeing this
behavior.

Here's what's happening:

In the "working" case, you ask for x.tcl to be built.  Because of
$(file_x), there is a prerequisite of f.q for this file.  f.q can be
rebuilt from f.tcl, but we can't use the %.tcl rule to build f.tcl since
that would be a pattern rule recursion (we're already using this pattern
rule to build x.tcl) so it's not allowed and we have no rule to build
f.tcl.  Thus we have no rule to build f.q, but since f.q already exists
it's OK, then we build x.tcl with $* set to "x" so $(build_$*) works as
expected.  Here's some debug output:

  Considering target file 'x.tcl'.
   File 'x.tcl' does not exist.
   Looking for an implicit rule for 'x.tcl'.
   Trying pattern rule with stem 'x'.
   Trying implicit prerequisite 'f.q'.
   Found an implicit rule for 'x.tcl'.
    Considering target file 'f.q'.
     Looking for an implicit rule for 'f.q'.
     Trying pattern rule with stem 'f'.
     Trying implicit prerequisite 'f.tcl'.
     Looking for a rule with intermediate file 'f.tcl'.
      Avoiding implicit rule recursion.
      Trying pattern rule with stem 'f'.
     Found an implicit rule for 'f.q'.
     Finished prerequisites of target file 'f.q'.
     Prerequisite 'f.tcl' of target 'f.q' does not exist.
    No need to remake target 'f.q'.
   Finished prerequisites of target file 'x.tcl'.
  Must remake target 'x.tcl'.


In the "non-working" case, you again ask for x.tcl to be built.  We
depend on file.txt (due to $(intermediate)) and make doesn't have a rule
to build it but it exists, so that's OK.  Then we try to build f.q and
see it can be rebuilt from f.tcl, as before.

Here's where the difference shows up.  f.tcl depends now on "file.txt"
and $(file_f) which is empty.  Make sees that this is a pattern rule
recursion as before, but for some reason the addition of the extra
prerequisite "file.txt" causes make to not throw away that rule; it
still decides to try to build f.tcl and because $(build_f) is not set,
that fails.  Here's some debug output:

  Considering target file 'x.tcl'.
   File 'x.tcl' does not exist.
   Looking for an implicit rule for 'x.tcl'.
   Trying pattern rule with stem 'x'.
   Trying implicit prerequisite 'file.txt'.
   Trying implicit prerequisite 'f.q'.
   Found an implicit rule for 'x.tcl'.
    Considering target file 'file.txt'.
     Looking for an implicit rule for 'file.txt'.
     No implicit rule found for 'file.txt'.
     Finished prerequisites of target file 'file.txt'.
    No need to remake target 'file.txt'.
    Considering target file 'f.q'.
     Looking for an implicit rule for 'f.q'.
     Trying pattern rule with stem 'f'.
     Trying implicit prerequisite 'f.tcl'.
     Looking for a rule with intermediate file 'f.tcl'.
      Avoiding implicit rule recursion.
      Trying pattern rule with stem 'f'.
      Trying implicit prerequisite 'file.txt'.
     Found an implicit rule for 'f.q'.
       Pruning file 'file.txt'.
     Considering target file 'f.tcl'.
      File 'f.tcl' does not exist.
       Pruning file 'file.txt'.
      Finished prerequisites of target file 'f.tcl'.
     Must remake target 'f.tcl'.
 <fails>

Note that even though we say "Avoiding implicit rule recursion" we still
think we found a rule to build f.tcl.

I think there's something wonky about implicit rule recursion and
secondary expansion.

Can you file a bug on Savannah about this?

But I suspect you are really not interested in this extra recursion for
f.tcl anyway; as a workaround you can define an explicit rule for it;
add:

  f.tcl: ;

and then it will work.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: secondexpansion interference

Martin d'Anjou-5
On 14-04-27 02:37 PM, Paul Smith wrote:
> On Fri, 2014-04-25 at 16:08 -0400, Martin d'Anjou wrote:
> I think there's something wonky about implicit rule recursion and
> secondary expansion.
> Can you file a bug on Savannah about this?

https://savannah.gnu.org/bugs/index.php?42215

Thanks Paul!

Martin

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make