ignore specific file in VPATH/vpath

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

ignore specific file in VPATH/vpath

Jannick
Hey,

I am having an issue with the make vpath/VPATH configuration where make
should ignore a specific existing file in vpath/VPATH. Here my case for
which any input from you is highly appreciated:

Cd'ed to the folder 'build', the objective is to build prog with the source
files prog.c and parse.y, both sitting in 'src':

+---build
|       makefile
|
\---src
        parse.c <<< disregard for a build in folder 'build'
        parse.y
        prog.c

The nasty thing is src/parse.c which unfortunately does exist, but should be
ignored for our project, since build/parse.c should be separately build
(from parse.y) and used instead.

Currently, I am not able - in an elegant way - to avoid that make uses
src/parse.c, unless make -B. The following makefile does the job, but it is
not very elegant to set up a vpath statement for each .c source file (this
is just a tiny example, in real life there might be many of those required).

$ cat makefile

src_dir = ../src
prog: prog.o parse.o
%.c: %.y
                        bison -o $@ $<

# vpath statement for each .c source file, not very elegant
vpath   %.y $(src_dir)
vpath prog.c $(src_dir)

Alternatives:
1 - Using VPATH = $(src_dir) instead of the vpath block does not work, since
then make uses src/parse.c unless make -B.
2 - Another solution could be to copy the source files to build and take it
from there.
3 - My question: Is there any (easy) tweak to the makefile above (without
the given vpath block) to make that happen - or should be a copy to build
folder & compile method applied or something else I was not thinking of?


Separately, an additional note: make's YACC built-in recipe

%.c: %.y
#  recipe to execute (built-in):
        $(YACC.y) $<
         mv -f y.tab.c $@

appears to assume that YACC/bison's output file name defaults to y.tab.c.
However, it is $*.tab.c for bison >= 3.0 (at least). Would that be worth an
enhancement to make to update to the new built-in recipe? Here I am not sure
which versions of yacc are out there and what their output files are named.

Many thanks for any input!

Regards,
J.

Signatures:
- GNU Make 4.2.1; Built for i686-pc-cygwin
- bison (GNU Bison) 3.0.4


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: ignore specific file in VPATH/vpath

Jannick
I figured out another solution: If make cannot distinguish the two generated
files ../parse.c and parse.c from each other, then the makefile could do
that by labelling parse.c, i.e. using a prefix (here: 'g-' for generated);
use g-parse.c instead of parse.c. This definitely routes the building path
of parse.y through the corresponding .c file built *in* the folder 'build'.

This is at the expense to have an explicit rule for .y files which is -
AFAICS - currently needed anyways, since the built-in YACC rule breaks with
bison >= 3.0 (at least); more as to why please see below.

src_dir = ../src
prog: prog.o g-parse.o

g-%.c: %.y
                        bison -o $@ $<

# solution 1
VPATH = $(src_dir)

# solution 2
#vpath   %.y $(src_dir)
#vpath %.c $(src_dir)

Nonetheless, I would be still interested in other sweet and short solutions.

Thanks,
J.

On Wed, 9 May 2018 14:30:15 +0200, Jannick wrote:
> Hey,
>
> I am having an issue with the make vpath/VPATH configuration where make
> should ignore a specific existing file in vpath/VPATH. Here my case for
which
> any input from you is highly appreciated:
>
> Cd'ed to the folder 'build', the objective is to build prog with the
source files

> prog.c and parse.y, both sitting in 'src':
>
> +---build
> |       makefile
> |
> \---src
>         parse.c <<< disregard for a build in folder 'build'
>         parse.y
>         prog.c
>
> The nasty thing is src/parse.c which unfortunately does exist, but should
be
> ignored for our project, since build/parse.c should be separately build
(from
> parse.y) and used instead.
>
> Currently, I am not able - in an elegant way - to avoid that make uses
> src/parse.c, unless make -B. The following makefile does the job, but it
is not
> very elegant to set up a vpath statement for each .c source file (this is
just a

> tiny example, in real life there might be many of those required).
>
> $ cat makefile
>
> src_dir = ../src
> prog: prog.o parse.o
> %.c: %.y
> bison -o $@ $<
>
> # vpath statement for each .c source file, not very elegant
> vpath   %.y $(src_dir)
> vpath prog.c $(src_dir)
>
> Alternatives:
> 1 - Using VPATH = $(src_dir) instead of the vpath block does not work,
since
> then make uses src/parse.c unless make -B.
> 2 - Another solution could be to copy the source files to build and take
it
> from there.
> 3 - My question: Is there any (easy) tweak to the makefile above (without
the
> given vpath block) to make that happen - or should be a copy to build
folder

> & compile method applied or something else I was not thinking of?
>
>
> Separately, an additional note: make's YACC built-in recipe
>
> %.c: %.y
> #  recipe to execute (built-in):
> $(YACC.y) $<
> mv -f y.tab.c $@
>
> appears to assume that YACC/bison's output file name defaults to y.tab.c.
> However, it is $*.tab.c for bison >= 3.0 (at least). Would that be worth
an

> enhancement to make to update to the new built-in recipe? Here I am not
> sure which versions of yacc are out there and what their output files are
> named.
>
> Many thanks for any input!
>
> Regards,
> J.
>
> Signatures:
> - GNU Make 4.2.1; Built for i686-pc-cygwin
> - bison (GNU Bison) 3.0.4


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: ignore specific file in VPATH/vpath

Paul Smith-20
In reply to this post by Jannick
On Wed, 2018-05-09 at 14:30 +0200, Jannick wrote:
> Separately, an additional note: make's YACC built-in recipe
>
> %.c: %.y
> #  recipe to execute (built-in):
>         $(YACC.y) $<
>          mv -f y.tab.c $@
>
> appears to assume that YACC/bison's output file name defaults to y.tab.c.
> However, it is $*.tab.c for bison >= 3.0 (at least).

Can you tell us what your makefile is using for a "yacc" command?

The default in GNU make is:

  YACC = yacc

and this default, and the behavior of the rules for generating source
and object files using yacc, is required by the POSIX specification for
GNU make: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html

Are you overriding this value to set it to "bison" instead in your
makefile?  Or, does your system actually provide a "yacc" command?

The "yacc" command is defined by POSIX including its generated output
file names: http://pubs.opengroup.org/onlinepubs/009604599/utilities/yacc.html

Basically it's up to you to either (a) use a standard-conforming "yacc"
command, or (b) modify the YACC variable so that it contains a
standard-conforming yacc command, or (c) rewrite the rule to use a
different yacc command.

If your system provides a "yacc" command (option (a)) and it does not
conform to POSIX (generates the wrong output) then that's a bug in your
OS or distribution and you should report that to them.

If you want to follow option (b) above and replace "yacc" with "bison",
then instead of using:

  YACC = bison

you need to use:

  YACC = bison -y

which tells Bison to use traditional output file naming.

Or of course you can use option (c), as you've done, and rewrite the
rule to use whatever output you like.

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: ignore specific file in VPATH/vpath

Jannick
Paul,

Many thanks for the clarifications.

After going through all that everything is just fine from my side as well!

On Thu, 10 May 2018 07:07:54 -0400, Paul Smith wrote:

> On Wed, 2018-05-09 at 14:30 +0200, Jannick wrote:
> > Separately, an additional note: make's YACC built-in recipe
> >
> > %.c: %.y
> > #  recipe to execute (built-in):
> >         $(YACC.y) $<
> >          mv -f y.tab.c $@
> >
> > appears to assume that YACC/bison's output file name defaults to y.tab.c.
> > However, it is $*.tab.c for bison >= 3.0 (at least).
>
> Can you tell us what your makefile is using for a "yacc" command?

In my makefile YACC is overridden by 'bison' which ultimately does not see any '-y' flag (!), such that the redefined YACC and the built-in rule need not be compatible with each other. Since they aren’t, a user-defined rule is required for '%.c: %y' (option (c) below).
 
> Basically it's up to you to either (a) use a standard-conforming "yacc"
> command, or (b) modify the YACC variable so that it contains a standard-
> conforming yacc command, or (c) rewrite the rule to use a different yacc
> command.

> Or of course you can use option (c), as you've done, and rewrite the rule to
> use whatever output you like.

BTW: Did you have by any chance some time to have a quick look at my example? I would be curious if there are other sweet and clean solutions for it probably using some deep make subtleties I am not aware of?

Many thanks again for the links, in particular, and sorry for the noise.

Regards,
J.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: ignore specific file in VPATH/vpath

Paul Smith-20
On Thu, 2018-05-10 at 14:01 +0200, Jannick wrote:
> In my makefile YACC is overridden by 'bison' which ultimately does
> not see any '-y' flag (!)

I'm not sure what you mean by "does not see"...?  It's definitely
available in Bison 3:

$ .../bison --version
bison (GNU Bison) 3.0.4
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ .../bison --help | grep .-y
  -y, --yacc                 emulate POSIX Yacc

I'll take a look at the rest of your email soon I hope.

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: ignore specific file in VPATH/vpath

Jannick
On Thu, 10 May 2018 08:12:08 -0400, Paul Smith wrote:
> On Thu, 2018-05-10 at 14:01 +0200, Jannick wrote:
> > In my makefile YACC is overridden by 'bison' which ultimately does not
> > see any '-y' flag (!)

What I meant is that when running my makefile and the YACC command is composed, there is no '-y' flag set for bison. Sorry for being a bit sloppy here. Of course, the '-y' flag is implemented for bison to invoke the yacc mode.

> I'll take a look at the rest of your email soon I hope.

Appreciated - thanks!


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: ignore specific file in VPATH/vpath

Paul Smith-20
In reply to this post by Jannick
On Wed, 2018-05-09 at 14:30 +0200, Jannick wrote:

> +---build
> >        makefile
> >
>
> \---src
>         parse.c <<< disregard for a build in folder 'build'
>         parse.y
>         prog.c
>
> The nasty thing is src/parse.c which unfortunately does exist, but
> should be ignored for our project, since build/parse.c should be
> separately build (from parse.y) and used instead.
>
> Currently, I am not able - in an elegant way - to avoid that make
> uses src/parse.c, unless make -B.

I think maybe it would help if you backed up a bit... I thought I
understood what you wanted but then I was totally thrown by this
reference to "make -B" at the end here.

Can you explain more clearly?  You seem to be saying that build/parse.c
is the output from building src/parse.y?  And that's what you want to
use to create build/parse.o?  Is src/parse.c also that output, or some
completely different code?  What do you want to do with src/parse.c?

Why do you want to NOT ignore src/parse.c if make -B is provided?  If
make -B is provided what do you want to use src/parse.c for?

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: ignore specific file in VPATH/vpath

Jannick
Hi Paul,

On Fri, 11 May 2018 12:46:57 -0400, Paul Smith wrote:

> On Wed, 2018-05-09 at 14:30 +0200, Jannick wrote:
> > +---build
> > >        makefile
> > >
> >
> > \---src
> >         parse.c <<< disregard for a build in folder 'build'
> >         parse.y
> >         prog.c
> >
> > The nasty thing is src/parse.c which unfortunately does exist, but
> > should be ignored for our project, since build/parse.c should be
> > separately build (from parse.y) and used instead.
> >
> > Currently, I am not able - in an elegant way - to avoid that make uses
> > src/parse.c, unless make -B.
>
> I think maybe it would help if you backed up a bit... I thought I understood
> what you wanted but then I was totally thrown by this reference to "make -
> B" at the end here.
>
> Can you explain more clearly?  You seem to be saying that build/parse.c is
> the output from building src/parse.y?  And that's what you want to use to
> create build/parse.o?  Is src/parse.c also that output, or some completely
> different code?  What do you want to do with src/parse.c?

Sorry for apparently being more than even a bit unclear. I am taking the time to elaborate in more detail what I want to achieve and what did not work when I was writing the posting above:

In general I am headed to a - what is called I think - "VPATH build". The following is meant to be a small generic example to discuss the issue. In reality, however, it applies to projects with many more source files like 'src/parse.y' together with 'src/parse.c'.
 
After cd'ing to the folder 'build', the goal is to compile src/parse.y and src/prog.c into
prog.exe using the following working steps

a - src/parse.y -[bison/yacc]-> build/parse.c -[gcc] -> build/parse.o
b - src/prog.c -[gcc]-> build/prog.o
c - build/prog.o build/parse.o -[gcc]-> prog.exe

(For clarity I added the redundant prefix 'build/' - redundant, because I said that we have already cd'ed to 'build'. I will stick to this notation.)
These steps ignore src/parse.c (it exists as a left-over of another different build in the real world case) and it should be ignored in any case in this exercise.

1 - Using the following makefile (picked from my first posting; I dropped the explicit %.c:%.y rule, since the rule itself is not important and we assume that we rely on the built-in yacc rule; this makes the script cleaner for the discussion) causes a problem if we say (1.1) a simple 'make' (the culprit src/parse.c is used unfortunately), but it is fine if we say (1.2) 'make -B':

src_dir = ../src
prog: prog.o parse.o
vpath   %.y $(src_dir)
vpath   %.c $(src_dir)

1.1 - A simple command 'make' does not meet our goal, since when make sees 'src/parse.c' in the course of the search algorithm, it stops and uses this file as a prerequisite for build/parse.o. This is because of our rule 'vpath %.c $(src_dir)'.

- src/parse.c -[gcc]-> build/parse.o   <<< the culprit!
- src/prog.c -[gcc]-> build/prog.o
- build/prog.o build/parse.o -[gcc]-> prog.exe

1.2 - 'make -B' is perfectly fine, since then make goes back to 'src/parse.y' (given vpath %.y $(src_dir)) to build 'build/parse.c' with the built-in yacc rule.

Now let us look at the following variations of the makefile in (1):

2 - Replacing the two vpath lines by 'VPATH = $(src_dir)' causes the same issues as in (1). So this does not solve the problem. (It might get even worse if src/parse.o existed; but let's drop that here, since this variant does not help anyway.)

3 - Replacing 'vpath %.c $(src_dir)' by the more specific line 'vpath prog.c $(src_dir)' does help, since then make is prohibited to consider src/parse.c' in the search algorithm. Then for making 'build/parse.o' make has to use either 'build/parse.c' - if this file exists already - or the rule sequence (a) above to build 'build/parse.o' from 'src/parse.y'.  The downside of this solution is that in bigger projects we need an explicit line 'vpath f.c $(src_dir)' for *each* source file f.c (here all in folder src).

Question - Is there any way to say something like "vpath %.c <list of folder names>, but exclude file_1.c, ..., file_n.c' from '%.c'" (i.e. something like 'A \minus B'? I believe the make syntax does not allow any valid "\minus statement" for the file scope, I have not seen any at least, but I might be wrong. Or could "$(filter-out ... )" help here with a loop across admissible .c files?  

4 - For now I do not have any solution using 'vpath' or 'VPATH', but using the prefix idea shields 'src/parse.c' away from make's search algorithm (an explicit rule for %.y is needed then):

src_dir = ../src
prog: prog.o gen-parse.o
gen-%.c: %.y
                        bison -o $@ $<
vpath   %.y $(src_dir)
vpath   %.c $(src_dir)

Using in this case 'VPATH=$(src_dir)' instead - or equivalently on top of - the two vpath lines in the makefile (4) above seems to solve the issue in the first place. But then it is falling over, because it is a bit too broad in the case in which there is a file src/parse.o sitting next to src/parse.c (which is very probable in real life). This could make term 'VPATH build' sound a bit misleading, since rather 'vpath' statements should be used to better be on the very safe side.

> Why do you want to NOT ignore src/parse.c if make -B is provided?

I don’t. My comment regarding 'make -B' refers to the undesired result (1.1), albeit (1.2) is fine.

> If make -B is provided what do you want to use src/parse.c for?

Let me reiterate that src/parse.c (together with any files dependent on it) should never be used in our exercise. What I wanted to say is that 'make -B' did what I wanted make to do, but a simple 'make' command did not and caused the trouble.


All in all I hope this can clarify your questions, although this got really lengthy. Please let me know otherwise, happy to revisit the explanations above or to provide more information. Apologies for the sloppy imprecise language, apparently English is not my mother tongue.

In any case many thanks for your time!

Regards,
J.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: ignore specific file in VPATH/vpath

Paul Smith-20
On Sat, 2018-05-12 at 11:34 +0200, Jannick wrote:
> Question - Is there any way to say something like "vpath %.c <list of
> folder names>, but exclude file_1.c, ..., file_n.c' from '%.c'" (i.e.
> something like 'A \minus B'? I believe the make syntax does not allow
> any valid "\minus statement" for the file scope, I have not seen any
> at least, but I might be wrong. Or could "$(filter-out ... )" help
> here with a loop across admissible .c files?

I see.  I think the comments about make -B were adding more confusion
than clarity, for what it's worth.

The answer to your question, as you've discovered, is no, there's no
capability to omit certain files from vpath.

My only suggestion other than what you've got (using pattern rules to
ensure that the build/ versions of the files have different names than
the src/ versions) is to force the right version by providing a
pathname, so that it won't match via vpath.  For example, you could
write your rule like this:

    src_dir = ../src
    prog: prog.o parse.o
    vpath %.y $(src_dir)
    vpath %.c $(CURDIR) $(src_dir)

    # remove the default rule for %.c:%.y
    %.c: %.y

    # Define a new rule that builds a local .c
    $(CURDIR)/%.c : %.y
            bison ...

I'm not exactly sure that you need to add $(CURDIR) to vpath %.c but
you probably do.

I didn't test this but it seems like it should work.

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: ignore specific file in VPATH/vpath

Jannick
On Tue, 15 May 2018 19:24:06 -0400, Paul Smith wrote:

> My only suggestion other than what you've got (using pattern rules to ensure
> that the build/ versions of the files have different names than the src/
> versions) is to force the right version by providing a pathname, so that it
> won't match via vpath.  For example, you could write your rule like this:
>
>     src_dir = ../src
>     prog: prog.o parse.o
>     vpath %.y $(src_dir)
>     vpath %.c $(CURDIR) $(src_dir)
>
>     # remove the default rule for %.c:%.y
>     %.c: %.y
>
>     # Define a new rule that builds a local .c
>     $(CURDIR)/%.c : %.y
>             bison ...
>
> I'm not exactly sure that you need to add $(CURDIR) to vpath %.c but you
> probably do.
>
> I didn't test this but it seems like it should work.

Thanks for this suggestion!

For the record here a simplified version which works and suits the objective to properly build in 'build'. The point is to prefix the target of c files to be generated with './' (was $(CURDIR) in your suggestion). That make make ignore src/parse.c. Moreover, the vpath statements can be replaced by one single VPATH statement.

VPATH = ../src
prog: prog.o parse.o

# remove the default rule for %.c:%.y
%.c: %.y

# Define a new rule that builds a local .c
./%.c : %.y
        bison -o $@ $<

This makes script pretty simple now.

Many thanks again for taking the time.

Regards,
J.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make