simple explanation for order-only prerequisite?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

simple explanation for order-only prerequisite?

Robert P. J. Day-2

  was asked on friday by a make newbie to explain order-only
prerequisite (OOP), as he was reading the explanation in the manual
and was having trouble getting a fix on what it really meant.

  i haven't used one for a while, so i reviewed the section and, AIUI
(and the way i will try to explain it), an OOP means nothing more
than, while the prerequisite will be timestamp checked to see if the
rule needs to be invoked, whether or not that happens is simply not
taken into account for the processing of *this* target.

  in short, while an OOP will be *processed* as usual, its invocation
has no further effect. is that a fundamentally simple (and accurate)
way to describe it? thanks muchly.

rday

Reply | Threaded
Open this post in threaded view
|

Re: simple explanation for order-only prerequisite?

Nicholas Clark
I'd describe OOPs as "just like a regular dependency, except that the
timestamp isn't checked. Only whether it exists on the filesystem."

On Sun, Nov 3, 2019 at 7:51 AM Robert P. J. Day <[hidden email]>
wrote:

>
>   was asked on friday by a make newbie to explain order-only
> prerequisite (OOP), as he was reading the explanation in the manual
> and was having trouble getting a fix on what it really meant.
>
>   i haven't used one for a while, so i reviewed the section and, AIUI
> (and the way i will try to explain it), an OOP means nothing more
> than, while the prerequisite will be timestamp checked to see if the
> rule needs to be invoked, whether or not that happens is simply not
> taken into account for the processing of *this* target.
>
>   in short, while an OOP will be *processed* as usual, its invocation
> has no further effect. is that a fundamentally simple (and accurate)
> way to describe it? thanks muchly.
>
> rday
>
>
Reply | Threaded
Open this post in threaded view
|

Re: simple explanation for order-only prerequisite?

Robert P. J. Day-2
On Sun, 3 Nov 2019, Nicholas Clark wrote:

> I'd describe OOPs as "just like a regular dependency, except that
> the timestamp isn't checked. Only whether it exists on the
> filesystem."

  is it really that simple? just a test for "does it exist or not?"
that suggests that make would not even bother consulting the rule for
that OOP.

  i thought it was a bit more complicated in that make would still
check the rule for the OOP and, if there were any prerequisites that
had a newer timestamp, the OOP's recipe would be invoked, but beyond
that, that would have no dependency effect on the original target. in
short, an OOP would still be *processed* normally, it would just have
no further effect.

  i'm trying to think of an example so what about this one. imagine a
massive SW project, one of whose dependencies is "docs", to update the
documentation. now if any of the (say) markdown files is updated, then
the "docs" recipe is invoked to update the online docs. however, since
documentation has no effect on the actual software, there is no point
letting that target have any further effect in terms of recompiling
anything.

  might not be the best example, but is it accurate?

rday

Reply | Threaded
Open this post in threaded view
|

Re: simple explanation for order-only prerequisite?

pacalet
Le 03/11/2019 à 16:17, Robert P. J. Day a écrit :

> On Sun, 3 Nov 2019, Nicholas Clark wrote:
>
>> I'd describe OOPs as "just like a regular dependency, except that
>> the timestamp isn't checked. Only whether it exists on the
>> filesystem."
>
>   is it really that simple? just a test for "does it exist or not?"
> that suggests that make would not even bother consulting the rule for
> that OOP.
>
>   i thought it was a bit more complicated in that make would still
> check the rule for the OOP and, if there were any prerequisites that
> had a newer timestamp, the OOP's recipe would be invoked, but beyond
> that, that would have no dependency effect on the original target. in
> short, an OOP would still be *processed* normally, it would just have
> no further effect.
>
>   i'm trying to think of an example so what about this one. imagine a
> massive SW project, one of whose dependencies is "docs", to update the
> documentation. now if any of the (say) markdown files is updated, then
> the "docs" recipe is invoked to update the online docs. however, since
> documentation has no effect on the actual software, there is no point
> letting that target have any further effect in terms of recompiling
> anything.
>
>   might not be the best example, but is it accurate?
Looks good to me (except your example, see below). An OOP is processed as any regular prerequisites: if it does not exist (or if it is out-of-date) it is (re)built before the targets for which it is an OOP. But whether it is (re)built or not has no impact on the firing of the rule it is an OOP of.

Simple demo:

$ cat Makefile
a: | b
b: c
a b c:
    touch $@
$ make
touch c
touch b
touch a
$ touch c; make
touch b
$ rm b; make
touch b
$ touch b; make
make: 'a' is up to date.

See? Whatever the reason why 'b' is (re)built, it does not cause 'a' to be rebuilt. And if 'b' is newer than 'a', 'a' is not rebuilt.

Your example is not the best we can imagine because in most cases the doc will not be a prerequisite at all of the executables. So there will be no need to use OOPs. In my humble opinion a better example is the one from the manual: if you want to put some of your targets in a subdirectory you must guarantee that this subdirectory exists before you build these targets. But you don't want these targets to be rebuilt just because the timestamp of the subdirectory changed. You want them to be rebuilt only if one of their regular prerequisites is newer. But this example is not ideal neither because if the subdirectory does not exist yet, the targets do not exist neither and so they will have to be rebuilt anyway. The example I personally prefer is the logging of some recipes in a 'log' directory. If it does not exist we must create the 'log' directory before we execute the recipes but we don't want to execute the recipes just because the 'log' directory does not exist or is newer than the targets:

apache-started: | /var/log
    systemctl start apache2.service
    touch $@

/var/log:
    mkdir $@
--
Renaud Pacalet
Télécom Paris
Campus SophiaTech
450 Route des Chappes, CS 50193
06904 Biot Sophia Antipolis cedex, FRANCE
Tel : +33 (0) 4 9300 8402
Web : http://www.telecom-paris.fr/


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: simple explanation for order-only prerequisite?

Kaz Kylheku (gmake)
In reply to this post by Robert P. J. Day-2
On 2019-11-03 06:50, Robert P. J. Day wrote:
> was asked on friday by a make newbie to explain order-only
> prerequisite (OOP), as he was reading the explanation in the manual
> and was having trouble getting a fix on what it really meant.

When we specify that P is an order-only prerequisite for T,

    T : | P

most of the regular target-prerequisite relationship between
them continues to hold, except that an update/creation of
P doesn't require T to be updated.

This is true even if the update to P is triggered by the fact
that T itself needs P.

Basically it means "If rule T is in the dependency tree
then add P as a dependency of T, ensuring that it's
up-to-date, before T, without any side effect of invalidating an
up-to-date T."

It's not entirely explained with time stamps, because T or P
can be phony targets.

Consider this Makefile with P as a phony, and two real file
targets T0 and T1

.PHONY: P

all: T0 T1

T0 : P
        touch T0

T1 : P
        touch T1

Each time we run, both recipes are executed: both touches take
place.  P is a phony target, and everything that requires it
requires unconditional update.

Now suppose we make it this:

.PHONY: P

all: T0 T1

T0 : | P
        touch T0

T1 : P
        touch T1

If we run this when T0 and T1 do not exist, both recipes run.
Then every subsequent time the T1 recipe is run, but not T0.

The P target is always updated because "all" depends on it
through T0 and T1, and it is phony.

T0 doesn't run if that file exists, because the update of P
doesn't require it to due to order-only.

T1 runs because the update of P causes its own update.

Then if we make it:

all: T0 T1

T0 : | P
        touch T0

T1 : | P
        touch T1

so that both are order-only, we get a useful-looking behavior:

$ rm T0 T1
$ make
touch T0
touch T1
$ make
make: Nothing to be done for 'all'.
$ rm T0
$ make
touch T0
$ make
make: Nothing to be done for 'all'.
$ rm T1
$ make
touch T1
$ make
make: Nothing to be done for 'all'.

Our Makefile now behaves exactly like this one:

all: T0 T1

T0:
        touch T0

T1:
        touch T1

Except that both T0 and T1 depends on the phony target P also.

We can hang some useful recipe on that target which is is
always executed, and which is executed before T0 or T1 is
updated:

Without order-only prerequisites, we couldn't do that; if we
introduce the phony target P we get the behavior that T0 and
T1 are always updated:

.PHONY: P

all: T0 T1

T0 : | P
        touch T0

T1 : | P
        touch T1

P :
        echo P first



$ rm T0 T1
$ make
echo P first
P first
touch T0
touch T1
$ rm T0
$ make
echo P first
P first
touch T0
$ make
echo P first
P first

Because P is phony and in the dependency tree for "all",
the "echo P first" recipe always executes. If either of T0
or T1 rules, or both, also execute, they necessarily
execute afterward.

Semi-useful example: T0 and T1 need some output directory:

all: dir/T0 dir/T1

dir/T0 : | dir
        touch dir/T0

dir/T1 : | dir
        touch dir/T1

dir :
        mkdir -p dir

Why would we make a directory order-only? Because it's not a
real input to the build.

We don't want to update either dir/T0 or dir/T1 just because
the directory is newer than they are; the directory is irrelevant!
It just has to exist before either one is made/updated.

A directory's modification time stamp is bumped each time
an entry in that directory is created or deleted, so if
we make it a real prerequisite, it could cause spurious
rebuilds of the intermediates.

The above trick saves us from having to repeat a "mkdir -p"
into every recipe. However, the value is fairly low because
macros can be used to do that sort of thing.

dir/T1 : | dir
        $(call TOUCH, $@) # TOUCH includes mkdir -p action

Another example is if you're debugging parallel build (make
-j) race conditions, it can be useful as a band-aid for
imposing processing order.

Suppose that rules T0 and T0 have some hidden effect on shared
state, creating a seldom reproduced build-breaking race condition.
But, suppose that they are otherwise independent.

T0 : P01 P02 P03 ...
        # e.g. suppose a temp file is used here

T1 : P11 P12 P13 ...
        # ... and the same named temp file is used here! oops!

We can pull out the order-only hammer and serialize things:

T0 : P01 P02 P03 ... | T1

T1 : P11 P12 P13 ...

We thus have T0 done before T1, in a way that updating T1
doesn't cause an unnecessary rebuild of T0.

T0 now requires T1, so they can't be independently built
targets; if we build T0 we get T1 whether we like it or not.
That situation is often acceptable; internal intermediate
targets are often not independently built.


Reply | Threaded
Open this post in threaded view
|

Re: simple explanation for order-only prerequisite?

David Deutsch
On 05.11.19 04:51, Kaz Kylheku (gmake) wrote:
> Why would we make a directory order-only? Because it's not a
> real input to the build.

That has always been the crux for me about understanding order-only
prerequisites: - It's an OOP if the prerequisite is not directly /used/
in the recipe for the target.

mytarget: [things that are used in the recipe] | [things that need to
"be there" before the recipe is executed]
    mybin $< > $@

-David