Looking for well-logged Make output

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Looking for well-logged Make output

Alex Nelson-4
Hello all,

This email is somewhat a cross-post of a Stack Overflow question:
http://stackoverflow.com/questions/16488581/looking-for-well-logged-make-output

The gist is, I run workflows focused on single input files, logging stdout,
stderr and shell exit status of each step.  I think a natural way to
express these workflows is with a Makefile.  However, in the event of an
error (which in my case is more frequently nothing of the program's fault -
the power dies, a disk fills, menial stuff), I'd like Make to be able to
resume a workflow based on the exit status of the intermediary target steps.

That is, if the Make rules include target "target4", I'd like Make to
interpret "target4.status.log" having the contents "0" or "0\n" as "Ok,
target4 built."  But if the log contained anything else, like "1\n" or
"Started", Make would treat that as not-ok and halt.

From the crickets except one person on Stack Overflow; the gmake man page;
Wikipedia's make-like programs page; and writing a mess of Bash, Python and
Make to do this before; I'm guessing this doesn't exist in any Make.
 However, I figure this list may provide a more authoritative answer.

Has anybody seen something like what I'm asking about before?

--Alex
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Looking for well-logged Make output

Martin d'Anjou-5
Alexander,

When you run make a second time in the same folder, it will continue
based on the presence of files, not based on their content. The response
from Eric Melski on stackoverflow is right: make sure the output file is
created only when the entire recipe has worked.

Martin


On 13-05-14 03:30 PM, Alexander Nelson wrote:

> Hello all,
>
> This email is somewhat a cross-post of a Stack Overflow question:
> http://stackoverflow.com/questions/16488581/looking-for-well-logged-make-output
>
> The gist is, I run workflows focused on single input files, logging stdout,
> stderr and shell exit status of each step.  I think a natural way to
> express these workflows is with a Makefile.  However, in the event of an
> error (which in my case is more frequently nothing of the program's fault -
> the power dies, a disk fills, menial stuff), I'd like Make to be able to
> resume a workflow based on the exit status of the intermediary target steps.
>
> That is, if the Make rules include target "target4", I'd like Make to
> interpret "target4.status.log" having the contents "0" or "0\n" as "Ok,
> target4 built."  But if the log contained anything else, like "1\n" or
> "Started", Make would treat that as not-ok and halt.
>
>  From the crickets except one person on Stack Overflow; the gmake man page;
> Wikipedia's make-like programs page; and writing a mess of Bash, Python and
> Make to do this before; I'm guessing this doesn't exist in any Make.
>   However, I figure this list may provide a more authoritative answer.
>
> Has anybody seen something like what I'm asking about before?
>
> --Alex
> _______________________________________________
> Help-make mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/help-make


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Looking for well-logged Make output

Alex Nelson-4
Hi Martin and list,

I was hoping there would be a way for Make to continue or halt based on the presence of a file _and_ its respective exit status log - i.e. the file and the declaration that the file's good.  Is this outside the scope of known Makes?

--Alex


On May 14, 2013, at 18:35 , Martin d'Anjou <[hidden email]> wrote:

> Alexander,
>
> When you run make a second time in the same folder, it will continue based on the presence of files, not based on their content. The response from Eric Melski on stackoverflow is right: make sure the output file is created only when the entire recipe has worked.
>
> Martin
>
>
> On 13-05-14 03:30 PM, Alexander Nelson wrote:
>> Hello all,
>>
>> This email is somewhat a cross-post of a Stack Overflow question:
>> http://stackoverflow.com/questions/16488581/looking-for-well-logged-make-output
>>
>> The gist is, I run workflows focused on single input files, logging stdout,
>> stderr and shell exit status of each step.  I think a natural way to
>> express these workflows is with a Makefile.  However, in the event of an
>> error (which in my case is more frequently nothing of the program's fault -
>> the power dies, a disk fills, menial stuff), I'd like Make to be able to
>> resume a workflow based on the exit status of the intermediary target steps.
>>
>> That is, if the Make rules include target "target4", I'd like Make to
>> interpret "target4.status.log" having the contents "0" or "0\n" as "Ok,
>> target4 built."  But if the log contained anything else, like "1\n" or
>> "Started", Make would treat that as not-ok and halt.
>>
>> From the crickets except one person on Stack Overflow; the gmake man page;
>> Wikipedia's make-like programs page; and writing a mess of Bash, Python and
>> Make to do this before; I'm guessing this doesn't exist in any Make.
>>  However, I figure this list may provide a more authoritative answer.
>>
>> Has anybody seen something like what I'm asking about before?
>>
>> --Alex
>> _______________________________________________
>> Help-make mailing list
>> [hidden email]
>> https://lists.gnu.org/mailman/listinfo/help-make
>

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Looking for well-logged Make output

Stephan Beal-2
On Wed, May 15, 2013 at 7:40 AM, Alex Nelson <[hidden email]> wrote:

> I was hoping there would be a way for Make to continue or halt based on
> the presence of a file _and_ its respective exit status log - i.e. the file
> and the declaration that the file's good.  Is this outside the scope of
> known Makes?
>

In a way that's what it does because only one exit status really counts for
anything: code 0. If it exits with code zero, do like Marin suggested:
touch $@

If it doesn't exit cleanly, the file won't be updated, so it will be out of
date on the next run. i did something similar to implement steps of a
long-running db anonymization process some years ago - it could run for
days at a time and be interrupted by, as you say, menial things. In my
case, each step of the process had a tag file associated with it ("step1",
"step17", etc.). On successful completion of the step, its file would be
created. That also allowed us, aside from error recovery, to easily
abort/continue the process at logic transition points in the process, as
well as "fake" steps by creating their marker files manually.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make