[bug #47399] include sometimes fails for built files

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
URL:
  <http://savannah.gnu.org/bugs/?47399>

                 Summary: include sometimes fails for built files
                 Project: make
            Submitted by: None
            Submitted on: Sat 12 Mar 2016 11:51:38 PM UTC
                Severity: 3 - Normal
              Item Group: Bug
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: 4.1
        Operating System: None
           Fixed Release: None
           Triage Status: None

    _______________________________________________________

Details:

See attached makefile for success/failure test results and other relevant
notes.

For files which are built by recipes in an included makefile, GNU make fails
catastrophically, even if all dependencies are satisfied and rules to build
the file to be included are defined.
This differs from the behavior of other implementations of make (as noted in
the attached makefile) as well as for the similar case of use of an internal
rule to generate the file to be included (e.g. using SCCS).

On an unrelated note: the supposedly-anti-spam filter annoyingly does not
accept the correct answer to "Please enter the title of George Orwell's famous
dystopian book (it's a date):", which is "Nineteen Hundred Eighty Four"



    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Sat 12 Mar 2016 11:51:38 PM UTC  Name: makefile.gmake-bug  Size: 1kB  
By: None

<http://savannah.gnu.org/bugs/download.php?file_id=36615>

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Update of bug #47399 (project make):

                  Status:                    None => Not A Bug              
             Open/Closed:                    Open => Closed                

    _______________________________________________________

Follow-up Comment #1:

This isn't a bug: this is how make is documented to work with building
included makefiles; from the make manual "How Makefiles are Remade":

_After all makefiles have been checked, if any have actually been changed,
make starts with a clean slate and reads all the makefiles over again._

So, GNU make doesn't stop after the first included file, try to make it, and
if it succeeds re-exec itself right away.  Instead it will try to make ALL the
included makefiles it knows about, then afterwards if any of them have been
remade it will re-exec itself.

Since when make first reads the makefile there is no file "baz" and no target
that can build "baz", it fails.  This is the way make has always worked, and
always been intended to work; this feature was already present in GNU make in
1992, when the first set of source code was added to source control.

You have various choices: first, you can change your "include baz" line to
"-include" or "sinclude" which will prevent GNU make from failing if the "baz"
makefile doesn't exist.  It will then remake "bar", ignore the fact that "baz"
doesn't exist, re-exec, find a rule to build "baz", build that, re-exec, and
complete as expected.

Alternatively, you can move the "include baz" line into the "bar" makefile so
that make cannot see it until after "bar" has been created and make has
re-exec'd to read it.  Or you can do a similar thing without moving "include
bar" but using make's if conditionals to make the "include baz" invisible
unless "bar" has been included: testing some variable defined only in "bar"
for example, or even just filtering MAKEFILE_LIST to see if "bar" is present
in it.

FWIW, the correct title of Orwell's work is "Nineteen Eighty-Four", not
"Nineteen Hundred Eight Four".  Although I have no idea if that would be
accepted or if the "it's a date" is supposed to be a hint that it wants the
numeric "1984".

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Update of bug #47399 (project make):

              Item Group:                     Bug => Enhancement            
                  Status:               Not A Bug => None                  

    _______________________________________________________

Follow-up Comment #2:

Actually I'm going to reopen this as an enhancement.

One thing GNU make could do would be, if some makefile cannot be included and
cannot be built, if any makefile has already been built then rather than
failing we'll try to re-exec.

I think this shouldn't be a backward-incompatibility.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Update of bug #47399 (project make):

             Open/Closed:                  Closed => Open                  
           Triage Status:                    None => Medium Effort          


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #3, bug #47399 (project make):

> This is the way make has always worked

See the notes in the sample makefile: make and dmake both work fine; gmake
fails.  It might be the case that gmake has always had this issue; that is an
incompatibility with make.

> you can change your "include baz" line to "-include" or "sinclude"

No, I cannot; that would cause the makefile to be non-portable, i.e. it would
no longer work with make, dmake, bmake,...

> you can move the "include baz" line into the "bar" makefile

Maintenance issues, see the notes in the attached makefile.
Note also that the sample makefile is just that; a brief example that
demonstrates the issue.  Typical cases such as the one where
this gmake incompatibility with make occur involves more than a dozen files,
several of which are built, with include directives
in one file except for the include directive in the main makefile, which
includes the file containing the other include directives (intentionally, for
ease of maintenance).

> [g]make's if conditionals [...] filtering MAKEFILE_LIST

More non-portable stuff.

Anyway, I look forward to seeing the enhancement. Thanks.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #4, bug #47399 (project make):

Well, we'll just have to agree to disagree over who implemented it
"incompatibly".  I've seen no evidence that any other implementation of make
provided this feature before GNU make (Roland added it in September 1988).

Also it's not clear what you mean when you say "make"; the original make
implemented by Feldman didn't have this feature, of course.  And the POSIX
definition of "make" doesn't mention it either (until recently POSIX didn't
require "include").  When I say "make" I clearly mean "GNU make".  I don't
know what variation of make you are referring to when you say "make".

I admire your attempt to create a single set of makefiles for a complex
project which could work across such disparate implementations of make; they
have barely more than the basic makefile structure in common.  It's an
impressive feat, but it must be a huge amount of frustrating effort.  Wouldn't
it be significantly less work to use a single implementation of make which is
portable across systems; it's much simpler to build a portable make one time
than to maintain makefiles that squeak through all the portability cracks in
different versions of make.  Or use a "meta language", which is what cmake and
automake do.

> Anyway, I look forward to seeing the enhancement. Thanks.

Honestly this is not high on my personal TODO list.  The current makefile
update behavior has been present for 27+ years and it's worked fine.  However
maybe someone will find it an interesting project.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #5, bug #47399 (project make):

> Also it's not clear what you mean when you say "make"

Tested versions are listed in the comments at the top of the test makefile.
Note that on OpenIndiana, /usr/bin/make, /usr/bin/dmake, and
/usr/xpg4/bin/make
all use the same executable, but with slightly varying semantics (and they
all work fine with the test makefile).


> Wouldn't it be significantly less work to use a single implementation of
make which is portable across systems;

That would most likely have to be dmake, which "just works", has been ported
to many systems,
is widely available (e.g. via pkgsrc), and does not carry onerous licensing
restrictions.
But it is impractical to force such a requirement ("you must use such-and-such
an implementation
of make"), as there are systems where administrative policy and/or legal
considerations would
preclude such.

IANAL, but although there is a version of gmake on OpenIndiana, its legal
status is
unclear (at least to me):

# uname -srvio
SunOS 5.11 illumos-bc0e913 i86pc Solaris
# gmake --version
GNU Make 4.1
Built for i386-pc-solaris2.11
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
# what $(whence gmake)
/bin/gmake:
        SunOS 5.11 illumos-33e8313 November 2015

Clearly, GPL'd gmake has been linked with CDDL illumos libraries (the linked
executable
contains illumos SCCS ids), even though the FSF says that such linking is not
permitted.
Their exact wording is stronger, and can be found at
http://www.gnu.org/licenses/license-list.html#CDDL )

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #6, bug #47399 (project make):

Regarding licensing: I can't say for sure what CDDL-covered code might have
been linked with GNU make on OpenIndiana.  However it's most likely simply
things such as the C runtime library, since that's about all that GNU make
relies on.

In that case, there's no problem because this falls under the "System Library"
exception in the GPL, both versions 2 and 3.  So the licensing does not need
to be compatible.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #7, bug #47399 (project make):

[comment #4:]
> Well, we'll just have to agree to disagree over who implemented it
"incompatibly".  I've seen no evidence that any other implementation of make
provided this feature before GNU make (Roland added it in September 1988).

This is a somewhat minor point, but 4 years and a global pandemic later, it's
bubbled up the list of things to look into, so I'm adding some information
here for reference by anybody who cares about makefile inclusion:

https://minnie.tuhs.org/cgi-bin/utree.pl has a vast repository of UNIX
sources, including various versions of make, notably including the V7 version
of Stu Feldman's original (at
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/make) [with a few
tweaks, it can be built...]

The Ultrix version of gram.y
(https://minnie.tuhs.org/cgi-bin/utree.pl?file=Ultrix-3.1/src/cmd/make/gram.y)
certainly includes support for "include" and has copyright dates of 1984-1986,
SCCS id 3.0 and date of "4/21/86", i.e. 1986-04-21. This version appears to
have been derived from System V (the executables are named starting with
"s5make").  That certainly pre-dates 1988.

The System III version of gram.y
(https://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/make/gram.y)
also has support for "include"; there's no date in the file itself (SCCS
version id 3.6), but it's supposedly dated 1980-04-27, which seems to indicate
that "include" support was added fairly early (the V7 version w/o it is dated
1979), at least in versions originating from USG/USL versions of UNIX.

On the other hand, it didn't find its way into Berkeley-derived systems until
much later; it's not in the BSD 4.3-Reno source
(https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.3BSD-Reno/src/old/make/gram.y)
with Berkeley SCCS id 4.2 dated 1987.

Side note: clearly the files followed diverging modification paths with few
updates being merged back, as can be seen by the disparate SCCS ids.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #47399] include sometimes fails for built files

David Boyce-5
Follow-up Comment #8, bug #47399 (project make):

Just to clarify, it's not support for the include keyword we're discussing
here.  I have no problem believing that various make implementations provided
some kind of include facility before GNU make even existed: it seems an
obviously useful extension.

Here we're talking about the "rebuild included makefiles" feature
specifically.

Perhaps one or all of these earlier incarnations DID provide for rebuilding
included makefiles, I don't know.  But we'd need to look more deeply than
checking to see if the grammar provides for an "include" keyword to find out.

But, that's certainly interesting!  I may pull some of those older sources
down just for historical purposes.  Thanks for digging these up!

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?47399>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/