[bug #57242] Non-recursive command passes invalid jobserver file descriptors

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
URL:
  <https://savannah.gnu.org/bugs/?57242>

                 Summary: Non-recursive command passes invalid jobserver file
descriptors
                 Project: make
            Submitted by: stefanbruens
            Submitted on: Fri 15 Nov 2019 07:41:39 PM UTC
                Severity: 3 - Normal
              Item Group: Bug
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: 4.2.1
        Operating System: POSIX-Based
           Fixed Release: None
           Triage Status: None

    _______________________________________________________

Details:

The problem can be reproduced using the following trivial
Makefile:
---
all:
    env | grep FLAGS
---

Using strace to get some diagnostics:
$> env -i strace -epipe,dup2,execve,fcntl -v -f -o nonrecursive.txt
/usr/bin/make -j2 -f  Makefile

Obivously, the jobserver file descriptors (3,4) are closed on exec by
fcntl({3,4}, F_SETFD, FD_CLOEXEC), but are still exported via
MFLAGS/MAKEFLAGS=--jobserver-auth=3,4.

When the "env | grep FLAGS" command is invoked via shell, the shell creates a
new pipe, reusing the no longer used file descriptors 3 and 4.

I.e. "env" (which serves as a placeholder here for some command implementing
the jobserver protocol) receives some file descriptors which are no jobserver
fds.

This does not happen when the command is marked as a recursive one (using the
'+' marker), but this is non-trivial to achieve in practice:

1. the Makefile may come from some generator
2. the command itself may be some script invoking "gmake" again.





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #1, bug #57242 (project make):

Stefan, the behavior you described is intended.
make closes the pipe unless the command has + or (MAKE) or {MAKE}.
This is necessary, because a command can mess up job server operation or a
command may expect a specific fd to be available.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #2, bug #57242 (project make):

[comment #1 comment #1:]
> Stefan, the behavior you described is intended.
> make closes the pipe unless the command has + or (MAKE) or {MAKE}.
> This is necessary, because a command can mess up job server operation or a
command may expect a specific fd to be available.

Then at least the information should be consistent and the MAKEFLAGS should
not mention jobserver FDs.

As is, it already messes up make itself - a make process invocated indirectly
may get valid filedescriptors *not* belonging to the jobserver, and start busy
looping on the "read(rfd, 1)" call (when the other end is closed and read
returns 0).

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #3, bug #57242 (project make):

I fully agree with Stafan that MAKEFLAGS should be aligned with jobserver
FDs.
My impression is that a much better approach would be the usage of named pipes
instead of file descriptors. I would consider the process relying on fixed
file descriptor numbers as a bad practice.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #4, bug #57242 (project make):

Hello, I wanted to add that I have written a jobserver client and am currently
getting burned by the issue described in this bug.

We have an LLVM plugin that is compute-intensive, so it wants to use multiple
cores. However, by the time execution reaches our plugin, fds 3 and 4 are in
use by LLVM even if they were closed on exec by make, making it hard to tell
whether these fds lead to the jobserver or not

Of course the problem only happens if people fail to mark LLVM as a recursive
make command, but this is a very easy mistake to make, and the consequences
(reading and writing bytes from whatever random files LLVM has open at the
moment) seem pretty unpleasant.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #5, bug #57242 (project make):

It is not TOO hard to provide a custom value (that doesn't include the
jobserver options) for the MAKEFLAGS environment variable we set when
forking/execing a command that is not a recursive make.

It's much trickier to reset the MAKEFLAGS make variable to not include those
options when expanding the command lines.  For example:


foo:
        +: $(MAKEFLAGS)
        +: $$MAKEFLAGS
        : $(MAKEFLAGS)
        : $$MAKEFLAGS


Ideally when run with -j you'd see the jobserver options in the first two
lines and you would NOT see them in the last two lines but this is hard.

Much simpler would be to show the jobserver options in the first three lines
but not show them in the last line.  For most uses this is probably sufficient
since many programs will be obtaining this information from their environment
anyway, not from the command line.  Still, it's an annoying "gotcha".

Regarding named pipes: yes this would be a good option (although it comes with
its own issues such as location and permissions); I didn't use this because I
wanted the solution to be maximally portable.  But, perhaps there are no
useful systems left that both (a) provide standard pipes sufficiently well to
support the jobserver and (b) don't provide named pipes.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #6, bug #57242 (project make):

I can't speak about the broader situation but Paul's "much simpler" option
would certainly be sufficient for my use case.

Something else that would be useful to us is a way to force make to consider
some or all subprograms to be recursive make invocations. The "+" convention
does not work well for us because we're working with third party (and usually
auto-generated) makefiles and it's no fun to mark these up with "+"
characters. I'm currently working around this using a one-line patch to make,
but that isn't something that I can generally expect other people to do.

Regarding the named pipe, I assume this would also solve the issue where
there's some uncertainty about the inherited jobserver fds being blocking vs.
non-blocking, because every client would simply open the pipe in whatever mode
suited it? If so, then I'm all for it.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


Reply | Threaded
Open this post in threaded view
|

[bug #57242] Non-recursive command passes invalid jobserver file descriptors

Paul D. Smith
Follow-up Comment #7, bug #57242 (project make):

About this statement below:

"This is necessary, because a command can mess up job server operation or a
command may expect a specific fd to be available."

a) I wonder how a command can mess up job server operation:

a1) One way would be that a program is looking at all open file descriptors it
inherited, and messing with them. I am not sure whether we want to make the
job server robust against such attacks.

a2) Another way is that the program has a bug and is not using the job server
properly. But I do not think that this scenario is worth much consideration
either.


b) If one really wants to stop inheriting the job server and the associated
flag, I wonder whether a makefile can filter out the job server options from
MAKEFLAGS.

I am actually doing the opposite, I am fetching the jobserver flags from
MAKEFLAGS and letting only those through. Check out variable
EXTRACT_SELECTED_FLAGS_FROM_MAKEFLAGS in this makefile:

https://github.com/rdiez/JtagDue/blob/master/Toolchain/Makefile

Would it be possible to use GNU Make function $(filter-out pattern...,text) to
remove the job server options?


c) I wonder how "a command may expect a specific fd to be available".

A while ago I wanted a way to always pass down the jobserver file
descriptors:

Passing jobserver file descriptors to all children
https://lists.gnu.org/archive/html/help-make/2020-02/msg00000.html

I encountered a similar statement then:

"Originally make did pass these file descriptors to all children, but there
were bugs filed because some tools invoked by make expected to have specific
file descriptors available; having make pass down open FDs caused them to
fail."

Can someone provide more detail about which file descriptors a program may
expect to be available? Because that is generally considered bad practice, as
far as I know.

If this mainly refers to Bash scripts, I found in its manual page that Bash
seems to use descriptors above 10 internally, as only the low 10 seem
guarantee stability for scripts:

"Redirections using file descriptors greater than 9 should be used with care,
as they may conflict with file descriptors the shell uses internally."

Maybe if GNU Make moved those job server descriptors to higher numbers, like
Bash does internally, there would be less problems in this area. I cannot
imagine that some program or script may assume that file descriptor number
12,345 must be available right now. But maybe I am missing something, I am no
expert in this area.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57242>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/