RE: Strange parallel behavior noted on Linux kernel sources wrt CPU threading

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: Strange parallel behavior noted on Linux kernel sources wrt CPU threading

Jason Mancini
So this has been observed as gnu make not reaping defunct make/sh processes on openSUSE Tumbleweed.  Presumably all the tokens in the shared jobserver read/write fds pipe become tied up by said defunct process, and then make behaves as -j1 at that point.   It doesn't happen every time, so it's very time consuming to debug.  Eventually the defunct processes are reaped and the build completes mostly in a linear fashion.  Am investigating if Ubuntu or Fedora demonstrate the same behavior.

Is there some new glibc or kernel code or compile options that could trigger this sort of (mis-)behavior?  Note that Tumbleweed is on gcc8.

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Strange parallel behavior noted on Linux kernel sources wrt CPU threading

Paul Smith-20
On Sat, 2018-07-07 at 10:01 +0000, Jason Mancini wrote:

> So this has been observed as gnu make not reaping defunct make/sh
> processes on openSUSE Tumbleweed.  Presumably all the tokens in the
> shared jobserver read/write fds pipe become tied up by said defunct
> process, and then make behaves as -j1 at that point.   It doesn't
> happen every time, so it's very time consuming to debug.  Eventually
> the defunct processes are reaped and the build completes mostly in a
> linear fashion.  Am investigating if Ubuntu or Fedora demonstrate the
> same behavior.
>
> Is there some new glibc or kernel code or compile options that could
> trigger this sort of (mis-)behavior?  Note that Tumbleweed is on
> gcc8.

My best guess is that this is related to an issue with pselect().

If you want to try building GNU make without pselect() support, you can
use this configure command:

  $ make clean
  $ ac_cv_func_pselect=no ./configure
  $ make

There has been a fix made to GNU make since 4.2.1 was released which
could be related to the problem you're seeing.

Unfortunately building GNU make from Git these days requires a bit more
infrastructure than it used to: you'd need to have a number of other
packages installed.  The steps are well-described (I hope!) in the
README.git file in the GNU make Git repository:

https://savannah.gnu.org/git/?group=make

Alternatively you can try applying just that change and see if it
helps; you can find it here:

http://git.savannah.gnu.org/cgit/make.git/commit/?id=b552b05251980f693c729e251f93f5225b400714

It may require some tweaking to apply correctly.

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

RE: Strange parallel behavior noted on Linux kernel sources wrt CPU threading

Jason Mancini
In reply to this post by Jason Mancini
Thanks Paul!  I pulled make from git, copied those latest *.{c,h} files over the top of a 4.2.1 tarball.  Make is working correctly now.  I'll check for / file an openSUSE Tumbleweed tracking bug.

-JasonM

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Strange parallel behavior noted on Linux kernel sources wrt CPU threading

Paul Smith-20
On Sat, 2018-07-07 at 18:54 +0000, Jason Mancini wrote:
> Thanks Paul!  I pulled make from git, copied those latest *.{c,h}
> files over the top of a 4.2.1 tarball.  Make is working correctly
> now.  I'll check for / file an openSUSE Tumbleweed tracking bug.
> -JasonM

That's good to hear.

FYI, your mail client is not preserving References headers properly
when you reply, which breaks thread tracking.  It would be good if you
could request that whomever is managing your mailer software fix that
issue... it's a problem when working on mailing lists.

Cheers!

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make