[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
URL:
  <https://savannah.gnu.org/bugs/?57014>

                 Summary: make-4.2.91 segfaults under Solaris 10 when many
files are involved
                 Project: make
            Submitted by: None
            Submitted on: Mon 07 Oct 2019 01:01:57 PM UTC
                Severity: 3 - Normal
              Item Group: Bug
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: SCM
        Operating System: POSIX-Based
           Fixed Release: None
           Triage Status: None

    _______________________________________________________

Details:

Hello,

With the following Makefile, make-4.2.91 segfaults on my Solaris 10:

% cat Makefile
include /dev/null
dummy: subdir/*.c
include /dev/null
%

This needs some preparation: many files in subdir, including at least one
subfolder:

% mkdir -p subdir/subsubdir
% touch `seq -f subdir/%5.0f 11001 12000` # seq does not exist on my solaris,
yet you understand the idea
% make -d
GNU Make 4.2.91
Built for sparc-sun-solaris2.10
Copyright (C) 1988-2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Reading makefile '/dev/null' (search path) (no ~ expansion)...
Segmentation fault
%

Of course the two include's are not necessary, but are useful to pinpoint the
bug.
This 'make' was built with '--disable-posix-spawn'.

I insist that when there is no subfolder in subdir, no segfault shows up.

I was unable to reproduce on linux and/or cygwin. I have no access on Solaris
11.

Regards,

Denis Excoffier.





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
Follow-up Comment #1, bug #57014 (project make):

Thanks for reporting.

That seems bad, but unfortunately there's nothing I can do with this report
as-is.  It doesn't fail on any system I have access to, so the only way it
will be addressed, other than by accident, is if someone can provide details
of the segmentation fault.

If someone can use a debugger to obtain a stacktrace where the segmentation
fault occurs that would be a great first step.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
Follow-up Comment #2, bug #57014 (project make):

Inserting many fprintf() incantations into the source code shows that
glob (name, GLOB_ALTDIRFUNC, NULL, &gl)
(line 3391 of src/read.c)
seems the responsible.




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: [bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

Gnu - Make - Bugs mailing list
In reply to this post by anonymous
On Mon, Oct 7, 2019 at 9:03 AM anonymous <[hidden email]> wrote:
> With the following Makefile, make-4.2.91 segfaults on my Solaris 10:
...
> include /dev/null
> dummy: subdir/*.c
> include /dev/null

This reproduces for me. This is the same read past the end of the
string inside sum_up_to_nul that is already fixed in git.
Denis, can you please pull the latest, built, run and report?

regards, Dmitry

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: [bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

Paul Smith-20
On Mon, 2019-10-07 at 11:26 -0400, Dmitry Goncharov wrote:

> On Mon, Oct 7, 2019 at 9:03 AM anonymous <[hidden email]>
> wrote:
> > With the following Makefile, make-4.2.91 segfaults on my Solaris
> > 10:
>
> ...
> > include /dev/null
> > dummy: subdir/*.c
> > include /dev/null
>
> This reproduces for me. This is the same read past the end of the
> string inside sum_up_to_nul that is already fixed in git.
> Denis, can you please pull the latest, built, run and report?

Unfortunately it's not easy to build from Git unless you have a full
suite of autotools available.

I am thinking of creating the official 4.3 release any day now (I was
going to do it yesterday but got sidetracked).  I can instead create a
new release candidate if people feel it's warranted.


I've often wished there was a straightforward way to generate "nightly
builds" (at least for nights when there have been changes pushed) that
we could point people at.

I have no problems making them, I just am not sure where to publish
them.  Pushing them to alpha.gnu.org is a lot of overhead but maybe
that's the right place anyway.


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: [bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

Gnu - Make - Bugs mailing list
On Mon, Oct 7, 2019 at 11:33 AM Paul Smith <[hidden email]> wrote:
> Unfortunately it's not easy to build from Git unless you have a full
> suite of autotools available.

Denis, if you cannot built from git, can you enable core dumps and run
dbx and post the backtrace here?
This is what is look like for me

$ dbx make core.make.1433.1570461665
Reading make
core file header read successfully
Reading ld.so.1
Reading libc.so.1
program terminated by signal SEGV (no mapping at the fault address)
0xffffffff7d882098: memcpy%sun4v-hwcap3+0x02a4: ldub     [%o1 - 2], %o3
Current function is jhash_string
  467       sum_up_to_nul(b, k, have_nul);
(dbx) where
  [1] memcpy%sun4v-hwcap3(0xffffffff7fffc1a0, 0xffffffff7d606002,
0xfffffffffffffffd, 0x0, 0xffffffff7fffdffe, 0x2), at
0xffffffff7d882098
=>[2] jhash_string(k = 0xffffffff7d605ffe "8"), line 467 in "hash.c"
  [3] dirfile_hash_1(key = 0xffffffff7fffc450), line 416 in "dir.c"
  [4] hash_find_slot(ht = 0x1001882a0, key = 0xffffffff7fffc450), line
89 in "hash.c"
  [5] dir_contents_file_exists_p(dir = 0x100188290, filename = (nil)),
line 719 in "dir.c"
  [6] open_dirstream(directory = 0xffffffff7fffc860 "subdir"), line
1202 in "dir.c"
  [7] glob_in_dir(pattern = 0x10017f087 "*.c", directory =
0xffffffff7fffc860 "subdir", flags = 512, errfunc = (nil), pglob =
0xffffffff7fffcc58), line 1299 in "glob.c"
  [8] glob(pattern = 0x10017f080 "subdir/*.c", flags = 512, errfunc =
(nil), pglob = 0xffffffff7fffcc58), line 1011 in "glob.c"
  [9] parse_file_seq(stringp = 0xffffffff7fffceb0, size = 40U, stopmap
= 257, prefix = (nil), flags = 0), line 3391 in "read.c"
  [10] split_prereqs(p = 0x100184ff0 "subdir/*.c"), line 445 in "file.c"
  [11] record_files(filenames = 0x100185010, are_also_makes = 0,
pattern = (nil), pattern_percent = (nil), depstr = 0x100184ff0
"subdir/*.c", cmds_started = 1U, commands = 0x1001860f0 "",
commands_idx = 0, two_colon = 0, prefix = '\t', flocp =
0xffffffff7fffd1d0), line 2006 in "read.c"
  [12] eval(ebuf = 0xffffffff7fffd480, set_default = 1), line 875 in "read.c"
  [13] eval_makefile(filename = 0x10017cfee "Makefile", flags = 0),
line 436 in "read.c"
  [14] read_all_makefiles(makefiles = 0x100161320), line 221 in "read.c"
  [15] main(argc = 3, argv = 0xffffffff7fffe6c8, envp =
0xffffffff7fffe6e8), line 1945 in "main.c"
(dbx) frame 2
Current function is jhash_string
  467       sum_up_to_nul(b, k, have_nul);
(dbx) print k
k = 0xffffffff7d605ffe "8"
(dbx) print k+4
k+4 = 0xffffffff7d606002 "<bad address 0xffffffff7d606002>"


regards, Dmitry

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: [bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

Dennis Clarke
In reply to this post by Paul Smith-20
On 2019-10-07 15:33, Paul Smith wrote:

> On Mon, 2019-10-07 at 11:26 -0400, Dmitry Goncharov wrote:
>> On Mon, Oct 7, 2019 at 9:03 AM anonymous <[hidden email]>
>> wrote:
>>> With the following Makefile, make-4.2.91 segfaults on my Solaris
>>> 10:
>>
>> ...
>>> include /dev/null
>>> dummy: subdir/*.c
>>> include /dev/null
>>
>> This reproduces for me. This is the same read past the end of the
>> string inside sum_up_to_nul that is already fixed in git.
>> Denis, can you please pull the latest, built, run and report?
>
> Unfortunately it's not easy to build from Git unless you have a full
> suite of autotools available.

Indeed this is true but having all those tools is mostly just a set of
steps and some effort.  Therefore let's not think of automake/autoconf
and friends as a barrier whatsoever.

> I am thinking of creating the official 4.3 release any day now (I was
> going to do it yesterday but got sidetracked).  I can instead create a
> new release candidate if people feel it's warranted.

Yes please.  With full enthusiasm.

> I've often wished there was a straightforward way to generate "nightly
> builds" (at least for nights when there have been changes pushed) that
> we could point people at.
>
> I have no problems making them, I just am not sure where to publish
> them.  Pushing them to alpha.gnu.org is a lot of overhead but maybe
> that's the right place anyway.

Firstly, this needs to be said. Thank you for your efforts and endless
diligence.  In my opinion GNU make is the beginning of all things to be
built on just about any system that is able.  It has become the defacto
standard 'make' and that has been true for at least a decade. Let's not
rush headlong without a great deal of care. Another release candidate is
a great idea. Consider that GNU Make 4.2.1 has done well for a long time
now and whatever release we work towards today will become the baseline
starting point of toolchains everywhere. We should take time and care.


--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
In reply to this post by anonymous
Follow-up Comment #3, bug #57014 (project make):

Please have a look into src/hash.c, in sum_up_to_nul(). The memcpy() (line
416) segfaults because the bytes after \0 are not readable, in the case where
the filename (or whatever is hashed here) ends immediately (or almost
immediately) before the boundary.

I replaced 'memcpy(&val, (p), 4);' with

val = 0;
if (p[0] == 0) {
  memcpy(&val, (p), 1);
} else if (p[1] == 0) {
  memcpy(&val, (p), 2);
} else if (p[2] == 0) {
  memcpy(&val, (p), 3);
} else {
  memcpy(&val, (p), 4);
};

with success.

Another solution is to always manage 3 bytes after \0 for the hashed strings.

I suppose this sheds some (more) light on the subject.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
Update of bug #57014 (project make):

                  Status:                    None => Duplicate              
             Open/Closed:                    Open => Closed                

    _______________________________________________________

Follow-up Comment #4:

Thanks for the investigation.  I suppose you are not subscribed to the
bug-make mailing list and without an email address here we couldn't make
contact directly.

There was some discussion on bug-make as to whether this was a duplicate of a
known issue; your comment below confirms that it is.  This problem has already
been fixed in the Git source repo and will be available in the next
release/release candidate.

Thanks for reporting and debugging!

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

[bug #57014] make-4.2.91 segfaults under Solaris 10 when many files are involved

anonymous
Follow-up Comment #5, bug #57014 (project make):

Thank you. Indeed it is solved in 4.2.92.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57014>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make