GMAKE 3.81 vs GMAKE 4.2

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Hi,

I am seeing very different behaviors in my build between make version 3.81
and 4.2

My build compiles correctly but fails with segmentation fault on 4.2
version.
My build compiles correctly and executes also correctly on 3.81 version.

The build failed with error message related to THREADS.

Is there some change in 3.81 vs 4.2 related to threads ?

Please reply. It is urgent for me to provide a Correct compiled product to
my users.
I do not want to switch back to 3.81.


Thanks
Nikhil
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Kaz Kylheku (gmake)
On 2019-09-27 13:23, nikhil jain wrote:
> Please reply. It is urgent for me to provide a Correct compiled product
> to
> my users.
> I do not want to switch back to 3.81.

Could this be related to parallelization with "-j <jobs>"?

If you put a

  .NOTPARALLEL:

declaration at the top of your Makefile, does the problem persist?
If not, that could be a workaround.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Do you mean to not use -j ?

That means a single rule execution at a time. It takes 2 hours to build.
With -j4 it takes 1 hour.

I have -j kind of unlimited. So it usually takes around just 5 minutes to
build. Less than that sometimes.

What exactly is the bug ? And what is the fix in 4.2 ?

Thanks
Nikhil

Need help.

On Sat, 28 Sep 2019, 03:58 Kaz Kylheku (gmake), <[hidden email]>
wrote:

> On 2019-09-27 13:23, nikhil jain wrote:
> > Please reply. It is urgent for me to provide a Correct compiled product
> > to
> > my users.
> > I do not want to switch back to 3.81.
>
> Could this be related to parallelization with "-j <jobs>"?
>
> If you put a
>
>   .NOTPARALLEL:
>
> declaration at the top of your Makefile, does the problem persist?
> If not, that could be a workaround.
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Gnu - Make - Help mailing list


> On 28 Sep 2019, at 16:21, nikhil jain <[hidden email]> wrote:
>
> Do you mean to not use -j ?
>
> That means a single rule execution at a time. It takes 2 hours to build.
> With -j4 it takes 1 hour.
>
> I have -j kind of unlimited. So it usually takes around just 5 minutes to
> build. Less than that sometimes.

Does the error occur if you set a large number of jobs like -j 64, or use the
--load-average (-l) flag as well?

> What exactly is the bug ? And what is the fix in 4.2 ?

Can you narrow it down? Does it always fail at the same targets? Is it a hand-written
Makefile or generated from automake/cmake? Does it succeed with subsequent invocations
of make?

Cheers,

Tony


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Hi Tony,

1) I always set -j 4.
2) No, I do not set -l flag.
3) The BUILD never fails. The binary which is generated causes segmentation
fault. It works perfectly If I build in 3.81 GMAKE.
4) Makefile is handwritten.
5) No, It never succeed on any number of invocation.

Just to again notify -

There are 2 parts to this problem -

1) Build the code using v3.81 and v4.2 -> In both cases, builds is
happening correctly.
2) Segmentation fault -> While running the binary generated from the build,
it segfaults with v3.81 and works fine with v4.2

I hope you will provide some resolution.

Thanks
Nikhil

On Sun, Sep 29, 2019 at 6:20 AM Tony Theodore <[hidden email]> wrote:

>
>
> > On 28 Sep 2019, at 16:21, nikhil jain <[hidden email]> wrote:
> >
> > Do you mean to not use -j ?
> >
> > That means a single rule execution at a time. It takes 2 hours to build.
> > With -j4 it takes 1 hour.
> >
> > I have -j kind of unlimited. So it usually takes around just 5 minutes to
> > build. Less than that sometimes.
>
> Does the error occur if you set a large number of jobs like -j 64, or use
> the
> --load-average (-l) flag as well?
>
> > What exactly is the bug ? And what is the fix in 4.2 ?
>
> Can you narrow it down? Does it always fail at the same targets? Is it a
> hand-written
> Makefile or generated from automake/cmake? Does it succeed with subsequent
> invocations
> of make?
>
> Cheers,
>
> Tony
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Paul Smith-20
On Mon, 2019-09-30 at 13:51 +0530, nikhil jain wrote:
> There are 2 parts to this problem -
>
> 1) Build the code using v3.81 and v4.2 -> In both cases, builds is
> happening correctly.
> 2) Segmentation fault -> While running the binary generated from the build,
> it segfaults with v3.81 and works fine with v4.2
>
> I hope you will provide some resolution.

I don't see how there's anything we can do about this.  It's not a
problem in GNU make, it's a problem in your makefile, which you haven't
shown us (and based on the reported size of the project, we don't have
the capacity to debug even if you did show us).

GNU make doesn't compile, assemble, or link any code: it simply invokes
shell commands that do whatever your recipes tell them to do.  If GNU
make doesn't fail then it successfully invoked all your recipes: that's
all it knows how to do and the only responsibility it has.

There are a number of reasons that using different GNU make versions
might have this effect but all of them are specific to your environment
and your makefiles and your code.

For example, maybe you're using $(wildcard ...) to gather filenames and
the difference in the order returned is causing your code to be linked
differently (in 3.81 I believe wildcard also sorted output while in
4.2.1 it doesn't--however in 4.3 it will again).

Or maybe something else even more obscure.

For a correct build system none of these things should matter (anywhere
order does matter, you should be specifying it explicitly) but clearly
there's something about your system which is not deterministic.

Your only choices are to either (a) continue to use 3.81, or (b)
investigate the crash with a debugger or similar and discover why it
happens and then figure out what part of your makefile is not
repeatable and fix it.

Or (c), you could try using the pre-release for the next version of GNU
make and see if the problem disappears as mysteriously as it appeared:

https://alpha.gnu.org/gnu/make/make-4.2.91.tar.gz

If it were me I'd prefer (b): just because it gets fixed in a newer
version doesn't mean it won't break again if you don't understand the
reason and stop it from happening.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
For example, maybe you're using $(wildcard ...) to gather filenames and
the difference in the order returned is causing your code to be linked
differently (in 3.81 I believe wildcard also sorted output while in
4.2.1 it doesn't--however in 4.3 it will again).

Do you confirm the above behavior can impact the binary we are building ?
If Yes, I was hoping all the Make Versions to be backward compatible. I had
compiled with make 4.0 without any issue. Looks like something has changed
significantly in 4.2 ?

Thanks
Nikhil

On Mon, Sep 30, 2019 at 6:30 PM Paul Smith <[hidden email]> wrote:

> On Mon, 2019-09-30 at 13:51 +0530, nikhil jain wrote:
> > There are 2 parts to this problem -
> >
> > 1) Build the code using v3.81 and v4.2 -> In both cases, builds is
> > happening correctly.
> > 2) Segmentation fault -> While running the binary generated from the
> build,
> > it segfaults with v3.81 and works fine with v4.2
> >
> > I hope you will provide some resolution.
>
> I don't see how there's anything we can do about this.  It's not a
> problem in GNU make, it's a problem in your makefile, which you haven't
> shown us (and based on the reported size of the project, we don't have
> the capacity to debug even if you did show us).
>
> GNU make doesn't compile, assemble, or link any code: it simply invokes
> shell commands that do whatever your recipes tell them to do.  If GNU
> make doesn't fail then it successfully invoked all your recipes: that's
> all it knows how to do and the only responsibility it has.
>
> There are a number of reasons that using different GNU make versions
> might have this effect but all of them are specific to your environment
> and your makefiles and your code.
>
> For example, maybe you're using $(wildcard ...) to gather filenames and
> the difference in the order returned is causing your code to be linked
> differently (in 3.81 I believe wildcard also sorted output while in
> 4.2.1 it doesn't--however in 4.3 it will again).
>
> Or maybe something else even more obscure.
>
> For a correct build system none of these things should matter (anywhere
> order does matter, you should be specifying it explicitly) but clearly
> there's something about your system which is not deterministic.
>
> Your only choices are to either (a) continue to use 3.81, or (b)
> investigate the crash with a debugger or similar and discover why it
> happens and then figure out what part of your makefile is not
> repeatable and fix it.
>
> Or (c), you could try using the pre-release for the next version of GNU
> make and see if the problem disappears as mysteriously as it appeared:
>
> https://alpha.gnu.org/gnu/make/make-4.2.91.tar.gz
>
> If it were me I'd prefer (b): just because it gets fixed in a newer
> version doesn't mean it won't break again if you don't understand the
> reason and stop it from happening.
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Kaz Kylheku (gmake)
In reply to this post by nikhil jain
On 2019-09-30 01:21, nikhil jain wrote:
> Hi Tony,
>
> 1) I always set -j 4.
> 2) No, I do not set -l flag.
> 3) The BUILD never fails. The binary which is generated causes
> segmentation
> fault. It works perfectly If I build in 3.81 GMAKE.

Possibly, what is going on that you have some $(wildcard ...) material
in
the Makefile. In 3.81, the output of wildcard is lexicographically
sorted.
In newer GNU Make, a change was made not to sort this. The users have to
apply
the $(sort ...) function explicitly.

This can make a difference because if you have, say, a list of object
files generated from source files, then the linking order of your
program will
change; you've scrambled the image compared to earlier builds. Moreover,
it will be different on different build machines.

Such a change can expose bugs.

> 4) Makefile is handwritten.
> 5) No, It never succeed on any number of invocation.

You can simply capture the full command sequence from the working build
with the old gmake and the newer build and compare. There has to be some
difference in how some toolchain command is being invoked.

That order issue is the first thing that comes to my mind; but it could
be something else.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Paul Smith-20
In reply to this post by nikhil jain
It would be helpful if you could teach your email client to properly
quote replies (or find a better one).  Also we prefer inline responses
rather than top-quoting.

On Mon, 2019-09-30 at 19:48 +0530, nikhil jain wrote:
> For example, maybe you're using $(wildcard ...) to gather filenames
> and the difference in the order returned is causing your code to be
> linked differently (in 3.81 I believe wildcard also sorted output
> while in 4.2.1 it doesn't--however in 4.3 it will again).
>
> Do you confirm the above behavior can impact the binary we are
> building ?

As I tried to point out before, make just runs commands.  It doesn't
even know which of those commands might be compiling code, linking
code, or generating documentation, etc.

So if your particular makefile is written to assume a specific order of
files, then of course changing the order of files will impact the
build.  If your makefile does depend on that order, then it should
hardcode the order of files rather than leaving it up to chance.  Only
you can find out whether that order does or does not matter.

> If Yes, I was hoping all the Make Versions to be backward compatible.
> I had compiled with make 4.0 without any issue. Looks like something
> has changed significantly in 4.2 ?

There are hundreds of thousands of programs (at least) using GNU make
out there, and they all still work.

If you are successful using GNU make 4.0, then it's not related to the
wildcard function sorting: that was changed in 3.82.

We've given you three options for how to find the problem:

1. Try the pre-release of the new version of GNU make.  The downside of
this is that if it works you still don't know what's wrong with your
makefiles and it might break again in the future.

2. Debug the crashing program and figure out why it's crashing, and
determine how that could have been caused by the way the code was
built.  Then either change your code to fix it, or change makefile to
be sure it can't happen again.

3. As Kaz suggested, look very carefully at the full output of the
"working" and "non-working" build logs (compile commands, arguments,
etc.) and check for differences and try to figure out which of those
differences might be causing the problem, then change your makefile to
make sure those differences don't happen in the future.

You might be able to combine #2 and #3.

That's all we can tell you.  You're going to have to do the work to
solve this yourself: it's your makefile and your build system.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
RELAX!

I will do that. Thanks for your help. I will revert back with my findings
and ask for help if required.

Thanks
Nikhil

On Mon, Sep 30, 2019 at 9:23 PM Paul Smith <[hidden email]> wrote:

> It would be helpful if you could teach your email client to properly
> quote replies (or find a better one).  Also we prefer inline responses
> rather than top-quoting.
>
> On Mon, 2019-09-30 at 19:48 +0530, nikhil jain wrote:
> > For example, maybe you're using $(wildcard ...) to gather filenames
> > and the difference in the order returned is causing your code to be
> > linked differently (in 3.81 I believe wildcard also sorted output
> > while in 4.2.1 it doesn't--however in 4.3 it will again).
> >
> > Do you confirm the above behavior can impact the binary we are
> > building ?
>
> As I tried to point out before, make just runs commands.  It doesn't
> even know which of those commands might be compiling code, linking
> code, or generating documentation, etc.
>
> So if your particular makefile is written to assume a specific order of
> files, then of course changing the order of files will impact the
> build.  If your makefile does depend on that order, then it should
> hardcode the order of files rather than leaving it up to chance.  Only
> you can find out whether that order does or does not matter.
>
> > If Yes, I was hoping all the Make Versions to be backward compatible.
> > I had compiled with make 4.0 without any issue. Looks like something
> > has changed significantly in 4.2 ?
>
> There are hundreds of thousands of programs (at least) using GNU make
> out there, and they all still work.
>
> If you are successful using GNU make 4.0, then it's not related to the
> wildcard function sorting: that was changed in 3.82.
>
> We've given you three options for how to find the problem:
>
> 1. Try the pre-release of the new version of GNU make.  The downside of
> this is that if it works you still don't know what's wrong with your
> makefiles and it might break again in the future.
>
> 2. Debug the crashing program and figure out why it's crashing, and
> determine how that could have been caused by the way the code was
> built.  Then either change your code to fix it, or change makefile to
> be sure it can't happen again.
>
> 3. As Kaz suggested, look very carefully at the full output of the
> "working" and "non-working" build logs (compile commands, arguments,
> etc.) and check for differences and try to figure out which of those
> differences might be causing the problem, then change your makefile to
> make sure those differences don't happen in the future.
>
> You might be able to combine #2 and #3.
>
> That's all we can tell you.  You're going to have to do the work to
> solve this yourself: it's your makefile and your build system.
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Christian Hujer
In reply to this post by nikhil jain
Hello Nikhil,

On Sat, Sep 28, 2019 at 1:54 AM nikhil jain <[hidden email]> wrote:

> I am seeing very different behaviors in my build between make version 3.81
> and 4.2
>
> My build compiles correctly but fails with segmentation fault on 4.2
> version.
> My build compiles correctly and executes also correctly on 3.81 version.
>
As others have pointed out, without knowing which process segmentation
faults, it is impossible to know.
Also, as others have pointed out, GNU make does not create any artifacts,
it only orchestrates the actual tools like preprocessors, compilers,
assemblers, librarians, linkers which create the actual artifacts.
It is much more likely that the problem was in the Makefile already, and
that the switch from GNU make 3.81 to GNU make 4.2 only triggered the issue.
For example, when running with -j, there could be a race condition from the
dependency tree. The update can shift the timing just slightly enough to
trigger the issue.

If you want to really know what's going on, it's probably best to install
GNU make 3.81 and GNU make 4.2 in parallel.
You can run your build in one directory with GNU make 3.81 and in the other
directory with GNU make 4.2.
Then use a tool like diff to compare the binary files created by the build.
Depending on your build tools, they may or may not include timestamps and
absolute paths in the files.
I hope your build tools don't, because that would make comparing more
difficult, you have to take that into account.
That comparison can show you what runs differently.

Another go could be to run make without -s.
I hope that your Makefile doesn't make extensive use of command echo
suppression.
Command echo suppression should not be used as a replacement of -s, it
makes Makefiles difficult to debug.
Run both versions of GNU make in parallel and compare the output and run
sequence to find out where your Makefile and your tools go wrong with the
new version of make.

I know you didn't want to install the old version of make.
But if you seriously want to understand what is going on and where the
problem is, I fear that there is no other way.
And in case your Makefile failes, I suggest you better do. Because if you
just hotfix the problem to go away without properly understanding it,
there's a chance it comes back with the next update or even other unrelated
changes to your setup.

The build failed with error message related to THREADS.
>
Is that message coming from GNU make itself or from one of the tools
invoked by GNU make?

Is there some change in 3.81 vs 4.2 related to threads ?
>
When it comes to threads, for sure there is, even if the logic is not
changed, the other changes will have an influence on the timing behavior.
However, such changes should be fine if the Makefile is written correctly,
such changes happen between all versions, and such changes in timing can
even happen with the same binary if you run on a different CPU, or with
different processes in parallel, or even just replace RAM.

Please reply. It is urgent for me to provide a Correct compiled product to
> my users.
> I do not want to switch back to 3.81.
>
Who wants to, GNU make 4.2 is so much better than 3.81.

--
Christian Hujer
E: [hidden email]
T: @christianhujer
+91 77 2003 6661
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Hi,

Thanks for the answers.

You know what!

It worked with GMAKE 4.2.91

It is the test version sent to me in the mail thread by a GMAKE expert. I
would probably move to it.
So, it looks like the wildcards issue then ?

thanks
Nikhil

On Tue, Oct 1, 2019 at 1:44 AM Christian Hujer <[hidden email]>
wrote:

> Hello Nikhil,
>
> On Sat, Sep 28, 2019 at 1:54 AM nikhil jain <[hidden email]>
> wrote:
>
>> I am seeing very different behaviors in my build between make version 3.81
>> and 4.2
>>
>> My build compiles correctly but fails with segmentation fault on 4.2
>> version.
>> My build compiles correctly and executes also correctly on 3.81 version.
>>
> As others have pointed out, without knowing which process segmentation
> faults, it is impossible to know.
> Also, as others have pointed out, GNU make does not create any artifacts,
> it only orchestrates the actual tools like preprocessors, compilers,
> assemblers, librarians, linkers which create the actual artifacts.
> It is much more likely that the problem was in the Makefile already, and
> that the switch from GNU make 3.81 to GNU make 4.2 only triggered the issue.
> For example, when running with -j, there could be a race condition from
> the dependency tree. The update can shift the timing just slightly enough
> to trigger the issue.
>
> If you want to really know what's going on, it's probably best to install
> GNU make 3.81 and GNU make 4.2 in parallel.
> You can run your build in one directory with GNU make 3.81 and in the
> other directory with GNU make 4.2.
> Then use a tool like diff to compare the binary files created by the build.
> Depending on your build tools, they may or may not include timestamps and
> absolute paths in the files.
> I hope your build tools don't, because that would make comparing more
> difficult, you have to take that into account.
> That comparison can show you what runs differently.
>
> Another go could be to run make without -s.
> I hope that your Makefile doesn't make extensive use of command echo
> suppression.
> Command echo suppression should not be used as a replacement of -s, it
> makes Makefiles difficult to debug.
> Run both versions of GNU make in parallel and compare the output and run
> sequence to find out where your Makefile and your tools go wrong with the
> new version of make.
>
> I know you didn't want to install the old version of make.
> But if you seriously want to understand what is going on and where the
> problem is, I fear that there is no other way.
> And in case your Makefile failes, I suggest you better do. Because if you
> just hotfix the problem to go away without properly understanding it,
> there's a chance it comes back with the next update or even other unrelated
> changes to your setup.
>
> The build failed with error message related to THREADS.
>>
> Is that message coming from GNU make itself or from one of the tools
> invoked by GNU make?
>
> Is there some change in 3.81 vs 4.2 related to threads ?
>>
> When it comes to threads, for sure there is, even if the logic is not
> changed, the other changes will have an influence on the timing behavior.
> However, such changes should be fine if the Makefile is written correctly,
> such changes happen between all versions, and such changes in timing can
> even happen with the same binary if you run on a different CPU, or with
> different processes in parallel, or even just replace RAM.
>
> Please reply. It is urgent for me to provide a Correct compiled product to
>> my users.
>> I do not want to switch back to 3.81.
>>
> Who wants to, GNU make 4.2 is so much better than 3.81.
>
> --
> Christian Hujer
> E: [hidden email]
> T: @christianhujer
> +91 77 2003 6661
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Also, is there some change in how the commands will be displayed while
running make.

I saw some strange behavior -

1) Starting version 3.82, the builds are failing. They were successful in
the latest alpha release of 4.2.91
2) Also till 3.81 the commands are being displayed properly when make is
running. Is there some change after 3.81 with respect to the commands/rules
being displayed while running the Makefile ?

Please respond.

Thanks
Nikhil

On Tue, Oct 1, 2019 at 10:09 AM nikhil jain <[hidden email]> wrote:

> Hi,
>
> Thanks for the answers.
>
> You know what!
>
> It worked with GMAKE 4.2.91
>
> It is the test version sent to me in the mail thread by a GMAKE expert. I
> would probably move to it.
> So, it looks like the wildcards issue then ?
>
> thanks
> Nikhil
>
> On Tue, Oct 1, 2019 at 1:44 AM Christian Hujer <[hidden email]>
> wrote:
>
>> Hello Nikhil,
>>
>> On Sat, Sep 28, 2019 at 1:54 AM nikhil jain <[hidden email]>
>> wrote:
>>
>>> I am seeing very different behaviors in my build between make version
>>> 3.81
>>> and 4.2
>>>
>>> My build compiles correctly but fails with segmentation fault on 4.2
>>> version.
>>> My build compiles correctly and executes also correctly on 3.81 version.
>>>
>> As others have pointed out, without knowing which process segmentation
>> faults, it is impossible to know.
>> Also, as others have pointed out, GNU make does not create any artifacts,
>> it only orchestrates the actual tools like preprocessors, compilers,
>> assemblers, librarians, linkers which create the actual artifacts.
>> It is much more likely that the problem was in the Makefile already, and
>> that the switch from GNU make 3.81 to GNU make 4.2 only triggered the issue.
>> For example, when running with -j, there could be a race condition from
>> the dependency tree. The update can shift the timing just slightly enough
>> to trigger the issue.
>>
>> If you want to really know what's going on, it's probably best to install
>> GNU make 3.81 and GNU make 4.2 in parallel.
>> You can run your build in one directory with GNU make 3.81 and in the
>> other directory with GNU make 4.2.
>> Then use a tool like diff to compare the binary files created by the
>> build.
>> Depending on your build tools, they may or may not include timestamps and
>> absolute paths in the files.
>> I hope your build tools don't, because that would make comparing more
>> difficult, you have to take that into account.
>> That comparison can show you what runs differently.
>>
>> Another go could be to run make without -s.
>> I hope that your Makefile doesn't make extensive use of command echo
>> suppression.
>> Command echo suppression should not be used as a replacement of -s, it
>> makes Makefiles difficult to debug.
>> Run both versions of GNU make in parallel and compare the output and run
>> sequence to find out where your Makefile and your tools go wrong with the
>> new version of make.
>>
>> I know you didn't want to install the old version of make.
>> But if you seriously want to understand what is going on and where the
>> problem is, I fear that there is no other way.
>> And in case your Makefile failes, I suggest you better do. Because if you
>> just hotfix the problem to go away without properly understanding it,
>> there's a chance it comes back with the next update or even other unrelated
>> changes to your setup.
>>
>> The build failed with error message related to THREADS.
>>>
>> Is that message coming from GNU make itself or from one of the tools
>> invoked by GNU make?
>>
>> Is there some change in 3.81 vs 4.2 related to threads ?
>>>
>> When it comes to threads, for sure there is, even if the logic is not
>> changed, the other changes will have an influence on the timing behavior.
>> However, such changes should be fine if the Makefile is written
>> correctly, such changes happen between all versions, and such changes in
>> timing can even happen with the same binary if you run on a different CPU,
>> or with different processes in parallel, or even just replace RAM.
>>
>> Please reply. It is urgent for me to provide a Correct compiled product to
>>> my users.
>>> I do not want to switch back to 3.81.
>>>
>> Who wants to, GNU make 4.2 is so much better than 3.81.
>>
>> --
>> Christian Hujer
>> E: [hidden email]
>> T: @christianhujer
>> +91 77 2003 6661
>>
>>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Paul Smith-20
On Thu, 2019-10-03 at 20:54 +0530, nikhil jain wrote:

> Also, is there some change in how the commands will be displayed
> while running make.
>
> I saw some strange behavior -
>
> 1) Starting version 3.82, the builds are failing. They were
> successful in the latest alpha release of 4.2.91
> 2) Also till 3.81 the commands are being displayed properly when make
> is running. Is there some change after 3.81 with respect to the
> commands/rules being displayed while running the Makefile ?

Since there is no information provided about the difference in output
that you see, all I can say is I can't think of any relevant change.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
In reply to this post by nikhil jain
Just to confirm -

My build works with all the make versions. It's the binary which fails.

Binary works with only 3.81 and the alpha test release you gave.

It fails with versions starting from 3.82 to 4.2.

So it looks like the wild card issue as you mention.

Unfortunately there is one more issue which is not solved in 4.2 and alpha
release.
I don't know if its a make issue or makefile issue.

I use command line option VERBOSE to enable and disable rule display on
screen while building. I usually check if I pass verbose as false then I
append @ to the command so make doesn't display it.

But when I pass true, still it doesn't display the command.

With 3.81, verbose true and false works.
So, I was just wondering if there is some change in command display
starting version 3.82 ?

Any information will be very useful to me at this point. I do not really
want to go back to 3.81 and implement the remote execution again.

Help.

Thanks
Nikhil





On Fri, 4 Oct 2019, 02:59 Paul Smith, <[hidden email]> wrote:

> On Tue, 2019-10-01 at 10:09 +0530, nikhil jain wrote:
> > It is the test version sent to me in the mail thread by a GMAKE
> > expert. I would probably move to it.  So, it looks like the wildcards
> > issue then ?
>
> If your build worked properly with 3.82 or 4.0, then it was not caused
> by the change to the wildcard function.  That change was made in GNU
> make 3.82.
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Kaz Kylheku (gmake)
On 2019-10-03 19:06, nikhil jain wrote:
> Just to confirm -
>
> My build works with all the make versions. It's the binary which fails.

Then, I think, perhaps it would be good to avoid phrasing like
"build fails". Clearly, that is the opposite of "build works" that
you're using here.

> With 3.81, verbose true and false works.
> So, I was just wondering if there is some change in command display
> starting version 3.82 ?

Since mechanisms like VERBOSE=y are implemented entirely inside the
Makefiles themselves, whoever is reproducing a problem like this
has to narrow it down more. Not necessarily root cause it, but find
a minimal-ish repro test case that shows the change in behavior.

Lots of projects have elaborate Makefiles with condensed command
printing and VERBOSE type flags. If the @ mechanism simply broke,
there would be widespread groaning.



_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Paul Smith-20
In reply to this post by nikhil jain
On Fri, 2019-10-04 at 07:36 +0530, nikhil jain wrote:
> So it looks like the wild card issue as you mention.

If that is indeed the case, it means your build system is relying on
the sorted order of some files retrieved by wildcard in order to build
properly.

That's a problem: it makes your build system very brittle.  What if new
files are added, or existing files are renamed, which break things
again?  I recommend that you make an attempt to discover why this is
the case and use a specific list of files with a well-known order
(probably with a comment or two in the makefile explaining why the
ordering requirement exists) rather than relying on the order returned
by wildcard.

You can test this by going through your makefiles and replacing
instances of $(wildcard ...) with $(sort $(wildcard ...)) then trying
to build and test with GNU make 3.82 or 4.2.1 etc. and determine when
it starts working.

> I use command line option VERBOSE to enable and disable rule display on
> screen while building. I usually check if I pass verbose as false then I
> append @ to the command so make doesn't display it.
>
> But when I pass true, still it doesn't display the command.

Unfortunately these abstract explanations of problems don't allow us to
help.

Given the above description, the way *I* would implement it works the
same in GNU make 3.81 and 4.2.1 and all other versions of GNU make.

We need to know exactly how you "use command line option VERBOSE",
exactly how you "check if verbose [is] false", and exactly how you
"append @ to the command".

Please construct a small reproducible case you can post here which
shows the problem, and show us the make command line you invoked, the
output you got, and explain how that's different from the output you
wanted.  Then we can help.

Cheers!


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
Thank you for explanation.
Actually these are legacy makefiles. Those who created them might not be in
the company even.

But still, we have asked R&D to find out what exactly makefile is doing and
why it is dependent on the sequence and how to avoid it.

Also, I have asked the user to create a minimal example of how to use
verbose option and provide a reproducible case.

Just for an update - I have moved to 3.81 for now with a blind guess that
all will work fine in 3.81 and nothing will break.

Can you tell when the make 4.3 will be official ?

I have to move to 4.3 after all the regression testing on the binaries
generated by the build.

If I share a makefile then can you debug that ?


On Sat, 5 Oct 2019, 19:35 Paul Smith, <[hidden email]> wrote:

> On Fri, 2019-10-04 at 07:36 +0530, nikhil jain wrote:
> > So it looks like the wild card issue as you mention.
>
> If that is indeed the case, it means your build system is relying on
> the sorted order of some files retrieved by wildcard in order to build
> properly.
>
> That's a problem: it makes your build system very brittle.  What if new
> files are added, or existing files are renamed, which break things
> again?  I recommend that you make an attempt to discover why this is
> the case and use a specific list of files with a well-known order
> (probably with a comment or two in the makefile explaining why the
> ordering requirement exists) rather than relying on the order returned
> by wildcard.
>
> You can test this by going through your makefiles and replacing
> instances of $(wildcard ...) with $(sort $(wildcard ...)) then trying
> to build and test with GNU make 3.82 or 4.2.1 etc. and determine when
> it starts working.
>
> > I use command line option VERBOSE to enable and disable rule display on
> > screen while building. I usually check if I pass verbose as false then I
> > append @ to the command so make doesn't display it.
> >
> > But when I pass true, still it doesn't display the command.
>
> Unfortunately these abstract explanations of problems don't allow us to
> help.
>
> Given the above description, the way *I* would implement it works the
> same in GNU make 3.81 and 4.2.1 and all other versions of GNU make.
>
> We need to know exactly how you "use command line option VERBOSE",
> exactly how you "check if verbose [is] false", and exactly how you
> "append @ to the command".
>
> Please construct a small reproducible case you can post here which
> shows the problem, and show us the make command line you invoked, the
> output you got, and explain how that's different from the output you
> wanted.  Then we can help.
>
> Cheers!
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

Paul Smith-20
On Sat, 2019-10-05 at 19:55 +0530, nikhil jain wrote:
> Can you tell when the make 4.3 will be official ?

Should be within a few weeks.

> If I share a makefile then can you debug that ?

Speaking for myself, I don't have the time to understand and debug
large makefile environments.  If you can cut and paste one of the makefile rules that doesn't behave properly and the part of the makefile that checks the verbose flag, then show how make is invoked and what the output of that rule is, then we can try to help.

If you're not familiar enough with your makefiles or using make to do
that then we can wait for someone else to generate a test case.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: GMAKE 3.81 vs GMAKE 4.2

nikhil jain
So,

Finally, I migrated all the code to GMAKE 3.81 from GMAKE 4.2.... I noticed
that my build time is improved considerably in make 3.81 compared to make
4.2. Is this expected ? Upgrading gmake comes with a hit on build time ?
This is something not what i wanted but no other option. Thanks all for
your help. I will keep you posted for any issues I come across.

thanks
Nikhil

On Sat, Oct 5, 2019 at 8:09 PM Paul Smith <[hidden email]> wrote:

> On Sat, 2019-10-05 at 19:55 +0530, nikhil jain wrote:
> > Can you tell when the make 4.3 will be official ?
>
> Should be within a few weeks.
>
> > If I share a makefile then can you debug that ?
>
> Speaking for myself, I don't have the time to understand and debug
> large makefile environments.  If you can cut and paste one of the makefile
> rules that doesn't behave properly and the part of the makefile that checks
> the verbose flag, then show how make is invoked and what the output of that
> rule is, then we can try to help.
>
> If you're not familiar enough with your makefiles or using make to do
> that then we can wait for someone else to generate a test case.
>
>
_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
12