GNU Make for Java projects

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

GNU Make for Java projects

Blake McBride
Greetings,

I have used GNU Make for many years on C projects.  Over the last 10 or more years, I have been working mainly in Java.  There are several build tools that are popular in the Java world such as Ant, Maven, and Gradle.

I have found Make very easy to understand and use.  Given the amount of work they do in the background, I have found build tools such as Maven and Gradle to be very confusing.  They are easy and work well within a very narrow scope, but as soon as you try to go outside their typical pattern, they become inordinately complex or can't even handle some out-of-pattern situations.

It is my incomplete understanding that certain aspects of Make don't lend themselves well to the building of Java projects.  This is what has driven the quest for better Java fitting build tools.

Being frustrated by the difficulty of going outside of the Java build tools scope, I wondered whether it might be possible to enhance certain aspects of GNU Make to better accommodate the needs of a Java environment.  I also wonder if this issue had been previously raised and addressed without my knowledge.

Anyway, I thought I would raise the issue and possibly spark an interesting conversation.

Thank you.

Blake McBride



_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Henrik Carlqvist
On Fri, 18 Jan 2019 05:42:44 -0600
Blake McBride <[hidden email]> wrote:
> I have found Make very easy to understand and use.  Given the amount of
> work they do in the background, I have found build tools such as Maven
> and Gradle to be very confusing.

When I some years ago wrote Android apps I found eclipse too buggy to be
usable. Point and click might be nice, but far too often eclipse crashed.

I resorted to editing the source files in emacs and build the Android
projects with gnu Make, just as I was used to with C projects.

> It is my incomplete understanding that certain aspects of Make don't
> lend themselves well to the building of Java projects.  This is what has
> driven the quest for better Java fitting build tools.

Gnu Make is very simple, it uses some command to create something and does
that if something else indicates that it is needed. This simplicity makes
Make very useful for far more things than just compiling programs.

> I wondered whether it might be possible to enhance certain
> aspects of GNU Make to better accommodate the needs of a Java
> environment.

What kind of need would that be? The hardest thing for me migrating from
eclipse to gnu Make was finding out everything that was done below the
sheets by eclipse. In the end it was really just about running different
commands with arguments to generate files.

> Anyway, I thought I would raise the issue and possibly spark an
> interesting conversation.

My Android app projects are today abandoned, but if you want to study my
Makefile to genereate the .apk files from Java code you can download it
as described at http://halttimer.cvs.sourceforge.net/

regards Henrik

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Blake McBride
Thank you for your kind response.  

It seems to me, and as reported by others, that what Make does and what the Java build tools do have some fundamental differences.  Some that come to mind are as follows:

1.  When building C programs one calls the compiler for each source file.  This is impractical with Java because of the javac startup time.  Typically javac is called once with many source files at a time - but not the ones that are not out of date.

2.  Typically, javac will be called to compile all out of date files in a directory rather than having to explicitly name each one in the build file.

3.  Java build tools have the ability to download and cache files from remote repositories over the net.

Although I have more than 10 years experience working with Java, I have to admit that I am not completely clear about the Java build process since, on large project, I've used the IDE, ANT, Maven, or Gradle to do the work for me.

It is my semi-ignorant opinion that:

A.  Make is not a good tool for building large, modern Java programs.

B.  The things that would have to be added to make Make a practical Java / JavaScript build tool is not that much.

C.  (For JavaScript build tools) The ability to run continuously.  When Make detects a source file change it automatically kicks off a target file build.

Thanks.

Blake

On Fri, Jan 18, 2019 at 1:51 PM Henrik Carlqvist <[hidden email]> wrote:
On Fri, 18 Jan 2019 05:42:44 -0600
Blake McBride <[hidden email]> wrote:
> I have found Make very easy to understand and use.  Given the amount of
> work they do in the background, I have found build tools such as Maven
> and Gradle to be very confusing.

When I some years ago wrote Android apps I found eclipse too buggy to be
usable. Point and click might be nice, but far too often eclipse crashed.

I resorted to editing the source files in emacs and build the Android
projects with gnu Make, just as I was used to with C projects.

> It is my incomplete understanding that certain aspects of Make don't
> lend themselves well to the building of Java projects.  This is what has
> driven the quest for better Java fitting build tools.

Gnu Make is very simple, it uses some command to create something and does
that if something else indicates that it is needed. This simplicity makes
Make very useful for far more things than just compiling programs.

> I wondered whether it might be possible to enhance certain
> aspects of GNU Make to better accommodate the needs of a Java
> environment.

What kind of need would that be? The hardest thing for me migrating from
eclipse to gnu Make was finding out everything that was done below the
sheets by eclipse. In the end it was really just about running different
commands with arguments to generate files.

> Anyway, I thought I would raise the issue and possibly spark an
> interesting conversation.

My Android app projects are today abandoned, but if you want to study my
Makefile to genereate the .apk files from Java code you can download it
as described at http://halttimer.cvs.sourceforge.net/

regards Henrik

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Andrea Venturoli
On 1/21/19 3:17 AM, Blake McBride wrote:

> 1.  When building C programs one calls the compiler for each source
> file.  This is impractical with Java because of the javac startup time.  
> Typically javac is called once with many source files at a time - but
> not the ones that are not out of date.

This is easy to overcome with a proper rule.

I've used make a lot in the past for some Java project which didn't
involve complex layouts.




> 3.  Java build tools have the ability to download and cache files from
> remote repositories over the net.

This is something that make does not do (and does not try to do).
Period.

If you are doing a complex project which involves a lot of dependencies
and you want them to be fetched and delivered, I guess make is not an
option.




Just my 2c.

  bye
        av.

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Brian Vandenberg-2
In reply to this post by Blake McBride
It's been years, but I'll throw in my 2 cents in case it helps.

I attempted to make some java stuff build using a makefile and ran into circular dependency issues: foo.java depended on bar.java & vice-versa.  When I attempted to compile foo.java by itself it complained about unsatisfied dependencies from bar.java.  The only way I could get it to build was like this:

blah.jar : foo.java bar.java
> javac ${FLAGS} ${^}
> jar ${FLAGS} $(addsuffix .javac,$(basename ${^})) -o ${@}

... or something to that effect.

-brian

On Fri, Jan 18, 2019 at 7:16 AM Blake McBride <[hidden email]> wrote:
Greetings,

I have used GNU Make for many years on C projects.  Over the last 10 or more years, I have been working mainly in Java.  There are several build tools that are popular in the Java world such as Ant, Maven, and Gradle.

I have found Make very easy to understand and use.  Given the amount of work they do in the background, I have found build tools such as Maven and Gradle to be very confusing.  They are easy and work well within a very narrow scope, but as soon as you try to go outside their typical pattern, they become inordinately complex or can't even handle some out-of-pattern situations.

It is my incomplete understanding that certain aspects of Make don't lend themselves well to the building of Java projects.  This is what has driven the quest for better Java fitting build tools.

Being frustrated by the difficulty of going outside of the Java build tools scope, I wondered whether it might be possible to enhance certain aspects of GNU Make to better accommodate the needs of a Java environment.  I also wonder if this issue had been previously raised and addressed without my knowledge.

Anyway, I thought I would raise the issue and possibly spark an interesting conversation.

Thank you.

Blake McBride


_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Blake McBride
In reply to this post by Andrea Venturoli
I appreciate all of the replies and comments, but I think a point is being missed.

Let's say my Java app is made up of 9,500 source files (which is actually true!).  Let's say I change 20 of them.  

1.  How could I create a dependency that basically says the 9,500 Java files in an entire source directory tree result in the 9,500 class files in the output directory tree?

2.  I need to create a text file to pass a lot of parameters to a single evocation of javac.  This file must contain all the updated java files and a pointer (classpath) to the already up-to-date class files.

What has been suggested works fine for 20 Java files.  But it is impractical for 10,000 files.

Make doesn't deal with source trees well.  Although I can use 'find' in a recipe, I can't for the target and dependency files.  I can't say, all of the class files in one tree is dependent on all of the java files in another - while also only compiling the out-of-date files - all in one javac evocation.

Perhaps I am wrong.  But I think these are the problems that drove the development of ant, maven, gradle, and others.

I intuit that make could be enhanced to deal with these other scenarios if the will existed.

Thanks.

Blake


On Mon, Jan 21, 2019 at 3:24 AM Andrea Venturoli <[hidden email]> wrote:
On 1/21/19 3:17 AM, Blake McBride wrote:

> 1.  When building C programs one calls the compiler for each source
> file.  This is impractical with Java because of the javac startup time. 
> Typically javac is called once with many source files at a time - but
> not the ones that are not out of date.

This is easy to overcome with a proper rule.

I've used make a lot in the past for some Java project which didn't
involve complex layouts.




> 3.  Java build tools have the ability to download and cache files from
> remote repositories over the net.

This is something that make does not do (and does not try to do).
Period.

If you are doing a complex project which involves a lot of dependencies
and you want them to be fetched and delivered, I guess make is not an
option.




Just my 2c.

  bye
        av.

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Tim Murphy-4
Targets with multiple outputs have never been well supported in GNU make and despite appeals and the problem having existed forever and despite examples of commercial make implementations that support it well, it would not surprise me if support for it was something we could only hope for for our children. I have, quite literally, gone grey haired waiting for it.

So my advice is to to do anything except wait. 

Regards, Tim


On Tue, 12 Feb 2019, 04:13 Blake McBride <[hidden email] wrote:
I appreciate all of the replies and comments, but I think a point is being missed.

Let's say my Java app is made up of 9,500 source files (which is actually true!).  Let's say I change 20 of them.  

1.  How could I create a dependency that basically says the 9,500 Java files in an entire source directory tree result in the 9,500 class files in the output directory tree?

2.  I need to create a text file to pass a lot of parameters to a single evocation of javac.  This file must contain all the updated java files and a pointer (classpath) to the already up-to-date class files.

What has been suggested works fine for 20 Java files.  But it is impractical for 10,000 files.

Make doesn't deal with source trees well.  Although I can use 'find' in a recipe, I can't for the target and dependency files.  I can't say, all of the class files in one tree is dependent on all of the java files in another - while also only compiling the out-of-date files - all in one javac evocation.

Perhaps I am wrong.  But I think these are the problems that drove the development of ant, maven, gradle, and others.

I intuit that make could be enhanced to deal with these other scenarios if the will existed.

Thanks.

Blake


On Mon, Jan 21, 2019 at 3:24 AM Andrea Venturoli <[hidden email]> wrote:
On 1/21/19 3:17 AM, Blake McBride wrote:

> 1.  When building C programs one calls the compiler for each source
> file.  This is impractical with Java because of the javac startup time. 
> Typically javac is called once with many source files at a time - but
> not the ones that are not out of date.

This is easy to overcome with a proper rule.

I've used make a lot in the past for some Java project which didn't
involve complex layouts.




> 3.  Java build tools have the ability to download and cache files from
> remote repositories over the net.

This is something that make does not do (and does not try to do).
Period.

If you are doing a complex project which involves a lot of dependencies
and you want them to be fetched and delivered, I guess make is not an
option.




Just my 2c.

  bye
        av.

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
Reply | Threaded
Open this post in threaded view
|

Re: GNU Make for Java projects

Blake McBride

On Mon, Feb 11, 2019 at 9:40 PM Tim Murphy <[hidden email]> wrote:
Targets with multiple outputs have never been well supported in GNU make and despite appeals and the problem having existed forever and despite examples of commercial make implementations that support it well, it would not surprise me if support for it was something we could only hope for for our children. I have, quite literally, gone grey haired waiting for it.

So my advice is to to do anything except wait. 

Regards, Tim


On Tue, 12 Feb 2019, 04:13 Blake McBride <[hidden email] wrote:
I appreciate all of the replies and comments, but I think a point is being missed.

Let's say my Java app is made up of 9,500 source files (which is actually true!).  Let's say I change 20 of them.  

1.  How could I create a dependency that basically says the 9,500 Java files in an entire source directory tree result in the 9,500 class files in the output directory tree?

2.  I need to create a text file to pass a lot of parameters to a single evocation of javac.  This file must contain all the updated java files and a pointer (classpath) to the already up-to-date class files.

What has been suggested works fine for 20 Java files.  But it is impractical for 10,000 files.

Make doesn't deal with source trees well.  Although I can use 'find' in a recipe, I can't for the target and dependency files.  I can't say, all of the class files in one tree is dependent on all of the java files in another - while also only compiling the out-of-date files - all in one javac evocation.

Perhaps I am wrong.  But I think these are the problems that drove the development of ant, maven, gradle, and others.

I intuit that make could be enhanced to deal with these other scenarios if the will existed.

Thanks.

Blake


On Mon, Jan 21, 2019 at 3:24 AM Andrea Venturoli <[hidden email]> wrote:
On 1/21/19 3:17 AM, Blake McBride wrote:

> 1.  When building C programs one calls the compiler for each source
> file.  This is impractical with Java because of the javac startup time. 
> Typically javac is called once with many source files at a time - but
> not the ones that are not out of date.

This is easy to overcome with a proper rule.

I've used make a lot in the past for some Java project which didn't
involve complex layouts.




> 3.  Java build tools have the ability to download and cache files from
> remote repositories over the net.

This is something that make does not do (and does not try to do).
Period.

If you are doing a complex project which involves a lot of dependencies
and you want them to be fetched and delivered, I guess make is not an
option.




Just my 2c.

  bye
        av.

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make
_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make

_______________________________________________
Bug-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/bug-make