Problem Creating Directories

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem Creating Directories

Bogdan
Hello,

I have the following problem which boils down to directory creation, in case anyone can help me with it.

I have a project makefile which generates and includes makefiles for all of the project's modules. By generation, I really mean they are processed with m4. Before anyone says anything, yes, I am aware that I could be using GNU Autotools to solve my problem but I would still like to figure out what I am doing wrong. Here is the directory structure:

project - The project directory
project/makefiles - Here is where the makefiles are to be generated
project/src - Here is where the modules go
project/src/foo - Example module
project/src/bar - Example module

There are four files in this dummy project:

project/makefile - The main makefile which includes everything from the makefiles directory
project/src/project.m4 - Generate makefiles/project.mk to select modules
project/src/foo/foo.m4 - Generate makefiles/foo/foo.mk to build component foo
project/src/bar/bar.m4 - Generate makefiles/bar/bar.mk to build component bar

Here is what I expect to be generated:

project/makefiles/project.mk
project/makefiles/foo/foo.mk
project/makefiles/bar/bar.mk

Here is project/makefile:

SRC_DIR = src
MAKE_DIR = makefiles

$(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4 | $(MAKE_DIR)/$(MODULE); m4 $< $@
$(MAKE_DIR)/$(MODULE): ; mkdir -p $@

include $(MAKE_DIR)/project.mk

I know MODULE isn't defined at the point of the inclusion; that is because I want the macro to expand to the empty string such that the "makefiles" subdirectory can be created. This macro will be relevant later and should expand on usage.

Since this is just a dummy test project, there isn't actually any m4 processing being done so project.m4 and project.mk will end up looking identical. Here are the contents:

MODULE = foo
include $(MAKE_DIR)/$(MODULE)/foo.mk

MODULE = bar
include $(MAKE_DIR)/$(MODULE)/bar.mk

Nothing else happens because foo.m4 and bar.m4 are empty. There is no target; I only expect this to generate the makefiles mentioned above for me (after all, make can't know that there is no target until it's done generating and including all the makefiles).

Here's what's wrong:

$ make
makefile:7: makefiles/project.mk: No such file or directory
mkdir -p makefiles/
m4 src/project.m4 >makefiles/project.mk
makefiles/project.mk:2: makefiles/foo/foo.mk: No such file or directory
makefiles/project.mk:5: makefiles/bar/bar.mk: No such file or directory
m4 src/bar/bar.m4 >makefiles/bar/bar.mk
/bin/sh: 1: cannot create makefiles/bar/bar/.mk: Directory nonexistent
makefile:4: recipe for target 'makefiles/bar/bar.mk' failed
make: *** [makefiles/bar/bar.mk] Error 2

The "No such file or directory" lines don't bother me. They are there because I didn't silence the include directives; make should attempt to generate what's being included and restart. As you can see, generating project.mk causes the makefiles subdirectory to be created. Why doesn't this happen in the case of foo.mk and bar.mk as well?

_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Problem Creating Directories

Paul Smith-20
On Sun, 2018-10-07 at 01:29 +0000, Bogdan wrote:
> $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4 | $(MAKE_DIR)/$(MODULE); m4 $< $@
> $(MAKE_DIR)/$(MODULE): ; mkdir -p $@
>
> I know MODULE isn't defined at the point of the inclusion; that is
> because I want the macro to expand to the empty string such that the
> "makefiles" subdirectory can be created. This macro will be relevant
> later and should expand on usage.

This is your problem.  You may want the macro to expand on usage, but
that's not how it works.

The GNU make manual section describing how variable expansion works is
here: https://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html

It shows that variables appearing in a target or prerequisite are
expanded immediately when the rule is parsed. At that time the variable
$(MODULE) is not set, so the rule is parsed like this:

  makefiles/%.mk: src/%.m4 | makefiles/ ; m4 $< $@

There are various ways to do what you want.  One option is secondary
expansion: https://www.gnu.org/software/make/manual/html_node/Secondary-Expansion.html

which would look like:

  .SECONDEXPANSION:
  $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4 | $(MAKE_DIR)/$$(MODULE); m4 $< $@
  $(MAKE_DIR)/$$(MODULE): ; mkdir -p $@

(note the extra "$" when referencing the MODULE variable)


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Problem Creating Directories

Paul Smith-20
On Sun, 2018-10-07 at 09:01 -0400, Paul Smith wrote:
>   $(MAKE_DIR)/$$(MODULE): ; mkdir -p $@
>
> (note the extra "$" when referencing the MODULE variable)

Actually I'm mistaken here: there's no secondary expansion for targets.

You might be able to write this as:

  $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4 | $(MAKE_DIR)/$$(MODULE)/.; m4 $< $@
  $(MAKE_DIR)/%/. : ; mkdir -p $@

but I'm not sure (untested).

Another option, obviously, is to simply put the mkdir inside the recipe
(that's how I'd personally do it):

  $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4
          mkdir -p $(MAKE_DIR)/$(MODULE)
          m4 $< $@

It may run extra instances of mkdir but I doubt it will be noticeable.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make
Reply | Threaded
Open this post in threaded view
|

Re: Problem Creating Directories

Bogdan
Yes, someone on IRC explained to me how macros expand inside rules shortly after posting that message and linking them to the mailing list archive. I actually came up with your second solution in response to that. However, the first one seems interesting as well. One thing I haven't mentioned is that although I'm using GNU Make to process the makefiles, I'm limiting myself to POSIX features and features that I expect will be eventually standardized by POSIX. I was unable to find anything on secondary expansion on the Austin Group's bug tracker. Since you're involved there (I've seen you discussing numerous make-related issues), perhaps you know anything going on on this front?

Somewhat related, is there any news on pattern rules (bug 0000513)? The related proposal on pattern substitution for macros (bug 0000519) seems to have been worked out.

Thanks,
Bogdan






On Sunday, October 7, 2018, 4:07:52 PM GMT+3, Paul Smith <[hidden email]> wrote:





On Sun, 2018-10-07 at 09:01 -0400, Paul Smith wrote:
>  $(MAKE_DIR)/$$(MODULE): ; mkdir -p $@
>
> (note the extra "$" when referencing the MODULE variable)

Actually I'm mistaken here: there's no secondary expansion for targets.

You might be able to write this as:

  $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4 | $(MAKE_DIR)/$$(MODULE)/.; m4 $< $@
  $(MAKE_DIR)/%/. : ; mkdir -p $@

but I'm not sure (untested).

Another option, obviously, is to simply put the mkdir inside the recipe
(that's how I'd personally do it):

  $(MAKE_DIR)/%.mk: $(SRC_DIR)/%.m4
          mkdir -p $(MAKE_DIR)/$(MODULE)

          m4 $< $@


It may run extra instances of mkdir but I doubt it will be noticeable.


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make


_______________________________________________
Help-make mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-make