MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0011808CMakeCPackpublic2011-02-06 16:382016-06-10 14:31
ReporterPetri Hodju 
Assigned ToKitware Robot 
PrioritynormalSeverityfeatureReproducibilityalways
StatusclosedResolutionmoved 
PlatformPCOSUbuntuOS Version10.10
Product VersionCMake-2-8 
Target VersionFixed in Version 
Summary0011808: Make CPack preinstall to take account current install project name
DescriptionThe sample & patch attached will introduce enhanced way to handle packaging in projects having multiple subprojects, which have a need to generated multiple binary packages each (binaries, development files, documentation etc.). I've seen that Eric is currently revisiting the packaging for DEB & similar generators and I thought this could be something interesting for that work.

I have taken a path where each subproject does whatever install they want to do and then mark those installs to be included in one or several packages based on the component names given to the install commands. On the top level CMakefile the information is then gathered from all subprojects and packages are defined. To my taste this gives the developer more control over the process than just including the CPack.cmake file in each subproject.

Please see more details from the attachment provided. It has a patch and a working example how the thing works. Well, it's a dummy one, but shows the workflow.

Br,
Petri Hodju
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0010751closed Kitware Robot CPack.cmake confused when used twice 
parent of 0010781closed Kitware Robot CPack infinite recursion with include 
Attached Filesgz cpack-preinstall-follow-cmake-install-project-name.tar.gz (5,025) 2011-02-06 16:38
https://public.kitware.com/Bug/file/3682/cpack-preinstall-follow-cmake-install-project-name.tar.gz
Issue History
Date ModifiedUsernameFieldChange
2011-02-06 16:38Petri HodjuNew Issue
2011-02-06 16:38Petri HodjuFile Added: cpack-preinstall-follow-cmake-install-project-name.tar.gz
2011-02-07 02:00Eric NOULARDNote Added: 0025307
2011-02-07 02:00Eric NOULARDRelationship addedrelated to 0010751
2011-02-07 02:00Eric NOULARDRelationship addedparent of 0010781
2011-02-07 07:41Petri HodjuNote Added: 0025309
2011-02-08 15:22Petri HodjuNote Added: 0025341
2011-02-08 17:44Eric NOULARDNote Added: 0025346
2011-02-09 16:08Petri HodjuNote Added: 0025359
2011-02-13 23:34Petri HodjuNote Added: 0025423
2011-02-14 02:21Eric NOULARDNote Added: 0025424
2011-02-26 15:08Eric NOULARDAssigned To => Eric NOULARD
2011-02-26 15:08Eric NOULARDStatusnew => assigned
2011-02-26 15:08Eric NOULARDNote Added: 0025590
2011-03-03 17:57Eric NOULARDNote Added: 0025665
2011-03-07 03:01Petri HodjuNote Added: 0025683
2012-08-14 14:37Eric NOULARDNote Added: 0030655
2012-08-14 14:38Eric NOULARDAssigned ToEric NOULARD =>
2012-08-14 14:38Eric NOULARDStatusassigned => backlog
2016-06-10 14:28Kitware RobotNote Added: 0041797
2016-06-10 14:28Kitware RobotStatusbacklog => resolved
2016-06-10 14:28Kitware RobotResolutionopen => moved
2016-06-10 14:28Kitware RobotAssigned To => Kitware Robot
2016-06-10 14:31Kitware RobotStatusresolved => closed

Notes
(0025307)
Eric NOULARD   
2011-02-07 02:00   
Hi Petri,

The idea is interesting.
I'll give it a try and provide you with feedback.

After a [very] fast reading:
  - CAR and CDR (even if they doesn't seem to be used here)
    should be replace with builtin "list" sub-commands

  - GET_DEBIAN_ARCHITECTURE deserve to be put directly in
    CPackDeb.cmake and if you look at
    what is currently done in CPackDeb.cmake you may
    find that it does just that.
    CPACK_DEBIAN_PACKAGE_ARCHITECTURE has a default value.
    see cmake --help-module CPackDeb

  - You'd better not SET(CMAKE_SET_DESTDIR "ON")
    from CMake 2.8.3 and up it shouldn't be necessary
    and may be doing more damage than help.

Now with the overall goal for this if I try to rephrase it,
you want more control on package generation than the current

1) set your vars
2) include(CPack)

is that it?

I add some links to related bugs.
(0025309)
Petri Hodju   
2011-02-07 07:41   
Thanks for the comments Eric,

Yes, the overall goal I am after here is to get more fine grained control over the package generation. Defining what goes in which package with the ADD_TO_PACKAGE function and generating the package with the ADD_PACKAGE (could be actually CREATE_PACKAGE) gives more freedom in the package definition and adds visibility to the packaging process.
(0025341)
Petri Hodju   
2011-02-08 15:22   
Hi Eric,

I'm cleaning up the helper file according to your suggestions and I have encountered some minor problems.

1) If I leave out the SET(CMAKE_SET_DESTDIR "ON") the generated packages will have the files installed based on "./usr" instead of "/opt/cpack-sample" as indicated by CMAKE_INSTALL_PREFIX (which is then propagated to CPACK_INSTALL_PREFIX in the debian-package.in). If not using the DESTDIR is recommended, am I missing some other CPACK_* variable(s) I should set to get the files installed correctly to CPACK_INSTALL_PREFIX?

2) If I leave out the GET_DEBIAN_ARCHITECTURE from my helpers and replace the @PKG_ARCHITECTURE@ with ${CPACK_DEBIAN_PACKAGE_ARCHITECTURE} in the debian-package.in, it won't be expanded by the cpack when run.
(0025346)
Eric NOULARD   
2011-02-08 17:44   
Hi,

Which version of CMake do you use for your tests?

1) You can override CPACK_INSTALL_PREFIX directly.
   I'll check wether if CMAKE_SET_DESTDIR is supposed to work
   on DEB too. I did work mostly on RPM lastly so I may be
   wrong about current DEB generator capabilities.

2) You can not replace @PKG_ARCHITECTURE@ with
   ${CPACK_DEBIAN_PACKAGE_ARCHITECTURE} in your cpack config file
   because when your generated config is evaluated by [generic] CPack generator
   the specific CPack generator (i.e. the DEB Generator) has not
   yet began its work...

CPack generators is a hierarchy of classes rooted in a base class
"cmCPackGenerator" i.e. generic generator and then daughter classes
like "cmCPackDebGenerator" i.e. specific generator implements specific behavior
and actions.

I'll think about a proper away to avoid to do the same computation
(like arch guess) at two different place and still have the flexibility you are looking for.

It may take me some time.

Do not hesitate to dig into CPack code in order to better
understand the intricated CPack workflow.

Remember that with CMake you have several "times":

CMake time - when cmake runs for configuring the project
Build time - when the build tool runs for building the project
CPack time - when CPack runs for building the package
Install time - when software is installed

the last time may be done without CPack (like 'make install')
or using the specific package built by CPack (like 'dpkg -i my.deb').

Different values are available (or not) at different 'time'.
Sometime it's even more complicated if you call CPack from within
an execute_process i.e. at CMake time or from add_custom_target
i.e. at Build time.

During CPack time some workflow is implied too you can learn
it from cmCPackXXX sources.
(0025359)
Petri Hodju   
2011-02-09 16:08   
Hi,

I'm using version "2.8.3.20110204-gaa78a-dirty", against which I made the patch. I got your point at 2), meanwhile I'm sticking with my copy of the arch guessing as I'm using the debian-package.in template straight without the CPack.cmake and its friends.

How about taking out those in-place evaluated script fragments from CPack.cmake and make functions out of them in another file, which is then included by CPack.cmake. This way the arch guessing would be available to everyone as a function and the whole system gets more modular. I'm sure there are more fragments that could be pulled out like that and be "functionalized"

If you feel like this scheme of ADD_TO_PACKAGE & CREATE_PACKAGE I'm proposing is something worthwhile, I could still work it towards more general solution in which the CREATE_PACKAGE would take also GENERATORS parameter (RPM & DEB at least) and create both types of packages with the same effort. Or would you maybe like to go after even more general solution?

Br,
Petri
(0025423)
Petri Hodju   
2011-02-13 23:34   
Hi,

Any feedback about the patch itself?

Br,
Petri
(0025424)
Eric NOULARD   
2011-02-14 02:21   
No not yet.
I am in a busy loop for a week or so.
And unfortunately CMake comes after...
If you want a wider audience you may post a mail on
cmake or cmake-developer ML.

I'll come back to you when free time open up again.
(0025590)
Eric NOULARD   
2011-02-26 15:08   
Hi Petri,

I'll have a look at your work during next week.
(0025665)
Eric NOULARD   
2011-03-03 17:57   
Hi Petri,

Now I did try your patch for real.
It implements something which has recently been requested:
http://www.cmake.org/pipermail/cmake/2011-February/043075.html [^]

I should admit that this is nice to control in distributed places
which component goes where and your solution nicely handles the
fact that sub-project may [blindly] decide to use same component NAME
for component which shouldn't end-up in the same package.

The trouble I have is to try to keep something "simple"
like I answered there:
http://www.cmake.org/pipermail/cmake/2011-February/043083.html [^]

CPack wasn't "package file" oriented but merely suppose there would
only be a single package per project.

Moreover CPack COMPONENT weren't designed with "multi-package" in mind
even less with the idea of multi-project.
The problem we will face will be backward compatibility...

To summarize:
1) The need for multi-package is there.
   
2) A way to handle mutiple projects will come soon after
   (you may be the first to ask but other will come soon for sure).

3) We should be going away from the
   include(CPack) way of thinking and use some macros like yours.
   see the beginning of the work in stage branch
   CPack-ModularizeCPack
see http://public.kitware.com/Bug/view.php?id=10751 [^] too.
   
The cross product of those ideas may become complex and we should discuss
this on cmake-developer ML.

your ADD_PACKAGE may be inferred by the
INSTALL_TYPES if we enhance the cpack_add_install_type found in
current CPack.cmake.

Currently what I don't like with ADD_PACKAGE and
ADD_TO_PACKAGE is the fact that the subproject should know about the
toplevel ADD_PACKAGE:

in project bin1 you do:
ADD_TO_PACKAGE(NAME "bin1" COMPONENT "bin")

I would rather let each [sub]-project declare all the components.
And then define at the top level the way to build

"INSTALL_TYPES" for which we can specify that

install type "BIN1" will contains say
- all components named bin or runtime in every [sub]-projects
- all components named bin or runtime in a sub-projects list

that way the package "merging" would only be specified
in one place
(we may ignore local packaging request from sub-projects)

I'll try to summarize the pending CPack issues on cmake-developper ML
in order to see others idea. I have mine but I may be wrong.

Thank for your patches and idea, they will certainly contribute to
make cpack better.
(0025683)
Petri Hodju   
2011-03-07 03:01   
Hi Eric,

I went through the pointers you provided and I like the idea and syntax of the cpack_add_* macros. To me they at first seem a bit more complicated to use, but on the other hand they provide more functionality than what I was seeking for : ) I see that they also address the needs of typical graphical installation schema like NSIS is using and that looks logical.

I agree that the sub-project to top-level project dependency in the macros I provided is not desired and should be implemented properly the way you proposed; let the sub-projects define their components and assign those components in the install types on the top-level project. In my example the sub-projects were kind of assigning the components directly to install types, which should be left to be decided on the top-level.

One thing what I like to have is full power for the CPACK_INSTALL_CMAKE_PROJECTS variable. That was actually the reason behind the patch I proposed. When I started to directly use the variable with cpack, I assumed that if I just list the project from which I want e.g. "dev" component, I will get only files from that project, which was not the case (I got files of "dev" component from all projects). This of course has two sides, but to me that was unexpected and a bit burden to have globally unique component names through all sub-projects to correct it.

To me the patch is just a low level enabler that gives us opportunity to do more fine grained selection of files to install. And that was actually what I was expecting from the CPACK_INSTALL_CMAKE_PROJECTS variable in the first place.

Being aware of backward compatibility the patch also introduced the CPACK_PREINSTALL_FOLLOWS_CMAKE_PROJECT_NAME variable to control whether to install "dev" files from all projects or to follow also the project name part of the CPACK_INSTALL_CMAKE_PROJECTS variable.

Let's continue the discussion on the mailing list.

Br,
Petri
(0030655)
Eric NOULARD   
2012-08-14 14:37   
Sorry no time to "really" work on this now.
(0041797)
Kitware Robot   
2016-06-10 14:28   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.