MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0012448CMakeCPackpublic2011-09-06 04:282016-06-10 14:31
ReporterAndreas Mohr 
Assigned ToKitware Robot 
PriorityhighSeveritymajorReproducibilityalways
StatusclosedResolutionmoved 
PlatformLinuxOSRHELOS Version5
Product VersionCMake-2-6 
Target VersionFixed in Version 
Summary0012448: install_manifest.txt syntax incompatible with RPM white space requirements --> easy/concise packaging handling mode not usable
DescriptionHi,

I'm using the RPM .spec
%files -f files.list
syntax (see http://www.rpm.org/max-rpm-snapshot/s1-rpm-specref-files-list.html [^] ),
(via a
set(SPECFILE_INSTALL_MANIFEST_FILE_template "${CMAKE_BINARY_DIR}/install_manifest.txt")
to have rpmbuild successfully reach the packaging-tree-external manifest file).

Unfortunately, with my awfully Windows-tainted source tree, this fails horribly due to many files with embedded whitespace.
For the result, see
"Re: spaces in file names." http://www.redhat.com/archives/rpm-list/2001-November/msg00051.html [^]
"Filenames with blanks" http://www.redhat.com/archives/rpm-list/2001-November/msg00051.html [^]

Note that it does not seem to be an issue on the RPM .spec side - since they support multiple files per line, they use spaces as separators, and since that is a conflict with embedded-space files, they do seem to support the usual quoting convention to properly indicate this.

One could argue that specifying an install manifest in %files line is exactly the way that packaging _should_ always be done - it's a manifest file as precisely created by the build/install tree, with all permissions declared via install() statements, and RPM .spec then simply adopting that compiled list and packaging it up. Thus no need to have any hassle whatsoever with manual, bit-rotting and maintenance-prone individual custom lines.
Thus it's rather shocking that such a precise way is the one that fails.

The question now would be how to fix it.
One could add a CMake variable (e.g. CMAKE_INSTALL_MANIFEST_WHITESPACE_QUOTING_MODE) which may support values such as 0 (never quote), 1 (quote intelligently, only for files with embedded whitespace), 2 (always quote).
And perhaps one should switch to having quoting activated by _default_ (although I'm sure some UNIX users will violently disagree :).
And perhaps this new handling needs to be governed by a new CMake policy.

Observed on CMake 2.6.4 (sorry), but from grepping git tree it doesn't seem like there's improved handling in this area.

Thanks!
Steps To Reproduce- have a project with files containing spaces
- have install() statements for such files
- create RPM .spec template
- add line %files -f @SPEC_INSTALL_MANIFEST_template@
- set(SPEC_INSTALL_MANIFEST_template "${CMAKE_BINARY_DIR}/install_manifest.txt")
- observe rpmbuild failure on cpack -G RPM
Additional Information
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2011-09-06 04:28Andreas MohrNew Issue
2011-09-06 04:45Eric NOULARDNote Added: 0027366
2011-09-06 06:05Andreas MohrNote Added: 0027367
2011-09-06 06:31Eric NOULARDNote Added: 0027368
2011-09-06 06:42Andreas MohrNote Added: 0027369
2011-09-06 06:55Andreas MohrNote Added: 0027370
2011-09-12 05:03Andreas MohrNote Added: 0027404
2012-08-11 11:38David ColeStatusnew => backlog
2012-08-11 11:38David ColeNote Added: 0030305
2012-12-20 11:03Eric NOULARDCategoryCMake => CPack
2016-06-10 14:28Kitware RobotNote Added: 0041902
2016-06-10 14:28Kitware RobotStatusbacklog => resolved
2016-06-10 14:28Kitware RobotResolutionopen => moved
2016-06-10 14:28Kitware RobotAssigned To => Kitware Robot
2016-06-10 14:31Kitware RobotStatusresolved => closed

Notes
(0027366)
Eric NOULARD   
2011-09-06 04:45   
Hi Andreas,

CPackRPM does not currently use install_manifest.txt but is using
an ugly:
# Use files tree to construct files command (spec file)
# We should not forget to include symlinks (thus -o -type l)
# We must remove the './' due to the local search and escape the
# file name by enclosing it between double quotes (thus the sed)
# Then we must authorize any man pages extension (adding * at the end)
# because rpmbuild may automatically compress those files
EXECUTE_PROCESS(COMMAND find . -type f -o -type l
                COMMAND sed s:.*/man.*/.*:&*:
                COMMAND sed s/\\.\\\(.*\\\)/\"\\1\"/
                WORKING_DIRECTORY "${WDIR}"
                OUTPUT_VARIABLE CPACK_RPM_INSTALL_FILES)


We could perfectly read install_manifest.txt instead of doing "find"
and keep the ugly quoting part as before.

You should know that CPackRPM is doing his best effort to automatically
handle absolute install file (like /etc/yourtool.conf) in order
to automagically insert %config on those.

simply using
%files -f files.list

would break this feature.

So in the end quoting in the manifest does not seem to be needed for CPackRPM.

Did I miss your point?
Why do you need to use manifest for CPackRPM in the first place?
(0027367)
Andreas Mohr   
2011-09-06 06:05   
I'm... currently using my own custom .spec file. Don't ask me why this is the case currently (I reviewed my copious CMake source comments, but there was no truly deal-breaking reason listed). I believe it's due to weak support for more advanced .spec parts, at least in older CMake versions.


That ugly find operation (as done _currently_) might easily turn out to be a deal-breaker (at least for my rather complex, un-partitioned install).

Actually doing CPackRPM.cmake processing via install_manifest.txt sounds like a better idea (but of course that wouldn't provide any help to the custom .spec file case). Perhaps one should make this branching configurable via a CPACK_RPM_ config variable? (and default to evaluating install_manifest.txt instead of doing raw find)


[automatic %config detection]

I'm not quite sure whether using a manifest file would break it:

"Re: %files -f and %defattr"
http://markmail.org/message/xoey3olqigxezx5b#query:+page:1+mid:v7sbfx3ogknzqbjd+state:results [^]

seems to indicate that the manifest file is used _in addition_ to the manual contents (but then ordering might still be wrong, so perhaps it really breaks).

Thank you for your very fast/precise reply!
(0027368)
Eric NOULARD   
2011-09-06 06:31   
The fact that CPackRPM has a feature to handle custom spec file
was done on purpose as a trap to get advanced spec file features
which are not handled by CPackRPM.

That said it's difficult for CPackRPM to handle any kind of custom
spec file without relying on user knowledge.

We may switch from find to manifest file
(I have to evaluate pros- & cons- first)
but in your custom spec file case
I think you'd better process the manifest file yourself for quoting.

You should be able to do that with a CPACK_PROJECT_CONFIG_FILE
see http://www.itk.org/Wiki/CMake:CPackPackageGenerators#Overall_usage_.28common_to_all_generators.29 [^]

this file is loaded BEFORE CPackRPM is run so if you do create
a quoted "install_manifest_forRPM.txt" in this script
CPackRPM should be able to use it.

This can be seen as a hack before you get a chance to run more
up to date CMake/CPack version.
(0027369)
Andreas Mohr   
2011-09-06 06:42   
Ah, right. I was afraid that I needed to create a custom CPackMyConfig.cmake for that, to be able to process it at exactly the right step.

One thing that's somewhat of a stumbling block is that there's probably no way to (within a specific build tree) designate a custom CPack config file to be the _default_ for a cpack invocation,
you _have_ to use cpack --config (and users are _required_ to not forget about that). Since this is not the first time that I've been thinking hard about creating a custom CPack config, it would probably be a useful feature to have.


I just realized that install_manifest.txt vs. ugly find may have a problem:
in case of more strongly componentized builds, one may have a multitude of .spec files to have each component properly packaged, whereas AFAIK install_manifest.txt is one _global_ build tree file. IOW this is a deal-breaker, handling perhaps cannot be changed due to changing from a .spec-directory-specific ugly find operation to gathering global install_manifest.txt data.
(0027370)
Andreas Mohr   
2011-09-06 06:55   
Rectification:
Ah screw it, my CPack config file rambling is bull. Seems I was completely unaware of actual CPACK_PROJECT_CONFIG_FILE behaviour, as described at http://www.itk.org/Wiki/CMake:CPackPackageGenerators#Overall_usage_.28common_to_all_generators.29 [^]
Plus, I somehow had thought that CPACK_PROJECT_CONFIG_FILE was not properly supported on old versions, but that does not seem to be the case (2.6.4 here).

Will try to add a hook immediately.

Thanks!
(0027404)
Andreas Mohr   
2011-09-12 05:03   
Hmm, unfortunately install_manifest.txt on a clean build
is not available yet on CPack startup (where the install phase has not happened yet). Would there be another hook
to be used to process the manifest subsequent to the "make install" as executed by CPack processing?
A (VERY ugly) workaround would be to change CPACK_GENERATOR sorting
to process simple archive types (TGZ etc.) first, in order to create
install_manifest.txt early enough.

IMHO this shows a very critical missing feature of CPack:
there seems to be (at least when going through source implementation) no _generic_ (independent of whether it's a CMake-based CPack invocation or a manual CPack setup) way at all to do some post-processing of installation between installation finished and packaging starting.
cmCPackGenerator::DoPackage() simply calls InstallProject(), which does not provide any post-install hook whatsoever.

And of course CPACK_INSTALL_COMMANDS, CPACK_INSTALL_SCRIPT are not usable for that purpose either since these are individual steps _within_ install handling at cmCPackGenerator::InstallProject() (there's nothing done after the end).

This should probably be filed as a high-prio new bug.
If that feature had properly existed, tweaking install_manifest.txt into a new format as required by certain subsequent packaging steps wouldn't have been an issue at all.

And, to view this in a more high-level aspect: IMHO it would be useful to have all CMake/CPack parts reviewed on whether they properly do support nicely generic pre/post handling hooks.
And, additionally, whether high-level handling of all install/packaging steps does follow a maximally correct flow at all, to be as generically useful as possible (i.e., a nicely generic flow as expressed by a good implementation of an UML Activity Diagram).


That's actually a very important issue: it doesn't help much if people implement various moderately featureful mechanisms and then forget adding the one single part that's most important to hapless users out in the cruel world: proper hooks to let people add their critical custom code to get around missing functionality that the original author did not have time for yet or even forgot about. The RPM .spec generator seems to be a fitting example here, too. My comments near the option() for choosing a custom .spec template:

  # Why do we want to use a custom RPM .spec template file?
  # Simple: because CPackRPM.cmake in semi-new (2.6.4) CMake versions
  # does NOT provide (specific _or_ generic, via hooks) support
  # for %post, %postun sections yet
  # (via specific variables CPACK_RPM_POST_INSTALL_SCRIPT_FILE,
  # CPACK_RPM_POST_UNINSTALL_SCRIPT_FILE).
  # And because their %files's %defattr does not indicate a specific user group.

Thanks!
(0030305)
David Cole   
2012-08-11 11:38   
Sending old, never assigned issues to the backlog.

(The age of the bug, plus the fact that it's never been assigned to anyone means that nobody is actively working on it...)

If an issue you care about is sent to the backlog when you feel it should have been addressed in a different manner, please bring it up on the CMake mailing list for discussion. Sign up for the mailing list here, if you're not already on it: http://www.cmake.org/mailman/listinfo/cmake [^]

It's easy to re-activate a bug here if you can find a CMake developer who has the bandwidth to take it on, and ferry a fix through to our 'next' branch for dashboard testing.
(0041902)
Kitware Robot   
2016-06-10 14:28   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.