--- App-Staticperl/staticperl.pod 2010/12/10 20:29:17 1.20 +++ App-Staticperl/staticperl.pod 2010/12/21 19:14:56 1.26 @@ -124,10 +124,12 @@ =head1 THE F SCRIPT This module installs a script called F into your perl -binary directory. The script is fully self-contained, and can be used -without perl (for example, in an uClibc chroot environment). In fact, -it can be extracted from the C distribution tarball as -F, without any installation. +binary directory. The script is fully self-contained, and can be +used without perl (for example, in an uClibc chroot environment). In +fact, it can be extracted from the C distribution +tarball as F, without any installation. The +newest (possibly alpha) version can also be downloaded from +L. F interprets the first argument as a command to execute, optionally followed by any parameters. @@ -147,12 +149,17 @@ staticperl install -Is normally all you need: It installs the perl interpreter in +is normally all you need: It installs the perl interpreter in F<~/.staticperl/perl>. It downloads, configures, builds and installs the perl interpreter if required. -Most of the following commands simply run one or more steps of this -sequence. +Most of the following F subcommands simply run one or more +steps of this sequence. + +If it fails, then most commonly because the compiler options I selected +are not supported by your compiler - either edit the F script +yourself or create F<~/.staticperl> shell script where your set working +C etc. variables. To force recompilation or reinstallation, you need to run F first. @@ -209,11 +216,12 @@ Deletes the perl source directory (and potentially cleans up other intermediate files). This can be used to clean up files only needed for -building perl, without removing the installed perl interpreter, or to -force a re-build from scratch. +building perl, without removing the installed perl interpreter. At the moment, it doesn't delete downloaded tarballs. +The exact semantics of this command will probably change. + =item F This wipes your complete F<~/.staticperl> directory. Be careful with this, @@ -272,13 +280,45 @@ # run it ./app +Here are the three phase 2 commands: + +=over 4 + +=item F args... + +The "default" bundle command - it interprets the given bundle options and +writes out F, F, F and F +files, useful for embedding. + +=item F args... + +Creates a bundle just like F (in fact, it's the same +as invoking F args...), but then compiles and +links a new perl interpreter that embeds the created bundle, then deletes +all intermediate files. + +=item F filename args... + +Does the same as F (in fact, it's the same as +invoking F filename args...), but then compiles +and links a new standalone application that simply initialises the perl +interpreter. + +The difference to F is that the standalone application +does not act like a perl interpreter would - in fact, by default it would +just do nothing and exit immediately, so you should specify some code to +be executed via the F<--boot> option. + +=back + =head3 OPTION PROCESSING All options can be given as arguments on the command line (typically using long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since -specifying a lot of modules can make the command line very cumbersome, -you can put all long options into a "bundle specification file" (with or -without C<--> prefix) and specify this bundle file instead. +specifying a lot of modules can make the command line very cumbersome, you +can put all long options into a "bundle specification file" (one option +per line, with or without C<--> prefix) and specify this bundle file +instead. For example, the command given earlier could also look like this: @@ -293,48 +333,22 @@ add eg/httpd httpd.pm All options that specify modules or files to be added are processed in the -order given on the command line (that affects the C<--use> and C<--eval> -options at the moment). +order given on the command line. -=head3 PACKAGE SELECTION WORKFLOW +=head3 BUNDLE CREATION WORKFLOW -F has a number of options to control package -selection. This section describes how they interact with each other. Also, -since I am still a newbie w.r.t. these issues, maybe future versions of -F will change this, so watch out :) - -The idiom "in order" means "in order that they are specified on the -commandline". If you use a bundle specification file, then the options -will be processed as if they were given in place of the bundle file name. +F works by first assembling a list of candidate +files and modules to include, then filtering them by include/exclude +patterns. The remaining modules (together with their direct depdendencies, +such as link libraries and AutoLoader files) are then converted into +bundle files suitable for embedding. Afterwards, F +can optionally build a new perl interpreter or a standalone application. =over 4 -=item 1. apply all C<--use>, C<--eval>, C<--add>, C<--addbin> and -C<--incglob> options, in order. - -In addition, C<--use> and C<--eval> dependencies will be added when the -options are processed. +=item Step 0: Generic argument processing. -=item 2. apply all C<--include> and C<--exclude> options, in order. - -All this step does is potentially reduce the number of files already -selected or found in phase 1. - -=item 3. find all modules (== F<.pm> files), gather their static archives -(F<.a>) and AutoLoader splitfiles (F<.ix> and F<.al> files), find any -extra libraries they need for linking (F) and optionally -evaluate any F<.packlist> files. - -This step is required to link against XS extensions and also adds files -required for L to do it's job. - -=back - -After this, all the files selected for bundling will be read and processed -(stripped), the bundle files will be written, and optionally a new F -or application binary will be linked. - -=head3 MKBUNDLE OPTIONS +The following options influence F itself. =over 4 @@ -346,67 +360,27 @@ Decreases the verbosity level by one. -=item --strip none|pod|ppi - -Specify the stripping method applied to reduce the file of the perl -sources included. - -The default is C, which uses the L module to remove all -pod documentation, which is very fast and reduces file size a lot. - -The C method uses L to parse and condense the perl sources. This -saves a lot more than just L, and is generally safer, -but is also a lot slower (some files take almost a minute to strip - -F maintains a cache of stripped files to speed up subsequent -runs for this reason). Note that this method doesn't optimise for raw file -size, but for best compression (that means that the uncompressed file size -is a bit larger, but the files compress better, e.g. with F). - -Last not least, if you need accurate line numbers in error messages, -or in the unlikely case where C is too slow, or some module gets -mistreated, you can specify C to not mangle included perl sources in -any way. - -=item --perl - -After writing out the bundle files, try to link a new perl interpreter. It -will be called F and will be left in the current working -directory. The bundle files will be removed. - -This switch is automatically used when F is invoked with the -C command (instead of C): - - # build a new ./perl with only common::sense in it - very small :) - staticperl mkperl -Mcommon::sense - -=item --app name - -After writing out the bundle files, try to link a new standalone -program. It will be called C, and the bundle files get removed after -linking it. +=item any other argument -The difference to the (mutually exclusive) C<--perl> option is that the -binary created by this option will not try to act as a perl interpreter - -instead it will simply initialise the perl interpreter, clean it up and -exit. +Any other argument is interpreted as a bundle specification file, which +supports most long options (without extra quoting), one option per line. -This switch is automatically used when F is invoked with the -C command (instead of C): +=back -To let it do something useful you I add some boot code, e.g. with -the C<--boot> option. +=item Step 1: gather candidate files and modules -Example: create a standalone perl binary that will execute F when -it is started. +In this step, modules, perl libraries (F<.pl> files) and other files are +selected for inclusion in the bundle. The relevant options are executed +in order (this makes a difference mostly for C<--eval>, which can rely on +earlier C<--use> options to have been executed). - staticperl mkbundle --app myexe --boot appfile +=over 4 -=item --use module | -Mmodule +=item C<--use> F | C<-M>F -Include the named module and all direct dependencies. This is done by +Include the named module and trace direct dependencies. This is done by C'ing the module in a subprocess and tracing which other modules -and files it actually loads. If the module uses L, then all -splitfiles will be included as well. +and files it actually loads. Example: include AnyEvent and AnyEvent::Impl::Perl. @@ -427,17 +401,17 @@ # bundle specification file use "Config_heavy.pl" -The C<-Mmodule> syntax is included as an alias that might be easier to -remember than C. Or maybe it confuses people. Time will tell. Or -maybe not. Argh. +The C<-M>module syntax is included as an alias that might be easier to +remember than C<--use>. Or maybe it confuses people. Time will tell. Or +maybe not. Sigh. -=item --eval "perl code" | -e "perl code" +=item C<--eval> "perl code" | C<-e> "perl code" Sometimes it is easier (or necessary) to specify dependencies using perl code, or maybe one of the modules you use need a special use statement. In -that case, you can use C to execute some perl snippet or set some -variables or whatever you need. All files C'd or C'd in the -script are included in the final bundle. +that case, you can use C<--eval> to execute some perl snippet or set some +variables or whatever you need. All files C'd or C'd while +executing the snippet are included in the final bundle. Keep in mind that F will only C the modules named by the C<--use> option, so do not expect the symbols from modules you @@ -449,22 +423,134 @@ staticperl mkbundle --eval 'use AnyEvent; AnyEvent::detect' # or like this - staticperl mkbundle -MAnyEvent --eval 'use AnyEvent; AnyEvent::detect' + staticperl mkbundle -MAnyEvent --eval 'AnyEvent::detect' Example: use a separate "bootstrap" script that C's lots of modules -and include this in the final bundle, to be executed automatically. +and also include this in the final bundle, to be executed automatically +when the interpreter is initialised. staticperl mkbundle --eval 'do "bootstrap"' --boot bootstrap -=item --boot filename +=item C<--boot> F + +Include the given file in the bundle and arrange for it to be +executed (using C) before the main program when the new perl +is initialised. This can be used to modify C<@INC> or do similar +modifications before the perl interpreter executes scripts given on the +command line (or via C<-e>). This works even in an embedded interpreter - +the file will be executed during interpreter initialisation in that case. + +=item C<--incglob> pattern + +This goes through all standard library directories and tries to match any +F<.pm> and F<.pl> files against the extended glob pattern (see below). If +a file matches, it is added. The pattern is matched against the full path +of the file (sans the library directory prefix), e.g. F. + +This is very useful to include "everything": + + --incglob '*' + +It is also useful for including perl libraries, or trees of those, such as +the unicode database files needed by some perl builtins, the regex engine +and other modules. + + --incglob '/unicore/**.pl' + +=item C<--add> F | C<--add> "F alias" + +Adds the given (perl) file into the bundle (and optionally call it +"alias"). The F is either an absolute path or a path relative to +the current directory. If an alias is specified, then this is the name it +will use for C<@INC> searches, otherfile the F will be used as the +internal name. + +This switch is used to include extra files into the bundle. + +Example: embed the file F in the current directory as F +when creating the bundle. + + staticperl mkperl --add "httpd httpd.pm" + +Example: add local files as extra modules in the bundle. + + # specification file + add file1 myfiles/file1.pm + add file2 myfiles/file2.pm + add file3 myfiles/file3.pl + + # then later, in perl, use + use myfiles::file1; + require myfiles::file2; + my $res = do "myfiles/file3.pl"; + +=item C<--binadd> F | C<--add> "F alias" + +Just like C<--add>, except that it treats the file as binary and adds it +without any postprocessing (perl files might get stripped to reduce their +size). + +You should probably add a C prefix to avoid clashing with embedded perl +files (whose paths do not start with C), and/or use a special directory +prefix, such as C. -Include the given file in the bundle and arrange for it to be executed -(using a C) before anything else when the new perl is -initialised. This can be used to modify C<@INC> or anything else before -the perl interpreter executes scripts given on the command line (or via -C<-e>). This works even in an embedded interpreter. +You can later get a copy of these files by calling C. + +An alternative way to embed binary files is to convert them to perl and +use C to get the contents - this method is a bit cumbersome, but works +both inside and outside of a staticperl bundle: + + # a "binary" file, call it "bindata.pl" + <<'SOME_MARKER' + binary data NOT containing SOME_MARKER + SOME_MARKER + + # load the binary + chomp (my $data = do "bindata.pl"); + +=back + +=item Step 2: filter all files using C<--include> and C<--exclude> options. + +After all candidate files and modules are added, they are I +by a combination of C<--include> and C<--exclude> patterns (there is an +implicit C<--include **> at the end, so if no filters are specified, all +files are included). + +All that this step does is potentially reduce the number of files that are +to be included - no new files are added during this step. + +=over 4 -=item --usepacklist +=item C<--include> pattern | C<-i> pattern | C<--exclude> pattern | C<-x> pattern + +These specify an include or exclude pattern to be applied to the candidate +file list. An include makes sure that the given files will be part of the +resulting file set, an exclude will exclude remaining files. The patterns +are "extended glob patterns" (see below). + +The patterns are applied "in order" - files included via earlier +C<--include> specifications cannot be removed by any following +C<--exclude>, and likewise, and file excluded by an earlier C<--exclude> +cannot be added by any following C<--include>. + +For example, to include everything except C modules, but still +include F, you could use this: + + --incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**' + +=back + +=item Step 3: add any extra or "hidden" dependencies. + +F currently knows about three extra types of depdendencies +that are added automatically. Only one (F<.packlist> files) is currently +optional and can be influenced, the others are always included: + +=over 4 + +=item C<--usepacklist> Read F<.packlist> files for each distribution that happens to match a module name you specified. Sounds weird, and it is, so expect semantics to @@ -481,75 +567,120 @@ all L submodules that have been installed via the CPAN distribution are included as well, so you don't have to manually specify them. -=item --incglob pattern +=item L splitfiles -This goes through all library directories and tries to match any F<.pm> -and F<.pl> files against the extended glob pattern (see below). If a file -matches, it is added. This switch will automatically detect L -files and the required link libraries for XS modules, but it will I -scan the file for dependencies (at the moment). +Some modules use L - less commonly (hopefully) used functions +are split into separate F<.al> files, and an index (F<.ix>) file contains +the prototypes. -This is mainly useful to include "everything": +Both F<.ix> and F<.al> files will be detected automatically and added to +the bundle. - --incglob '*' +=item link libraries (F<.a> files) -Or to include perl libraries, or trees of those, such as the unicode -database files needed by many other modules: +Modules using XS (or any other non-perl language extension compiled at +installation time) will have a static archive (typically F<.a>). These +will automatically be added to the linker options in F. - --incglob '/unicore/**.pl' +Should F find a dynamic link library (typically F<.so>) it +will warn about it - obviously this shouldn't happen unless you use +F on the wrong perl, or one (probably wrongly) configured to +use dynamic loading. -=item --add file | --add "file alias" +=item extra libraries (F) -Adds the given (perl) file into the bundle (and optionally call it -"alias"). This is useful to include any custom files into the bundle. +Some modules need linking against external libraries - these are found in +F and added to F. -Example: embed the file F as F when creating the bundle. +=back - staticperl mkperl --add "httpd httpd.pm" +=item Step 4: write bundle files and optionally link a program -It is also a great way to add any custom modules: +At this point, the select files will be read, processed (stripped) and +finally the bundle files get written to disk, and F +is normally finished. Optionally, it can go a step further and either link +a new F binary with all selected modules and files inside, or build +a standalone application. - # specification file - add file1 myfiles/file1 - add file2 myfiles/file2 - add file3 myfiles/file3 +Both the contents of the bundle files and any extra linking is controlled +by these options: -=item --binadd file | --add "file alias" +=over 4 -Just like C<--add>, except that it treats the file as binary and adds it -without any processing. +=item C<--strip> C|C|C -You should probably add a C prefix to avoid clashing with embedded -perl files (whose paths do not start with C), and/or use a special -directory, such as C. +Specify the stripping method applied to reduce the file of the perl +sources included. -You can later get a copy of these files by calling C. +The default is C, which uses the L module to remove all +pod documentation, which is very fast and reduces file size a lot. + +The C method uses L to parse and condense the perl sources. This +saves a lot more than just L, and is generally safer, +but is also a lot slower (some files take almost a minute to strip - +F maintains a cache of stripped files to speed up subsequent +runs for this reason). Note that this method doesn't optimise for raw file +size, but for best compression (that means that the uncompressed file size +is a bit larger, but the files compress better, e.g. with F). -=item --include pattern | -i pattern | --exclude pattern | -x pattern +Last not least, if you need accurate line numbers in error messages, +or in the unlikely case where C is too slow, or some module gets +mistreated, you can specify C to not mangle included perl sources in +any way. -These two options define an include/exclude filter that is used after all -files selected by the other options have been found. Each include/exclude -is applied to all files found so far - an include makes sure that the -given files will be part of the resulting file set, an exclude will -exclude files. The patterns are "extended glob patterns" (see below). +=item --perl -For example, to include everything, except C modules, but still -include F, you could use this: +After writing out the bundle files, try to link a new perl interpreter. It +will be called F and will be left in the current working +directory. The bundle files will be removed. - --incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**' +This switch is automatically used when F is invoked with the +C command instead of C. + +Example: build a new F<./perl> binary with only L inside - +it will be even smaller than the standard perl interpreter as none of the +modules of the base distribution (such as L) will be included. + + staticperl mkperl -Mcommon::sense + +=item --app name + +After writing out the bundle files, try to link a new standalone +program. It will be called C, and the bundle files get removed after +linking it. + +This switch is automatically used when F is invoked with the +C command instead of C. + +The difference to the (mutually exclusive) C<--perl> option is that the +binary created by this option will not try to act as a perl interpreter - +instead it will simply initialise the perl interpreter, clean it up and +exit. + +This means that, by default, it will do nothing but burna few CPU cycles +- for it to do something useful you I add some boot code, e.g. with +the C<--boot> option. + +Example: create a standalone perl binary called F<./myexe> that will +execute F when it is started. + + staticperl mkbundle --app myexe --boot appfile =item --static -When C<--perl> is also given, link statically instead of dynamically. The -default is to link the new perl interpreter fully dynamic (that means all -perl modules are linked statically, but all external libraries are still +Add C<-static> to F, which means a fully static (if +supported by the OS) executable will be created. This is not immensely +useful when just creating the bundle files, but is most useful when +linking a binary with the C<--perl> or C<--app> options. + +The default is to link the new binary dynamically (that means all perl +modules are linked statically, but all external libraries are still referenced dynamically). Keep in mind that Solaris doesn't support static linking at all, and -systems based on GNU libc don't really support it in a usable fashion -either. Try uClibc if you want to create fully statically linked -executables, or try the C<--staticlibs> option to link only some libraries +systems based on GNU libc don't really support it in a very usable +fashion either. Try uClibc if you want to create fully statically linked +executables, or try the C<--staticlib> option to link only some libraries statically. =item --staticlib libname @@ -567,13 +698,10 @@ staticperl mkperl -MIO::AIO --staticlib crypt - # ldopts might nwo contain: + # ldopts might now contain: # -lm -Wl,-Bstatic -lcrypt -Wl,-Bdynamic -lpthread -=item any other argument - -Any other argument is interpreted as a bundle specification file, which -supports most long options (without extra quoting), one option per line. +=back =back @@ -703,13 +831,17 @@ (C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to reduce filesize further. -=item C, C, C, C +=item C, C, C, C, C These flags are passed to perl's F script, and are generally optimised for small size (at the cost of performance). Since they also contain subtle workarounds around various build issues, changing these -usually requires understanding their default values - best look at the top -of the F script for more info on these. +usually requires understanding their default values - best look at +the top of the F script for more info on these, and use a +F<~/.staticperlrc> to override them. + +Most of the variables override (or modify) the corresponding F +variable, except C, which gets appended. =back @@ -717,6 +849,10 @@ =over 4 +=item C + +The make command to use - default is C. + =item C Where F writes the C command to @@ -1053,7 +1189,7 @@ } This mostly gains space when linking staticaly, as the functions will -liekly not be linked in. The gain for dynamically-linked binaries is +likely not be linked in. The gain for dynamically-linked binaries is smaller. Also, this leaves C in - not only is it actually used