--- App-Staticperl/staticperl.pod 2010/12/07 13:23:07 1.12 +++ App-Staticperl/staticperl.pod 2010/12/13 17:25:17 1.22 @@ -16,6 +16,7 @@ staticperl instcpan modulename... # install modules from CPAN staticperl mkbundle # see documentation staticperl mkperl # see documentation + staticperl mkapp appname # see documentation Typical Examples: @@ -24,20 +25,27 @@ staticperl mkperl -M '"Config_heavy.pl"' # build a perl that supports -V staticperl mkperl -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI -MURI::http # build a perl with the above modules linked in + staticperl mkapp myapp --boot mainprog mymodules + # build a binary "myapp" from mainprog and mymodules =head1 DESCRIPTION -This script helps you creating single-file perl interpreters, or embedding -a perl interpreter in your applications. Single-file means that it is -fully self-contained - no separate shared objects, no autoload fragments, -no .pm or .pl files are needed. And when linking statically, you can -create (or embed) a single file that contains perl interpreter, libc, all -the modules you need and all the libraries you need. +This script helps you to create single-file perl interpreters +or applications, or embedding a perl interpreter in your +applications. Single-file means that it is fully self-contained - no +separate shared objects, no autoload fragments, no .pm or .pl files are +needed. And when linking statically, you can create (or embed) a single +file that contains perl interpreter, libc, all the modules you need, all +the libraries you need and of course your actual program. With F and F on x86, you can create a single 500kb binary that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, Coro and so on. Or any other choice of modules. +To see how this turns out, you can try out smallperl and bigperl, two +pre-built static and compressed perl binaries with many and even more +modules: just follow the links at L. + The created files do not need write access to the file system (like PAR does). In fact, since this script is in many ways similar to PAR::Packer, here are the differences: @@ -65,17 +73,21 @@ F loads all required files directly from memory. There is no need to unpack files into a temporary directory. -=item * More control over included files. +=item * More control over included files, more burden. PAR tries to be maintenance and hassle-free - it tries to include more -files than necessary to make sure everything works out of the box. The -extra files (such as the unicode database) can take substantial amounts of -memory and file size. +files than necessary to make sure everything works out of the box. It +mostly succeeds at this, but he extra files (such as the unicode database) +can take substantial amounts of memory and file size. With F, the burden is mostly with the developer - only direct compile-time dependencies and L are handled automatically. This means the modules to include often need to be tweaked manually. +All this does not preclude more permissive modes to be implemented in +the future, but right now, you have to resolve state hidden dependencies +manually. + =item * PAR works out of the box, F does not. Maintaining your own custom perl build can be a pain in the ass, and while @@ -83,6 +95,11 @@ build and possibly fiddling with some modules. PAR is likely to produce results faster. +Ok, PAR never has worked for me out of the box, and for some people, +F does work out of the box, as they don't count "fiddling with +module use lists" against it, but nevertheless, F is certainly +a bit more difficult to use. + =back =head1 HOW DOES IT WORK? @@ -100,17 +117,19 @@ except everything is compiled in), or you create bundle files (basically C sources you can use to embed all files into your project). -This step is very fast (a few seconds if PPI is not used for stripping, -more seconds otherwise, as PPI is very slow), and can be tweaked and -repeated as often as necessary. +This step is very fast (a few seconds if PPI is not used for stripping, or +the stripped files are in the cache), and can be tweaked and repeated as +often as necessary. =head1 THE F SCRIPT This module installs a script called F into your perl -binary directory. The script is fully self-contained, and can be used -without perl (for example, in an uClibc chroot environment). In fact, -it can be extracted from the C distribution tarball as -F, without any installation. +binary directory. The script is fully self-contained, and can be +used without perl (for example, in an uClibc chroot environment). In +fact, it can be extracted from the C distribution +tarball as F, without any installation. The +newest (possibly alpha) version can also be downloaded from +L. F interprets the first argument as a command to execute, optionally followed by any parameters. @@ -142,6 +161,10 @@ =over 4 +=item F + +Prints some info about the version of the F script you are using. + =item F Runs only the download and unpack phase, unless this has already happened. @@ -241,6 +264,16 @@ to include that module. I found out about these dependencies by carefully watching any error messages about missing modules... +Instead of building a new perl binary, you can also build a standalone +application: + + # build the app + staticperl mkapp app --boot eg/httpd \ + -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI::http + + # run it + ./app + =head3 OPTION PROCESSING All options can be given as arguments on the command line (typically @@ -265,6 +298,44 @@ order given on the command line (that affects the C<--use> and C<--eval> options at the moment). +=head3 PACKAGE SELECTION WORKFLOW + +F has a number of options to control package +selection. This section describes how they interact with each other. Also, +since I am still a newbie w.r.t. these issues, maybe future versions of +F will change this, so watch out :) + +The idiom "in order" means "in order that they are specified on the +commandline". If you use a bundle specification file, then the options +will be processed as if they were given in place of the bundle file name. + +=over 4 + +=item 1. apply all C<--use>, C<--eval>, C<--add>, C<--addbin> and +C<--incglob> options, in order. + +In addition, C<--use> and C<--eval> dependencies will be added when the +options are processed. + +=item 2. apply all C<--include> and C<--exclude> options, in order. + +All this step does is potentially reduce the number of files already +selected or found in phase 1. + +=item 3. find all modules (== F<.pm> files), gather their static archives +(F<.a>) and AutoLoader splitfiles (F<.ix> and F<.al> files), find any +extra libraries they need for linking (F) and optionally +evaluate any F<.packlist> files. + +This step is required to link against XS extensions and also adds files +required for L to do it's job. + +=back + +After this, all the files selected for bundling will be read and processed +(stripped), the bundle files will be written, and optionally a new F +or application binary will be linked. + =head3 MKBUNDLE OPTIONS =over 4 @@ -286,11 +357,12 @@ pod documentation, which is very fast and reduces file size a lot. The C method uses L to parse and condense the perl sources. This -saves a lot more than just L, and is generally safer, but -is also a lot slower, so is best used for production builds. Note that -this method doesn't optimise for raw file size, but for best compression -(that means that the uncompressed file size is a bit larger, but the files -compress better, e.g. with F). +saves a lot more than just L, and is generally safer, +but is also a lot slower (some files take almost a minute to strip - +F maintains a cache of stripped files to speed up subsequent +runs for this reason). Note that this method doesn't optimise for raw file +size, but for best compression (that means that the uncompressed file size +is a bit larger, but the files compress better, e.g. with F). Last not least, if you need accurate line numbers in error messages, or in the unlikely case where C is too slow, or some module gets @@ -309,6 +381,28 @@ # build a new ./perl with only common::sense in it - very small :) staticperl mkperl -Mcommon::sense +=item --app name + +After writing out the bundle files, try to link a new standalone +program. It will be called C, and the bundle files get removed after +linking it. + +The difference to the (mutually exclusive) C<--perl> option is that the +binary created by this option will not try to act as a perl interpreter - +instead it will simply initialise the perl interpreter, clean it up and +exit. + +This switch is automatically used when F is invoked with the +C command (instead of C): + +To let it do something useful you I add some boot code, e.g. with +the C<--boot> option. + +Example: create a standalone perl binary that will execute F when +it is started. + + staticperl mkbundle --app myexe --boot appfile + =item --use module | -Mmodule Include the named module and all direct dependencies. This is done by @@ -372,7 +466,41 @@ the perl interpreter executes scripts given on the command line (or via C<-e>). This works even in an embedded interpreter. -=item --add "file" | --add "file alias" +=item --usepacklist + +Read F<.packlist> files for each distribution that happens to match a +module name you specified. Sounds weird, and it is, so expect semantics to +change somehow in the future. + +The idea is that most CPAN distributions have a F<.pm> file that matches +the name of the distribution (which is rather reasonable after all). + +If this switch is enabled, then if any of the F<.pm> files that have been +selected match an install distribution, then all F<.pm>, F<.pl>, F<.al> +and F<.ix> files installed by this distribution are also included. + +For example, using this switch, when the L module is specified, then +all L submodules that have been installed via the CPAN distribution +are included as well, so you don't have to manually specify them. + +=item --incglob pattern + +This goes through all library directories and tries to match any F<.pm> +and F<.pl> files against the extended glob pattern (see below). If a file +matches, it is added. This switch will automatically detect L +files and the required link libraries for XS modules, but it will I +scan the file for dependencies (at the moment). + +This is mainly useful to include "everything": + + --incglob '*' + +Or to include perl libraries, or trees of those, such as the unicode +database files needed by many other modules: + + --incglob '/unicore/**.pl' + +=item --add file | --add "file alias" Adds the given (perl) file into the bundle (and optionally call it "alias"). This is useful to include any custom files into the bundle. @@ -388,7 +516,7 @@ add file2 myfiles/file2 add file3 myfiles/file3 -=item --binadd "file" | --add "file alias" +=item --binadd file | --add "file alias" Just like C<--add>, except that it treats the file as binary and adds it without any processing. @@ -400,6 +528,19 @@ You can later get a copy of these files by calling C. +=item --include pattern | -i pattern | --exclude pattern | -x pattern + +These two options define an include/exclude filter that is used after all +files selected by the other options have been found. Each include/exclude +is applied to all files found so far - an include makes sure that the +given files will be part of the resulting file set, an exclude will +exclude files. The patterns are "extended glob patterns" (see below). + +For example, to include everything, except C modules, but still +include F, you could use this: + + --incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**' + =item --static When C<--perl> is also given, link statically instead of dynamically. The @@ -413,6 +554,24 @@ executables, or try the C<--staticlibs> option to link only some libraries statically. +=item --staticlib libname + +When not linking fully statically, this option allows you to link specific +libraries statically. What it does is simply replace all occurances of +C<-llibname> with the GCC-specific C<-Wl,-Bstatic -llibname -Wl,-Bdynamic> +option. + +This will have no effect unless the library is actually linked against, +specifically, C<--staticlib> will not link against the named library +unless it would be linked against anyway. + +Example: link libcrypt statically into the binary. + + staticperl mkperl -MIO::AIO --staticlib crypt + + # ldopts might nwo contain: + # -lm -Wl,-Bstatic -lcrypt -Wl,-Bdynamic -lpthread + =item any other argument Any other argument is interpreted as a bundle specification file, which @@ -420,18 +579,62 @@ =back -=head2 F CONFIGURATION AND HOOKS +=head3 EXTENDED GLOB PATTERNS + +Some options of F expect an I. This is neither a normal shell glob nor a regex, but something +in between. The idea has been copied from rsync, and there are the current +matching rules: + +=over 4 + +=item Patterns starting with F will be a anchored at the root of the library tree. + +That is, F will match the F directory in C<@INC>, but +nothing inside, and neither any other file or directory called F +anywhere else in the hierarchy. + +=item Patterns not starting with F will be anchored at the end of the path. + +That is, F will match any file called F anywhere in the +hierarchy, but not any directories of the same name. + +=item A F<*> matches any single component. + +That is, F would match all F<.pl> files directly inside +C, not any deeper level F<.pl> files. Or in other words, F<*> +will not match slashes. + +=item A F<**> matches anything. + +That is, F would match all F<.pl> files under F, +no matter how deeply nested they are inside subdirectories. + +=item A F matches a single character within a component. + +That is, F matches F, but not the +hypothetical F, as F does not match F. + +=back + +=head2 F CONFIGURATION AND HOOKS -During (each) startup, F tries to source the following shell -files in order: +During (each) startup, F tries to source some shell files to +allow you to fine-tune/override configuration settings. + +In them you can override shell variables, or define shell functions +("hooks") to be called at specific phases during installation. For +example, you could define a C hook to install additional +modules from CPAN each time you start from scratch. + +If the env variable C<$STATICPERLRC> is set, then F will try +to source the file named with it only. Otherwise, it tries the following +shell files in order: /etc/staticperlrc ~/.staticperlrc $STATICPERL/rc -They can be used to override shell variables, or define functions to be -called at specific phases. - Note that the last file is erased during F, so generally should not be used. @@ -502,7 +705,7 @@ (C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to reduce filesize further. -=item C, C, C, C +=item C, C, C, C, C These flags are passed to perl's F script, and are generally optimised for small size (at the cost of performance). Since they also @@ -744,6 +947,124 @@ perl f directory (if you installed it) into the F filesystem, chroot inside and run it. +=head1 RECIPES / SPECIFIC MODULES + +This section contains some common(?) recipes and information about +problems with some common modules or perl constructs that require extra +files to be included. + +=head2 MODULES + +=over 4 + +=item utf8 + +Some functionality in the utf8 module, such as swash handling (used +for unicode character ranges in regexes) is implemented in the +C<"utf8_heavy.pl"> library: + + -M'"utf8_heavy.pl"' + +Many Unicode properties in turn are defined in separate modules, +such as C<"unicore/Heavy.pl"> and more specific data tables such as +C<"unicore/To/Digit.pl"> or C<"unicore/lib/Perl/Word.pl">. These tables +are big (7MB uncompressed, although F contains special +handling for those files), so including them on demand by your application +only might pay off. + +To simply include the whole unicode database, use: + + --incglob '/unicore/*.pl' + +=item AnyEvent + +AnyEvent needs a backend implementation that it will load in a delayed +fashion. The L backend is the default choice +for AnyEvent if it can't find anything else, and is usually a safe +fallback. If you plan to use e.g. L (L...), then you need to +include the L (L...) backend as +well. + +If you want to handle IRIs or IDNs (L punycode and idn +functions), you also need to include C<"AnyEvent/Util/idna.pl"> and +C<"AnyEvent/Util/uts46data.pl">. + +Or you can use C<--usepacklist> and specify C<-MAnyEvent> to include +everything. + +=item Carp + +Carp had (in older versions of perl) a dependency on L. As of +perl 5.12.2 (maybe earlier), this dependency no longer exists. + +=item Config + +The F switch (as well as many modules) needs L, which in +turn might need L<"Config_heavy.pl">. Including the latter gives you +both. + +=item Term::ReadLine::Perl + +Also needs L, or C<--usepacklist>. + +=item URI + +URI implements schemes as separate modules - the generic URL scheme is +implemented in L, HTTP is implemented in L. If +you need to use any of these schemes, you should include these manually, +or use C<--usepacklist>. + +=back + +=head2 RECIPES + +=over 4 + +=item Linking everything in + +To link just about everything installed in the perl library into a new +perl, try this: + + staticperl mkperl --strip ppi --incglob '*' + +=item Getting rid of netdb function + +The perl core has lots of netdb functions (C, C +and so on) that few applications use. You can avoid compiling them in by +putting the following fragment into a C hook: + + preconfigure() { + for sym in \ + d_getgrnam_r d_endgrent d_endgrent_r d_endhent \ + d_endhostent_r d_endnent d_endnetent_r d_endpent \ + d_endprotoent_r d_endpwent d_endpwent_r d_endsent \ + d_endservent_r d_getgrent d_getgrent_r d_getgrgid_r \ + d_getgrnam_r d_gethbyaddr d_gethent d_getsbyport \ + d_gethostbyaddr_r d_gethostbyname_r d_gethostent_r \ + d_getlogin_r d_getnbyaddr d_getnbyname d_getnent \ + d_getnetbyaddr_r d_getnetbyname_r d_getnetent_r \ + d_getpent d_getpbyname d_getpbynumber d_getprotobyname_r \ + d_getprotobynumber_r d_getprotoent_r d_getpwent \ + d_getpwent_r d_getpwnam_r d_getpwuid_r d_getsent \ + d_getservbyname_r d_getservbyport_r d_getservent_r \ + d_getspnam_r d_getsbyname + # d_gethbyname + do + PERL_CONFIGURE="$PERL_CONFIGURE -U$sym" + done + } + +This mostly gains space when linking staticaly, as the functions will +likely not be linked in. The gain for dynamically-linked binaries is +smaller. + +Also, this leaves C in - not only is it actually used +often, the L module also exposes it, so leaving it out usually +gains little. Why Socket exposes a C function that is in the core already +is anybody's guess. + +=back + =head1 AUTHOR Marc Lehmann