--- App-Staticperl/staticperl.pod 2010/12/06 21:30:05 1.8 +++ App-Staticperl/staticperl.pod 2011/07/13 17:47:40 1.50 @@ -11,32 +11,41 @@ staticperl install # build and then install perl staticperl clean # clean most intermediate files (restart at configure) staticperl distclean # delete everything installed by this script + staticperl perl ... # invoke the perlinterpreter staticperl cpan # invoke CPAN shell - staticperl instmod path... # install unpacked modules + staticperl instsrc path... # install unpacked modules staticperl instcpan modulename... # install modules from CPAN staticperl mkbundle # see documentation staticperl mkperl # see documentation + staticperl mkapp appname # see documentation Typical Examples: staticperl install # fetch, configure, build and install perl staticperl cpan # run interactive cpan shell - staticperl mkperl -M '"Config_heavy.pl"' # build a perl that supports -V + staticperl mkperl -MConfig_heavy.pl # build a perl that supports -V staticperl mkperl -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI -MURI::http # build a perl with the above modules linked in + staticperl mkapp myapp --boot mainprog mymodules + # build a binary "myapp" from mainprog and mymodules =head1 DESCRIPTION -This script helps you creating single-file perl interpreters, or embedding -a perl interpreter in your applications. Single-file means that it is -fully self-contained - no separate shared objects, no autoload fragments, -no .pm or .pl files are needed. And when linking statically, you can -create (or embed) a single file that contains perl interpreter, libc, all -the modules you need and all the libraries you need. +This script helps you to create single-file perl interpreters +or applications, or embedding a perl interpreter in your +applications. Single-file means that it is fully self-contained - no +separate shared objects, no autoload fragments, no .pm or .pl files are +needed. And when linking statically, you can create (or embed) a single +file that contains perl interpreter, libc, all the modules you need, all +the libraries you need and of course your actual program. With F and F on x86, you can create a single 500kb binary that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, -Coro and so on. Or any other choice of modules. +Coro and so on. Or any other choice of modules (and some other size :). + +To see how this turns out, you can try out smallperl and bigperl, two +pre-built static and compressed perl binaries with many and even more +modules: just follow the links at L. The created files do not need write access to the file system (like PAR does). In fact, since this script is in many ways similar to PAR::Packer, @@ -65,17 +74,21 @@ F loads all required files directly from memory. There is no need to unpack files into a temporary directory. -=item * More control over included files. +=item * More control over included files, more burden. PAR tries to be maintenance and hassle-free - it tries to include more -files than necessary to make sure everything works out of the box. The -extra files (such as the unicode database) can take substantial amounts of -memory and file size. +files than necessary to make sure everything works out of the box. It +mostly succeeds at this, but he extra files (such as the unicode database) +can take substantial amounts of memory and file size. With F, the burden is mostly with the developer - only direct compile-time dependencies and L are handled automatically. This means the modules to include often need to be tweaked manually. +All this does not preclude more permissive modes to be implemented in +the future, but right now, you have to resolve hidden dependencies +manually. + =item * PAR works out of the box, F does not. Maintaining your own custom perl build can be a pain in the ass, and while @@ -83,6 +96,11 @@ build and possibly fiddling with some modules. PAR is likely to produce results faster. +Ok, PAR never has worked for me out of the box, and for some people, +F does work out of the box, as they don't count "fiddling with +module use lists" against it, but nevertheless, F is certainly +a bit more difficult to use. + =back =head1 HOW DOES IT WORK? @@ -100,17 +118,19 @@ except everything is compiled in), or you create bundle files (basically C sources you can use to embed all files into your project). -This step is very fast (a few seconds if PPI is not used for stripping, -more seconds otherwise, as PPI is very slow), and can be tweaked and -repeated as often as necessary. +This step is very fast (a few seconds if PPI is not used for stripping, or +the stripped files are in the cache), and can be tweaked and repeated as +often as necessary. =head1 THE F SCRIPT This module installs a script called F into your perl -binary directory. The script is fully self-contained, and can be used -without perl (for example, in an uClibc chroot environment). In fact, -it can be extracted from the C distribution tarball as -F, without any installation. +binary directory. The script is fully self-contained, and can be +used without perl (for example, in an uClibc chroot environment). In +fact, it can be extracted from the C distribution +tarball as F, without any installation. The +newest (possibly alpha) version can also be downloaded from +L. F interprets the first argument as a command to execute, optionally followed by any parameters. @@ -122,7 +142,7 @@ =head2 PHASE 1 COMMANDS: INSTALLING PERL The most important command is F, which does basically -everything. The default is to download and install perl 5.12.2 and a few +everything. The default is to download and install perl 5.12.3 and a few modules required by F itself, but all this can (and should) be changed - see L, below. @@ -130,18 +150,27 @@ staticperl install -Is normally all you need: It installs the perl interpreter in +is normally all you need: It installs the perl interpreter in F<~/.staticperl/perl>. It downloads, configures, builds and installs the perl interpreter if required. -Most of the following commands simply run one or more steps of this -sequence. +Most of the following F subcommands simply run one or more +steps of this sequence. + +If it fails, then most commonly because the compiler options I selected +are not supported by your compiler - either edit the F script +yourself or create F<~/.staticperl> shell script where your set working +C etc. variables. To force recompilation or reinstallation, you need to run F first. =over 4 +=item F + +Prints some info about the version of the F script you are using. + =item F Runs only the download and unpack phase, unless this has already happened. @@ -160,12 +189,24 @@ Wipes the perl installation directory (usually F<~/.staticperl/perl>) and installs the perl distribution, potentially after building it first. +=item F [args...] + +Invokes the compiled perl interpreter with the given args. Basically the +same as starting perl directly (usually via F<~/.staticperl/bin/perl>), +but beats typing the path sometimes. + +Example: check that the Gtk2 module is installed and loadable. + + staticperl perl -MGtk2 -e0 + =item F [args...] Starts an interactive CPAN shell that you can use to install further modules. Installs the perl first if necessary, but apart from that, no magic is involved: you could just as well run it manually via -F<~/.staticperl/perl/bin/cpan>. +F<~/.staticperl/perl/bin/cpan>, except that F additionally +sets the environment variable C<$PERL> to the path of the perl +interpreter, which is handy in subshells. Any additional arguments are simply passed to the F command. @@ -186,9 +227,13 @@ =item F -Runs F in the perl source directory (and potentially -cleans up other intermediate files). This can be used to clean up -intermediate files without removing the installed perl interpreter. +Deletes the perl source directory (and potentially cleans up other +intermediate files). This can be used to clean up files only needed for +building perl, without removing the installed perl interpreter. + +At the moment, it doesn't delete downloaded tarballs. + +The exact semantics of this command will probably change. =item F @@ -222,7 +267,7 @@ staticperl instcpan AnyEvent::HTTPD # now build the perl - staticperl mkperl -M'"Config_heavy.pl"' -MAnyEvent::Impl::Perl \ + staticperl mkperl -MConfig_heavy.pl -MAnyEvent::Impl::Perl \ -MAnyEvent::HTTPD -MURI::http \ --add 'eg/httpd httpd.pm' @@ -238,20 +283,64 @@ to include that module. I found out about these dependencies by carefully watching any error messages about missing modules... +Instead of building a new perl binary, you can also build a standalone +application: + + # build the app + staticperl mkapp app --boot eg/httpd \ + -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI::http + + # run it + ./app + +Here are the three phase 2 commands: + +=over 4 + +=item F args... + +The "default" bundle command - it interprets the given bundle options and +writes out F, F, F and F +files, useful for embedding. + +=item F args... + +Creates a bundle just like F (in fact, it's the same +as invoking F args...), but then compiles and +links a new perl interpreter that embeds the created bundle, then deletes +all intermediate files. + +=item F filename args... + +Does the same as F (in fact, it's the same as +invoking F filename args...), but then compiles +and links a new standalone application that simply initialises the perl +interpreter. + +The difference to F is that the standalone application +does not act like a perl interpreter would - in fact, by default it would +just do nothing and exit immediately, so you should specify some code to +be executed via the F<--boot> option. + +=back + =head3 OPTION PROCESSING All options can be given as arguments on the command line (typically using long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since -specifying a lot of modules can make the command line very cumbersome, -you can put all long options into a "bundle specification file" (with or -without C<--> prefix) and specify this bundle file instead. +specifying a lot of options can make the command line very long and +unwieldy, you can put all long options into a "bundle specification file" +(one option per line, with or without C<--> prefix) and specify this +bundle file instead. -For example, the command given earlier could also look like this: +For example, the command given earlier to link a new F could also +look like this: staticperl mkperl httpd.bundle -And all options could be in F: - +With all options stored in the F file (one option per line, +everything after the option is an argument): + use "Config_heavy.pl" use AnyEvent::Impl::Perl use AnyEvent::HTTPD @@ -259,93 +348,127 @@ add eg/httpd httpd.pm All options that specify modules or files to be added are processed in the -order given on the command line (that affects the C<--use> and C<--eval> -options at the moment). +order given on the command line. + +=head3 BUNDLE CREATION WORKFLOW / STATICPELR MKBUNDLE OPTIONS -=head3 MKBUNDLE OPTIONS +F works by first assembling a list of candidate +files and modules to include, then filtering them by include/exclude +patterns. The remaining modules (together with their direct dependencies, +such as link libraries and L files) are then converted into +bundle files suitable for embedding. F can then +optionally build a new perl interpreter or a standalone application. =over 4 -=item --verbose | -v +=item Step 0: Generic argument processing. -Increases the verbosity level by one (the default is C<1>). +The following options influence F itself. -=item --quiet | -q +=over 4 -Decreases the verbosity level by one. +=item C<--verbose> | C<-v> -=item --strip none|pod|ppi +Increases the verbosity level by one (the default is C<1>). -Specify the stripping method applied to reduce the file of the perl -sources included. +=item C<--quiet> | C<-q> -The default is C, which uses the L module to remove all -pod documentation, which is very fast and reduces file size a lot. +Decreases the verbosity level by one. -The C method uses L to parse and condense the perl sources. This -saves a lot more than just L, and is generally safer, but -is also a lot slower, so is best used for production builds. Note that -this method doesn't optimise for raw file size, but for best compression -(that means that the uncompressed file size is a bit larger, but the files -compress better, e.g. with F). - -Last not least, in the unlikely case where C is too slow, or some -module gets mistreated, you can specify C to not mangle included -perl sources in any way. +=item any other argument -=item --perl +Any other argument is interpreted as a bundle specification file, which +supports all options (without extra quoting), one option per line, in the +format C, using it's +built-in ash shell. + +Finally, you need F inside the chroot for many scripts to work +- either F or bind-mounting your F +will provide this. + +After you have compiled and set up your buildroot target, you can copy +F from the C distribution or from your +perl F directory (if you installed it) into the F +filesystem, chroot inside and run it. + +=head1 RECIPES / SPECIFIC MODULES + +This section contains some common(?) recipes and information about +problems with some common modules or perl constructs that require extra +files to be included. + +=head2 MODULES + +=over 4 + +=item utf8 + +Some functionality in the utf8 module, such as swash handling (used +for unicode character ranges in regexes) is implemented in the +C<"utf8_heavy.pl"> library: + + -Mutf8_heavy.pl + +Many Unicode properties in turn are defined in separate modules, +such as C<"unicore/Heavy.pl"> and more specific data tables such as +C<"unicore/To/Digit.pl"> or C<"unicore/lib/Perl/Word.pl">. These tables +are big (7MB uncompressed, although F contains special +handling for those files), so including them on demand by your application +only might pay off. + +To simply include the whole unicode database, use: + + --incglob '/unicore/**.pl' + +=item AnyEvent + +AnyEvent needs a backend implementation that it will load in a delayed +fashion. The L backend is the default choice +for AnyEvent if it can't find anything else, and is usually a safe +fallback. If you plan to use e.g. L (L...), then you need to +include the L (L...) backend as +well. + +If you want to handle IRIs or IDNs (L punycode and idn +functions), you also need to include C<"AnyEvent/Util/idna.pl"> and +C<"AnyEvent/Util/uts46data.pl">. + +Or you can use C<--usepacklists> and specify C<-MAnyEvent> to include +everything. + +=item Cairo + +See Glib, same problem, same solution. + +=item Carp + +Carp had (in older versions of perl) a dependency on L. As of +perl 5.12.2 (maybe earlier), this dependency no longer exists. + +=item Config + +The F switch (as well as many modules) needs L, which in +turn might need L<"Config_heavy.pl">. Including the latter gives you +both. + +=item Glib + +Glib literally requires Glib to be installed already to build - it tries +to fake this by running Glib out of the build directory before being +built. F tries to work around this by forcing C and +C to be empty via the C environment variable. + +=item Gtk2 + +See Pango, same problems, same solution. + +=item Pango + +In addition to the C problem in Glib, Pango also routes around +L by compiling its files on its own. F +tries to patch L to route around Pango. + +=item Term::ReadLine::Perl + +Also needs L, or C<--usepacklists>. + +=item URI + +URI implements schemes as separate modules - the generic URL scheme is +implemented in L, HTTP is implemented in L. If +you need to use any of these schemes, you should include these manually, +or use C<--usepacklists>. + +=back + +=head2 RECIPES + +=over 4 + +=item Just link everything in + +To link just about everything installed in the perl library into a new +perl, try this (the first time this runs it will take a long time, as a +lot of files need to be parsed): + + staticperl mkperl -v --strip ppi --incglob '*' + +If you don't mind the extra megabytes, this can be a very effective way of +creating bundles without having to worry about forgetting any modules. + +You get even more useful variants of this method by first selecting +everything, and then excluding stuff you are reasonable sure not to need - +L uses this approach. + +=item Getting rid of netdb functions + +The perl core has lots of netdb functions (C, C +and so on) that few applications use. You can avoid compiling them in by +putting the following fragment into a C hook: + + preconfigure() { + for sym in \ + d_getgrnam_r d_endgrent d_endgrent_r d_endhent \ + d_endhostent_r d_endnent d_endnetent_r d_endpent \ + d_endprotoent_r d_endpwent d_endpwent_r d_endsent \ + d_endservent_r d_getgrent d_getgrent_r d_getgrgid_r \ + d_getgrnam_r d_gethbyaddr d_gethent d_getsbyport \ + d_gethostbyaddr_r d_gethostbyname_r d_gethostent_r \ + d_getlogin_r d_getnbyaddr d_getnbyname d_getnent \ + d_getnetbyaddr_r d_getnetbyname_r d_getnetent_r \ + d_getpent d_getpbyname d_getpbynumber d_getprotobyname_r \ + d_getprotobynumber_r d_getprotoent_r d_getpwent \ + d_getpwent_r d_getpwnam_r d_getpwuid_r d_getsent \ + d_getservbyname_r d_getservbyport_r d_getservent_r \ + d_getspnam_r d_getsbyname + # d_gethbyname + do + PERL_CONFIGURE="$PERL_CONFIGURE -U$sym" + done + } + +This mostly gains space when linking statically, as the functions will +likely not be linked in. The gain for dynamically-linked binaries is +smaller. + +Also, this leaves C in - not only is it actually used +often, the L module also exposes it, so leaving it out usually +gains little. Why Socket exposes a C function that is in the core already +is anybody's guess. + +=back + =head1 AUTHOR Marc Lehmann