[ViewVC] Diff of: cvs/App-Staticperl/staticperl.pod

Comparing App-Staticperl/staticperl.pod (file contents):
Revision 1.17 by root, Thu Dec 9 08:55:52 2010 UTC vs.
Revision 1.24 by root, Wed Dec 15 00:17:47 2010 UTC

 With F<uClibc> and F<upx> on x86, you can create a single 500kb binary
 that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO,
 Coro and so on. Or any other choice of modules.
+To see how this turns out, you can try out smallperl and bigperl, two
+pre-built static and compressed perl binaries with many and even more
+modules: just follow the links at L<http://staticperl.schmorp.de/>.
 The created files do not need write access to the file system (like PAR
 does). In fact, since this script is in many ways similar to PAR::Packer,
 here are the differences:
 =over 4
 Afterwards, you create a list of files and modules you want to include,
 and then either build a new perl binary (that acts just like a normal perl
 except everything is compiled in), or you create bundle files (basically C
 sources you can use to embed all files into your project).
-This step is very fast (a few seconds if PPI is not used for stripping,
+This step is very fast (a few seconds if PPI is not used for stripping, or
-more seconds otherwise, as PPI is very slow), and can be tweaked and
+the stripped files are in the cache), and can be tweaked and repeated as
-repeated as often as necessary.
+often as necessary.
 =head1 THE F<STATICPERL> SCRIPT
 This module installs a script called F<staticperl> into your perl
-binary directory. The script is fully self-contained, and can be used
+binary directory. The script is fully self-contained, and can be
-without perl (for example, in an uClibc chroot environment). In fact,
+used without perl (for example, in an uClibc chroot environment). In
-it can be extracted from the C<App::Staticperl> distribution tarball as
+fact, it can be extracted from the C<App::Staticperl> distribution
-F<bin/staticperl>, without any installation.
+tarball as F<bin/staticperl>, without any installation. The
+newest (possibly alpha) version can also be downloaded from
+L<http://staticperl.schmorp.de/staticperl>.
 F<staticperl> interprets the first argument as a command to execute,
 optionally followed by any parameters.
 There are two command categories: the "phase 1" commands which deal with
 The command
    staticperl install
-Is normally all you need: It installs the perl interpreter in
+is normally all you need: It installs the perl interpreter in
 F<~/.staticperl/perl>. It downloads, configures, builds and installs the
 perl interpreter if required.
-Most of the following commands simply run one or more steps of this
+Most of the following F<staticperl> subcommands simply run one or more
-sequence.
+steps of this sequence.
+If it fails, then most commonly because the compiler options I selected
+are not supported by your compiler - either edit the F<staticperl> script
+yourself or create F<~/.staticperl> shell script where your set working
+C<PERL_CCFLAGS> etc. variables.
 To force recompilation or reinstallation, you need to run F<staticperl
 distclean> first.
 =over 4
+=item F<staticperl version>
+Prints some info about the version of the F<staticperl> script you are using.
 =item F<staticperl fetch>
 Runs only the download and unpack phase, unless this has already happened.
 =item F<staticperl clean>
 Deletes the perl source directory (and potentially cleans up other
 intermediate files). This can be used to clean up files only needed for
-building perl, without removing the installed perl interpreter, or to
+building perl, without removing the installed perl interpreter.
-force a re-build from scratch.
 At the moment, it doesn't delete downloaded tarballs.
+The exact semantics of this command will probably change.
 =item F<staticperl distclean>
 This wipes your complete F<~/.staticperl> directory. Be careful with this,
 it nukes your perl download, perl sources, perl distribution and any
 All options that specify modules or files to be added are processed in the
 order given on the command line (that affects the C<--use> and C<--eval>
 options at the moment).
+=head3 PACKAGE SELECTION WORKFLOW
+F<staticperl mkbundle> has a number of options to control package
+selection. This section describes how they interact with each other. Also,
+since I am still a newbie w.r.t. these issues, maybe future versions of
+F<staticperl> will change this, so watch out :)
+The idiom "in order" means "in order that they are specified on the
+commandline". If you use a bundle specification file, then the options
+will be processed as if they were given in place of the bundle file name.
+=over 4
+=item 1. apply all C<--use>, C<--eval>, C<--add>, C<--addbin> and
+C<--incglob> options, in order.
+In addition, C<--use> and C<--eval> dependencies will be added when the
+options are processed.
+=item 2. apply all C<--include> and C<--exclude> options, in order.
+All this step does is potentially reduce the number of files already
+selected or found in phase 1.
+=item 3. find all modules (== F<.pm> files), gather their static archives
+(F<.a>) and AutoLoader splitfiles (F<.ix> and F<.al> files), find any
+extra libraries they need for linking (F<extralibs.ld>) and optionally
+evaluate any F<.packlist> files.
+This step is required to link against XS extensions and also adds files
+required for L<AutoLoader> to do it's job.
+=back
+After this, all the files selected for bundling will be read and processed
+(stripped), the bundle files will be written, and optionally a new F<perl>
+or application binary will be linked.
 =head3 MKBUNDLE OPTIONS
 =over 4
 =item --verbose | -v
 The default is C<pod>, which uses the L<Pod::Strip> module to remove all
 pod documentation, which is very fast and reduces file size a lot.
 The C<ppi> method uses L<PPI> to parse and condense the perl sources. This
-saves a lot more than just L<Pod::Strip>, and is generally safer, but
+saves a lot more than just L<Pod::Strip>, and is generally safer,
-is also a lot slower, so is best used for production builds. Note that
+but is also a lot slower (some files take almost a minute to strip -
-this method doesn't optimise for raw file size, but for best compression
+F<staticperl> maintains a cache of stripped files to speed up subsequent
-(that means that the uncompressed file size is a bit larger, but the files
+runs for this reason). Note that this method doesn't optimise for raw file
-compress better, e.g. with F<upx>).
+size, but for best compression (that means that the uncompressed file size
+is a bit larger, but the files compress better, e.g. with F<upx>).
 Last not least, if you need accurate line numbers in error messages,
 or in the unlikely case where C<pod> is too slow, or some module gets
 mistreated, you can specify C<none> to not mangle included perl sources in
 any way.
 (using a C<require>) before anything else when the new perl is
 initialised. This can be used to modify C<@INC> or anything else before
 the perl interpreter executes scripts given on the command line (or via
 C<-e>). This works even in an embedded interpreter.
-=item --add "file" | --add "file alias"
+=item --usepacklist
+Read F<.packlist> files for each distribution that happens to match a
+module name you specified. Sounds weird, and it is, so expect semantics to
+change somehow in the future.
+The idea is that most CPAN distributions have a F<.pm> file that matches
+the name of the distribution (which is rather reasonable after all).
+If this switch is enabled, then if any of the F<.pm> files that have been
+selected match an install distribution, then all F<.pm>, F<.pl>, F<.al>
+and F<.ix> files installed by this distribution are also included.
+For example, using this switch, when the L<URI> module is specified, then
+all L<URI> submodules that have been installed via the CPAN distribution
+are included as well, so you don't have to manually specify them.
+=item --incglob pattern
+This goes through all library directories and tries to match any F<.pm>
+and F<.pl> files against the extended glob pattern (see below). If a file
+matches, it is added. This switch will automatically detect L<AutoLoader>
+files and the required link libraries for XS modules, but it will I<not>
+scan the file for dependencies (at the moment).
+This is mainly useful to include "everything":
+   --incglob '*'
+Or to include perl libraries, or trees of those, such as the unicode
+database files needed by many other modules:
+   --incglob '/unicore/**.pl'
+=item --add file | --add "file alias"
 Adds the given (perl) file into the bundle (and optionally call it
 "alias"). This is useful to include any custom files into the bundle.
 Example: embed the file F<httpd> as F<httpd.pm> when creating the bundle.
    # specification file
    add file1 myfiles/file1
    add file2 myfiles/file2
    add file3 myfiles/file3
-=item --binadd "file" | --add "file alias"
+=item --binadd file | --add "file alias"
 Just like C<--add>, except that it treats the file as binary and adds it
 without any processing.
 You should probably add a C</> prefix to avoid clashing with embedded
 perl files (whose paths do not start with C</>), and/or use a special
 directory, such as C</res/name>.
 You can later get a copy of these files by calling C<staticperl::find
 "alias">.
+=item --include pattern | -i pattern | --exclude pattern | -x pattern
+These two options define an include/exclude filter that is used after all
+files selected by the other options have been found. Each include/exclude
+is applied to all files found so far - an include makes sure that the
+given files will be part of the resulting file set, an exclude will
+exclude files. The patterns are "extended glob patterns" (see below).
+For example, to include everything, except C<Devel> modules, but still
+include F<Devel::PPPort>, you could use this:
+   --incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**'
 =item --static
 When C<--perl> is also given, link statically instead of dynamically. The
 default is to link the new perl interpreter fully dynamic (that means all
 systems based on GNU libc don't really support it in a usable fashion
 either. Try uClibc if you want to create fully statically linked
 executables, or try the C<--staticlibs> option to link only some libraries
 statically.
+=item --staticlib libname
+When not linking fully statically, this option allows you to link specific
+libraries statically. What it does is simply replace all occurances of
+C<-llibname> with the GCC-specific C<-Wl,-Bstatic -llibname -Wl,-Bdynamic>
+option.
+This will have no effect unless the library is actually linked against,
+specifically, C<--staticlib> will not link against the named library
+unless it would be linked against anyway.
+Example: link libcrypt statically into the binary.
+   staticperl mkperl -MIO::AIO --staticlib crypt
+   # ldopts might nwo contain:
+   # -lm -Wl,-Bstatic -lcrypt -Wl,-Bdynamic -lpthread
 =item any other argument
 Any other argument is interpreted as a bundle specification file, which
 supports most long options (without extra quoting), one option per line.
 =back
+=head3 EXTENDED GLOB PATTERNS
+Some options of F<staticperl mkbundle> expect an I<extended glob
+pattern>. This is neither a normal shell glob nor a regex, but something
+in between. The idea has been copied from rsync, and there are the current
+matching rules:
+=over 4
+=item Patterns starting with F</> will be a anchored at the root of the library tree.
+That is, F</unicore> will match the F<unicore> directory in C<@INC>, but
+nothing inside, and neither any other file or directory called F<unicore>
+anywhere else in the hierarchy.
+=item Patterns not starting with F</> will be anchored at the end of the path.
+That is, F<idna.pl> will match any file called F<idna.pl> anywhere in the
+hierarchy, but not any directories of the same name.
+=item A F<*> matches any single component.
+That is, F</unicore/*.pl> would match all F<.pl> files directly inside
+C</unicore>, not any deeper level F<.pl> files. Or in other words, F<*>
+will not match slashes.
+=item A F<**> matches anything.
+That is, F</unicore/**.pl> would match all F<.pl> files under F</unicore>,
+no matter how deeply nested they are inside subdirectories.
+=item A F<?> matches a single character within a component.
+That is, F</Encode/??.pm> matches F</Encode/JP.pm>, but not the
+hypothetical F</Encode/J/.pm>, as F<?> does not match F</>.
+=back
 =head2 F<STATICPERL> CONFIGURATION AND HOOKS
-During (each) startup, F<staticperl> tries to source the following shell
+During (each) startup, F<staticperl> tries to source some shell files to
+allow you to fine-tune/override configuration settings.
+In them you can override shell variables, or define shell functions
+("hooks") to be called at specific phases during installation. For
+example, you could define a C<postinstall> hook to install additional
+modules from CPAN each time you start from scratch.
+If the env variable C<$STATICPERLRC> is set, then F<staticperl> will try
+to source the file named with it only. Otherwise, it tries the following
-files in order:
+shell files in order:
    /etc/staticperlrc
    ~/.staticperlrc
    $STATICPERL/rc
-They can be used to override shell variables, or define functions to be
-called at specific phases.
 Note that the last file is erased during F<staticperl distclean>, so
 generally should not be used.
 =head3 CONFIGURATION VARIABLES
 More commonly, you would either activate 64 bit integer support
 (C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to
 reduce filesize further.
-=item C<PERL_CPPFLAGS>, C<PERL_OPTIMIZE>, C<PERL_LDFLAGS>, C<PERL_LIBS>
+=item C<PERL_CC>, C<PERL_CCFLAGS>, C<PERL_OPTIMIZE>, C<PERL_LDFLAGS>, C<PERL_LIBS>
 These flags are passed to perl's F<Configure> script, and are generally
 optimised for small size (at the cost of performance). Since they also
 contain subtle workarounds around various build issues, changing these
-usually requires understanding their default values - best look at the top
+usually requires understanding their default values - best look at
-of the F<staticperl> script for more info on these.
+the top of the F<staticperl> script for more info on these, and use a
+F<~/.staticperlrc> to override them.
+Most of the variables override (or modify) the corresponding F<Configure>
+variable, except C<PERL_CCFLAGS>, which gets appended.
 =back
 =head4 Variables you probably I<do not want> to override
 =over 4
+=item C<MAKE>
+The make command to use - default is C<make>.
 =item C<MKBUNDLE>
 Where F<staticperl> writes the C<mkbundle> command to
 (default: F<$STATICPERL/mkbundle>).
 =item utf8
 Some functionality in the utf8 module, such as swash handling (used
 for unicode character ranges in regexes) is implemented in the
-C<"utf8_heavy.pl"> library.
+C<"utf8_heavy.pl"> library:
+   -M'"utf8_heavy.pl"'
 Many Unicode properties in turn are defined in separate modules,
 such as C<"unicore/Heavy.pl"> and more specific data tables such as
-C<"unicore/To/Digit.pl"> or C<"unicore/lib/Perl/Word.pl">. These
+C<"unicore/To/Digit.pl"> or C<"unicore/lib/Perl/Word.pl">. These tables
-tables are big (7MB uncompressed), so including them on demand by your
+are big (7MB uncompressed, although F<staticperl> contains special
+handling for those files), so including them on demand by your application
-applciation only might pay off.
+only might pay off.
-=item Carp
+To simply include the whole unicode database, use:
-Carp had (in older versions of perl) a dependency on L<Carp::Heavy>. As of
+   --incglob '/unicore/*.pl'
-perl 5.12.2 (maybe earlier), this dependency no longer exists.
-=item Config
-The F<perl -V> switch (as well as many modules) needs L<Config>, which in
-turn might need L<"Config_heavy.pl">. Including the latter gives you
-both.
 =item AnyEvent
 AnyEvent needs a backend implementation that it will load in a delayed
 fashion. The L<AnyEvent::Impl::Perl> backend is the default choice
 If you want to handle IRIs or IDNs (L<AnyEvent::Util> punycode and idn
 functions), you also need to include C<"AnyEvent/Util/idna.pl"> and
 C<"AnyEvent/Util/uts46data.pl">.
+Or you can use C<--usepacklist> and specify C<-MAnyEvent> to include
+everything.
+=item Carp
+Carp had (in older versions of perl) a dependency on L<Carp::Heavy>. As of
+perl 5.12.2 (maybe earlier), this dependency no longer exists.
+=item Config
+The F<perl -V> switch (as well as many modules) needs L<Config>, which in
+turn might need L<"Config_heavy.pl">. Including the latter gives you
+both.
+=item Term::ReadLine::Perl
+Also needs L<Term::ReadLine::readline>, or C<--usepacklist>.
 =item URI
 URI implements schemes as separate modules - the generic URL scheme is
 implemented in L<URI::_generic>, HTTP is implemented in L<URI::http>. If
-you need to use any of these schemes, you should include these manually.
+you need to use any of these schemes, you should include these manually,
+or use C<--usepacklist>.
 =back
 =head2 RECIPES
 =over 4
+=item Linking everything in
+To link just about everything installed in the perl library into a new
+perl, try this:
+   staticperl mkperl --strip ppi --incglob '*'
 =item Getting rid of netdb function
 The perl core has lots of netdb functions (C<getnetbyname>, C<getgrent>
 and so on) that few applications use. You can avoid compiling them in by
          PERL_CONFIGURE="$PERL_CONFIGURE -U$sym"
       done
    }
 This mostly gains space when linking staticaly, as the functions will
-liekly not be linked in. The gain for dynamically-linked binaries is
+likely not be linked in. The gain for dynamically-linked binaries is
 smaller.
 Also, this leaves C<gethostbyname> in - not only is it actually used
 often, the L<Socket> module also exposes it, so leaving it out usually
 gains little. Why Socket exposes a C function that is in the core already

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing App-Staticperl/staticperl.pod (file contents): Revision 1.17 by root, Thu Dec 9 08:55:52 2010 UTC vs. Revision 1.24 by root, Wed Dec 15 00:17:47 2010 UTC

Diff Legend

Comparing App-Staticperl/staticperl.pod (file contents):
Revision 1.17 by root, Thu Dec 9 08:55:52 2010 UTC vs.
Revision 1.24 by root, Wed Dec 15 00:17:47 2010 UTC