[ViewVC] Diff of: cvs/App-Staticperl/staticperl.pod

Comparing App-Staticperl/staticperl.pod (file contents):
Revision 1.2 by root, Mon Dec 6 20:53:44 2010 UTC vs.
Revision 1.13 by root, Tue Dec 7 19:55:56 2010 UTC

 =head1 NAME
-staticperl - perl, libc, 50 modules all in one 500kb file
+staticperl - perl, libc, 100 modules, all in one 500kb file
 =head1 SYNOPSIS
    staticperl help      # print the embedded documentation
    staticperl fetch     # fetch and unpack perl sources
                         # build a perl with the above modules linked in
 =head1 DESCRIPTION
 This script helps you creating single-file perl interpreters, or embedding
-a pelr interpreter in your apps. Single-file means that it is fully
+a perl interpreter in your applications. Single-file means that it is
-self-contained - no separate shared objects, no autoload fragments, no .pm
+fully self-contained - no separate shared objects, no autoload fragments,
-or .pl files are needed. And when linking statically, you can create (or
+no .pm or .pl files are needed. And when linking statically, you can
-embed) a single file that contains perl interpreter, libc, all the modules
+create (or embed) a single file that contains perl interpreter, libc, all
-you need and all the libraries you need.
+the modules you need and all the libraries you need.
-With uclibc and upx on x86, you can create a single 500kb binary that
+With F<uClibc> and F<upx> on x86, you can create a single 500kb binary
-contains perl and 50 modules such as AnyEvent, EV, IO::AIO, Coro and so
+that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO,
-on. Or any other choice of modules.
+Coro and so on. Or any other choice of modules.
-The created files do not need write access to the filesystem (like PAR
+The created files do not need write access to the file system (like PAR
 does). In fact, since this script is in many ways similar to PAR::Packer,
 here are the differences:
 =over 4
 F<staticperl> loads all required files directly from memory. There is no
 need to unpack files into a temporary directory.
 =item * More control over included files.
-PAR tries to be maintainance and hassle-free - it tries to include more files
+PAR tries to be maintenance and hassle-free - it tries to include more
-than necessary to make sure everything works out of the box. The extra files
+files than necessary to make sure everything works out of the box. The
-(such as the unicode database) can take substantial amounts of memory and filesize.
+extra files (such as the unicode database) can take substantial amounts of
+memory and file size.
 With F<staticperl>, the burden is mostly with the developer - only direct
 compile-time dependencies and L<AutoLoader> are handled automatically.
 This means the modules to include often need to be tweaked manually.
 Maintaining your own custom perl build can be a pain in the ass, and while
 F<staticperl> tries to make this easy, it still requires a custom perl
 build and possibly fiddling with some modules. PAR is likely to produce
 results faster.
+Ok, PAR never has worked for me out of the box, and for some people,
+F<staticperl> does work out of the box, as they don't count "fiddling with
+module use lists" against it, but nevertheless, F<staticperl> is certainly
+a bit more difficult to use.
 =back
 =head1 HOW DOES IT WORK?
 Simple: F<staticperl> downloads, compile and installs a perl version of
 your choice in F<~/.staticperl>. You can add extra modules either by
 letting F<staticperl> install them for you automatically, or by using CPAN
 and doing it interactively. This usually takes 5-10 minutes, depending on
-the speed of your computer and your internet conenction.
+the speed of your computer and your internet connection.
 It is possible to do program development at this stage, too.
 Afterwards, you create a list of files and modules you want to include,
-and then either build a new perl binary (that acts just like a normla perl
+and then either build a new perl binary (that acts just like a normal perl
 except everything is compiled in), or you create bundle files (basically C
 sources you can use to embed all files into your project).
 This step is very fast (a few seconds if PPI is not used for stripping,
 more seconds otherwise, as PPI is very slow), and can be tweaked and
 perl interpreter if required.
 Most of the following commands simply run one or more steps of this
 sequence.
-To force recompilation or reinstalaltion, you need to run F<staticperl
+To force recompilation or reinstallation, you need to run F<staticperl
 distclean> first.
 =over 4
 =item F<staticperl fetch>
 Builds the configured perl sources, potentially after automatically
 configuring them.
 =item F<staticperl install>
-Wipes the perl installation directory (usually F<~/.staticperl/perl>) and installs
+Wipes the perl installation directory (usually F<~/.staticperl/perl>) and
-the perl distribution, potentially aftering building it first.
+installs the perl distribution, potentially after building it first.
 =item F<staticperl cpan> [args...]
-Starts an interactive CPAN shell that you cna use to install further
+Starts an interactive CPAN shell that you can use to install further
-modules. Installs the perl first if neccessary, but apart from that,
+modules. Installs the perl first if necessary, but apart from that,
 no magic is involved: you could just as well run it manually via
 F<~/.staticperl/perl/bin/cpan>.
 Any additional arguments are simply passed to the F<cpan> command.
    staticperl instcpan EV AnyEvent::HTTPD Coro
 =item F<staticperl instsrc> directory...
 In the unlikely case that you have unpacked perl modules around and want
-to install from these instead of from CPAN, you cna do this using this
+to install from these instead of from CPAN, you can do this using this
 command by specifying all the directories with modules in them that you
 want to have built.
 =item F<staticperl clean>
-Runs F<make distclean> in the perl source directory (and potentially
+Deletes the perl source directory (and potentially cleans up other
-cleans up other intermediate files). This can be used to clean up
+intermediate files). This can be used to clean up files only needed for
-intermediate files without removing the installed perl interpreter.
+building perl, without removing the installed perl interpreter, or to
+force a re-build from scratch.
+At the moment, it doesn't delete downloaded tarballs.
 =item F<staticperl distclean>
 This wipes your complete F<~/.staticperl> directory. Be careful with this,
 it nukes your perl download, perl sources, perl distribution and any
 In the oh so unlikely case of something not working here, you
 can run the script manually as well (by default it is written to
 F<~/.staticperl/mkbundle>).
 F<mkbundle> is a more conventional command and expect the argument
-syntax commonly used on unix clones. For example, this command builds
+syntax commonly used on UNIX clones. For example, this command builds
 a new F<perl> binary and includes F<Config.pm> (for F<perl -V>),
 F<AnyEvent::HTTPD>, F<URI> and a custom F<httpd> script (from F<eg/httpd>
 in this distribution):
    # first make sure we have perl and the required modules
    ./perl -Mhttpd
 As you can see, things are not quite as trivial: the L<Config> module has
 a hidden dependency which is not even a perl module (F<Config_heavy.pl>),
 L<AnyEvent> needs at least one event loop backend that we have to
-specifymanually (here L<AnyEvent::Impl::Perl>), and the F<URI> module
+specify manually (here L<AnyEvent::Impl::Perl>), and the F<URI> module
 (required by L<AnyEvent::HTTPD>) implements various URI schemes as extra
 modules - since L<AnyEvent::HTTPD> only needs C<http> URIs, we only need
-to include that module.
+to include that module. I found out about these dependencies by carefully
+watching any error messages about missing modules...
 =head3 OPTION PROCESSING
-All options can be given as arguments on the commandline (typically using
+All options can be given as arguments on the command line (typically
-long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since
+using long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since
-specifying a lot of modules can make the commandlien very cumbersome,
+specifying a lot of modules can make the command line very cumbersome,
 you can put all long options into a "bundle specification file" (with or
 without C<--> prefix) and specify this bundle file instead.
 For example, the command given earlier could also look like this:
    use AnyEvent::HTTPD
    use URI::http
    add eg/httpd httpd.pm
 All options that specify modules or files to be added are processed in the
-order given on the commandline (that affects the C<--use> and C<--eval>
+order given on the command line (that affects the C<--use> and C<--eval>
 options at the moment).
 =head3 MKBUNDLE OPTIONS
 =over 4
 Specify the stripping method applied to reduce the file of the perl
 sources included.
 The default is C<pod>, which uses the L<Pod::Strip> module to remove all
-pod documenatiton, which is very fast and reduces filesize a lot.
+pod documentation, which is very fast and reduces file size a lot.
 The C<ppi> method uses L<PPI> to parse and condense the perl sources. This
-saves a lot more than just L<Pod::Strip>, and is generally safer, but is
+saves a lot more than just L<Pod::Strip>, and is generally safer, but
-also a lot slower, so is best used for production builds.
+is also a lot slower, so is best used for production builds. Note that
+this method doesn't optimise for raw file size, but for best compression
+(that means that the uncompressed file size is a bit larger, but the files
+compress better, e.g. with F<upx>).
+Last not least, if you need accurate line numbers in error messages,
-Last not least, in the unlikely case where C<pod> is too slow, or some
+or in the unlikely case where C<pod> is too slow, or some module gets
-module gets mistreated, you can specify C<none> to not mangle included
+mistreated, you can specify C<none> to not mangle included perl sources in
-perl sources in any way.
+any way.
 =item --perl
 After writing out the bundle files, try to link a new perl interpreter. It
 will be called F<perl> and will be left in the current working
 directory. The bundle files will be removed.
-This switch is automatically ued when F<staticperl> is invoked with the
+This switch is automatically used when F<staticperl> is invoked with the
 C<mkperl> command (instead of C<mkbundle>):
    # build a new ./perl with only common::sense in it - very small :)
    staticperl mkperl -Mcommon::sense
    staticperl mkbundle --use AnyEvent --use AnyEvent::Impl::Perl
 Sometimes you want to load old-style "perl libraries" (F<.pl> files), or
 maybe other weirdly named files. To do that, you need to quote the name in
-single or double quoutes. When given on the commandline, you probably need
+single or double quotes. When given on the command line, you probably need
 to quote once more to avoid your shell interpreting it. Common cases that
 need this are F<Config_heavy.pl> and F<utf8_heavy.pl>.
 Example: include the required files for F<perl -V> to work in all its
 glory (F<Config.pm> is included automatically by this).
 variables or whatever you need. All files C<require>'d or C<use>'d in the
 script are included in the final bundle.
 Keep in mind that F<mkbundle> will only C<require> the modules named
 by the C<--use> option, so do not expect the symbols from modules you
-C<--use>'d earlier on the commandlien to be available.
+C<--use>'d earlier on the command line to be available.
 Example: force L<AnyEvent> to detect a backend and therefore include it
 in the final bundle.
    staticperl mkbundle --eval 'use AnyEvent; AnyEvent::detect'
 =item --boot filename
 Include the given file in the bundle and arrange for it to be executed
 (using a C<require>) before anything else when the new perl is
 initialised. This can be used to modify C<@INC> or anything else before
-the perl interpreter executes scripts given on the commandline (or via
+the perl interpreter executes scripts given on the command line (or via
 C<-e>). This works even in an embedded interpreter.
 =item --add "file" | --add "file alias"
 Adds the given (perl) file into the bundle (and optionally call it
    # specification file
    add file1 myfiles/file1
    add file2 myfiles/file2
    add file3 myfiles/file3
+=item --binadd "file" | --add "file alias"
+Just like C<--add>, except that it treats the file as binary and adds it
+without any processing.
+You should probably add a C</> prefix to avoid clashing with embedded
+perl files (whose paths do not start with C</>), and/or use a special
+directory, such as C</res/name>.
+You can later get a copy of these files by calling C<staticperl::find
+"alias">.
 =item --static
 When C<--perl> is also given, link statically instead of dynamically. The
 default is to link the new perl interpreter fully dynamic (that means all
 =item C<EMAIL>
 The e-mail address of the person who built this binary. Has no good
 default, so should be specified by you.
-=back
+=item C<CPAN>
+The URL of the CPAN mirror to use (e.g. L<http://mirror.netcologne.de/cpan/>).
+=item C<EXTRA_MODULES>
+Additional modules installed during F<staticperl install>. Here you can
+set which modules you want have to installed from CPAN.
+Example: I really really need EV, AnyEvent, Coro and AnyEvent::AIO.
+   EXTRA_MODULES="EV AnyEvent Coro AnyEvent::AIO"
+Note that you can also use a C<postinstall> hook to achieve this, and
+more.
+=back
-=head4 Variables you I<might want> to override
+=head4 Variables you might I<want> to override
 =over 4
+=item C<STATICPERL>
+The directory where staticperl stores all its files
+(default: F<~/.staticperl>).
+=item C<PERL_MM_USE_DEFAULT>, C<EV_EXTRA_DEFS>, ...
+Usually set to C<1> to make modules "less inquisitive" during their
+installation, you can set any environment variable you want - some modules
+(such as L<Coro> or L<EV>) use environment variables for further tweaking.
-=item C<PERLVER>
+=item C<PERL_VERSION>
 The perl version to install - default is currently C<5.12.2>, but C<5.8.9>
 is also a good choice (5.8.9 is much smaller than 5.12.2, while 5.10.1 is
 about as big as 5.12.2).
-=item C<CPAN>
+=item C<PERL_PREFIX>
-The URL of the CPAN mirror to use (e.g. L<http://mirror.netcologne.de/cpan/>).
+The prefix where perl gets installed (default: F<$STATICPERL/perl>),
+i.e. where the F<bin> and F<lib> subdirectories will end up.
+=item C<PERL_CONFIGURE>
+Additional Configure options - these are simply passed to the perl
+Configure script. For example, if you wanted to enable dynamic loading,
+you could pass C<-Dusedl>. To enable ithreads (Why would you want that
+insanity? Don't! Use L<forks> instead!) you would pass C<-Duseithreads>
+and so on.
+More commonly, you would either activate 64 bit integer support
+(C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to
+reduce filesize further.
 =item C<PERL_CPPFLAGS>, C<PERL_OPTIMIZE>, C<PERL_LDFLAGS>, C<PERL_LIBS>
 These flags are passed to perl's F<Configure> script, and are generally
 optimised for small size (at the cost of performance). Since they also
 contain subtle workarounds around various build issues, changing these
 usually requires understanding their default values - best look at the top
 of the F<staticperl> script for more info on these.
-=item C<STATICPERL>
-The directory where staticperl stores all its files
-(default: F<~/.staticperl>).
-=item C<PREFIX>
-The prefix where perl get's installed (default: F<$STATICPERL/perl>),
-i.e. where the F<bin> and F<lib> subdirectories will end up.
-=item C<PERL_MM_USE_DEFAULT>, C<EV_EXTRA_DEFS>, others
-Usually set to C<1> to make modules "less inquisitive" during their
-installation, you can set any environment variable you want - some modules
-(such as L<Coro> or L<EV>) use environment variables for further tweaking.
-=item C<EXTRA_MODULES>
-Additional modules installed during F<staticperl install>. Here you can
-set which modules you want have to installed from CPAN.
-Example: I really really need EV, AnyEvent, Coro and IO::AIO.
-   EXTRA_MODULES="EV AnyEvent Coro IO::AIO"
-Note that you cna also use a C<postinstall> hook to achieve this, and
-more.
 =back
-=head4 Variables you I<probably do not want> to override
+=head4 Variables you probably I<do not want> to override
 =over 4
 =item C<MKBUNDLE>
 =head3 OVERRIDABLE HOOKS
 In addition to environment variables, it is possible to provide some
 shell functions that are called at specific times. To provide your own
-commands, justd efine the corresponding function.
+commands, just define the corresponding function.
 Example: install extra modules from CPAN and from some directories
 at F<staticperl install> time.
    postinstall() {
-      rm -rf lib/threads.* # weg mit Schaden
+      rm -rf lib/threads* # weg mit Schaden
       instcpan IO::AIO EV
       instsrc ~/src/AnyEvent
       instsrc ~/src/XML-Sablotron-1.0100001
-      instcpan AnyEvent::HTTPD
+      instcpan Anyevent::AIO AnyEvent::HTTPD
    }
 =over 4
+=item preconfigure
+Called just before running F<./Configur> in the perl source
+directory. Current working directory is the perl source directory.
+This can be used to set any C<PERL_xxx> variables, which might be costly
+to compute.
 =item postconfigure
 Called after configuring, but before building perl. Current working
 directory is the perl source directory.
-Could be used to tailor/patch config.sh (followed by F<./Configure -S>) or
+Could be used to tailor/patch config.sh (followed by F<sh Configure -S>)
-do any other modifications.
+or do any other modifications.
 =item postbuild
 Called after building, but before installing perl. Current working
 directory is the perl source directory.
 The script must return with a zero exit status, or the installation will
 fail.
 =back
+=head1 ANATOMY OF A BUNDLE
+When not building a new perl binary, C<mkbundle> will leave a number of
+files in the current working directory, which can be used to embed a perl
+interpreter in your program.
+Intimate knowledge of L<perlembed> and preferably some experience with
+embedding perl is highly recommended.
+C<mkperl> (or the C<--perl> option) basically does this to link the new
+interpreter (it also adds a main program to F<bundle.>):
+   $Config{cc} $(cat bundle.ccopts) -o perl bundle.c $(cat bundle.ldopts)
+=over 4
+=item bundle.h
+A header file that contains the prototypes of the few symbols "exported"
+by bundle.c, and also exposes the perl headers to the application.
+=over 4
+=item staticperl_init ()
+Initialises the perl interpreter. You can use the normal perl functions
+after calling this function, for example, to define extra functions or
+to load a .pm file that contains some initialisation code, or the main
+program function:
+   XS (xsfunction)
+   {
+     dXSARGS;
+     // now we have items, ST(i) etc.
+   }
+   static void
+   run_myapp(void)
+   {
+      staticperl_init ();
+      newXSproto ("myapp::xsfunction", xsfunction, __FILE__, "$$;$");
+      eval_pv ("require myapp::main", 1); // executes "myapp/main.pm"
+   }
+=item staticperl_xs_init (pTHX)
+Sometimes you need direct control over C<perl_parse> and C<perl_run>, in
+which case you do not want to use C<staticperl_init> but call them on your
+own.
+Then you need this function - either pass it directly as the C<xs_init>
+function to C<perl_parse>, or call it from your own C<xs_init> function.
+=item staticperl_cleanup ()
+In the unlikely case that you want to destroy the perl interpreter, here
+is the corresponding function.
+=item PerlInterpreter *staticperl
+The perl interpreter pointer used by staticperl. Not normally so useful,
+but there it is.
+=back
+=item bundle.ccopts
+Contains the compiler options required to compile at least F<bundle.c> and
+any file that includes F<bundle.h> - you should probably use it in your
+C<CFLAGS>.
+=item bundle.ldopts
+The linker options needed to link the final program.
+=back
+=head1 RUNTIME FUNCTIONALITY
+Binaries created with C<mkbundle>/C<mkperl> contain extra functions, which
+are required to access the bundled perl sources, but might be useful for
+other purposes.
+In addition, for the embedded loading of perl files to work, F<staticperl>
+overrides the C<@INC> array.
+=over 4
+=item $file = staticperl::find $path
+Returns the data associated with the given C<$path>
+(e.g. C<Digest/MD5.pm>, C<auto/POSIX/autosplit.ix>), which is basically
+the UNIX path relative to the perl library directory.
+Returns C<undef> if the file isn't embedded.
+=item @paths = staticperl::list
+Returns the list of all paths embedded in this binary.
+=back
+=head1 FULLY STATIC BINARIES - BUILDROOT
+To make truly static (Linux-) libraries, you might want to have a look at
+buildroot (L<http://buildroot.uclibc.org/>).
+Buildroot is primarily meant to set up a cross-compile environment (which
+is not so useful as perl doesn't quite like cross compiles), but it can also compile
+a chroot environment where you can use F<staticperl>.
+To do so, download buildroot, and enable "Build options => development
+files in target filesystem" and optionally "Build options => gcc
+optimization level (optimize for size)". At the time of writing, I had
+good experiences with GCC 4.4.x but not GCC 4.5.
+To minimise code size, I used C<-pipe -ffunction-sections -fdata-sections
+-finline-limit=8 -fno-builtin-strlen -mtune=i386>. The C<-mtune=i386>
+doesn't decrease codesize much, but it makes the file much more
+compressible.
+If you don't need Coro or threads, you can go with "linuxthreads.old" (or
+no thread support). For Coro, it is highly recommended to switch to a
+uClibc newer than 0.9.31 (at the time of this writing, I used the 20101201
+snapshot) and enable NPTL, otherwise Coro needs to be configured with the
+ultra-slow pthreads backend to work around linuxthreads bugs (it also uses
+twice the address space needed for stacks).
+If you use C<linuxthreads.old>, then you should also be aware that
+uClibc shares C<errno> between all threads when statically linking. See
+L<http://lists.uclibc.org/pipermail/uclibc/2010-June/044157.html> for a
+workaround (And L<https://bugs.uclibc.org/2089> for discussion).
+C<ccache> support is also recommended, especially if you want
+to play around with buildroot options. Enabling the C<miniperl>
+package will probably enable all options required for a successful
+perl build. F<staticperl> itself additionally needs either C<wget>
+(recommended, for CPAN) or C<curl>.
+As for shells, busybox should provide all that is needed, but the default
+busybox configuration doesn't include F<comm> which is needed by perl -
+either make a custom busybox config, or compile coreutils.
+For the latter route, you might find that bash has some bugs that keep
+it from working properly in a chroot - either use dash (and link it to
+F</bin/sh> inside the chroot) or link busybox to F</bin/sh>, using it's
+built-in ash shell.
+Finally, you need F</dev/null> inside the chroot for many scripts to work
+- F<cp /dev/null output/target/dev> or bind-mounting your F</dev> will
+both provide this.
+After you have compiled and set up your buildroot target, you can copy
+F<staticperl> from the C<App::Staticperl> distribution or from your
+perl f<bin> directory (if you installed it) into the F<output/target>
+filesystem, chroot inside and run it.
 =head1 AUTHOR
  Marc Lehmann <schmorp@schmorp.de>
  http://software.schmorp.de/pkg/staticperl.html

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing App-Staticperl/staticperl.pod (file contents): Revision 1.2 by root, Mon Dec 6 20:53:44 2010 UTC vs. Revision 1.13 by root, Tue Dec 7 19:55:56 2010 UTC

Diff Legend

Comparing App-Staticperl/staticperl.pod (file contents):
Revision 1.2 by root, Mon Dec 6 20:53:44 2010 UTC vs.
Revision 1.13 by root, Tue Dec 7 19:55:56 2010 UTC