--- App-Staticperl/staticperl.pod 2010/12/06 20:53:44 1.2 +++ App-Staticperl/staticperl.pod 2010/12/07 09:27:54 1.10 @@ -1,6 +1,6 @@ =head1 NAME -staticperl - perl, libc, 50 modules all in one 500kb file +staticperl - perl, libc, 100 modules, all in one 500kb file =head1 SYNOPSIS @@ -28,17 +28,17 @@ =head1 DESCRIPTION This script helps you creating single-file perl interpreters, or embedding -a pelr interpreter in your apps. Single-file means that it is fully -self-contained - no separate shared objects, no autoload fragments, no .pm -or .pl files are needed. And when linking statically, you can create (or -embed) a single file that contains perl interpreter, libc, all the modules -you need and all the libraries you need. - -With uclibc and upx on x86, you can create a single 500kb binary that -contains perl and 50 modules such as AnyEvent, EV, IO::AIO, Coro and so -on. Or any other choice of modules. +a perl interpreter in your applications. Single-file means that it is +fully self-contained - no separate shared objects, no autoload fragments, +no .pm or .pl files are needed. And when linking statically, you can +create (or embed) a single file that contains perl interpreter, libc, all +the modules you need and all the libraries you need. + +With F and F on x86, you can create a single 500kb binary +that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, +Coro and so on. Or any other choice of modules. -The created files do not need write access to the filesystem (like PAR +The created files do not need write access to the file system (like PAR does). In fact, since this script is in many ways similar to PAR::Packer, here are the differences: @@ -67,9 +67,10 @@ =item * More control over included files. -PAR tries to be maintainance and hassle-free - it tries to include more files -than necessary to make sure everything works out of the box. The extra files -(such as the unicode database) can take substantial amounts of memory and filesize. +PAR tries to be maintenance and hassle-free - it tries to include more +files than necessary to make sure everything works out of the box. The +extra files (such as the unicode database) can take substantial amounts of +memory and file size. With F, the burden is mostly with the developer - only direct compile-time dependencies and L are handled automatically. @@ -90,12 +91,12 @@ your choice in F<~/.staticperl>. You can add extra modules either by letting F install them for you automatically, or by using CPAN and doing it interactively. This usually takes 5-10 minutes, depending on -the speed of your computer and your internet conenction. +the speed of your computer and your internet connection. It is possible to do program development at this stage, too. Afterwards, you create a list of files and modules you want to include, -and then either build a new perl binary (that acts just like a normla perl +and then either build a new perl binary (that acts just like a normal perl except everything is compiled in), or you create bundle files (basically C sources you can use to embed all files into your project). @@ -136,7 +137,7 @@ Most of the following commands simply run one or more steps of this sequence. -To force recompilation or reinstalaltion, you need to run F first. =over 4 @@ -156,13 +157,13 @@ =item F -Wipes the perl installation directory (usually F<~/.staticperl/perl>) and installs -the perl distribution, potentially aftering building it first. +Wipes the perl installation directory (usually F<~/.staticperl/perl>) and +installs the perl distribution, potentially after building it first. =item F [args...] -Starts an interactive CPAN shell that you cna use to install further -modules. Installs the perl first if neccessary, but apart from that, +Starts an interactive CPAN shell that you can use to install further +modules. Installs the perl first if necessary, but apart from that, no magic is involved: you could just as well run it manually via F<~/.staticperl/perl/bin/cpan>. @@ -179,7 +180,7 @@ =item F directory... In the unlikely case that you have unpacked perl modules around and want -to install from these instead of from CPAN, you cna do this using this +to install from these instead of from CPAN, you can do this using this command by specifying all the directories with modules in them that you want to have built. @@ -212,7 +213,7 @@ F<~/.staticperl/mkbundle>). F is a more conventional command and expect the argument -syntax commonly used on unix clones. For example, this command builds +syntax commonly used on UNIX clones. For example, this command builds a new F binary and includes F (for F), F, F and a custom F script (from F in this distribution): @@ -231,16 +232,17 @@ As you can see, things are not quite as trivial: the L module has a hidden dependency which is not even a perl module (F), L needs at least one event loop backend that we have to -specifymanually (here L), and the F module +specify manually (here L), and the F module (required by L) implements various URI schemes as extra modules - since L only needs C URIs, we only need -to include that module. +to include that module. I found out about these dependencies by carefully +watching any error messages about missing modules... =head3 OPTION PROCESSING -All options can be given as arguments on the commandline (typically using -long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since -specifying a lot of modules can make the commandlien very cumbersome, +All options can be given as arguments on the command line (typically +using long (e.g. C<--verbose>) or short option (e.g. C<-v>) style). Since +specifying a lot of modules can make the command line very cumbersome, you can put all long options into a "bundle specification file" (with or without C<--> prefix) and specify this bundle file instead. @@ -257,7 +259,7 @@ add eg/httpd httpd.pm All options that specify modules or files to be added are processed in the -order given on the commandline (that affects the C<--use> and C<--eval> +order given on the command line (that affects the C<--use> and C<--eval> options at the moment). =head3 MKBUNDLE OPTIONS @@ -278,15 +280,19 @@ sources included. The default is C, which uses the L module to remove all -pod documenatiton, which is very fast and reduces filesize a lot. +pod documentation, which is very fast and reduces file size a lot. The C method uses L to parse and condense the perl sources. This -saves a lot more than just L, and is generally safer, but is -also a lot slower, so is best used for production builds. - -Last not least, in the unlikely case where C is too slow, or some -module gets mistreated, you can specify C to not mangle included -perl sources in any way. +saves a lot more than just L, and is generally safer, but +is also a lot slower, so is best used for production builds. Note that +this method doesn't optimise for raw file size, but for best compression +(that means that the uncompressed file size is a bit larger, but the files +compress better, e.g. with F). + +Last not least, if you need accurate line numbers in error messages, +or in the unlikely case where C is too slow, or some module gets +mistreated, you can specify C to not mangle included perl sources in +any way. =item --perl @@ -294,7 +300,7 @@ will be called F and will be left in the current working directory. The bundle files will be removed. -This switch is automatically ued when F is invoked with the +This switch is automatically used when F is invoked with the C command (instead of C): # build a new ./perl with only common::sense in it - very small :) @@ -313,7 +319,7 @@ Sometimes you want to load old-style "perl libraries" (F<.pl> files), or maybe other weirdly named files. To do that, you need to quote the name in -single or double quoutes. When given on the commandline, you probably need +single or double quotes. When given on the command line, you probably need to quote once more to avoid your shell interpreting it. Common cases that need this are F and F. @@ -340,7 +346,7 @@ Keep in mind that F will only C the modules named by the C<--use> option, so do not expect the symbols from modules you -C<--use>'d earlier on the commandlien to be available. +C<--use>'d earlier on the command line to be available. Example: force L to detect a backend and therefore include it in the final bundle. @@ -360,7 +366,7 @@ Include the given file in the bundle and arrange for it to be executed (using a C) before anything else when the new perl is initialised. This can be used to modify C<@INC> or anything else before -the perl interpreter executes scripts given on the commandline (or via +the perl interpreter executes scripts given on the command line (or via C<-e>). This works even in an embedded interpreter. =item --add "file" | --add "file alias" @@ -379,6 +385,18 @@ add file2 myfiles/file2 add file3 myfiles/file3 +=item --binadd "file" | --add "file alias" + +Just like C<--add>, except that it treats the file as binary and adds it +without any processing. + +You should probably add a C prefix to avoid clashing with embedded +perl files (whose paths do not start with C), and/or use a special +directory, such as C. + +You can later get a copy of these files by calling C. + =item --static When C<--perl> is also given, link statically instead of dynamically. The @@ -427,7 +445,7 @@ =back -=head4 Variables you I to override +=head4 Variables you might I to override =over 4 @@ -441,45 +459,57 @@ The URL of the CPAN mirror to use (e.g. L). -=item C, C, C, C - -These flags are passed to perl's F script, and are generally -optimised for small size (at the cost of performance). Since they also -contain subtle workarounds around various build issues, changing these -usually requires understanding their default values - best look at the top -of the F script for more info on these. +=item C -=item C +Additional modules installed during F. Here you can +set which modules you want have to installed from CPAN. -The directory where staticperl stores all its files -(default: F<~/.staticperl>). +Example: I really really need EV, AnyEvent, Coro and IO::AIO. -=item C + EXTRA_MODULES="EV AnyEvent Coro IO::AIO" -The prefix where perl get's installed (default: F<$STATICPERL/perl>), -i.e. where the F and F subdirectories will end up. +Note that you can also use a C hook to achieve this, and +more. -=item C, C, others +=item C, C, ... Usually set to C<1> to make modules "less inquisitive" during their installation, you can set any environment variable you want - some modules (such as L or L) use environment variables for further tweaking. -=item C +=item C -Additional modules installed during F. Here you can -set which modules you want have to installed from CPAN. +The directory where staticperl stores all its files +(default: F<~/.staticperl>). -Example: I really really need EV, AnyEvent, Coro and IO::AIO. +=item C - EXTRA_MODULES="EV AnyEvent Coro IO::AIO" +The prefix where perl gets installed (default: F<$STATICPERL/perl>), +i.e. where the F and F subdirectories will end up. -Note that you cna also use a C hook to achieve this, and -more. +=item C + +Additional Configure options - these are simply passed to the perl +Configure script. For example, if you wanted to enable dynamic loading, +you could pass C<-Dusedl>. To enable ithreads (Why would you want that +insanity? Don't! Use L instead!) you would pass C<-Duseithreads> +and so on. + +More commonly, you would either activate 64 bit integer support +(C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to +reduce filesize further. + +=item C, C, C, C + +These flags are passed to perl's F script, and are generally +optimised for small size (at the cost of performance). Since they also +contain subtle workarounds around various build issues, changing these +usually requires understanding their default values - best look at the top +of the F script for more info on these. =back -=head4 Variables you I to override +=head4 Variables you probably I to override =over 4 @@ -499,17 +529,17 @@ In addition to environment variables, it is possible to provide some shell functions that are called at specific times. To provide your own -commands, justd efine the corresponding function. +commands, just define the corresponding function. Example: install extra modules from CPAN and from some directories at F time. postinstall() { - rm -rf lib/threads.* # weg mit Schaden + rm -rf lib/threads* # weg mit Schaden instcpan IO::AIO EV instsrc ~/src/AnyEvent instsrc ~/src/XML-Sablotron-1.0100001 - instcpan AnyEvent::HTTPD + instcpan Anyevent::AIO AnyEvent::HTTPD } =over 4 @@ -545,6 +575,163 @@ =back +=head1 ANATOMY OF A BUNDLE + +When not building a new perl binary, C will leave a number of +files in the current working directory, which can be used to embed a perl +interpreter in your program. + +Intimate knowledge of L and preferably some experience with +embedding perl is highly recommended. + +C (or the C<--perl> option) basically does this to link the new +interpreter (it also adds a main program to F): + + $Config{cc} $(cat bundle.ccopts) -o perl bundle.c $(cat bundle.ldopts) + +=over 4 + +=item bundle.h + +A header file that contains the prototypes of the few symbols "exported" +by bundle.c, and also exposes the perl headers to the application. + +=over 4 + +=item staticperl_init () + +Initialises the perl interpreter. You can use the normal perl functions +after calling this function, for example, to define extra functions or +to load a .pm file that contains some initialisation code, or the main +program function: + + XS (xsfunction) + { + dXSARGS; + + // now we have items, ST(i) etc. + } + + static void + run_myapp(void) + { + staticperl_init (); + newXSproto ("myapp::xsfunction", xsfunction, __FILE__, "$$;$"); + eval_pv ("require myapp::main", 1); // executes "myapp/main.pm" + } + +=item staticperl_xs_init (pTHX) + +Sometimes you need direct control over C and C, in +which case you do not want to use C but call them on your +own. + +Then you need this function - either pass it directly as the C +function to C, or call it from your own C function. + +=item staticperl_cleanup () + +In the unlikely case that you want to destroy the perl interpreter, here +is the corresponding function. + +=item PerlInterpreter *staticperl + +The perl interpreter pointer used by staticperl. Not normally so useful, +but there it is. + +=back + +=item bundle.ccopts + +Contains the compiler options required to compile at least F and +any file that includes F - you should probably use it in your +C. + +=item bundle.ldopts + +The linker options needed to link the final program. + +=back + +=head1 RUNTIME FUNCTIONALITY + +Binaries created with C/C contain extra functions, which +are required to access the bundled perl sources, but might be useful for +other purposes. + +In addition, for the embedded loading of perl files to work, F +overrides the C<@INC> array. + +=over 4 + +=item $file = staticperl::find $path + +Returns the data associated with the given C<$path> +(e.g. C, C), which is basically +the UNIX path relative to the perl library directory. + +Returns C if the file isn't embedded. + +=item @paths = staticperl::list + +Returns the list of all paths embedded in this binary. + +=back + +=head1 FULLY STATIC BINARIES - BUILDROOT + +To make truly static (Linux-) libraries, you might want to have a look at +buildroot (L). + +Buildroot is primarily meant to set up a cross-compile environment (which +is not so useful as perl doesn't quite like cross compiles), but it can also compile +a chroot environment where you can use F. + +To do so, download buildroot, and enable "Build options => development +files in target filesystem" and optionally "Build options => gcc +optimization level (optimize for size)". At the time of writing, I had +good experiences with GCC 4.4.x but not GCC 4.5. + +To minimise code size, I used C<-pipe -ffunction-sections -fdata-sections +-finline-limit=8 -fno-builtin-strlen -mtune=i386>. The C<-mtune=i386> +doesn't decrease codesize much, but it makes the file much more +compressible. + +If you don't need Coro or threads, you can go with "linuxthreads.old" (or +no thread support). For Coro, it is highly recommended to switch to a +uClibc newer than 0.9.31 (at the time of this writing, I used the 20101201 +snapshot) and enable NPTL, otherwise Coro needs to be configured with the +ultra-slow pthreads backend to work around linuxthreads bugs (it also uses +twice the address space needed for stacks). + +If you use C, then you should also be aware that +uClibc shares C between all threads when statically linking. See +L for a +workaround (And L for discussion). + +C support is also recommended, especially if you want to +play around with buildroot options. Enabling the C package +will probably enable all options required for a successful perl +build. F itself additionally needs either C or C. + +As for shells, busybox should provide all that is needed, but the default +busybox configuration doesn't include F which is needed by perl - +either make a custom busybox config, or compile coreutils. + +For the latter route, you might find that bash has some bugs that keep +it from working properly in a chroot - either use dash (and link it to +F inside the chroot) or link busybox to F, using it's +built-in ash shell. + +Finally, you need F inside the chroot for many scripts to work +- F or bind-mounting your F will +both provide this. + +After you have compiled and set up your buildroot target, you can copy +F from the C distribution or from your +perl f directory (if you installed it) into the F +filesystem, chroot inside and run it. + =head1 AUTHOR Marc Lehmann