--- App-Staticperl/staticperl.pod 2010/12/22 01:23:37 1.29 +++ App-Staticperl/staticperl.pod 2012/12/05 15:19:52 1.57 @@ -1,6 +1,6 @@ =head1 NAME -staticperl - perl, libc, 100 modules, all in one 500kb file +staticperl - perl, libc, 100 modules, all in one standalone 500kb file =head1 SYNOPSIS @@ -11,8 +11,9 @@ staticperl install # build and then install perl staticperl clean # clean most intermediate files (restart at configure) staticperl distclean # delete everything installed by this script + staticperl perl ... # invoke the perlinterpreter staticperl cpan # invoke CPAN shell - staticperl instmod path... # install unpacked modules + staticperl instsrc path... # install unpacked modules staticperl instcpan modulename... # install modules from CPAN staticperl mkbundle # see documentation staticperl mkperl # see documentation @@ -22,7 +23,7 @@ staticperl install # fetch, configure, build and install perl staticperl cpan # run interactive cpan shell - staticperl mkperl -M '"Config_heavy.pl"' # build a perl that supports -V + staticperl mkperl -MConfig_heavy.pl # build a perl that supports -V staticperl mkperl -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI -MURI::http # build a perl with the above modules linked in staticperl mkapp myapp --boot mainprog mymodules @@ -40,7 +41,7 @@ With F and F on x86, you can create a single 500kb binary that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, -Coro and so on. Or any other choice of modules. +Coro and so on. Or any other choice of modules (and some other size :). To see how this turns out, you can try out smallperl and bigperl, two pre-built static and compressed perl binaries with many and even more @@ -85,7 +86,7 @@ This means the modules to include often need to be tweaked manually. All this does not preclude more permissive modes to be implemented in -the future, but right now, you have to resolve state hidden dependencies +the future, but right now, you have to resolve hidden dependencies manually. =item * PAR works out of the box, F does not. @@ -141,7 +142,7 @@ =head2 PHASE 1 COMMANDS: INSTALLING PERL The most important command is F, which does basically -everything. The default is to download and install perl 5.12.2 and a few +everything. The default is to download and install perl 5.12.3 and a few modules required by F itself, but all this can (and should) be changed - see L, below. @@ -188,12 +189,24 @@ Wipes the perl installation directory (usually F<~/.staticperl/perl>) and installs the perl distribution, potentially after building it first. +=item F [args...] + +Invokes the compiled perl interpreter with the given args. Basically the +same as starting perl directly (usually via F<~/.staticperl/bin/perl>), +but beats typing the path sometimes. + +Example: check that the Gtk2 module is installed and loadable. + + staticperl perl -MGtk2 -e0 + =item F [args...] Starts an interactive CPAN shell that you can use to install further modules. Installs the perl first if necessary, but apart from that, no magic is involved: you could just as well run it manually via -F<~/.staticperl/perl/bin/cpan>. +F<~/.staticperl/perl/bin/cpan>, except that F additionally +sets the environment variable C<$PERL> to the path of the perl +interpreter, which is handy in subshells. Any additional arguments are simply passed to the F command. @@ -254,7 +267,7 @@ staticperl instcpan AnyEvent::HTTPD # now build the perl - staticperl mkperl -M'"Config_heavy.pl"' -MAnyEvent::Impl::Perl \ + staticperl mkperl -MConfig_heavy.pl -MAnyEvent::Impl::Perl \ -MAnyEvent::HTTPD -MURI::http \ --add 'eg/httpd httpd.pm' @@ -337,7 +350,7 @@ All options that specify modules or files to be added are processed in the order given on the command line. -=head3 BUNDLE CREATION WORKFLOW / STATICPELR MKBUNDLE OPTIONS +=head3 BUNDLE CREATION WORKFLOW / STATICPERL MKBUNDLE OPTIONS F works by first assembling a list of candidate files and modules to include, then filtering them by include/exclude @@ -383,31 +396,62 @@ =item C<--use> F | C<-M>F -Include the named module and trace direct dependencies. This is done by -C'ing the module in a subprocess and tracing which other modules -and files it actually loads. +Include the named module or perl library and trace direct +dependencies. This is done by loading the module in a subprocess and +tracing which other modules and files it actually loads. Example: include AnyEvent and AnyEvent::Impl::Perl. staticperl mkbundle --use AnyEvent --use AnyEvent::Impl::Perl -Sometimes you want to load old-style "perl libraries" (F<.pl> files), -or maybe other weirdly named files. To do that, you need to quote -the name in single or double quotes (this is because F -I just adds the string after the C - which acts -different when confronted with quoted vs. unquoted strings). When given on -the command line, you probably need to quote once more to avoid your shell -interpreting it. Common cases that need this are F and -F. +Sometimes you want to load old-style "perl libraries" (F<.pl> files), or +maybe other weirdly named files. To support this, the C<--use> option +actually tries to do what you mean, depending on the string you specify: + +=over 4 + +=item a possibly valid module name, e.g. F, F, +F. + +If the string contains no quotes, no F and no F<.>, then C<--use> +assumes that it is a normal module name. It will create a new package and +evaluate a C in it, i.e. it will load the package and do a +default import. + +The import step is done because many modules trigger more dependencies +when something is imported than without. + +=item anything that contains F or F<.> characters, +e.g. F, F. + +The string will be quoted and passed to require, as if you used C. Nothing will be imported. + +=item "path" or 'path', e.g. C<"utf8_heavy.pl">. + +If you enclose the name into single or double quotes, then the quotes will +be removed and the resulting string will be passed to require. This syntax +is form compatibility with older versions of staticperl and should not be +used anymore. + +=back + +Example: C AnyEvent::Socket, once using C (importing the +symbols), and once via C, not importing any symbols. The first +form is preferred as many modules load some extra dependencies when asked +to export symbols. + + staticperl mkbundle -MAnyEvent::Socket # use + import + staticperl mkbundle -MAnyEvent/Socket.pm # require only Example: include the required files for F to work in all its -glory (F is included automatically by this). +glory (F is included automatically by the dependency tracker). - # bourne shell - staticperl mkbundle --use '"Config_heavy.pl"' + # shell command + staticperl mkbundle -MConfig_heavy.pl # bundle specification file - use "Config_heavy.pl" + use Config_heavy.pl The C<-M>module syntax is included as a convenience that might be easier to remember than C<--use> - it's the same switch as perl itself uses @@ -422,9 +466,9 @@ variables or whatever you need. All files C'd or C'd while executing the snippet are included in the final bundle. -Keep in mind that F will only C the modules named -by the C<--use> option, so do not expect the symbols from modules you -C<--use>'d earlier on the command line to be available. +Keep in mind that F will not import any symbols from the modules +named by the C<--use> option, so do not expect the symbols from modules +you C<--use>'d earlier on the command line to be available. Example: force L to detect a backend and therefore include it in the final bundle. @@ -469,9 +513,9 @@ =item C<--add> F | C<--add> "F alias" Adds the given (perl) file into the bundle (and optionally call it -"alias"). The F is either an absolute path or a path relative to -the current directory. If an alias is specified, then this is the name it -will use for C<@INC> searches, otherwise the F will be used as the +"alias"). The F is either an absolute path or a path relative to the +current directory. If an alias is specified, then this is the name it will +use for C<@INC> searches, otherwise the path F will be used as the internal name. This switch is used to include extra files into the bundle. @@ -481,6 +525,14 @@ staticperl mkperl --add "httpd httpd.pm" + # can be accessed via "use httpd" + +Example: add a file F from the current directory. + + staticperl mkperl --add 'initcode &initcode' + + # can be accessed via "do '&initcode'" + Example: add local files as extra modules in the bundle. # specification file @@ -493,22 +545,22 @@ require myfiles::file2; my $res = do "myfiles/file3.pl"; -=item C<--binadd> F | C<--add> "F alias" +=item C<--binadd> F | C<--binadd> "F alias" Just like C<--add>, except that it treats the file as binary and adds it without any postprocessing (perl files might get stripped to reduce their size). -You should probably add a C prefix to avoid clashing with embedded perl -files (whose paths do not start with C), and/or use a special directory -prefix, such as C. +If you specify an alias you should probably add a C prefix to avoid +clashing with embedded perl files (whose paths never start with C), +and/or use a special directory prefix, such as C. -You can later get a copy of these files by calling C. An alternative way to embed binary files is to convert them to perl and use C to get the contents - this method is a bit cumbersome, but works -both inside and outside of a staticperl bundle: +both inside and outside of a staticperl bundle, without extra ado: # a "binary" file, call it "bindata.pl" <<'SOME_MARKER' @@ -518,6 +570,23 @@ # load the binary chomp (my $data = do "bindata.pl"); +=item C<--allow-dynamic> + +By default, when F hits a dynamic perl extension (e.g. a F<.so> +or F<.dll> file), it will stop with a fatal error. + +When this option is enabled, F packages the shared +object into the bundle instead, with a prefix of F +(e.g. F). What you do with that is currently up +to you, F has no special support for this at the moment, apart +from working around the lack of availability of F while +bootstrapping, at a speed cost. + +One way to deal with this is to write all files starting with F into +some directory and then C that path onto C<@INC>. + +#TODO: example + =back =item Step 2: filter all files using C<--include> and C<--exclude> options. @@ -666,7 +735,7 @@ instead it will simply initialise the perl interpreter, clean it up and exit. -This means that, by default, it will do nothing but burna few CPU cycles +This means that, by default, it will do nothing but burn a few CPU cycles - for it to do something useful you I add some boot code, e.g. with the C<--boot> option. @@ -675,6 +744,29 @@ staticperl mkbundle --app myexe --boot appfile +=item C<--ignore-env> + +Generates extra code to unset some environment variables before +initialising/running perl. Perl supports a lot of environment variables +that might alter execution in ways that might be undesirablre for +standalone applications, and this option removes those known to cause +trouble. + +Specifically, these are removed: + +C and C can cause undesirable +output, C, C, C and +C can alter execution significantly, and C, +C and C can affect input and output. + +The variables C and C are always ignored because the +startup code used by F overrides C<@INC> in all cases. + +This option will not make your program more secure (unless you are +running with elevated privileges), but it will reduce the surprise effect +when a user has these environment variables set and doesn't expect your +standalone program to act like a perl interpreter. + =item C<--static> Add C<-static> to F, which means a fully static (if @@ -811,22 +903,30 @@ The directory where staticperl stores all its files (default: F<~/.staticperl>). -=item C, C, ... +=item C -Usually set to C<1> to make modules "less inquisitive" during their -installation, you can set any environment variable you want - some modules -(such as L or L) use environment variables for further tweaking. +The path to a directory (will be created if it doesn't exist) where +downloaded perl sources are being cached, to avoid downloading them +again. The default is empty, which means there is no cache. =item C -The perl version to install - default is currently C<5.12.2>, but C<5.8.9> -is also a good choice (5.8.9 is much smaller than 5.12.2, while 5.10.1 is -about as big as 5.12.2). +The perl version to install - default is currently C<5.12.3>, but C<5.8.9> +is also a good choice (5.8.9 is much smaller than 5.12.3, while 5.10.1 is +about as big as 5.12.3). + +=item C, C, ... + +Usually set to C<1> to make modules "less inquisitive" during their +installation. You can set (and export!) any environment variable you want +- some modules (such as L or L) use environment variables for +further tweaking. =item C -The prefix where perl gets installed (default: F<$STATICPERL/perl>), -i.e. where the F and F subdirectories will end up. +The directory where perl gets installed (default: F<$STATICPERL/perl>), +i.e. where the F and F subdirectories will end up. Previous +contents will be removed on installation. =item C @@ -852,6 +952,24 @@ Most of the variables override (or modify) the corresponding F variable, except C, which gets appended. +The default for C is C<-Os> (assuming gcc), and for +C is C<-lm -lcrypt>, which should be good for most (but not +all) systems. + +For other compilers or more customised optimisation settings, you need to +adjust these, e.g. in your F<~/.staticperlrc>. + +With gcc on x86 and amd64, you can get more space-savings by using: + + -Os -ffunction-sections -fdata-sections -finline-limit=8 -mpush-args + -mno-inline-stringops-dynamically -mno-align-stringops + +And on x86 and pentium3 and newer (basically everything you might ever +want to run on), adding these is even better for space-savings (use +-mtune=core2 or something newer for much faster code, too): + + -fomit-frame-pointer -march=pentium3 -mtune=i386 + =back =head4 Variables you probably I to override @@ -880,6 +998,10 @@ shell functions that are called at specific times. To provide your own commands, just define the corresponding function. +The actual order in which hooks are invoked during a full install +from scratch is C, C, C, +C, C. + Example: install extra modules from CPAN and from some directories at F time. @@ -895,20 +1017,25 @@ =item preconfigure -Called just before running F<./Configur> in the perl source +Called just before running F<./Configure> in the perl source directory. Current working directory is the perl source directory. This can be used to set any C variables, which might be costly to compute. +=item patchconfig + +Called after running F<./Configure> in the perl source directory to create +F<./config.sh>, but before running F<./Configure -S> to actually apply the +config. Current working directory is the perl source directory. + +Can be used to tailor/patch F or do any other modifications. + =item postconfigure Called after configuring, but before building perl. Current working directory is the perl source directory. -Could be used to tailor/patch config.sh (followed by F) -or do any other modifications. - =item postbuild Called after building, but before installing perl. Current working @@ -955,7 +1082,7 @@ =over 4 -=item staticperl_init () +=item staticperl_init (xs_init = 0) Initialises the perl interpreter. You can use the normal perl functions after calling this function, for example, to define extra functions or @@ -972,25 +1099,43 @@ static void run_myapp(void) { - staticperl_init (); + staticperl_init (0); newXSproto ("myapp::xsfunction", xsfunction, __FILE__, "$$;$"); eval_pv ("require myapp::main", 1); // executes "myapp/main.pm" } -=item staticperl_xs_init (pTHX) +When your bootcode already wants to access some XS functions at +compiletime, then you need to supply an C function pointer that +is called as soon as perl is initialised enough to define XS functions, +but before the preamble code is executed: -Sometimes you need direct control over C and C, in -which case you do not want to use C but call them on your -own. + static void + xs_init (pTHX) + { + newXSproto ("myapp::xsfunction", xsfunction, __FILE__, "$$;$"); + } -Then you need this function - either pass it directly as the C -function to C, or call it from your own C function. + static void + run_myapp(void) + { + staticperl_init (xs_init); + } =item staticperl_cleanup () In the unlikely case that you want to destroy the perl interpreter, here is the corresponding function. +=item staticperl_xs_init (pTHX) + +Sometimes you need direct control over C and C, in +which case you do not want to use C but call them on your +own. + +Then you need this function - either pass it directly as the C +function to C, or call it as one of the first things from your +own C function. + =item PerlInterpreter *staticperl The perl interpreter pointer used by staticperl. Not normally so useful, @@ -1012,29 +1157,80 @@ =head1 RUNTIME FUNCTIONALITY -Binaries created with C/C contain extra functions, which -are required to access the bundled perl sources, but might be useful for -other purposes. +Binaries created with C/C contain extra functionality, +mostly related to the extra files bundled in the binary (the virtual +filesystem). All of this data is statically compiled into the binary, and +accessing means copying it from a read-only section of your binary. Data +pages in this way is usually freed by the operating system, as it isn't +use more the onace. + +=head2 VIRTUAL FILESYSTEM + +Every bundle has a virtual filesystem. The only information stored in it +is the path and contents of each file that was bundled. + +=head3 LAYOUT + +Any path starting with an ampersand (F<&>) or exclamation mark (F) are +reserved by F. They must only be used as described in this +section. -In addition, for the embedded loading of perl files to work, F -overrides the C<@INC> array. +=over 4 + +=item ! + +All files that typically cannot be loaded from memory (such as dynamic +objects or shared libraries), but have to reside in the filesystem, are +prefixed with F. Typically these files get written out to some +(semi-)temporary directory shortly after program startup, or before being +used. + +=item !boot + +The bootstrap file, if specified during bundling. + +=item !auto/ + +Shared objects or dlls corresponding to dynamically-linked perl extensions +are stored with an F prefix. + +=item !lib/ + +External shared libraries are stored in this directory. + +=item any letter + +Any path starting with a letter is a perl library file. For example, +F corresponds to the file loaded by C, and +F corresponds to C. + +Obviously, module names shouldn't start with any other characters than +letters :) + +=back + +=head3 FUNCTIONS =over 4 -=item $file = staticperl::find $path +=item $file = static::find $path Returns the data associated with the given C<$path> -(e.g. C, C), which is basically -the UNIX path relative to the perl library directory. +(e.g. C, C). Returns C if the file isn't embedded. -=item @paths = staticperl::list +=item @paths = static::list Returns the list of all paths embedded in this binary. =back +=head2 EXTRA FEATURES + +In addition, for the embedded loading of perl files to work, F +overrides the C<@INC> array. + =head1 FULLY STATIC BINARIES - UCLIBC AND BUILDROOT To make truly static (Linux-) libraries, you might want to have a look at @@ -1052,7 +1248,7 @@ To minimise code size, I used C<-pipe -ffunction-sections -fdata-sections -finline-limit=8 -fno-builtin-strlen -mtune=i386>. The C<-mtune=i386> doesn't decrease codesize much, but it makes the file much more -compressible. +compressible (and the execution a lot slower...). If you don't need Coro or threads, you can go with "linuxthreads.old" (or no thread support). For Coro, it is highly recommended to switch to a @@ -1064,7 +1260,7 @@ If you use C, then you should also be aware that uClibc shares C between all threads when statically linking. See L for a -workaround (And L for discussion). +workaround (and L for discussion). C support is also recommended, especially if you want to play around with buildroot options. Enabling the C @@ -1082,12 +1278,12 @@ built-in ash shell. Finally, you need F inside the chroot for many scripts to work -- F or bind-mounting your F will -both provide this. +- either F or bind-mounting your F +will provide this. After you have compiled and set up your buildroot target, you can copy F from the C distribution or from your -perl f directory (if you installed it) into the F +perl F directory (if you installed it) into the F filesystem, chroot inside and run it. =head1 RECIPES / SPECIFIC MODULES @@ -1106,7 +1302,7 @@ for unicode character ranges in regexes) is implemented in the C<"utf8_heavy.pl"> library: - -M'"utf8_heavy.pl"' + -Mutf8_heavy.pl Many Unicode properties in turn are defined in separate modules, such as C<"unicore/Heavy.pl"> and more specific data tables such as @@ -1117,7 +1313,7 @@ To simply include the whole unicode database, use: - --incglob '/unicore/*.pl' + --incglob '/unicore/**.pl' =item AnyEvent @@ -1135,6 +1331,10 @@ Or you can use C<--usepacklists> and specify C<-MAnyEvent> to include everything. +=item Cairo + +See Glib, same problem, same solution. + =item Carp Carp had (in older versions of perl) a dependency on L. As of @@ -1146,6 +1346,45 @@ turn might need L<"Config_heavy.pl">. Including the latter gives you both. +=item Glib + +Glib literally requires Glib to be installed already to build - it tries +to fake this by running Glib out of the build directory before being +built. F tries to work around this by forcing C and +C to be empty via the C environment variable. + +=item Gtk2 + +See Pango, same problems, same solution. + +=item Net::SSLeay + +This module hasn't been significantly updated since OpenSSL is called +OpenSSL, and fails to properly link against dependent libraries, most +commonly, it forgets to specify -ldl when linking. + +On GNU/Linux systems this usually goes undetected, as perl usually links +against -ldl itself and OpenSSL just happens to pick it up that way, by +chance. + +For static builds, you either have to configure -ldl manually, or you +cna use the following snippet in your C hook which patches +Net::SSLeay after installation, which happens to work most of the time: + + postinstall() { + # first install it + instcpan Net::SSLeay + # then add -ldl for future linking + chmod u+w "$PERL_PREFIX"/lib/auto/Net/SSLeay/extralibs.ld + echo " -ldl" >>"$PERL_PREFIX"/lib/auto/Net/SSLeay/extralibs.ld + } + +=item Pango + +In addition to the C problem in Glib, Pango also routes around +L by compiling its files on its own. F +tries to patch L to route around Pango. + =item Term::ReadLine::Perl Also needs L, or C<--usepacklists>. @@ -1205,7 +1444,7 @@ done } -This mostly gains space when linking staticaly, as the functions will +This mostly gains space when linking statically, as the functions will likely not be linked in. The gain for dynamically-linked binaries is smaller.