ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/App-Staticperl/staticperl.pod
(Generate patch)

Comparing App-Staticperl/staticperl.pod (file contents):
Revision 1.16 by root, Wed Dec 8 23:03:21 2010 UTC vs.
Revision 1.22 by root, Mon Dec 13 17:25:17 2010 UTC

40 40
41With F<uClibc> and F<upx> on x86, you can create a single 500kb binary 41With F<uClibc> and F<upx> on x86, you can create a single 500kb binary
42that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, 42that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO,
43Coro and so on. Or any other choice of modules. 43Coro and so on. Or any other choice of modules.
44 44
45To see how this turns out, you can try out smallperl and bigperl, two
46pre-built static and compressed perl binaries with many and even more
47modules: just follow the links at L<http://staticperl.schmorp.de/>.
48
45The created files do not need write access to the file system (like PAR 49The created files do not need write access to the file system (like PAR
46does). In fact, since this script is in many ways similar to PAR::Packer, 50does). In fact, since this script is in many ways similar to PAR::Packer,
47here are the differences: 51here are the differences:
48 52
49=over 4 53=over 4
67=item * The generated executables don't need a writable filesystem. 71=item * The generated executables don't need a writable filesystem.
68 72
69F<staticperl> loads all required files directly from memory. There is no 73F<staticperl> loads all required files directly from memory. There is no
70need to unpack files into a temporary directory. 74need to unpack files into a temporary directory.
71 75
72=item * More control over included files. 76=item * More control over included files, more burden.
73 77
74PAR tries to be maintenance and hassle-free - it tries to include more 78PAR tries to be maintenance and hassle-free - it tries to include more
75files than necessary to make sure everything works out of the box. The 79files than necessary to make sure everything works out of the box. It
76extra files (such as the unicode database) can take substantial amounts of 80mostly succeeds at this, but he extra files (such as the unicode database)
77memory and file size. 81can take substantial amounts of memory and file size.
78 82
79With F<staticperl>, the burden is mostly with the developer - only direct 83With F<staticperl>, the burden is mostly with the developer - only direct
80compile-time dependencies and L<AutoLoader> are handled automatically. 84compile-time dependencies and L<AutoLoader> are handled automatically.
81This means the modules to include often need to be tweaked manually. 85This means the modules to include often need to be tweaked manually.
86
87All this does not preclude more permissive modes to be implemented in
88the future, but right now, you have to resolve state hidden dependencies
89manually.
82 90
83=item * PAR works out of the box, F<staticperl> does not. 91=item * PAR works out of the box, F<staticperl> does not.
84 92
85Maintaining your own custom perl build can be a pain in the ass, and while 93Maintaining your own custom perl build can be a pain in the ass, and while
86F<staticperl> tries to make this easy, it still requires a custom perl 94F<staticperl> tries to make this easy, it still requires a custom perl
107Afterwards, you create a list of files and modules you want to include, 115Afterwards, you create a list of files and modules you want to include,
108and then either build a new perl binary (that acts just like a normal perl 116and then either build a new perl binary (that acts just like a normal perl
109except everything is compiled in), or you create bundle files (basically C 117except everything is compiled in), or you create bundle files (basically C
110sources you can use to embed all files into your project). 118sources you can use to embed all files into your project).
111 119
112This step is very fast (a few seconds if PPI is not used for stripping, 120This step is very fast (a few seconds if PPI is not used for stripping, or
113more seconds otherwise, as PPI is very slow), and can be tweaked and 121the stripped files are in the cache), and can be tweaked and repeated as
114repeated as often as necessary. 122often as necessary.
115 123
116=head1 THE F<STATICPERL> SCRIPT 124=head1 THE F<STATICPERL> SCRIPT
117 125
118This module installs a script called F<staticperl> into your perl 126This module installs a script called F<staticperl> into your perl
119binary directory. The script is fully self-contained, and can be used 127binary directory. The script is fully self-contained, and can be
120without perl (for example, in an uClibc chroot environment). In fact, 128used without perl (for example, in an uClibc chroot environment). In
121it can be extracted from the C<App::Staticperl> distribution tarball as 129fact, it can be extracted from the C<App::Staticperl> distribution
122F<bin/staticperl>, without any installation. 130tarball as F<bin/staticperl>, without any installation. The
131newest (possibly alpha) version can also be downloaded from
132L<http://staticperl.schmorp.de/staticperl>.
123 133
124F<staticperl> interprets the first argument as a command to execute, 134F<staticperl> interprets the first argument as a command to execute,
125optionally followed by any parameters. 135optionally followed by any parameters.
126 136
127There are two command categories: the "phase 1" commands which deal with 137There are two command categories: the "phase 1" commands which deal with
148 158
149To force recompilation or reinstallation, you need to run F<staticperl 159To force recompilation or reinstallation, you need to run F<staticperl
150distclean> first. 160distclean> first.
151 161
152=over 4 162=over 4
163
164=item F<staticperl version>
165
166Prints some info about the version of the F<staticperl> script you are using.
153 167
154=item F<staticperl fetch> 168=item F<staticperl fetch>
155 169
156Runs only the download and unpack phase, unless this has already happened. 170Runs only the download and unpack phase, unless this has already happened.
157 171
282 296
283All options that specify modules or files to be added are processed in the 297All options that specify modules or files to be added are processed in the
284order given on the command line (that affects the C<--use> and C<--eval> 298order given on the command line (that affects the C<--use> and C<--eval>
285options at the moment). 299options at the moment).
286 300
301=head3 PACKAGE SELECTION WORKFLOW
302
303F<staticperl mkbundle> has a number of options to control package
304selection. This section describes how they interact with each other. Also,
305since I am still a newbie w.r.t. these issues, maybe future versions of
306F<staticperl> will change this, so watch out :)
307
308The idiom "in order" means "in order that they are specified on the
309commandline". If you use a bundle specification file, then the options
310will be processed as if they were given in place of the bundle file name.
311
312=over 4
313
314=item 1. apply all C<--use>, C<--eval>, C<--add>, C<--addbin> and
315C<--incglob> options, in order.
316
317In addition, C<--use> and C<--eval> dependencies will be added when the
318options are processed.
319
320=item 2. apply all C<--include> and C<--exclude> options, in order.
321
322All this step does is potentially reduce the number of files already
323selected or found in phase 1.
324
325=item 3. find all modules (== F<.pm> files), gather their static archives
326(F<.a>) and AutoLoader splitfiles (F<.ix> and F<.al> files), find any
327extra libraries they need for linking (F<extralibs.ld>) and optionally
328evaluate any F<.packlist> files.
329
330This step is required to link against XS extensions and also adds files
331required for L<AutoLoader> to do it's job.
332
333=back
334
335After this, all the files selected for bundling will be read and processed
336(stripped), the bundle files will be written, and optionally a new F<perl>
337or application binary will be linked.
338
287=head3 MKBUNDLE OPTIONS 339=head3 MKBUNDLE OPTIONS
288 340
289=over 4 341=over 4
290 342
291=item --verbose | -v 343=item --verbose | -v
303 355
304The default is C<pod>, which uses the L<Pod::Strip> module to remove all 356The default is C<pod>, which uses the L<Pod::Strip> module to remove all
305pod documentation, which is very fast and reduces file size a lot. 357pod documentation, which is very fast and reduces file size a lot.
306 358
307The C<ppi> method uses L<PPI> to parse and condense the perl sources. This 359The C<ppi> method uses L<PPI> to parse and condense the perl sources. This
308saves a lot more than just L<Pod::Strip>, and is generally safer, but 360saves a lot more than just L<Pod::Strip>, and is generally safer,
309is also a lot slower, so is best used for production builds. Note that 361but is also a lot slower (some files take almost a minute to strip -
310this method doesn't optimise for raw file size, but for best compression 362F<staticperl> maintains a cache of stripped files to speed up subsequent
311(that means that the uncompressed file size is a bit larger, but the files 363runs for this reason). Note that this method doesn't optimise for raw file
312compress better, e.g. with F<upx>). 364size, but for best compression (that means that the uncompressed file size
365is a bit larger, but the files compress better, e.g. with F<upx>).
313 366
314Last not least, if you need accurate line numbers in error messages, 367Last not least, if you need accurate line numbers in error messages,
315or in the unlikely case where C<pod> is too slow, or some module gets 368or in the unlikely case where C<pod> is too slow, or some module gets
316mistreated, you can specify C<none> to not mangle included perl sources in 369mistreated, you can specify C<none> to not mangle included perl sources in
317any way. 370any way.
411(using a C<require>) before anything else when the new perl is 464(using a C<require>) before anything else when the new perl is
412initialised. This can be used to modify C<@INC> or anything else before 465initialised. This can be used to modify C<@INC> or anything else before
413the perl interpreter executes scripts given on the command line (or via 466the perl interpreter executes scripts given on the command line (or via
414C<-e>). This works even in an embedded interpreter. 467C<-e>). This works even in an embedded interpreter.
415 468
416=item --add "file" | --add "file alias" 469=item --usepacklist
470
471Read F<.packlist> files for each distribution that happens to match a
472module name you specified. Sounds weird, and it is, so expect semantics to
473change somehow in the future.
474
475The idea is that most CPAN distributions have a F<.pm> file that matches
476the name of the distribution (which is rather reasonable after all).
477
478If this switch is enabled, then if any of the F<.pm> files that have been
479selected match an install distribution, then all F<.pm>, F<.pl>, F<.al>
480and F<.ix> files installed by this distribution are also included.
481
482For example, using this switch, when the L<URI> module is specified, then
483all L<URI> submodules that have been installed via the CPAN distribution
484are included as well, so you don't have to manually specify them.
485
486=item --incglob pattern
487
488This goes through all library directories and tries to match any F<.pm>
489and F<.pl> files against the extended glob pattern (see below). If a file
490matches, it is added. This switch will automatically detect L<AutoLoader>
491files and the required link libraries for XS modules, but it will I<not>
492scan the file for dependencies (at the moment).
493
494This is mainly useful to include "everything":
495
496 --incglob '*'
497
498Or to include perl libraries, or trees of those, such as the unicode
499database files needed by many other modules:
500
501 --incglob '/unicore/**.pl'
502
503=item --add file | --add "file alias"
417 504
418Adds the given (perl) file into the bundle (and optionally call it 505Adds the given (perl) file into the bundle (and optionally call it
419"alias"). This is useful to include any custom files into the bundle. 506"alias"). This is useful to include any custom files into the bundle.
420 507
421Example: embed the file F<httpd> as F<httpd.pm> when creating the bundle. 508Example: embed the file F<httpd> as F<httpd.pm> when creating the bundle.
427 # specification file 514 # specification file
428 add file1 myfiles/file1 515 add file1 myfiles/file1
429 add file2 myfiles/file2 516 add file2 myfiles/file2
430 add file3 myfiles/file3 517 add file3 myfiles/file3
431 518
432=item --binadd "file" | --add "file alias" 519=item --binadd file | --add "file alias"
433 520
434Just like C<--add>, except that it treats the file as binary and adds it 521Just like C<--add>, except that it treats the file as binary and adds it
435without any processing. 522without any processing.
436 523
437You should probably add a C</> prefix to avoid clashing with embedded 524You should probably add a C</> prefix to avoid clashing with embedded
438perl files (whose paths do not start with C</>), and/or use a special 525perl files (whose paths do not start with C</>), and/or use a special
439directory, such as C</res/name>. 526directory, such as C</res/name>.
440 527
441You can later get a copy of these files by calling C<staticperl::find 528You can later get a copy of these files by calling C<staticperl::find
442"alias">. 529"alias">.
530
531=item --include pattern | -i pattern | --exclude pattern | -x pattern
532
533These two options define an include/exclude filter that is used after all
534files selected by the other options have been found. Each include/exclude
535is applied to all files found so far - an include makes sure that the
536given files will be part of the resulting file set, an exclude will
537exclude files. The patterns are "extended glob patterns" (see below).
538
539For example, to include everything, except C<Devel> modules, but still
540include F<Devel::PPPort>, you could use this:
541
542 --incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**'
443 543
444=item --static 544=item --static
445 545
446When C<--perl> is also given, link statically instead of dynamically. The 546When C<--perl> is also given, link statically instead of dynamically. The
447default is to link the new perl interpreter fully dynamic (that means all 547default is to link the new perl interpreter fully dynamic (that means all
452systems based on GNU libc don't really support it in a usable fashion 552systems based on GNU libc don't really support it in a usable fashion
453either. Try uClibc if you want to create fully statically linked 553either. Try uClibc if you want to create fully statically linked
454executables, or try the C<--staticlibs> option to link only some libraries 554executables, or try the C<--staticlibs> option to link only some libraries
455statically. 555statically.
456 556
557=item --staticlib libname
558
559When not linking fully statically, this option allows you to link specific
560libraries statically. What it does is simply replace all occurances of
561C<-llibname> with the GCC-specific C<-Wl,-Bstatic -llibname -Wl,-Bdynamic>
562option.
563
564This will have no effect unless the library is actually linked against,
565specifically, C<--staticlib> will not link against the named library
566unless it would be linked against anyway.
567
568Example: link libcrypt statically into the binary.
569
570 staticperl mkperl -MIO::AIO --staticlib crypt
571
572 # ldopts might nwo contain:
573 # -lm -Wl,-Bstatic -lcrypt -Wl,-Bdynamic -lpthread
574
457=item any other argument 575=item any other argument
458 576
459Any other argument is interpreted as a bundle specification file, which 577Any other argument is interpreted as a bundle specification file, which
460supports most long options (without extra quoting), one option per line. 578supports most long options (without extra quoting), one option per line.
461 579
462=back 580=back
463 581
582=head3 EXTENDED GLOB PATTERNS
583
584Some options of F<staticperl mkbundle> expect an I<extended glob
585pattern>. This is neither a normal shell glob nor a regex, but something
586in between. The idea has been copied from rsync, and there are the current
587matching rules:
588
589=over 4
590
591=item Patterns starting with F</> will be a anchored at the root of the library tree.
592
593That is, F</unicore> will match the F<unicore> directory in C<@INC>, but
594nothing inside, and neither any other file or directory called F<unicore>
595anywhere else in the hierarchy.
596
597=item Patterns not starting with F</> will be anchored at the end of the path.
598
599That is, F<idna.pl> will match any file called F<idna.pl> anywhere in the
600hierarchy, but not any directories of the same name.
601
602=item A F<*> matches any single component.
603
604That is, F</unicore/*.pl> would match all F<.pl> files directly inside
605C</unicore>, not any deeper level F<.pl> files. Or in other words, F<*>
606will not match slashes.
607
608=item A F<**> matches anything.
609
610That is, F</unicore/**.pl> would match all F<.pl> files under F</unicore>,
611no matter how deeply nested they are inside subdirectories.
612
613=item A F<?> matches a single character within a component.
614
615That is, F</Encode/??.pm> matches F</Encode/JP.pm>, but not the
616hypothetical F</Encode/J/.pm>, as F<?> does not match F</>.
617
618=back
619
464=head2 F<STATICPERL> CONFIGURATION AND HOOKS 620=head2 F<STATICPERL> CONFIGURATION AND HOOKS
465 621
466During (each) startup, F<staticperl> tries to source the following shell 622During (each) startup, F<staticperl> tries to source some shell files to
623allow you to fine-tune/override configuration settings.
624
625In them you can override shell variables, or define shell functions
626("hooks") to be called at specific phases during installation. For
627example, you could define a C<postinstall> hook to install additional
628modules from CPAN each time you start from scratch.
629
630If the env variable C<$STATICPERLRC> is set, then F<staticperl> will try
631to source the file named with it only. Otherwise, it tries the following
467files in order: 632shell files in order:
468 633
469 /etc/staticperlrc 634 /etc/staticperlrc
470 ~/.staticperlrc 635 ~/.staticperlrc
471 $STATICPERL/rc 636 $STATICPERL/rc
472
473They can be used to override shell variables, or define functions to be
474called at specific phases.
475 637
476Note that the last file is erased during F<staticperl distclean>, so 638Note that the last file is erased during F<staticperl distclean>, so
477generally should not be used. 639generally should not be used.
478 640
479=head3 CONFIGURATION VARIABLES 641=head3 CONFIGURATION VARIABLES
541 703
542More commonly, you would either activate 64 bit integer support 704More commonly, you would either activate 64 bit integer support
543(C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to 705(C<-Duse64bitint>), or disable large files support (-Uuselargefiles), to
544reduce filesize further. 706reduce filesize further.
545 707
546=item C<PERL_CPPFLAGS>, C<PERL_OPTIMIZE>, C<PERL_LDFLAGS>, C<PERL_LIBS> 708=item C<PERL_CC>, C<PERL_CPPFLAGS>, C<PERL_OPTIMIZE>, C<PERL_LDFLAGS>, C<PERL_LIBS>
547 709
548These flags are passed to perl's F<Configure> script, and are generally 710These flags are passed to perl's F<Configure> script, and are generally
549optimised for small size (at the cost of performance). Since they also 711optimised for small size (at the cost of performance). Since they also
550contain subtle workarounds around various build issues, changing these 712contain subtle workarounds around various build issues, changing these
551usually requires understanding their default values - best look at the top 713usually requires understanding their default values - best look at the top
783After you have compiled and set up your buildroot target, you can copy 945After you have compiled and set up your buildroot target, you can copy
784F<staticperl> from the C<App::Staticperl> distribution or from your 946F<staticperl> from the C<App::Staticperl> distribution or from your
785perl f<bin> directory (if you installed it) into the F<output/target> 947perl f<bin> directory (if you installed it) into the F<output/target>
786filesystem, chroot inside and run it. 948filesystem, chroot inside and run it.
787 949
950=head1 RECIPES / SPECIFIC MODULES
951
952This section contains some common(?) recipes and information about
953problems with some common modules or perl constructs that require extra
954files to be included.
955
956=head2 MODULES
957
958=over 4
959
960=item utf8
961
962Some functionality in the utf8 module, such as swash handling (used
963for unicode character ranges in regexes) is implemented in the
964C<"utf8_heavy.pl"> library:
965
966 -M'"utf8_heavy.pl"'
967
968Many Unicode properties in turn are defined in separate modules,
969such as C<"unicore/Heavy.pl"> and more specific data tables such as
970C<"unicore/To/Digit.pl"> or C<"unicore/lib/Perl/Word.pl">. These tables
971are big (7MB uncompressed, although F<staticperl> contains special
972handling for those files), so including them on demand by your application
973only might pay off.
974
975To simply include the whole unicode database, use:
976
977 --incglob '/unicore/*.pl'
978
979=item AnyEvent
980
981AnyEvent needs a backend implementation that it will load in a delayed
982fashion. The L<AnyEvent::Impl::Perl> backend is the default choice
983for AnyEvent if it can't find anything else, and is usually a safe
984fallback. If you plan to use e.g. L<EV> (L<POE>...), then you need to
985include the L<AnyEvent::Impl::EV> (L<AnyEvent::Impl::POE>...) backend as
986well.
987
988If you want to handle IRIs or IDNs (L<AnyEvent::Util> punycode and idn
989functions), you also need to include C<"AnyEvent/Util/idna.pl"> and
990C<"AnyEvent/Util/uts46data.pl">.
991
992Or you can use C<--usepacklist> and specify C<-MAnyEvent> to include
993everything.
994
995=item Carp
996
997Carp had (in older versions of perl) a dependency on L<Carp::Heavy>. As of
998perl 5.12.2 (maybe earlier), this dependency no longer exists.
999
1000=item Config
1001
1002The F<perl -V> switch (as well as many modules) needs L<Config>, which in
1003turn might need L<"Config_heavy.pl">. Including the latter gives you
1004both.
1005
1006=item Term::ReadLine::Perl
1007
1008Also needs L<Term::ReadLine::readline>, or C<--usepacklist>.
1009
1010=item URI
1011
1012URI implements schemes as separate modules - the generic URL scheme is
1013implemented in L<URI::_generic>, HTTP is implemented in L<URI::http>. If
1014you need to use any of these schemes, you should include these manually,
1015or use C<--usepacklist>.
1016
1017=back
1018
1019=head2 RECIPES
1020
1021=over 4
1022
1023=item Linking everything in
1024
1025To link just about everything installed in the perl library into a new
1026perl, try this:
1027
1028 staticperl mkperl --strip ppi --incglob '*'
1029
1030=item Getting rid of netdb function
1031
1032The perl core has lots of netdb functions (C<getnetbyname>, C<getgrent>
1033and so on) that few applications use. You can avoid compiling them in by
1034putting the following fragment into a C<preconfigure> hook:
1035
1036 preconfigure() {
1037 for sym in \
1038 d_getgrnam_r d_endgrent d_endgrent_r d_endhent \
1039 d_endhostent_r d_endnent d_endnetent_r d_endpent \
1040 d_endprotoent_r d_endpwent d_endpwent_r d_endsent \
1041 d_endservent_r d_getgrent d_getgrent_r d_getgrgid_r \
1042 d_getgrnam_r d_gethbyaddr d_gethent d_getsbyport \
1043 d_gethostbyaddr_r d_gethostbyname_r d_gethostent_r \
1044 d_getlogin_r d_getnbyaddr d_getnbyname d_getnent \
1045 d_getnetbyaddr_r d_getnetbyname_r d_getnetent_r \
1046 d_getpent d_getpbyname d_getpbynumber d_getprotobyname_r \
1047 d_getprotobynumber_r d_getprotoent_r d_getpwent \
1048 d_getpwent_r d_getpwnam_r d_getpwuid_r d_getsent \
1049 d_getservbyname_r d_getservbyport_r d_getservent_r \
1050 d_getspnam_r d_getsbyname
1051 # d_gethbyname
1052 do
1053 PERL_CONFIGURE="$PERL_CONFIGURE -U$sym"
1054 done
1055 }
1056
1057This mostly gains space when linking staticaly, as the functions will
1058likely not be linked in. The gain for dynamically-linked binaries is
1059smaller.
1060
1061Also, this leaves C<gethostbyname> in - not only is it actually used
1062often, the L<Socket> module also exposes it, so leaving it out usually
1063gains little. Why Socket exposes a C function that is in the core already
1064is anybody's guess.
1065
1066=back
1067
788=head1 AUTHOR 1068=head1 AUTHOR
789 1069
790 Marc Lehmann <schmorp@schmorp.de> 1070 Marc Lehmann <schmorp@schmorp.de>
791 http://software.schmorp.de/pkg/staticperl.html 1071 http://software.schmorp.de/pkg/staticperl.html

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines