1 | =head1 NAME |
1 | =head1 NAME |
2 | |
2 | |
3 | staticperl - perl, libc, 100 modules, all in one 500kb file |
3 | staticperl - perl, libc, 100 modules, all in one standalone 500kb file |
4 | |
4 | |
5 | =head1 SYNOPSIS |
5 | =head1 SYNOPSIS |
6 | |
6 | |
7 | staticperl help # print the embedded documentation |
7 | staticperl help # print the embedded documentation |
8 | staticperl fetch # fetch and unpack perl sources |
8 | staticperl fetch # fetch and unpack perl sources |
… | |
… | |
11 | staticperl install # build and then install perl |
11 | staticperl install # build and then install perl |
12 | staticperl clean # clean most intermediate files (restart at configure) |
12 | staticperl clean # clean most intermediate files (restart at configure) |
13 | staticperl distclean # delete everything installed by this script |
13 | staticperl distclean # delete everything installed by this script |
14 | staticperl perl ... # invoke the perlinterpreter |
14 | staticperl perl ... # invoke the perlinterpreter |
15 | staticperl cpan # invoke CPAN shell |
15 | staticperl cpan # invoke CPAN shell |
16 | staticperl instmod path... # install unpacked modules |
16 | staticperl instsrc path... # install unpacked modules |
17 | staticperl instcpan modulename... # install modules from CPAN |
17 | staticperl instcpan modulename... # install modules from CPAN |
18 | staticperl mkbundle <bundle-args...> # see documentation |
18 | staticperl mkbundle <bundle-args...> # see documentation |
19 | staticperl mkperl <bundle-args...> # see documentation |
19 | staticperl mkperl <bundle-args...> # see documentation |
20 | staticperl mkapp appname <bundle-args...> # see documentation |
20 | staticperl mkapp appname <bundle-args...> # see documentation |
21 | |
21 | |
… | |
… | |
39 | file that contains perl interpreter, libc, all the modules you need, all |
39 | file that contains perl interpreter, libc, all the modules you need, all |
40 | the libraries you need and of course your actual program. |
40 | the libraries you need and of course your actual program. |
41 | |
41 | |
42 | With F<uClibc> and F<upx> on x86, you can create a single 500kb binary |
42 | With F<uClibc> and F<upx> on x86, you can create a single 500kb binary |
43 | that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, |
43 | that contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, |
44 | Coro and so on. Or any other choice of modules. |
44 | Coro and so on. Or any other choice of modules (and some other size :). |
45 | |
45 | |
46 | To see how this turns out, you can try out smallperl and bigperl, two |
46 | To see how this turns out, you can try out smallperl and bigperl, two |
47 | pre-built static and compressed perl binaries with many and even more |
47 | pre-built static and compressed perl binaries with many and even more |
48 | modules: just follow the links at L<http://staticperl.schmorp.de/>. |
48 | modules: just follow the links at L<http://staticperl.schmorp.de/>. |
49 | |
49 | |
… | |
… | |
84 | With F<staticperl>, the burden is mostly with the developer - only direct |
84 | With F<staticperl>, the burden is mostly with the developer - only direct |
85 | compile-time dependencies and L<AutoLoader> are handled automatically. |
85 | compile-time dependencies and L<AutoLoader> are handled automatically. |
86 | This means the modules to include often need to be tweaked manually. |
86 | This means the modules to include often need to be tweaked manually. |
87 | |
87 | |
88 | All this does not preclude more permissive modes to be implemented in |
88 | All this does not preclude more permissive modes to be implemented in |
89 | the future, but right now, you have to resolve state hidden dependencies |
89 | the future, but right now, you have to resolve hidden dependencies |
90 | manually. |
90 | manually. |
91 | |
91 | |
92 | =item * PAR works out of the box, F<staticperl> does not. |
92 | =item * PAR works out of the box, F<staticperl> does not. |
93 | |
93 | |
94 | Maintaining your own custom perl build can be a pain in the ass, and while |
94 | Maintaining your own custom perl build can be a pain in the ass, and while |
… | |
… | |
202 | =item F<staticperl cpan> [args...] |
202 | =item F<staticperl cpan> [args...] |
203 | |
203 | |
204 | Starts an interactive CPAN shell that you can use to install further |
204 | Starts an interactive CPAN shell that you can use to install further |
205 | modules. Installs the perl first if necessary, but apart from that, |
205 | modules. Installs the perl first if necessary, but apart from that, |
206 | no magic is involved: you could just as well run it manually via |
206 | no magic is involved: you could just as well run it manually via |
207 | F<~/.staticperl/perl/bin/cpan>. |
207 | F<~/.staticperl/perl/bin/cpan>, except that F<staticperl> additionally |
|
|
208 | sets the environment variable C<$PERL> to the path of the perl |
|
|
209 | interpreter, which is handy in subshells. |
208 | |
210 | |
209 | Any additional arguments are simply passed to the F<cpan> command. |
211 | Any additional arguments are simply passed to the F<cpan> command. |
210 | |
212 | |
211 | =item F<staticperl instcpan> module... |
213 | =item F<staticperl instcpan> module... |
212 | |
214 | |
… | |
… | |
346 | add eg/httpd httpd.pm |
348 | add eg/httpd httpd.pm |
347 | |
349 | |
348 | All options that specify modules or files to be added are processed in the |
350 | All options that specify modules or files to be added are processed in the |
349 | order given on the command line. |
351 | order given on the command line. |
350 | |
352 | |
351 | =head3 BUNDLE CREATION WORKFLOW / STATICPELR MKBUNDLE OPTIONS |
353 | =head3 BUNDLE CREATION WORKFLOW / STATICPERL MKBUNDLE OPTIONS |
352 | |
354 | |
353 | F<staticperl mkbundle> works by first assembling a list of candidate |
355 | F<staticperl mkbundle> works by first assembling a list of candidate |
354 | files and modules to include, then filtering them by include/exclude |
356 | files and modules to include, then filtering them by include/exclude |
355 | patterns. The remaining modules (together with their direct dependencies, |
357 | patterns. The remaining modules (together with their direct dependencies, |
356 | such as link libraries and L<AutoLoader> files) are then converted into |
358 | such as link libraries and L<AutoLoader> files) are then converted into |
… | |
… | |
547 | |
549 | |
548 | Just like C<--add>, except that it treats the file as binary and adds it |
550 | Just like C<--add>, except that it treats the file as binary and adds it |
549 | without any postprocessing (perl files might get stripped to reduce their |
551 | without any postprocessing (perl files might get stripped to reduce their |
550 | size). |
552 | size). |
551 | |
553 | |
552 | If you specify an alias you should probably add a C<&> prefix to avoid |
554 | If you specify an alias you should probably add a C</> prefix to avoid |
553 | clashing with embedded perl files (whose paths never start with C<&>), |
555 | clashing with embedded perl files (whose paths never start with C</>), |
554 | and/or use a special directory prefix, such as C<&res/name>. |
556 | and/or use a special directory prefix, such as C</res/name>. |
555 | |
557 | |
556 | You can later get a copy of these files by calling C<staticperl::find |
558 | You can later get a copy of these files by calling C<static::find |
557 | "alias">. |
559 | "alias">. |
558 | |
560 | |
559 | An alternative way to embed binary files is to convert them to perl and |
561 | An alternative way to embed binary files is to convert them to perl and |
560 | use C<do> to get the contents - this method is a bit cumbersome, but works |
562 | use C<do> to get the contents - this method is a bit cumbersome, but works |
561 | both inside and outside of a staticperl bundle: |
563 | both inside and outside of a staticperl bundle, without extra ado: |
562 | |
564 | |
563 | # a "binary" file, call it "bindata.pl" |
565 | # a "binary" file, call it "bindata.pl" |
564 | <<'SOME_MARKER' |
566 | <<'SOME_MARKER' |
565 | binary data NOT containing SOME_MARKER |
567 | binary data NOT containing SOME_MARKER |
566 | SOME_MARKER |
568 | SOME_MARKER |
567 | |
569 | |
568 | # load the binary |
570 | # load the binary |
569 | chomp (my $data = do "bindata.pl"); |
571 | chomp (my $data = do "bindata.pl"); |
|
|
572 | |
|
|
573 | =item C<--allow-dynamic> |
|
|
574 | |
|
|
575 | By default, when F<mkbundle> hits a dynamic perl extension (e.g. a F<.so> |
|
|
576 | or F<.dll> file), it will stop with a fatal error. |
|
|
577 | |
|
|
578 | When this option is enabled, F<mkbundle> packages the shared |
|
|
579 | object into the bundle instead, with a prefix of F<!> |
|
|
580 | (e.g. F<!auto/List/Util/Util.so>). What you do with that is currently up |
|
|
581 | to you, F<staticperl> has no special support for this at the moment, apart |
|
|
582 | from working around the lack of availability of F<PerlIO::scalar> while |
|
|
583 | bootstrapping, at a speed cost. |
|
|
584 | |
|
|
585 | One way to deal with this is to write all files starting with F<!> into |
|
|
586 | some directory and then C<unshift> that path onto C<@INC>. |
|
|
587 | |
|
|
588 | #TODO: example |
570 | |
589 | |
571 | =back |
590 | =back |
572 | |
591 | |
573 | =item Step 2: filter all files using C<--include> and C<--exclude> options. |
592 | =item Step 2: filter all files using C<--include> and C<--exclude> options. |
574 | |
593 | |
… | |
… | |
733 | standalone applications, and this option removes those known to cause |
752 | standalone applications, and this option removes those known to cause |
734 | trouble. |
753 | trouble. |
735 | |
754 | |
736 | Specifically, these are removed: |
755 | Specifically, these are removed: |
737 | |
756 | |
738 | C<PERL_HASH_SEED_DEBUG> and C<PERL_DEBUG_MSTATS> can cause underaible |
757 | C<PERL_HASH_SEED_DEBUG> and C<PERL_DEBUG_MSTATS> can cause undesirable |
739 | output, C<PERL5OPT>, C<PERL_DESTRUCT_LEVEL>, C<PERL_HASH_SEED> and |
758 | output, C<PERL5OPT>, C<PERL_DESTRUCT_LEVEL>, C<PERL_HASH_SEED> and |
740 | C<PERL_SIGNALS> can alter execution significantly, and C<PERL_UNICODE>, |
759 | C<PERL_SIGNALS> can alter execution significantly, and C<PERL_UNICODE>, |
741 | C<PERLIO_DEBUG> and C<PERLIO> can affect input and output. |
760 | C<PERLIO_DEBUG> and C<PERLIO> can affect input and output. |
742 | |
761 | |
743 | The variables C<PERL_LIB> and C<PERL5_LIB> are always ignored because the |
762 | The variables C<PERL_LIB> and C<PERL5_LIB> are always ignored because the |
… | |
… | |
882 | =item C<STATICPERL> |
901 | =item C<STATICPERL> |
883 | |
902 | |
884 | The directory where staticperl stores all its files |
903 | The directory where staticperl stores all its files |
885 | (default: F<~/.staticperl>). |
904 | (default: F<~/.staticperl>). |
886 | |
905 | |
887 | =item C<PERL_MM_USE_DEFAULT>, C<EV_EXTRA_DEFS>, ... |
906 | =item C<DLCACHE> |
888 | |
907 | |
889 | Usually set to C<1> to make modules "less inquisitive" during their |
908 | The path to a directory (will be created if it doesn't exist) where |
890 | installation, you can set any environment variable you want - some modules |
909 | downloaded perl sources are being cached, to avoid downloading them |
891 | (such as L<Coro> or L<EV>) use environment variables for further tweaking. |
910 | again. The default is empty, which means there is no cache. |
892 | |
911 | |
893 | =item C<PERL_VERSION> |
912 | =item C<PERL_VERSION> |
894 | |
913 | |
895 | The perl version to install - default is currently C<5.12.3>, but C<5.8.9> |
914 | The perl version to install - default is currently C<5.12.3>, but C<5.8.9> |
896 | is also a good choice (5.8.9 is much smaller than 5.12.3, while 5.10.1 is |
915 | is also a good choice (5.8.9 is much smaller than 5.12.3, while 5.10.1 is |
897 | about as big as 5.12.3). |
916 | about as big as 5.12.3). |
|
|
917 | |
|
|
918 | =item C<PERL_MM_USE_DEFAULT>, C<EV_EXTRA_DEFS>, ... |
|
|
919 | |
|
|
920 | Usually set to C<1> to make modules "less inquisitive" during their |
|
|
921 | installation. You can set (and export!) any environment variable you want |
|
|
922 | - some modules (such as L<Coro> or L<EV>) use environment variables for |
|
|
923 | further tweaking. |
898 | |
924 | |
899 | =item C<PERL_PREFIX> |
925 | =item C<PERL_PREFIX> |
900 | |
926 | |
901 | The prefix where perl gets installed (default: F<$STATICPERL/perl>), |
927 | The prefix where perl gets installed (default: F<$STATICPERL/perl>), |
902 | i.e. where the F<bin> and F<lib> subdirectories will end up. |
928 | i.e. where the F<bin> and F<lib> subdirectories will end up. |
… | |
… | |
922 | the top of the F<staticperl> script for more info on these, and use a |
948 | the top of the F<staticperl> script for more info on these, and use a |
923 | F<~/.staticperlrc> to override them. |
949 | F<~/.staticperlrc> to override them. |
924 | |
950 | |
925 | Most of the variables override (or modify) the corresponding F<Configure> |
951 | Most of the variables override (or modify) the corresponding F<Configure> |
926 | variable, except C<PERL_CCFLAGS>, which gets appended. |
952 | variable, except C<PERL_CCFLAGS>, which gets appended. |
|
|
953 | |
|
|
954 | The default for C<PERL_OPTIMIZE> is C<-Os> (assuming gcc), and for |
|
|
955 | C<PERL_LIBS> is C<-lm -lcrypt>, which should be good for most (but not |
|
|
956 | all) systems. |
|
|
957 | |
|
|
958 | For other compilers or more customised optimisation settings, you need to |
|
|
959 | adjust these, e.g. in your F<~/.staticperlrc>. |
|
|
960 | |
|
|
961 | With gcc on x86 and amd64, you can get more space-savings by using: |
|
|
962 | |
|
|
963 | -Os -ffunction-sections -fdata-sections -finline-limit=8 -mpush-args |
|
|
964 | -mno-inline-stringops-dynamically -mno-align-stringops |
|
|
965 | |
|
|
966 | And on x86 and pentium3 and newer (basically everything you might ever |
|
|
967 | want to run on), adding these is even better for space-savings (use |
|
|
968 | -mtune=core2 or something newer for much faster code, too): |
|
|
969 | |
|
|
970 | -fomit-frame-pointer -march=pentium3 -mtune=i386 |
927 | |
971 | |
928 | =back |
972 | =back |
929 | |
973 | |
930 | =head4 Variables you probably I<do not want> to override |
974 | =head4 Variables you probably I<do not want> to override |
931 | |
975 | |
… | |
… | |
1110 | |
1154 | |
1111 | =back |
1155 | =back |
1112 | |
1156 | |
1113 | =head1 RUNTIME FUNCTIONALITY |
1157 | =head1 RUNTIME FUNCTIONALITY |
1114 | |
1158 | |
1115 | Binaries created with C<mkbundle>/C<mkperl> contain extra functions, which |
1159 | Binaries created with C<mkbundle>/C<mkperl> contain extra functionality, |
1116 | are required to access the bundled perl sources, but might be useful for |
1160 | mostly related to the extra files bundled in the binary (the virtual |
1117 | other purposes. |
1161 | filesystem). All of this data is statically compiled into the binary, and |
|
|
1162 | accessing means copying it from a read-only section of your binary. Data |
|
|
1163 | pages in this way is usually freed by the operating system, as it isn't |
|
|
1164 | use more the onace. |
|
|
1165 | |
|
|
1166 | =head2 VIRTUAL FILESYSTEM |
|
|
1167 | |
|
|
1168 | Every bundle has a virtual filesystem. The only information stored in it |
|
|
1169 | is the path and contents of each file that was bundled. |
|
|
1170 | |
|
|
1171 | =head3 LAYOUT |
|
|
1172 | |
|
|
1173 | Any path starting with an ampersand (F<&>) or exclamation mark (F<!>) are |
|
|
1174 | reserved by F<staticperl>. They must only be used as described in this |
|
|
1175 | section. |
|
|
1176 | |
|
|
1177 | =over 4 |
|
|
1178 | |
|
|
1179 | =item ! |
|
|
1180 | |
|
|
1181 | All files that typically cannot be loaded from memory (such as dynamic |
|
|
1182 | objects or shared libraries), but have to reside in the filesystem, are |
|
|
1183 | prefixed with F<!>. Typically these files get written out to some |
|
|
1184 | (semi-)temporary directory shortly after program startup, or before being |
|
|
1185 | used. |
|
|
1186 | |
|
|
1187 | =item !boot |
|
|
1188 | |
|
|
1189 | The bootstrap file, if specified during bundling. |
|
|
1190 | |
|
|
1191 | =item !auto/ |
|
|
1192 | |
|
|
1193 | Shared objects or dlls corresponding to dynamically-linked perl extensions |
|
|
1194 | are stored with an F<!auto/> prefix. |
|
|
1195 | |
|
|
1196 | =item !lib/ |
|
|
1197 | |
|
|
1198 | External shared libraries are stored in this directory. |
|
|
1199 | |
|
|
1200 | =item any letter |
|
|
1201 | |
|
|
1202 | Any path starting with a letter is a perl library file. For example, |
|
|
1203 | F<Coro/AIO.pm> corresponds to the file loaded by C<use Coro::AIO>, and |
|
|
1204 | F<Coro/jit.pl> corresponds to C<require "Coro/jit.pl">. |
|
|
1205 | |
|
|
1206 | Obviously, module names shouldn't start with any other characters than |
|
|
1207 | letters :) |
|
|
1208 | |
|
|
1209 | =back |
|
|
1210 | |
|
|
1211 | =head3 FUNCTIONS |
|
|
1212 | |
|
|
1213 | =over 4 |
|
|
1214 | |
|
|
1215 | =item $file = static::find $path |
|
|
1216 | |
|
|
1217 | Returns the data associated with the given C<$path> |
|
|
1218 | (e.g. C<Digest/MD5.pm>, C<auto/POSIX/autosplit.ix>). |
|
|
1219 | |
|
|
1220 | Returns C<undef> if the file isn't embedded. |
|
|
1221 | |
|
|
1222 | =item @paths = static::list |
|
|
1223 | |
|
|
1224 | Returns the list of all paths embedded in this binary. |
|
|
1225 | |
|
|
1226 | =back |
|
|
1227 | |
|
|
1228 | =head2 EXTRA FEATURES |
1118 | |
1229 | |
1119 | In addition, for the embedded loading of perl files to work, F<staticperl> |
1230 | In addition, for the embedded loading of perl files to work, F<staticperl> |
1120 | overrides the C<@INC> array. |
1231 | overrides the C<@INC> array. |
1121 | |
|
|
1122 | =over 4 |
|
|
1123 | |
|
|
1124 | =item $file = staticperl::find $path |
|
|
1125 | |
|
|
1126 | Returns the data associated with the given C<$path> |
|
|
1127 | (e.g. C<Digest/MD5.pm>, C<auto/POSIX/autosplit.ix>), which is basically |
|
|
1128 | the UNIX path relative to the perl library directory. |
|
|
1129 | |
|
|
1130 | Returns C<undef> if the file isn't embedded. |
|
|
1131 | |
|
|
1132 | =item @paths = staticperl::list |
|
|
1133 | |
|
|
1134 | Returns the list of all paths embedded in this binary. |
|
|
1135 | |
|
|
1136 | =back |
|
|
1137 | |
1232 | |
1138 | =head1 FULLY STATIC BINARIES - UCLIBC AND BUILDROOT |
1233 | =head1 FULLY STATIC BINARIES - UCLIBC AND BUILDROOT |
1139 | |
1234 | |
1140 | To make truly static (Linux-) libraries, you might want to have a look at |
1235 | To make truly static (Linux-) libraries, you might want to have a look at |
1141 | buildroot (L<http://buildroot.uclibc.org/>). |
1236 | buildroot (L<http://buildroot.uclibc.org/>). |
… | |
… | |
1150 | good experiences with GCC 4.4.x but not GCC 4.5. |
1245 | good experiences with GCC 4.4.x but not GCC 4.5. |
1151 | |
1246 | |
1152 | To minimise code size, I used C<-pipe -ffunction-sections -fdata-sections |
1247 | To minimise code size, I used C<-pipe -ffunction-sections -fdata-sections |
1153 | -finline-limit=8 -fno-builtin-strlen -mtune=i386>. The C<-mtune=i386> |
1248 | -finline-limit=8 -fno-builtin-strlen -mtune=i386>. The C<-mtune=i386> |
1154 | doesn't decrease codesize much, but it makes the file much more |
1249 | doesn't decrease codesize much, but it makes the file much more |
1155 | compressible. |
1250 | compressible (and the execution a lot slower...). |
1156 | |
1251 | |
1157 | If you don't need Coro or threads, you can go with "linuxthreads.old" (or |
1252 | If you don't need Coro or threads, you can go with "linuxthreads.old" (or |
1158 | no thread support). For Coro, it is highly recommended to switch to a |
1253 | no thread support). For Coro, it is highly recommended to switch to a |
1159 | uClibc newer than 0.9.31 (at the time of this writing, I used the 20101201 |
1254 | uClibc newer than 0.9.31 (at the time of this writing, I used the 20101201 |
1160 | snapshot) and enable NPTL, otherwise Coro needs to be configured with the |
1255 | snapshot) and enable NPTL, otherwise Coro needs to be configured with the |
… | |
… | |
1162 | twice the address space needed for stacks). |
1257 | twice the address space needed for stacks). |
1163 | |
1258 | |
1164 | If you use C<linuxthreads.old>, then you should also be aware that |
1259 | If you use C<linuxthreads.old>, then you should also be aware that |
1165 | uClibc shares C<errno> between all threads when statically linking. See |
1260 | uClibc shares C<errno> between all threads when statically linking. See |
1166 | L<http://lists.uclibc.org/pipermail/uclibc/2010-June/044157.html> for a |
1261 | L<http://lists.uclibc.org/pipermail/uclibc/2010-June/044157.html> for a |
1167 | workaround (And L<https://bugs.uclibc.org/2089> for discussion). |
1262 | workaround (and L<https://bugs.uclibc.org/2089> for discussion). |
1168 | |
1263 | |
1169 | C<ccache> support is also recommended, especially if you want |
1264 | C<ccache> support is also recommended, especially if you want |
1170 | to play around with buildroot options. Enabling the C<miniperl> |
1265 | to play around with buildroot options. Enabling the C<miniperl> |
1171 | package will probably enable all options required for a successful |
1266 | package will probably enable all options required for a successful |
1172 | perl build. F<staticperl> itself additionally needs either C<wget> |
1267 | perl build. F<staticperl> itself additionally needs either C<wget> |
… | |
… | |
1180 | it from working properly in a chroot - either use dash (and link it to |
1275 | it from working properly in a chroot - either use dash (and link it to |
1181 | F</bin/sh> inside the chroot) or link busybox to F</bin/sh>, using it's |
1276 | F</bin/sh> inside the chroot) or link busybox to F</bin/sh>, using it's |
1182 | built-in ash shell. |
1277 | built-in ash shell. |
1183 | |
1278 | |
1184 | Finally, you need F</dev/null> inside the chroot for many scripts to work |
1279 | Finally, you need F</dev/null> inside the chroot for many scripts to work |
1185 | - F<cp /dev/null output/target/dev> or bind-mounting your F</dev> will |
1280 | - either F<cp /dev/null output/target/dev> or bind-mounting your F</dev> |
1186 | both provide this. |
1281 | will provide this. |
1187 | |
1282 | |
1188 | After you have compiled and set up your buildroot target, you can copy |
1283 | After you have compiled and set up your buildroot target, you can copy |
1189 | F<staticperl> from the C<App::Staticperl> distribution or from your |
1284 | F<staticperl> from the C<App::Staticperl> distribution or from your |
1190 | perl f<bin> directory (if you installed it) into the F<output/target> |
1285 | perl F<bin> directory (if you installed it) into the F<output/target> |
1191 | filesystem, chroot inside and run it. |
1286 | filesystem, chroot inside and run it. |
1192 | |
1287 | |
1193 | =head1 RECIPES / SPECIFIC MODULES |
1288 | =head1 RECIPES / SPECIFIC MODULES |
1194 | |
1289 | |
1195 | This section contains some common(?) recipes and information about |
1290 | This section contains some common(?) recipes and information about |