1 |
NAME |
2 |
staticperl - perl, libc, 100 modules, all in one 500kb file |
3 |
|
4 |
SYNOPSIS |
5 |
staticperl help # print the embedded documentation |
6 |
staticperl fetch # fetch and unpack perl sources |
7 |
staticperl configure # fetch and then configure perl |
8 |
staticperl build # configure and then build perl |
9 |
staticperl install # build and then install perl |
10 |
staticperl clean # clean most intermediate files (restart at configure) |
11 |
staticperl distclean # delete everything installed by this script |
12 |
staticperl cpan # invoke CPAN shell |
13 |
staticperl instmod path... # install unpacked modules |
14 |
staticperl instcpan modulename... # install modules from CPAN |
15 |
staticperl mkbundle <bundle-args...> # see documentation |
16 |
staticperl mkperl <bundle-args...> # see documentation |
17 |
staticperl mkapp appname <bundle-args...> # see documentation |
18 |
|
19 |
Typical Examples: |
20 |
|
21 |
staticperl install # fetch, configure, build and install perl |
22 |
staticperl cpan # run interactive cpan shell |
23 |
staticperl mkperl -M '"Config_heavy.pl"' # build a perl that supports -V |
24 |
staticperl mkperl -MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI -MURI::http |
25 |
# build a perl with the above modules linked in |
26 |
staticperl mkapp myapp --boot mainprog mymodules |
27 |
# build a binary "myapp" from mainprog and mymodules |
28 |
|
29 |
DESCRIPTION |
30 |
This script helps you to create single-file perl interpreters or |
31 |
applications, or embedding a perl interpreter in your applications. |
32 |
Single-file means that it is fully self-contained - no separate shared |
33 |
objects, no autoload fragments, no .pm or .pl files are needed. And when |
34 |
linking statically, you can create (or embed) a single file that |
35 |
contains perl interpreter, libc, all the modules you need, all the |
36 |
libraries you need and of course your actual program. |
37 |
|
38 |
With uClibc and upx on x86, you can create a single 500kb binary that |
39 |
contains perl and 100 modules such as POSIX, AnyEvent, EV, IO::AIO, Coro |
40 |
and so on. Or any other choice of modules. |
41 |
|
42 |
To see how this turns out, you can try out smallperl and bigperl, two |
43 |
pre-built static and compressed perl binaries with many and even more |
44 |
modules: just follow the links at <http://staticperl.schmorp.de/>. |
45 |
|
46 |
The created files do not need write access to the file system (like PAR |
47 |
does). In fact, since this script is in many ways similar to |
48 |
PAR::Packer, here are the differences: |
49 |
|
50 |
* The generated executables are much smaller than PAR created ones. |
51 |
|
52 |
Shared objects and the perl binary contain a lot of extra info, |
53 |
while the static nature of staticperl allows the linker to remove |
54 |
all functionality and meta-info not required by the final |
55 |
executable. Even extensions statically compiled into perl at build |
56 |
time will only be present in the final executable when needed. |
57 |
|
58 |
In addition, staticperl can strip perl sources much more effectively |
59 |
than PAR. |
60 |
|
61 |
* The generated executables start much faster. |
62 |
|
63 |
There is no need to unpack files, or even to parse Zip archives |
64 |
(which is slow and memory-consuming business). |
65 |
|
66 |
* The generated executables don't need a writable filesystem. |
67 |
|
68 |
staticperl loads all required files directly from memory. There is |
69 |
no need to unpack files into a temporary directory. |
70 |
|
71 |
* More control over included files, more burden. |
72 |
|
73 |
PAR tries to be maintenance and hassle-free - it tries to include |
74 |
more files than necessary to make sure everything works out of the |
75 |
box. It mostly succeeds at this, but he extra files (such as the |
76 |
unicode database) can take substantial amounts of memory and file |
77 |
size. |
78 |
|
79 |
With staticperl, the burden is mostly with the developer - only |
80 |
direct compile-time dependencies and AutoLoader are handled |
81 |
automatically. This means the modules to include often need to be |
82 |
tweaked manually. |
83 |
|
84 |
All this does not preclude more permissive modes to be implemented |
85 |
in the future, but right now, you have to resolve state hidden |
86 |
dependencies manually. |
87 |
|
88 |
* PAR works out of the box, staticperl does not. |
89 |
|
90 |
Maintaining your own custom perl build can be a pain in the ass, and |
91 |
while staticperl tries to make this easy, it still requires a custom |
92 |
perl build and possibly fiddling with some modules. PAR is likely to |
93 |
produce results faster. |
94 |
|
95 |
Ok, PAR never has worked for me out of the box, and for some people, |
96 |
staticperl does work out of the box, as they don't count "fiddling |
97 |
with module use lists" against it, but nevertheless, staticperl is |
98 |
certainly a bit more difficult to use. |
99 |
|
100 |
HOW DOES IT WORK? |
101 |
Simple: staticperl downloads, compile and installs a perl version of |
102 |
your choice in ~/.staticperl. You can add extra modules either by |
103 |
letting staticperl install them for you automatically, or by using CPAN |
104 |
and doing it interactively. This usually takes 5-10 minutes, depending |
105 |
on the speed of your computer and your internet connection. |
106 |
|
107 |
It is possible to do program development at this stage, too. |
108 |
|
109 |
Afterwards, you create a list of files and modules you want to include, |
110 |
and then either build a new perl binary (that acts just like a normal |
111 |
perl except everything is compiled in), or you create bundle files |
112 |
(basically C sources you can use to embed all files into your project). |
113 |
|
114 |
This step is very fast (a few seconds if PPI is not used for stripping, |
115 |
or the stripped files are in the cache), and can be tweaked and repeated |
116 |
as often as necessary. |
117 |
|
118 |
THE STATICPERL SCRIPT |
119 |
This module installs a script called staticperl into your perl binary |
120 |
directory. The script is fully self-contained, and can be used without |
121 |
perl (for example, in an uClibc chroot environment). In fact, it can be |
122 |
extracted from the "App::Staticperl" distribution tarball as |
123 |
bin/staticperl, without any installation. The newest (possibly alpha) |
124 |
version can also be downloaded from |
125 |
<http://staticperl.schmorp.de/staticperl>. |
126 |
|
127 |
staticperl interprets the first argument as a command to execute, |
128 |
optionally followed by any parameters. |
129 |
|
130 |
There are two command categories: the "phase 1" commands which deal with |
131 |
installing perl and perl modules, and the "phase 2" commands, which deal |
132 |
with creating binaries and bundle files. |
133 |
|
134 |
PHASE 1 COMMANDS: INSTALLING PERL |
135 |
The most important command is install, which does basically everything. |
136 |
The default is to download and install perl 5.12.2 and a few modules |
137 |
required by staticperl itself, but all this can (and should) be changed |
138 |
- see CONFIGURATION, below. |
139 |
|
140 |
The command |
141 |
|
142 |
staticperl install |
143 |
|
144 |
is normally all you need: It installs the perl interpreter in |
145 |
~/.staticperl/perl. It downloads, configures, builds and installs the |
146 |
perl interpreter if required. |
147 |
|
148 |
Most of the following staticperl subcommands simply run one or more |
149 |
steps of this sequence. |
150 |
|
151 |
If it fails, then most commonly because the compiler options I selected |
152 |
are not supported by your compiler - either edit the staticperl script |
153 |
yourself or create ~/.staticperl shell script where your set working |
154 |
"PERL_CCFLAGS" etc. variables. |
155 |
|
156 |
To force recompilation or reinstallation, you need to run staticperl |
157 |
distclean first. |
158 |
|
159 |
staticperl version |
160 |
Prints some info about the version of the staticperl script you are |
161 |
using. |
162 |
|
163 |
staticperl fetch |
164 |
Runs only the download and unpack phase, unless this has already |
165 |
happened. |
166 |
|
167 |
staticperl configure |
168 |
Configures the unpacked perl sources, potentially after downloading |
169 |
them first. |
170 |
|
171 |
staticperl build |
172 |
Builds the configured perl sources, potentially after automatically |
173 |
configuring them. |
174 |
|
175 |
staticperl install |
176 |
Wipes the perl installation directory (usually ~/.staticperl/perl) |
177 |
and installs the perl distribution, potentially after building it |
178 |
first. |
179 |
|
180 |
staticperl cpan [args...] |
181 |
Starts an interactive CPAN shell that you can use to install further |
182 |
modules. Installs the perl first if necessary, but apart from that, |
183 |
no magic is involved: you could just as well run it manually via |
184 |
~/.staticperl/perl/bin/cpan. |
185 |
|
186 |
Any additional arguments are simply passed to the cpan command. |
187 |
|
188 |
staticperl instcpan module... |
189 |
Tries to install all the modules given and their dependencies, using |
190 |
CPAN. |
191 |
|
192 |
Example: |
193 |
|
194 |
staticperl instcpan EV AnyEvent::HTTPD Coro |
195 |
|
196 |
staticperl instsrc directory... |
197 |
In the unlikely case that you have unpacked perl modules around and |
198 |
want to install from these instead of from CPAN, you can do this |
199 |
using this command by specifying all the directories with modules in |
200 |
them that you want to have built. |
201 |
|
202 |
staticperl clean |
203 |
Deletes the perl source directory (and potentially cleans up other |
204 |
intermediate files). This can be used to clean up files only needed |
205 |
for building perl, without removing the installed perl interpreter. |
206 |
|
207 |
At the moment, it doesn't delete downloaded tarballs. |
208 |
|
209 |
The exact semantics of this command will probably change. |
210 |
|
211 |
staticperl distclean |
212 |
This wipes your complete ~/.staticperl directory. Be careful with |
213 |
this, it nukes your perl download, perl sources, perl distribution |
214 |
and any installed modules. It is useful if you wish to start over |
215 |
"from scratch" or when you want to uninstall staticperl. |
216 |
|
217 |
PHASE 2 COMMANDS: BUILDING PERL BUNDLES |
218 |
Building (linking) a new perl binary is handled by a separate script. To |
219 |
make it easy to use staticperl from a chroot, the script is embedded |
220 |
into staticperl, which will write it out and call for you with any |
221 |
arguments you pass: |
222 |
|
223 |
staticperl mkbundle mkbundle-args... |
224 |
|
225 |
In the oh so unlikely case of something not working here, you can run |
226 |
the script manually as well (by default it is written to |
227 |
~/.staticperl/mkbundle). |
228 |
|
229 |
mkbundle is a more conventional command and expect the argument syntax |
230 |
commonly used on UNIX clones. For example, this command builds a new |
231 |
perl binary and includes Config.pm (for perl -V), AnyEvent::HTTPD, URI |
232 |
and a custom httpd script (from eg/httpd in this distribution): |
233 |
|
234 |
# first make sure we have perl and the required modules |
235 |
staticperl instcpan AnyEvent::HTTPD |
236 |
|
237 |
# now build the perl |
238 |
staticperl mkperl -M'"Config_heavy.pl"' -MAnyEvent::Impl::Perl \ |
239 |
-MAnyEvent::HTTPD -MURI::http \ |
240 |
--add 'eg/httpd httpd.pm' |
241 |
|
242 |
# finally, invoke it |
243 |
./perl -Mhttpd |
244 |
|
245 |
As you can see, things are not quite as trivial: the Config module has a |
246 |
hidden dependency which is not even a perl module (Config_heavy.pl), |
247 |
AnyEvent needs at least one event loop backend that we have to specify |
248 |
manually (here AnyEvent::Impl::Perl), and the URI module (required by |
249 |
AnyEvent::HTTPD) implements various URI schemes as extra modules - since |
250 |
AnyEvent::HTTPD only needs "http" URIs, we only need to include that |
251 |
module. I found out about these dependencies by carefully watching any |
252 |
error messages about missing modules... |
253 |
|
254 |
Instead of building a new perl binary, you can also build a standalone |
255 |
application: |
256 |
|
257 |
# build the app |
258 |
staticperl mkapp app --boot eg/httpd \ |
259 |
-MAnyEvent::Impl::Perl -MAnyEvent::HTTPD -MURI::http |
260 |
|
261 |
# run it |
262 |
./app |
263 |
|
264 |
Here are the three phase 2 commands: |
265 |
|
266 |
staticperl mkbundle args... |
267 |
The "default" bundle command - it interprets the given bundle |
268 |
options and writes out bundle.h, bundle.c, bundle.ccopts and |
269 |
bundle.ldopts files, useful for embedding. |
270 |
|
271 |
staticperl mkperl args... |
272 |
Creates a bundle just like staticperl mkbundle (in fact, it's the |
273 |
same as invoking staticperl mkbundle --perl args...), but then |
274 |
compiles and links a new perl interpreter that embeds the created |
275 |
bundle, then deletes all intermediate files. |
276 |
|
277 |
staticperl mkapp filename args... |
278 |
Does the same as staticperl mkbundle (in fact, it's the same as |
279 |
invoking staticperl mkbundle --app filename args...), but then |
280 |
compiles and links a new standalone application that simply |
281 |
initialises the perl interpreter. |
282 |
|
283 |
The difference to staticperl mkperl is that the standalone |
284 |
application does not act like a perl interpreter would - in fact, by |
285 |
default it would just do nothing and exit immediately, so you should |
286 |
specify some code to be executed via the --boot option. |
287 |
|
288 |
OPTION PROCESSING |
289 |
All options can be given as arguments on the command line (typically |
290 |
using long (e.g. "--verbose") or short option (e.g. "-v") style). Since |
291 |
specifying a lot of options can make the command line very long and |
292 |
unwieldy, you can put all long options into a "bundle specification |
293 |
file" (one option per line, with or without "--" prefix) and specify |
294 |
this bundle file instead. |
295 |
|
296 |
For example, the command given earlier to link a new perl could also |
297 |
look like this: |
298 |
|
299 |
staticperl mkperl httpd.bundle |
300 |
|
301 |
With all options stored in the httpd.bundle file (one option per line, |
302 |
everything after the option is an argument): |
303 |
|
304 |
use "Config_heavy.pl" |
305 |
use AnyEvent::Impl::Perl |
306 |
use AnyEvent::HTTPD |
307 |
use URI::http |
308 |
add eg/httpd httpd.pm |
309 |
|
310 |
All options that specify modules or files to be added are processed in |
311 |
the order given on the command line. |
312 |
|
313 |
BUNDLE CREATION WORKFLOW / STATICPELR MKBUNDLE OPTIONS |
314 |
staticperl mkbundle works by first assembling a list of candidate files |
315 |
and modules to include, then filtering them by include/exclude patterns. |
316 |
The remaining modules (together with their direct dependencies, such as |
317 |
link libraries and AutoLoader files) are then converted into bundle |
318 |
files suitable for embedding. staticperl mkbundle can then optionally |
319 |
build a new perl interpreter or a standalone application. |
320 |
|
321 |
Step 0: Generic argument processing. |
322 |
The following options influence staticperl mkbundle itself. |
323 |
|
324 |
"--verbose" | "-v" |
325 |
Increases the verbosity level by one (the default is 1). |
326 |
|
327 |
"--quiet" | "-q" |
328 |
Decreases the verbosity level by one. |
329 |
|
330 |
any other argument |
331 |
Any other argument is interpreted as a bundle specification |
332 |
file, which supports all options (without extra quoting), one |
333 |
option per line, in the format "option" or "option argument". |
334 |
They will effectively be expanded and processed as if they were |
335 |
directly written on the command line, in place of the file name. |
336 |
|
337 |
Step 1: gather candidate files and modules |
338 |
In this step, modules, perl libraries (.pl files) and other files |
339 |
are selected for inclusion in the bundle. The relevant options are |
340 |
executed in order (this makes a difference mostly for "--eval", |
341 |
which can rely on earlier "--use" options to have been executed). |
342 |
|
343 |
"--use" module | "-M"module |
344 |
Include the named module and trace direct dependencies. This is |
345 |
done by "require"'ing the module in a subprocess and tracing |
346 |
which other modules and files it actually loads. |
347 |
|
348 |
Example: include AnyEvent and AnyEvent::Impl::Perl. |
349 |
|
350 |
staticperl mkbundle --use AnyEvent --use AnyEvent::Impl::Perl |
351 |
|
352 |
Sometimes you want to load old-style "perl libraries" (.pl |
353 |
files), or maybe other weirdly named files. To do that, you need |
354 |
to quote the name in single or double quotes (this is because |
355 |
staticperl *literally* just adds the string after the "require" |
356 |
- which acts different when confronted with quoted vs. unquoted |
357 |
strings). When given on the command line, you probably need to |
358 |
quote once more to avoid your shell interpreting it. Common |
359 |
cases that need this are Config_heavy.pl and utf8_heavy.pl. |
360 |
|
361 |
Example: include the required files for perl -V to work in all |
362 |
its glory (Config.pm is included automatically by this). |
363 |
|
364 |
# bourne shell |
365 |
staticperl mkbundle --use '"Config_heavy.pl"' |
366 |
|
367 |
# bundle specification file |
368 |
use "Config_heavy.pl" |
369 |
|
370 |
The "-M"module syntax is included as a convenience that might be |
371 |
easier to remember than "--use" - it's the same switch as perl |
372 |
itself uses to load modules. Or maybe it confuses people. Time |
373 |
will tell. Or maybe not. Sigh. |
374 |
|
375 |
"--eval" "perl code" | "-e" "perl code" |
376 |
Sometimes it is easier (or necessary) to specify dependencies |
377 |
using perl code, or maybe one of the modules you use need a |
378 |
special use statement. In that case, you can use "--eval" to |
379 |
execute some perl snippet or set some variables or whatever you |
380 |
need. All files "require"'d or "use"'d while executing the |
381 |
snippet are included in the final bundle. |
382 |
|
383 |
Keep in mind that mkbundle will only "require" the modules named |
384 |
by the "--use" option, so do not expect the symbols from modules |
385 |
you "--use"'d earlier on the command line to be available. |
386 |
|
387 |
Example: force AnyEvent to detect a backend and therefore |
388 |
include it in the final bundle. |
389 |
|
390 |
staticperl mkbundle --eval 'use AnyEvent; AnyEvent::detect' |
391 |
|
392 |
# or like this |
393 |
staticperl mkbundle -MAnyEvent --eval 'AnyEvent::detect' |
394 |
|
395 |
Example: use a separate "bootstrap" script that "use"'s lots of |
396 |
modules and also include this in the final bundle, to be |
397 |
executed automatically when the interpreter is initialised. |
398 |
|
399 |
staticperl mkbundle --eval 'do "bootstrap"' --boot bootstrap |
400 |
|
401 |
"--boot" filename |
402 |
Include the given file in the bundle and arrange for it to be |
403 |
executed (using "require") before the main program when the new |
404 |
perl is initialised. This can be used to modify @INC or do |
405 |
similar modifications before the perl interpreter executes |
406 |
scripts given on the command line (or via "-e"). This works even |
407 |
in an embedded interpreter - the file will be executed during |
408 |
interpreter initialisation in that case. |
409 |
|
410 |
"--incglob" pattern |
411 |
This goes through all standard library directories and tries to |
412 |
match any .pm and .pl files against the extended glob pattern |
413 |
(see below). If a file matches, it is added. The pattern is |
414 |
matched against the full path of the file (sans the library |
415 |
directory prefix), e.g. Sys/Syslog.pm. |
416 |
|
417 |
This is very useful to include "everything": |
418 |
|
419 |
--incglob '*' |
420 |
|
421 |
It is also useful for including perl libraries, or trees of |
422 |
those, such as the unicode database files needed by some perl |
423 |
built-ins, the regex engine and other modules. |
424 |
|
425 |
--incglob '/unicore/**.pl' |
426 |
|
427 |
"--add" file | "--add" "file alias" |
428 |
Adds the given (perl) file into the bundle (and optionally call |
429 |
it "alias"). The file is either an absolute path or a path |
430 |
relative to the current directory. If an alias is specified, |
431 |
then this is the name it will use for @INC searches, otherwise |
432 |
the file will be used as the internal name. |
433 |
|
434 |
This switch is used to include extra files into the bundle. |
435 |
|
436 |
Example: embed the file httpd in the current directory as |
437 |
httpd.pm when creating the bundle. |
438 |
|
439 |
staticperl mkperl --add "httpd httpd.pm" |
440 |
|
441 |
Example: add local files as extra modules in the bundle. |
442 |
|
443 |
# specification file |
444 |
add file1 myfiles/file1.pm |
445 |
add file2 myfiles/file2.pm |
446 |
add file3 myfiles/file3.pl |
447 |
|
448 |
# then later, in perl, use |
449 |
use myfiles::file1; |
450 |
require myfiles::file2; |
451 |
my $res = do "myfiles/file3.pl"; |
452 |
|
453 |
"--binadd" file | "--add" "file alias" |
454 |
Just like "--add", except that it treats the file as binary and |
455 |
adds it without any postprocessing (perl files might get |
456 |
stripped to reduce their size). |
457 |
|
458 |
You should probably add a "/" prefix to avoid clashing with |
459 |
embedded perl files (whose paths do not start with "/"), and/or |
460 |
use a special directory prefix, such as "/res/name". |
461 |
|
462 |
You can later get a copy of these files by calling |
463 |
"staticperl::find "alias"". |
464 |
|
465 |
An alternative way to embed binary files is to convert them to |
466 |
perl and use "do" to get the contents - this method is a bit |
467 |
cumbersome, but works both inside and outside of a staticperl |
468 |
bundle: |
469 |
|
470 |
# a "binary" file, call it "bindata.pl" |
471 |
<<'SOME_MARKER' |
472 |
binary data NOT containing SOME_MARKER |
473 |
SOME_MARKER |
474 |
|
475 |
# load the binary |
476 |
chomp (my $data = do "bindata.pl"); |
477 |
|
478 |
Step 2: filter all files using "--include" and "--exclude" options. |
479 |
After all candidate files and modules are added, they are *filtered* |
480 |
by a combination of "--include" and "--exclude" patterns (there is |
481 |
an implicit "--include *" at the end, so if no filters are |
482 |
specified, all files are included). |
483 |
|
484 |
All that this step does is potentially reduce the number of files |
485 |
that are to be included - no new files are added during this step. |
486 |
|
487 |
"--include" pattern | "-i" pattern | "--exclude" pattern | "-x" |
488 |
pattern |
489 |
These specify an include or exclude pattern to be applied to the |
490 |
candidate file list. An include makes sure that the given files |
491 |
will be part of the resulting file set, an exclude will exclude |
492 |
remaining files. The patterns are "extended glob patterns" (see |
493 |
below). |
494 |
|
495 |
The patterns are applied "in order" - files included via earlier |
496 |
"--include" specifications cannot be removed by any following |
497 |
"--exclude", and likewise, and file excluded by an earlier |
498 |
"--exclude" cannot be added by any following "--include". |
499 |
|
500 |
For example, to include everything except "Devel" modules, but |
501 |
still include Devel::PPPort, you could use this: |
502 |
|
503 |
--incglob '*' -i '/Devel/PPPort.pm' -x '/Devel/**' |
504 |
|
505 |
Step 3: add any extra or "hidden" dependencies. |
506 |
staticperl currently knows about three extra types of depdendencies |
507 |
that are added automatically. Only one (.packlist files) is |
508 |
currently optional and can be influenced, the others are always |
509 |
included: |
510 |
|
511 |
"--usepacklists" |
512 |
Read .packlist files for each distribution that happens to match |
513 |
a module name you specified. Sounds weird, and it is, so expect |
514 |
semantics to change somehow in the future. |
515 |
|
516 |
The idea is that most CPAN distributions have a .pm file that |
517 |
matches the name of the distribution (which is rather reasonable |
518 |
after all). |
519 |
|
520 |
If this switch is enabled, then if any of the .pm files that |
521 |
have been selected match an install distribution, then all .pm, |
522 |
.pl, .al and .ix files installed by this distribution are also |
523 |
included. |
524 |
|
525 |
For example, using this switch, when the URI module is |
526 |
specified, then all URI submodules that have been installed via |
527 |
the CPAN distribution are included as well, so you don't have to |
528 |
manually specify them. |
529 |
|
530 |
AutoLoader splitfiles |
531 |
Some modules use AutoLoader - less commonly (hopefully) used |
532 |
functions are split into separate .al files, and an index (.ix) |
533 |
file contains the prototypes. |
534 |
|
535 |
Both .ix and .al files will be detected automatically and added |
536 |
to the bundle. |
537 |
|
538 |
link libraries (.a files) |
539 |
Modules using XS (or any other non-perl language extension |
540 |
compiled at installation time) will have a static archive |
541 |
(typically .a). These will automatically be added to the linker |
542 |
options in bundle.ldopts. |
543 |
|
544 |
Should staticperl find a dynamic link library (typically .so) it |
545 |
will warn about it - obviously this shouldn't happen unless you |
546 |
use staticperl on the wrong perl, or one (probably wrongly) |
547 |
configured to use dynamic loading. |
548 |
|
549 |
extra libraries (extralibs.ld) |
550 |
Some modules need linking against external libraries - these are |
551 |
found in extralibs.ld and added to bundle.ldopts. |
552 |
|
553 |
Step 4: write bundle files and optionally link a program |
554 |
At this point, the select files will be read, processed (stripped) |
555 |
and finally the bundle files get written to disk, and staticperl |
556 |
mkbundle is normally finished. Optionally, it can go a step further |
557 |
and either link a new perl binary with all selected modules and |
558 |
files inside, or build a standalone application. |
559 |
|
560 |
Both the contents of the bundle files and any extra linking is |
561 |
controlled by these options: |
562 |
|
563 |
"--strip" "none"|"pod"|"ppi" |
564 |
Specify the stripping method applied to reduce the file of the |
565 |
perl sources included. |
566 |
|
567 |
The default is "pod", which uses the Pod::Strip module to remove |
568 |
all pod documentation, which is very fast and reduces file size |
569 |
a lot. |
570 |
|
571 |
The "ppi" method uses PPI to parse and condense the perl |
572 |
sources. This saves a lot more than just Pod::Strip, and is |
573 |
generally safer, but is also a lot slower (some files take |
574 |
almost a minute to strip - staticperl maintains a cache of |
575 |
stripped files to speed up subsequent runs for this reason). |
576 |
Note that this method doesn't optimise for raw file size, but |
577 |
for best compression (that means that the uncompressed file size |
578 |
is a bit larger, but the files compress better, e.g. with upx). |
579 |
|
580 |
Last not least, if you need accurate line numbers in error |
581 |
messages, or in the unlikely case where "pod" is too slow, or |
582 |
some module gets mistreated, you can specify "none" to not |
583 |
mangle included perl sources in any way. |
584 |
|
585 |
"--perl" |
586 |
After writing out the bundle files, try to link a new perl |
587 |
interpreter. It will be called perl and will be left in the |
588 |
current working directory. The bundle files will be removed. |
589 |
|
590 |
This switch is automatically used when staticperl is invoked |
591 |
with the "mkperl" command instead of "mkbundle". |
592 |
|
593 |
Example: build a new ./perl binary with only common::sense |
594 |
inside - it will be even smaller than the standard perl |
595 |
interpreter as none of the modules of the base distribution |
596 |
(such as Fcntl) will be included. |
597 |
|
598 |
staticperl mkperl -Mcommon::sense |
599 |
|
600 |
"--app" name |
601 |
After writing out the bundle files, try to link a new standalone |
602 |
program. It will be called "name", and the bundle files get |
603 |
removed after linking it. |
604 |
|
605 |
This switch is automatically used when staticperl is invoked |
606 |
with the "mkapp" command instead of "mkbundle". |
607 |
|
608 |
The difference to the (mutually exclusive) "--perl" option is |
609 |
that the binary created by this option will not try to act as a |
610 |
perl interpreter - instead it will simply initialise the perl |
611 |
interpreter, clean it up and exit. |
612 |
|
613 |
This means that, by default, it will do nothing but burna few |
614 |
CPU cycles - for it to do something useful you *must* add some |
615 |
boot code, e.g. with the "--boot" option. |
616 |
|
617 |
Example: create a standalone perl binary called ./myexe that |
618 |
will execute appfile when it is started. |
619 |
|
620 |
staticperl mkbundle --app myexe --boot appfile |
621 |
|
622 |
"--static" |
623 |
Add "-static" to bundle.ldopts, which means a fully static (if |
624 |
supported by the OS) executable will be created. This is not |
625 |
immensely useful when just creating the bundle files, but is |
626 |
most useful when linking a binary with the "--perl" or "--app" |
627 |
options. |
628 |
|
629 |
The default is to link the new binary dynamically (that means |
630 |
all perl modules are linked statically, but all external |
631 |
libraries are still referenced dynamically). |
632 |
|
633 |
Keep in mind that Solaris doesn't support static linking at all, |
634 |
and systems based on GNU libc don't really support it in a very |
635 |
usable fashion either. Try uClibc if you want to create fully |
636 |
statically linked executables, or try the "--staticlib" option |
637 |
to link only some libraries statically. |
638 |
|
639 |
"--staticlib" libname |
640 |
When not linking fully statically, this option allows you to |
641 |
link specific libraries statically. What it does is simply |
642 |
replace all occurrences of "-llibname" with the GCC-specific |
643 |
"-Wl,-Bstatic -llibname -Wl,-Bdynamic" option. |
644 |
|
645 |
This will have no effect unless the library is actually linked |
646 |
against, specifically, "--staticlib" will not link against the |
647 |
named library unless it would be linked against anyway. |
648 |
|
649 |
Example: link libcrypt statically into the final binary. |
650 |
|
651 |
staticperl mkperl -MIO::AIO --staticlib crypt |
652 |
|
653 |
# ldopts might now contain: |
654 |
# -lm -Wl,-Bstatic -lcrypt -Wl,-Bdynamic -lpthread |
655 |
|
656 |
EXTENDED GLOB PATTERNS |
657 |
Some options of staticperl mkbundle expect an *extended glob pattern*. |
658 |
This is neither a normal shell glob nor a regex, but something in |
659 |
between. The idea has been copied from rsync, and there are the current |
660 |
matching rules: |
661 |
|
662 |
Patterns starting with / will be a anchored at the root of the library |
663 |
tree. |
664 |
That is, /unicore will match the unicore directory in @INC, but |
665 |
nothing inside, and neither any other file or directory called |
666 |
unicore anywhere else in the hierarchy. |
667 |
|
668 |
Patterns not starting with / will be anchored at the end of the path. |
669 |
That is, idna.pl will match any file called idna.pl anywhere in the |
670 |
hierarchy, but not any directories of the same name. |
671 |
|
672 |
A * matches anything within a single path component. |
673 |
That is, /unicore/*.pl would match all .pl files directly inside |
674 |
"/unicore", not any deeper level .pl files. Or in other words, * |
675 |
will not match slashes. |
676 |
|
677 |
A ** matches anything. |
678 |
That is, /unicore/**.pl would match all .pl files under /unicore, no |
679 |
matter how deeply nested they are inside subdirectories. |
680 |
|
681 |
A ? matches a single character within a component. |
682 |
That is, /Encode/??.pm matches /Encode/JP.pm, but not the |
683 |
hypothetical /Encode/J/.pm, as ? does not match /. |
684 |
|
685 |
STATICPERL CONFIGURATION AND HOOKS |
686 |
During (each) startup, staticperl tries to source some shell files to |
687 |
allow you to fine-tune/override configuration settings. |
688 |
|
689 |
In them you can override shell variables, or define shell functions |
690 |
("hooks") to be called at specific phases during installation. For |
691 |
example, you could define a "postinstall" hook to install additional |
692 |
modules from CPAN each time you start from scratch. |
693 |
|
694 |
If the env variable $STATICPERLRC is set, then staticperl will try to |
695 |
source the file named with it only. Otherwise, it tries the following |
696 |
shell files in order: |
697 |
|
698 |
/etc/staticperlrc |
699 |
~/.staticperlrc |
700 |
$STATICPERL/rc |
701 |
|
702 |
Note that the last file is erased during staticperl distclean, so |
703 |
generally should not be used. |
704 |
|
705 |
CONFIGURATION VARIABLES |
706 |
Variables you *should* override |
707 |
"EMAIL" |
708 |
The e-mail address of the person who built this binary. Has no good |
709 |
default, so should be specified by you. |
710 |
|
711 |
"CPAN" |
712 |
The URL of the CPAN mirror to use (e.g. |
713 |
<http://mirror.netcologne.de/cpan/>). |
714 |
|
715 |
"EXTRA_MODULES" |
716 |
Additional modules installed during staticperl install. Here you can |
717 |
set which modules you want have to installed from CPAN. |
718 |
|
719 |
Example: I really really need EV, AnyEvent, Coro and AnyEvent::AIO. |
720 |
|
721 |
EXTRA_MODULES="EV AnyEvent Coro AnyEvent::AIO" |
722 |
|
723 |
Note that you can also use a "postinstall" hook to achieve this, and |
724 |
more. |
725 |
|
726 |
Variables you might *want* to override |
727 |
"STATICPERL" |
728 |
The directory where staticperl stores all its files (default: |
729 |
~/.staticperl). |
730 |
|
731 |
"PERL_MM_USE_DEFAULT", "EV_EXTRA_DEFS", ... |
732 |
Usually set to 1 to make modules "less inquisitive" during their |
733 |
installation, you can set any environment variable you want - some |
734 |
modules (such as Coro or EV) use environment variables for further |
735 |
tweaking. |
736 |
|
737 |
"PERL_VERSION" |
738 |
The perl version to install - default is currently 5.12.2, but 5.8.9 |
739 |
is also a good choice (5.8.9 is much smaller than 5.12.2, while |
740 |
5.10.1 is about as big as 5.12.2). |
741 |
|
742 |
"PERL_PREFIX" |
743 |
The prefix where perl gets installed (default: $STATICPERL/perl), |
744 |
i.e. where the bin and lib subdirectories will end up. |
745 |
|
746 |
"PERL_CONFIGURE" |
747 |
Additional Configure options - these are simply passed to the perl |
748 |
Configure script. For example, if you wanted to enable dynamic |
749 |
loading, you could pass "-Dusedl". To enable ithreads (Why would you |
750 |
want that insanity? Don't! Use forks instead!) you would pass |
751 |
"-Duseithreads" and so on. |
752 |
|
753 |
More commonly, you would either activate 64 bit integer support |
754 |
("-Duse64bitint"), or disable large files support (-Uuselargefiles), |
755 |
to reduce filesize further. |
756 |
|
757 |
"PERL_CC", "PERL_CCFLAGS", "PERL_OPTIMIZE", "PERL_LDFLAGS", "PERL_LIBS" |
758 |
These flags are passed to perl's Configure script, and are generally |
759 |
optimised for small size (at the cost of performance). Since they |
760 |
also contain subtle workarounds around various build issues, |
761 |
changing these usually requires understanding their default values - |
762 |
best look at the top of the staticperl script for more info on |
763 |
these, and use a ~/.staticperlrc to override them. |
764 |
|
765 |
Most of the variables override (or modify) the corresponding |
766 |
Configure variable, except "PERL_CCFLAGS", which gets appended. |
767 |
|
768 |
Variables you probably *do not want* to override |
769 |
"MAKE" |
770 |
The make command to use - default is "make". |
771 |
|
772 |
"MKBUNDLE" |
773 |
Where staticperl writes the "mkbundle" command to (default: |
774 |
$STATICPERL/mkbundle). |
775 |
|
776 |
"STATICPERL_MODULES" |
777 |
Additional modules needed by "mkbundle" - should therefore not be |
778 |
changed unless you know what you are doing. |
779 |
|
780 |
OVERRIDABLE HOOKS |
781 |
In addition to environment variables, it is possible to provide some |
782 |
shell functions that are called at specific times. To provide your own |
783 |
commands, just define the corresponding function. |
784 |
|
785 |
Example: install extra modules from CPAN and from some directories at |
786 |
staticperl install time. |
787 |
|
788 |
postinstall() { |
789 |
rm -rf lib/threads* # weg mit Schaden |
790 |
instcpan IO::AIO EV |
791 |
instsrc ~/src/AnyEvent |
792 |
instsrc ~/src/XML-Sablotron-1.0100001 |
793 |
instcpan Anyevent::AIO AnyEvent::HTTPD |
794 |
} |
795 |
|
796 |
preconfigure |
797 |
Called just before running ./Configur in the perl source directory. |
798 |
Current working directory is the perl source directory. |
799 |
|
800 |
This can be used to set any "PERL_xxx" variables, which might be |
801 |
costly to compute. |
802 |
|
803 |
postconfigure |
804 |
Called after configuring, but before building perl. Current working |
805 |
directory is the perl source directory. |
806 |
|
807 |
Could be used to tailor/patch config.sh (followed by sh Configure |
808 |
-S) or do any other modifications. |
809 |
|
810 |
postbuild |
811 |
Called after building, but before installing perl. Current working |
812 |
directory is the perl source directory. |
813 |
|
814 |
I have no clue what this could be used for - tell me. |
815 |
|
816 |
postinstall |
817 |
Called after perl and any extra modules have been installed in |
818 |
$PREFIX, but before setting the "installation O.K." flag. |
819 |
|
820 |
The current working directory is $PREFIX, but maybe you should not |
821 |
rely on that. |
822 |
|
823 |
This hook is most useful to customise the installation, by deleting |
824 |
files, or installing extra modules using the "instcpan" or "instsrc" |
825 |
functions. |
826 |
|
827 |
The script must return with a zero exit status, or the installation |
828 |
will fail. |
829 |
|
830 |
ANATOMY OF A BUNDLE |
831 |
When not building a new perl binary, "mkbundle" will leave a number of |
832 |
files in the current working directory, which can be used to embed a |
833 |
perl interpreter in your program. |
834 |
|
835 |
Intimate knowledge of perlembed and preferably some experience with |
836 |
embedding perl is highly recommended. |
837 |
|
838 |
"mkperl" (or the "--perl" option) basically does this to link the new |
839 |
interpreter (it also adds a main program to bundle.): |
840 |
|
841 |
$Config{cc} $(cat bundle.ccopts) -o perl bundle.c $(cat bundle.ldopts) |
842 |
|
843 |
bundle.h |
844 |
A header file that contains the prototypes of the few symbols |
845 |
"exported" by bundle.c, and also exposes the perl headers to the |
846 |
application. |
847 |
|
848 |
staticperl_init () |
849 |
Initialises the perl interpreter. You can use the normal perl |
850 |
functions after calling this function, for example, to define |
851 |
extra functions or to load a .pm file that contains some |
852 |
initialisation code, or the main program function: |
853 |
|
854 |
XS (xsfunction) |
855 |
{ |
856 |
dXSARGS; |
857 |
|
858 |
// now we have items, ST(i) etc. |
859 |
} |
860 |
|
861 |
static void |
862 |
run_myapp(void) |
863 |
{ |
864 |
staticperl_init (); |
865 |
newXSproto ("myapp::xsfunction", xsfunction, __FILE__, "$$;$"); |
866 |
eval_pv ("require myapp::main", 1); // executes "myapp/main.pm" |
867 |
} |
868 |
|
869 |
staticperl_xs_init (pTHX) |
870 |
Sometimes you need direct control over "perl_parse" and |
871 |
"perl_run", in which case you do not want to use |
872 |
"staticperl_init" but call them on your own. |
873 |
|
874 |
Then you need this function - either pass it directly as the |
875 |
"xs_init" function to "perl_parse", or call it from your own |
876 |
"xs_init" function. |
877 |
|
878 |
staticperl_cleanup () |
879 |
In the unlikely case that you want to destroy the perl |
880 |
interpreter, here is the corresponding function. |
881 |
|
882 |
PerlInterpreter *staticperl |
883 |
The perl interpreter pointer used by staticperl. Not normally so |
884 |
useful, but there it is. |
885 |
|
886 |
bundle.ccopts |
887 |
Contains the compiler options required to compile at least bundle.c |
888 |
and any file that includes bundle.h - you should probably use it in |
889 |
your "CFLAGS". |
890 |
|
891 |
bundle.ldopts |
892 |
The linker options needed to link the final program. |
893 |
|
894 |
RUNTIME FUNCTIONALITY |
895 |
Binaries created with "mkbundle"/"mkperl" contain extra functions, which |
896 |
are required to access the bundled perl sources, but might be useful for |
897 |
other purposes. |
898 |
|
899 |
In addition, for the embedded loading of perl files to work, staticperl |
900 |
overrides the @INC array. |
901 |
|
902 |
$file = staticperl::find $path |
903 |
Returns the data associated with the given $path (e.g. |
904 |
"Digest/MD5.pm", "auto/POSIX/autosplit.ix"), which is basically the |
905 |
UNIX path relative to the perl library directory. |
906 |
|
907 |
Returns "undef" if the file isn't embedded. |
908 |
|
909 |
@paths = staticperl::list |
910 |
Returns the list of all paths embedded in this binary. |
911 |
|
912 |
FULLY STATIC BINARIES - UCLIBC AND BUILDROOT |
913 |
To make truly static (Linux-) libraries, you might want to have a look |
914 |
at buildroot (<http://buildroot.uclibc.org/>). |
915 |
|
916 |
Buildroot is primarily meant to set up a cross-compile environment |
917 |
(which is not so useful as perl doesn't quite like cross compiles), but |
918 |
it can also compile a chroot environment where you can use staticperl. |
919 |
|
920 |
To do so, download buildroot, and enable "Build options => development |
921 |
files in target filesystem" and optionally "Build options => gcc |
922 |
optimization level (optimize for size)". At the time of writing, I had |
923 |
good experiences with GCC 4.4.x but not GCC 4.5. |
924 |
|
925 |
To minimise code size, I used "-pipe -ffunction-sections -fdata-sections |
926 |
-finline-limit=8 -fno-builtin-strlen -mtune=i386". The "-mtune=i386" |
927 |
doesn't decrease codesize much, but it makes the file much more |
928 |
compressible. |
929 |
|
930 |
If you don't need Coro or threads, you can go with "linuxthreads.old" |
931 |
(or no thread support). For Coro, it is highly recommended to switch to |
932 |
a uClibc newer than 0.9.31 (at the time of this writing, I used the |
933 |
20101201 snapshot) and enable NPTL, otherwise Coro needs to be |
934 |
configured with the ultra-slow pthreads backend to work around |
935 |
linuxthreads bugs (it also uses twice the address space needed for |
936 |
stacks). |
937 |
|
938 |
If you use "linuxthreads.old", then you should also be aware that uClibc |
939 |
shares "errno" between all threads when statically linking. See |
940 |
<http://lists.uclibc.org/pipermail/uclibc/2010-June/044157.html> for a |
941 |
workaround (And <https://bugs.uclibc.org/2089> for discussion). |
942 |
|
943 |
"ccache" support is also recommended, especially if you want to play |
944 |
around with buildroot options. Enabling the "miniperl" package will |
945 |
probably enable all options required for a successful perl build. |
946 |
staticperl itself additionally needs either "wget" (recommended, for |
947 |
CPAN) or "curl". |
948 |
|
949 |
As for shells, busybox should provide all that is needed, but the |
950 |
default busybox configuration doesn't include comm which is needed by |
951 |
perl - either make a custom busybox config, or compile coreutils. |
952 |
|
953 |
For the latter route, you might find that bash has some bugs that keep |
954 |
it from working properly in a chroot - either use dash (and link it to |
955 |
/bin/sh inside the chroot) or link busybox to /bin/sh, using it's |
956 |
built-in ash shell. |
957 |
|
958 |
Finally, you need /dev/null inside the chroot for many scripts to work - |
959 |
cp /dev/null output/target/dev or bind-mounting your /dev will both |
960 |
provide this. |
961 |
|
962 |
After you have compiled and set up your buildroot target, you can copy |
963 |
staticperl from the "App::Staticperl" distribution or from your perl |
964 |
f<bin> directory (if you installed it) into the output/target |
965 |
filesystem, chroot inside and run it. |
966 |
|
967 |
RECIPES / SPECIFIC MODULES |
968 |
This section contains some common(?) recipes and information about |
969 |
problems with some common modules or perl constructs that require extra |
970 |
files to be included. |
971 |
|
972 |
MODULES |
973 |
utf8 |
974 |
Some functionality in the utf8 module, such as swash handling (used |
975 |
for unicode character ranges in regexes) is implemented in the |
976 |
"utf8_heavy.pl" library: |
977 |
|
978 |
-M'"utf8_heavy.pl"' |
979 |
|
980 |
Many Unicode properties in turn are defined in separate modules, |
981 |
such as "unicore/Heavy.pl" and more specific data tables such as |
982 |
"unicore/To/Digit.pl" or "unicore/lib/Perl/Word.pl". These tables |
983 |
are big (7MB uncompressed, although staticperl contains special |
984 |
handling for those files), so including them on demand by your |
985 |
application only might pay off. |
986 |
|
987 |
To simply include the whole unicode database, use: |
988 |
|
989 |
--incglob '/unicore/**.pl' |
990 |
|
991 |
AnyEvent |
992 |
AnyEvent needs a backend implementation that it will load in a |
993 |
delayed fashion. The AnyEvent::Impl::Perl backend is the default |
994 |
choice for AnyEvent if it can't find anything else, and is usually a |
995 |
safe fallback. If you plan to use e.g. EV (POE...), then you need to |
996 |
include the AnyEvent::Impl::EV (AnyEvent::Impl::POE...) backend as |
997 |
well. |
998 |
|
999 |
If you want to handle IRIs or IDNs (AnyEvent::Util punycode and idn |
1000 |
functions), you also need to include "AnyEvent/Util/idna.pl" and |
1001 |
"AnyEvent/Util/uts46data.pl". |
1002 |
|
1003 |
Or you can use "--usepacklists" and specify "-MAnyEvent" to include |
1004 |
everything. |
1005 |
|
1006 |
Carp |
1007 |
Carp had (in older versions of perl) a dependency on Carp::Heavy. As |
1008 |
of perl 5.12.2 (maybe earlier), this dependency no longer exists. |
1009 |
|
1010 |
Config |
1011 |
The perl -V switch (as well as many modules) needs Config, which in |
1012 |
turn might need "Config_heavy.pl". Including the latter gives you |
1013 |
both. |
1014 |
|
1015 |
Term::ReadLine::Perl |
1016 |
Also needs Term::ReadLine::readline, or "--usepacklists". |
1017 |
|
1018 |
URI URI implements schemes as separate modules - the generic URL scheme |
1019 |
is implemented in URI::_generic, HTTP is implemented in URI::http. |
1020 |
If you need to use any of these schemes, you should include these |
1021 |
manually, or use "--usepacklists". |
1022 |
|
1023 |
RECIPES |
1024 |
Just link everything in |
1025 |
To link just about everything installed in the perl library into a |
1026 |
new perl, try this (the first time this runs it will take a long |
1027 |
time, as a lot of files need to be parsed): |
1028 |
|
1029 |
staticperl mkperl -v --strip ppi --incglob '*' |
1030 |
|
1031 |
If you don't mind the extra megabytes, this can be a very effective |
1032 |
way of creating bundles without having to worry about forgetting any |
1033 |
modules. |
1034 |
|
1035 |
You get even more useful variants of this method by first selecting |
1036 |
everything, and then excluding stuff you are reasonable sure not to |
1037 |
need - bigperl <http://staticperl.schmorp.de/bigperl.html> uses this |
1038 |
approach. |
1039 |
|
1040 |
Getting rid of netdb functions |
1041 |
The perl core has lots of netdb functions ("getnetbyname", |
1042 |
"getgrent" and so on) that few applications use. You can avoid |
1043 |
compiling them in by putting the following fragment into a |
1044 |
"preconfigure" hook: |
1045 |
|
1046 |
preconfigure() { |
1047 |
for sym in \ |
1048 |
d_getgrnam_r d_endgrent d_endgrent_r d_endhent \ |
1049 |
d_endhostent_r d_endnent d_endnetent_r d_endpent \ |
1050 |
d_endprotoent_r d_endpwent d_endpwent_r d_endsent \ |
1051 |
d_endservent_r d_getgrent d_getgrent_r d_getgrgid_r \ |
1052 |
d_getgrnam_r d_gethbyaddr d_gethent d_getsbyport \ |
1053 |
d_gethostbyaddr_r d_gethostbyname_r d_gethostent_r \ |
1054 |
d_getlogin_r d_getnbyaddr d_getnbyname d_getnent \ |
1055 |
d_getnetbyaddr_r d_getnetbyname_r d_getnetent_r \ |
1056 |
d_getpent d_getpbyname d_getpbynumber d_getprotobyname_r \ |
1057 |
d_getprotobynumber_r d_getprotoent_r d_getpwent \ |
1058 |
d_getpwent_r d_getpwnam_r d_getpwuid_r d_getsent \ |
1059 |
d_getservbyname_r d_getservbyport_r d_getservent_r \ |
1060 |
d_getspnam_r d_getsbyname |
1061 |
# d_gethbyname |
1062 |
do |
1063 |
PERL_CONFIGURE="$PERL_CONFIGURE -U$sym" |
1064 |
done |
1065 |
} |
1066 |
|
1067 |
This mostly gains space when linking statically, as the functions |
1068 |
will likely not be linked in. The gain for dynamically-linked |
1069 |
binaries is smaller. |
1070 |
|
1071 |
Also, this leaves "gethostbyname" in - not only is it actually used |
1072 |
often, the Socket module also exposes it, so leaving it out usually |
1073 |
gains little. Why Socket exposes a C function that is in the core |
1074 |
already is anybody's guess. |
1075 |
|
1076 |
AUTHOR |
1077 |
Marc Lehmann <schmorp@schmorp.de> |
1078 |
http://software.schmorp.de/pkg/staticperl.html |
1079 |
|