ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Perl-LibExtractor/README
Revision: 1.2
Committed: Fri Jan 27 20:39:07 2012 UTC (12 years, 4 months ago) by root
Branch: MAIN
CVS Tags: rel-1_1, rel-1_0, HEAD
Changes since 1.1: +402 -0 lines
Log Message:
1.0

File Contents

# User Rev Content
1 root 1.2 NAME
2     Perl::LibExtractor - determine perl library subsets for building
3     distributions
4    
5     SYNOPSIS
6     use Perl::LibExtractor;
7    
8     DESCRIPTION
9     The purpose of this module is to determine subsets of your perl library,
10     that is, a set of files needed to satisfy certain dependencies (e.g. of
11     a program).
12    
13     The goal is to extract a part of your perl installation including
14     dependencies. A typical use case for this module would be to find out
15     which files are needed to be build a PAR distribution, to link into an
16     App::Staticperl binary, or to pack with Urlader, to create stand-alone
17     distributions tailormade to run your app.
18    
19     METHODS
20     To use this module, first call the "new"-constructor and then as many
21     other methods as you want, to generate a set of files. Then query the
22     set of files and do whatever you want with them.
23    
24     The command-line utility perl-libextract can be a convenient alternative
25     to using this module directly, and offers a few extra options, such as
26     to copy out the files into a new directory, strip them and/or manipulate
27     them in other ways.
28    
29     CREATION
30     $extractor = new Perl::LibExtractor [key => value...]
31     Creates a new extractor object. Each extractor object stores some
32     configuration options and a subset of files that can be queried at
33     any time,.
34    
35     Binary executables (such as the perl interpreter) are stored inside
36     bin/, perl scripts are stored under script/, perl library files are
37     stored under lib/ and shared libraries are stored under dll/.
38    
39     The following key-value pairs exist, with default values as
40     specified.
41    
42     inc => \@INC without "."
43     An arrayref with paths to perl library directories. The default
44     is "\@INC", with . removed.
45    
46     To prepend custom dirs just do this:
47    
48     inc => ["mydir", @INC],
49    
50     use_packlist => 1
51     Enable (if true) or disable the use of ".packlist" files. If
52     enabled, then each time a file is traced, the complete
53     distribution that contains it is included (but not traced).
54    
55     If disabled, only shared objects and autoload files will be
56     added.
57    
58     Debian GNU/Linux doesn't completely package perl or any perl
59     modules, so this option will fail. Other perls should be fine.
60    
61     extra_deps => { file => [files...] }
62     Some (mainly runtime dependencies in the perl core library)
63     cannot be detected automatically by this module, especially if
64     you don't use packlists and "add_core".
65    
66     This module comes with a set of default dependencies (such as
67     Carp requiring Carp::Heavy), which you cna override with this
68     parameter.
69    
70     To see the default set of dependencies that come with this
71     module, use this:
72    
73     perl -MPerl::LibExtractor -MData::Dumper -e 'print Dumper $Perl::LibExtractor::EXTRA_DEPS'
74    
75     TRACE/PACKLIST BASED ADDING
76     The following methods add various things to the set of files.
77    
78     Each time a perl file is added, it is scanned by tracing either loading,
79     execution or compiling it, and seeing which other perl modules and
80     libraries have been loaded.
81    
82     For each library file found this way, additional dependencies are added:
83     if packlists are enabled, then all files of the distribution that
84     contains the file will be added. If packlists are disabled, then only
85     shared objects and autoload files for modules will be added.
86    
87     Only files from perl library directories will be added automatically.
88     Any other files (such as manpages or scripts installed in the bin
89     directory) are skipped.
90    
91     If there is an error, such as a module not being found, then this module
92     croaks (as opposed to silently skipping). If you want to add something
93     of which you are not sure it exists, then you can wrap the call into
94     "eval {}". In some cases, you can avoid this by executing the code you
95     want to work later using "add_eval" - see "add_core_support" for an
96     actual example of this technique.
97    
98     Note that packlists are meant to add files not covered by other
99     mechanisms, such as resource files and other data files loaded directly
100     by a module - they are not meant to add dependencies that are missed
101     because they only happen at runtime.
102    
103     For example, with packlists, when using AnyEvent, then all event loop
104     backends are automatically added as well, but *not* any event loops
105     (i.e. AnyEvent::Impl::POE is added, but POE itself is not). Without
106     packlists, only the backend that is being used is added (i.e. normally
107     none, as loading AnyEvent does not instantly load any backend).
108    
109     To catch the extra event loop dependencies, you can either initialise
110     AnyEvent so it picks a suitable backend:
111    
112     $extractor->add_eval ("use AnyEvent; AnyEvent::detect");
113    
114     Or you can directly load the backend modules you plan to use:
115    
116     $extractor->add_mod ("AnyEvent::Impl::EV", "AnyEvent::Impl::Perl");
117    
118     An example of a program (or module) that has extra resource files is
119     Deliantra::Client - the normal tracing (without packlist usage) will
120     correctly add all submodules, but miss the fonts and textures. By using
121     the packlist, those files are added correctly.
122    
123     $extractor->add_mod ($module[, $module...])
124     Adds the given module(s) to the file set - the module name must be
125     specified as in "use", i.e. with "::" as separators and without .pm.
126    
127     The program will be loaded with the default import list, any
128     dependent files, such as the shared object implementing xs
129     functions, or autoload files, will also be added.
130    
131     If you want to use a different import list (for those rare modules
132     wghere import lists trigger different backend modules to be loaded
133     for example), you can use "add_eval" instead:
134    
135     $extractor->add_eval ("use Module qw(a b c)");
136    
137     Example: add Coro.pm and AnyEvent/AIO.pm, and all relevant files
138     from the distribution they are part of.
139    
140     $extractor->add_mod ("Coro", "AnyEvent::AIO");
141    
142     $extractor->add_require ($name[, $name...])
143     Works like "add_mod", but uses "require $name" to load the module,
144     i.e. the name must be a filename.
145    
146     Example: load Coro and AnyEvent::AIO, but using "add_require"
147     instead of "add_mod".
148    
149     $extractor->add_require ("Coro.pm", "AnyEvent/AIO.pm");
150    
151     $extractor->add_bin ($name[, $name...])
152     Adds the given (perl) program(s) to the file set, that is, a program
153     installed by some perl module, written in perl (an example would be
154     the perl-libextract program that is part of the "Perl::LibExtractor"
155     distribution).
156    
157     Example: add the deliantra client program installed by the
158     Deliantra::Client module and put it under bin/deliantra.
159    
160     $extractor->add_bin ("deliantra");
161    
162     $extractor->add_eval ($string)
163     Evaluates the string as perl code and adds all modules that are
164     loaded by it. For example, this would add AnyEvent and the default
165     backend implementation module and event loop module:
166    
167     $extractor->add_eval ("use AnyEvent; AnyEvent::detect");
168    
169     Each code snippet will be executed in its own package and under "use
170     strict".
171    
172     OTHER METHODS FOR ADDING FILES
173     The following methods add commonly used files that are either not
174     covered by other methods or add commonly-used dependencies.
175    
176     $extractor->add_perl
177     Adds the perl binary itself to the file set, including the libperl
178     dll, if needed.
179    
180     For example, on UNIX systems, this usually adds a exe/perl and
181     possibly some dll/libperl.so.XXX.
182    
183     $extractor->add_core_support
184     Try to add modules and files needed to support commonly-used builtin
185     language features. For example to open a scalar for I/O you need the
186     PerlIO::scalar module:
187    
188     open $fh, "<", \$scalar
189    
190     A number of regex and string features (e.g. "ucfirst") need some
191     unicore files, e.g.:
192    
193     'my $x = chr 1234; "\u$x\U$x\l$x\L$x"; $x =~ /\d|\w|\s|\b|$x/i';
194    
195     This call adds these files (simply by executing code similar to the
196     above code fragments).
197    
198     Notable things that are missing are other PerlIO layers, such as
199     PerlIO::encoding, and named character and character class matches.
200    
201     $extractor->add_unicore
202     Adds (hopefully) all files from the unicore database that will ever
203     be needed.
204    
205     If you are not sure which unicode character classes and similar
206     unicore databases you need, and you do not care about an extra one
207     thousand(!) files comprising 4MB of data, then you can just call
208     this method, which adds basically all files from perl's unicode
209     database.
210    
211     Note that "add_core_support" also adds some unicore files, but it's
212     not a subset of "add_unicore" - the former adds all files neccessary
213     to support core builtins (which includes some unicore files and
214     other things), while the latter adds all unicore files (but nothing
215     else).
216    
217     When in doubt, use both.
218    
219     $extractor->add_core
220     This adds all files from the perl core distribution, that is, all
221     library files that come with perl.
222    
223     This is a superset of "add_core_support" and "add_unicore".
224    
225     This is quite a lot, but on the plus side, you can be sure nothing
226     is missing.
227    
228     This requires a full perl installation - Debian GNU/Linux doesn't
229     package the full perl library, so this function will not work there.
230    
231     GLOB-BASED ADDING AND FILTERING
232     These methods add or manipulate files by using glob-based patterns.
233    
234     These glob patterns work similarly to glob patterns in the shell:
235    
236     / A / at the start of the pattern interprets the pattern as a file
237     path inside the file set, almost the same as in the shell. For
238     example, /bin/perl* would match all files whose names starting with
239     perl inside the bin directory in the set.
240    
241     If the / is missing, then the pattern is interpreted as a module
242     name (a .pm file). For example, Coro matches the file lib/Coro.pm ,
243     while Coro::* would match lib/Coro/*.pm.
244    
245     * A single star matches anything inside a single directory component.
246     For example, /lib/Coro/*.pm would match all .pm files inside the
247     lib/Coro/ directory, but not any files deeper in the hierarchy.
248    
249     Another way to look at it is that a single star matches anything but
250     a slash (/).
251    
252     ** A double star matches any number of characters in the path,
253     including /.
254    
255     For example, AnyEvent::** would match all modules whose names start
256     with "AnyEvent::", no matter how deep in the hierarchy they are.
257    
258     $extractor->add_glob ($modglob[, $modglob...])
259     Adds all files from the perl library that match the given glob
260     pattern.
261    
262     For example, you could implement "add_unicore" yourself like this:
263    
264     $extractor->add_glob ("/unicore/**.pl");
265    
266     $extractor->filter ($pattern[, $pattern...])
267     Applies a series of include/exclude filters. Each filter must start
268     with either "+" or "-", to designate the pattern as *include* or
269     *exclude* pattern. The rest of the pattern is a normal glob pattern.
270    
271     An exclude pattern ("-") instantly removes all matching files from
272     the set. An include pattern ("+") protects matching files from later
273     removals.
274    
275     That is, if you have an include pattern then all files that were
276     matched by it will be included in the set, regardless of any further
277     exclude patterns matching the same files.
278    
279     Likewise, any file excluded by a pattern will not be included in the
280     set, even if matched by later include patterns.
281    
282     Any files not matched by any expression will simply stay in the set.
283    
284     For example, to remove most of the useless autoload functions by the
285     POSIX module (they either do the same thing as a builtin or always
286     raise an error), you would use this:
287    
288     $extractor->filter ("-/lib/auto/POSIX/*.al");
289    
290     This does not remove all autoload files, only the ones not defined
291     by a subclass (e.g. it leaves "POSIX::SigRt::xxx" alone).
292    
293     $extractor->runtime_only
294     This removes all files that are not needed at runtime, such as
295     static archives, header and other files needed only for compilation
296     of modules, and pod and html files (which are unlikely to be needed
297     at runtime).
298    
299     This is quite useful when you want to have only files actually
300     needed to execute a program.
301    
302     RESULT SET
303     $set = $extractor->set
304     Returns a hash reference that represents the result set. The hash is
305     the actual internal storage hash and can only be modified as
306     described below.
307    
308     Each key in the hash is the path inside the set, without a leading
309     slash, e.g.:
310    
311     bin/perl
312     lib/unicore/lib/Blk/Superscr.pl
313     lib/AnyEvent/Impl/EV.pm
314    
315     The value is an array reference with mostly unspecified contents,
316     except the first element, which is the file system path where the
317     actual file can be found.
318    
319     This code snippet lists all files inside the set:
320    
321     print "$_\n"
322     for sort keys %{ $extractor->set });
323    
324     This code fragment prints "filesystem_path => set_path" pairs for
325     all files in the set:
326    
327     my $set = $extractor->set;
328     while (my ($set,$fspath) = each %$set) {
329     print "$fspath => $set\n";
330     }
331    
332     You can implement your own filtering by asking for the result set
333     with "$extractor->set", and then deleting keys from the referenced
334     hash - since you can ask for the result set at any time you can add
335     things, filter them out this way, and add additional things.
336    
337     EXAMPLE
338     To package he deliantra client (Deliantra::Client), finding all (perl)
339     files needed to run it is a first step. This can be done by using
340     something like the following code snippet:
341    
342     my $ex = new Perl::LibExtractor;
343    
344     $ex->add_perl;
345     $ex->add_core_support;
346     $ex->add_bin ("deliantra");
347     $ex->add_mod ("AnyEvent::Impl::EV");
348     $ex->add_mod ("AnyEvent::Impl::Perl");
349     $ex->add_mod ("Urlader");
350     $ex->filter ("-/*/auto/POSIX/**.al");
351     $ex->runtime_only;
352    
353     First it sets the perl library directory to pm and . (the latter to work
354     around some AutoLoader bugs), so perl uses only the perl library files
355     that came with the binary package.
356    
357     Then it sets some environment variable to override the system default
358     (which might be incompatible).
359    
360     Then it runs the client itself, using "require". Since "require" only
361     looks in the perl library directory this is the reaosn why the scripts
362     were put there (of course, since . is also included it doesn't matter,
363     but I refuse to yield to bugs).
364    
365     Finally it exits with a clean status to signal "ok" to Urlader.
366    
367     Back to the original "Perl::LibExtractor" script: after initialising a
368     new set, the script simply adds the perl interpreter and core support
369     files (just in case, not all are needed, but some are, and I am too lazy
370     to find out which ones exactly).
371    
372     Then it adds the deliantra executable itself, which in turn adds most of
373     the required modules. After that, the AnyEvent implementation modules
374     are added because these dependencies are not picked up automatically.
375    
376     The Urlader module is added because the client itself does not depend on
377     it at all, but the wrapper does.
378    
379     At this point, all required files are present, and it's time to slim
380     down: most of the ueseless POSIX autoloaded functions are removed, not
381     because they are so big, but because creating files is a costly
382     operation in itself, so even small fiels have considerable overhead when
383     unpacking. Then files not required for running the client are removed.
384    
385     And that concludes it, the set is now ready.
386    
387     SEE ALSO
388     The utility program that comes with this module: perl-libextract.
389    
390     App::Staticperl, Urlader, Perl::Squish.
391    
392     LICENSE
393     This software package is licensed under the GPL version 3 or any later
394     version, see COPYING for details.
395    
396     This license does not, of course, apply to any output generated by this
397     software.
398    
399     AUTHOR
400     Marc Lehmann <schmorp@schmorp.de>
401     http://home.schmorp.de/
402