ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Urlader/Urlader.pm
Revision: 1.12
Committed: Mon Jun 10 06:16:16 2013 UTC (10 years, 10 months ago) by root
Branch: MAIN
CVS Tags: HEAD
Changes since 1.11: +4 -0 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 NAME
2
3 Urlader - installer-less single-file independent executables
4
5 =head1 SYNOPSIS
6
7 use Urlader;
8
9 =head1 DESCRIPTION
10
11 Urlader (that's german for "bootloader" btw.) was created out of
12 frustration over PAR always being horribly slow, again not working, again
13 not being flexible enough for simple things such as software upgrades, and
14 again causing mysterious missing file issues on various platforms.
15
16 That doesn't mean this module replaces PAR, in fact, you should stay with
17 PAR for many reasons right now, user-friendlyness is one of them.
18
19 However, if you want to make single-file distributions out of your perl
20 programs (or python, or C or whatever), and you are prepared to fiddle
21 a LOT, this module might provide a faster and more versatile deployment
22 technique then PAR. Well, if it ever gets finished.
23
24 Also, I<nothing in this module is considered very stable yet>, and it's
25 far from feature-complete.
26
27 Having said all that, Urlader basically provides three services:
28
29 =over 4
30
31 =item A simple archiver that packs a directory tree into a single file.
32
33 =item A small C program that works on windows and unix, which unpacks an attached
34 archive and runs a program (perl, python, whatever...).
35
36 =item A perl module support module (I<this one>), that can be used to query
37 the runtime environment, find out where to install updates and so on.
38
39 =back
40
41 =head1 EXAMPLE
42
43 How can it be used to provide single-file executables?
44
45 So simple, create a directory with everything that's needed, e.g.:
46
47 # find bintree
48 bintree/perl
49 bintree/libperl.so.5.10
50 bintree/run
51 bintree/pm/Guard.pm
52 bintree/pm/auto/Guard/Guard.so
53 bintree/pm/XSLoader.pm
54 bintree/pm/DynaLoader.pm
55 bintree/pm/Config.pm
56 bintree/pm/strict.pm
57 bintree/pm/vars.pm
58 bintree/pm/warnings.pm
59
60 # cat bintree/run
61 @INC = ("pm", "."); # "." works around buggy AutoLoader
62 use Guard;
63 guard { warn "hello, world!\n" }; # just to show off
64 exit 0; # tell the urlader that everything was fine
65
66 Then pack it:
67
68 # wget http://urlader.schmorp.de/prebuilt/1.0/linux-x86
69 # urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \
70 LD_LIBRARY_PATH=. ./perl run \
71 >myprog
72 # chmod 755 myprog
73
74 The packing step takes a binary loader and appends all the files in the
75 directory tree, plus some meta information.
76
77 The resulting file is an executable that, when run, will unpack all the
78 files and run the embedded program.
79
80 =head1 CONCEPTS
81
82 =over 4
83
84 =item urlader
85
86 A small (hopefully) and relatively portable (hopefully) binary that is
87 prepended to a pack file to make it executable.
88
89 You can build it yourself from sources (see F<prebuilt/Makefile> in the
90 distribution) or use one of the precompiled ones at:
91
92 http://urlader.schmorp.de/prebuilt/1.0/
93
94 The F<README> there has further information on the binaries provided.
95
96 =item exe_id
97
98 A string that uniquely identifies your program - all branches of it. It
99 must consist of the characters C<A-Za-z0-9_-> only and should be a valid
100 directory name on all systems you want to deploy on.
101
102 =item exe_ver
103
104 A string the uniquely identifies the contents of the archive, i.e. the
105 version. It has the same restrictions as the C<exe_id>, and should be
106 fixed-length, as Urlader assumes lexicographically higher versions are
107 newer, and thus preferable.
108
109 =item pack file (archive)
110
111 This contains the C<exe_id>, the C<exe_ver>, a number of environment
112 variable assignments, the program name to execute, the initial arguments
113 it receives, and finally, a list of files (with contents :) and
114 directories.
115
116 =item override
117
118 When the urlader starts, it first finds out what C<exe_id> is
119 embedded in it. It then looks for an override file for this id
120 (F<$URLADER_EXE_DIR/override>) and verifies that it is for the same
121 C<exe_id>, and the version is newer. If this is the case, then it will
122 unpack and run the override file instead of unpacking the files attched to
123 itself.
124
125 This way one can implement software upgrades - download a new executable,
126 write it safely to disk and move it to the override path.
127
128 =back
129
130 =head1 ENVIRONMENT VARIABLES
131
132 The urlader sets and maintains the following environment variables, in
133 addition to any variables specified on the commandline. The values in
134 parentheses are typical (but not gauranteed) values for unix - on windows,
135 F<~/.urlader> is replaced by F<%AppData%/urlader>.
136
137 =over 4
138
139 =item URLADER_VERSION (C<1.0>)
140
141 Set to the version of the urlader binary itself. All versions with the
142 same major number should be compatible to older versions with the same
143 major number.
144
145 =item URLADER_DATADIR (F<~/.urlader>)
146
147 The data directory used to store whatever urlader needs to store.
148
149 =item URLADER_CURRDIR
150
151 This is set to the full path of the current working directory where
152 the urlader was started. Atfer unpacking, the urlader changes to the
153 C<URLADER_EXECDIR>, so any relative paths should be resolved via this
154 path.
155
156 =item URLADER_EXEPATH
157
158 This is set to the path of the urlader executable itself, usually relative
159 to F<$URLADER_CURRDIR>.
160
161 =item URLADER_EXE_ID
162
163 This stores the executable id of the pack file attached to the urlader.
164
165 =item URLADER_EXE_VER
166
167 This is the executable version of the pack file attached to the urlader,
168 or the override, whichever was newer. Or in other words, this is the
169 version of the application running at the moment.
170
171 =item URLADER_EXE_DIR (F<~/.urlader/$URLADER_EXE_ID>>
172
173 The directory where urlader stores files related to the executable with
174 the given id.
175
176 =item URLADER_EXECDIR (F<~/.urlader/$URLADER_EXE_ID/i-$URLADER_EXE_VER>)
177
178 The directory where the files from the pack file are unpacked and the
179 program is being run. Also the working directory of the program when it is
180 run.
181
182 =item URLADER_OVERRIDE (empty or F<override>)
183
184 The override file used, if any, relative to F<$URLADER_EXECDIR>. This is
185 either missing, when no override was used, or the string F<override>, as
186 thta is currently the only override file urlader is looking for.
187
188 =back
189
190 =head1 FUNCTIONS AND VARIABLES IN THIS MODULE
191
192 =over 4
193
194 =cut
195
196 package Urlader;
197
198 use common::sense;
199
200 BEGIN {
201 our $VERSION = '1.01';
202
203 use XSLoader;
204 XSLoader::load __PACKAGE__, $VERSION;
205 }
206
207 =item $Urlader::URLADER_VERSION
208
209 Set to the urlader version (C<URLADER_VERSION>) when the program is
210 running form within urlader, undef otherwise.
211
212 =item $Urlader::DATADIR, $Urlader::EXE_ID, $Urlader::EXE_VER, $Urlader::EXE_DIR, $Urlader::EXECDIR
213
214 Contain the same value as the environment variable of the (almost) same
215 name. You should prefer these, though, as these might even be set to
216 correct values when not running form within an urlader environment.
217
218 =cut
219
220 our $URLADER_VERSION; # only set when running under urlader
221 our $DATADIR;
222 our $EXE_ID;
223 our $EXE_VER;
224 our $EXE_DIR; # %AppData%/urlader/EXE_ID
225 our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER
226
227 sub _get_env {
228 $URLADER_VERSION = getenv "URLADER_VERSION";
229 $DATADIR = getenv "URLADER_DATADIR";
230 $EXE_ID = getenv "URLADER_EXE_ID";
231 $EXE_VER = getenv "URLADER_EXE_VER";
232 $EXE_DIR = getenv "URLADER_EXE_DIR"; # %AppData%/urlader/EXE_ID
233 $EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER
234 }
235
236 _get_env;
237
238 =item Urlader::set_exe_info $exe_id, $exe_ver
239
240 Sets up the paths and variables as if running the given executable and
241 version from within urlader.
242
243 =cut
244
245 sub set_exe_info($$) {
246 _set_datadir unless defined getenv "URLADER_DATADIR";
247 &_set_exe_info;
248 _get_env;
249 }
250
251 =item $lock = Urlader::lock $path, $exclusive, $wait
252
253 Tries to acquire a lock on the given path (which must specify a file which
254 will be created if necessary). If C<$exclusive> is true, then it tries to
255 acquire an exclusive lock, otherwise the lock will be shared. If C<$wait>
256 is true, then it will wait until the lock can be acquired, otherwise it
257 only attempts to acquire it and returns immediately if it can't.
258
259 If successful it returns a lock object - the lock will be given up when
260 the lock object is destroyed or when the process exits (even on a crash)
261 and has a good chance of working on network drives as well.
262
263 If the lock could not be acquired, C<undef> is returned.
264
265 This function is provided to assist applications that want to clean up old
266 versions, see "TIPS AND TRICKS", below.
267
268 =cut
269
270 1;
271
272 =back
273
274 =head1 TIPS AND TRICKS
275
276 =over 4
277
278 =item Gathering files
279
280 Gathering all the files needed for distribution can be a big
281 problem. Right now, Urlader does not assist you in this task in any way,
282 however, just like perl source stripping, it is planned to unbundle the
283 relevant technology from B<staticperl> (L<http://staticperl.plan9.de>) for
284 use with this module.
285
286 You could always use par to find all library files, unpack the bundle and
287 add F<perl>, F<libperl> and other support libraries (e.g. F<libgcc_s>).
288
289 =item Software update
290
291 Updating the software can be done by downloading a new packfile (with the
292 same C<exe_id> but a higher C<exe_ver> - this can simply be the executable
293 you create when making a release) and replacing the F<override> file in
294 the F<$URLADER_EXE_DIR>.
295
296 When looking for updates, you should include C<$URLADER_VERSION>,
297 C<$URLADER_EXE_ID> and C<$URLADER_EXE_VER> - the first two must be
298 identical for update and currently running program, while the last one
299 should be lexicographically higher.
300
301 Replacing the override file can be done like this:
302
303 rename "new-override.tmp", "$Urlader::EXE_DIR/override"
304 or die "could not replace override";
305
306 This can fail on windows when another urlader currently reads it, but
307 should work on all platforms even when other urlader programs execute
308 concurrently.
309
310 =item Cleaning up old directories
311
312 Urlader only packs executables once and then caches them in the
313 F<$URLADER_EXECDIR>. After upgrades there will be old versions in there
314 that are not being used anymore. Or are they?
315
316 Each instance directory (F<i-*>) in the F<$URLADER_EXE_DIR>) has an
317 associated lock file (F<i-*.lck>) - while urlader executes an app it keeps
318 a shared lock on this file.
319
320 To detect whether a version is in use or not, you must try to acquire an
321 exclusive lock, i.e.:
322
323 my $lock = Urlader::lock "~/.urlader/myexe/i-ver01.lck", 1, 0;
324 if (!$lock) {
325 # instance dir is not in use and can be safely deleted
326 }
327
328 If an older urlader wants to use an instance that was deleted or is
329 currently being deleted it will wait until it's gone and simply recreate
330 it, so while less efficient, deleting instance directories should always
331 be safe.
332
333 The lockfile itself can be deleted as long as you have an exclusive lock
334 on it (if your platform allows this).
335
336 =item A real world project
337
338 The only real world project using this that I know of at the moment is the
339 deliantra client (http://www.deliantra.net for more info).
340
341 It uses some scary scripts to build the client and some dependnet modules
342 (F<build.*>), to gather perl source files into a distribution tree, shared
343 objects and system shared libraries (some of which have to be patched or,
344 due to the horrible dll hell on OS X, even renamed), called C<gatherer>,
345 and a script called C<gendist> to build executable distributions.
346
347 These can be found at
348 L<http://cvs.schmorp.de/deliantra/Deliantra-Client/util/>, but looking at
349 them can lead to premature blindless.
350
351 =item Shared Libraries
352
353 It is often desirable to package shared libraries - for example the
354 Deliantra client packages SD>, Berkely DB, Pango and amny other libraries
355 that are unlikely to be available on the target system.
356
357 This usually requires some fiddling (see below), and additionally some
358 environment variables to be set.
359
360 For example, on ELF systems you usually want F<LD_LIBRARY_PATH=.> and on
361 OS X, you want F<DYLD_LIBRARY_PATH=.> (these are effectively the default
362 on windows).
363
364 These can most easily be specified when building the packfile:
365
366 urlader-util ... LD_LIBRARY_PATH=. ./perl run
367
368 =item Portability: RPATH
369
370 Often F<perl> is linked against a shared F<libperl.so> - and might be so
371 using an rpath. Perl extensikns likewise might use an rpath, which means
372 the binary will mostly ignore LD_LIBRARY_PATH, which leads to trouble.
373
374 There is an utility called F<chrpath>, whose F<-d> option can remove the
375 rpath from binaries, shared library and shared objects.
376
377 =item Portability: OS X DLL HELL
378
379 OS X has the most severe form of DLL hell I have seen - if you link
380 against system libraries, which is practically unavoidable, you get
381 libraries of well-known names (e.g. libjpeg) that have nothing to do with
382 what you normally expect libjpeg to be, and there is no way to get your
383 version of libjpeg into your program.
384
385 Moreover, even if apple ships well-known libraries (e.g. libiconv), they
386 often ship patched versions which have a different ABI or even API then
387 the real releases.
388
389 The only way aorund this I found was to change all library names
390 in my releases (libjpeg.dylib becomes libdeliantra-jpeg.dylin and
391 so on), by patching the paths in the share dlibraries and shared
392 objects. F<install-name-tool> (with F<-id> and F<-change>) works in many
393 cases, but often paths are embedded indirectly, so you might have to use a
394 I<dirty> string replacement.
395
396 =back
397
398 =head1 SECURITY CONSIDERATIONS
399
400 The urlader executable itself does not support setuig/setgid operation, or
401 running with elevated privileges - it does no input sanitisation, and is
402 trivially exploitable.
403
404 =head1 SEE ALSO
405
406 F<< L<urlader-util> >>(1), used to pack urlader payloads.
407
408 =head1 AUTHOR
409
410 Marc Lehmann <schmorp@schmorp.de>
411 http://home.schmorp.de/
412
413 =cut
414