1 |
=head1 NAME |
2 |
|
3 |
Urlader - installer-less single-file independent executables |
4 |
|
5 |
=head1 SYNOPSIS |
6 |
|
7 |
use Urlader; |
8 |
|
9 |
=head1 DESCRIPTION |
10 |
|
11 |
Urlader (that's german for "bootloader" btw.) was created out of |
12 |
frustration over PAR always being horribly slow, again not working, again |
13 |
not being flexible enough for simple things such as software upgrades, and |
14 |
again causing mysterious missing file issues on various platforms. |
15 |
|
16 |
That doesn't mean this module replaces PAR, in fact, you should stay with |
17 |
PAR for many reasons right now, user-friendlyness is one of them. |
18 |
|
19 |
However, if you want to make single-file distributions out of your perl |
20 |
programs (or python, or C or whatever), and you are prepared to fiddle |
21 |
a LOT, this module might provide a faster and more versatile deployment |
22 |
technique then PAR. Well, if it ever gets finished. |
23 |
|
24 |
Also, I<nothing in this module is considered very stable yet>, and it's |
25 |
far from feature-complete. |
26 |
|
27 |
Having said all that, Urlader basically provides three services: |
28 |
|
29 |
=over 4 |
30 |
|
31 |
=item A simple archiver that packs a directory tree into a single file. |
32 |
|
33 |
=item A small C program that works on windows and unix, which unpacks an attached |
34 |
archive and runs a program (perl, python, whatever...). |
35 |
|
36 |
=item A perl module support module (I<this one>), that can be used to query |
37 |
the runtime environment, find out where to install updates and so on. |
38 |
|
39 |
=back |
40 |
|
41 |
=head1 EXAMPLE |
42 |
|
43 |
How can it be used to provide single-file executables? |
44 |
|
45 |
So simple, create a directory with everything that's needed, e.g.: |
46 |
|
47 |
# find bintree |
48 |
bintree/perl |
49 |
bintree/libperl.so.5.10 |
50 |
bintree/run |
51 |
bintree/pm/Guard.pm |
52 |
bintree/pm/auto/Guard/Guard.so |
53 |
bintree/pm/XSLoader.pm |
54 |
bintree/pm/DynaLoader.pm |
55 |
bintree/pm/Config.pm |
56 |
bintree/pm/strict.pm |
57 |
bintree/pm/vars.pm |
58 |
bintree/pm/warnings.pm |
59 |
|
60 |
# cat bintree/run |
61 |
@INC = ("pm", "."); # "." works around buggy AutoLoader |
62 |
use Guard; |
63 |
guard { warn "hello, world!\n" }; # just to show off |
64 |
exit 0; # tell the urlader that everything was fine |
65 |
|
66 |
Then pack it: |
67 |
|
68 |
# wget http://urlader.schmorp.de/prebuilt/1.0/linux-x86 |
69 |
# urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \ |
70 |
LD_LIBRARY_PATH=. ./perl run \ |
71 |
>myprog |
72 |
# chmod 755 myprog |
73 |
|
74 |
The packing step takes a binary loader and appends all the files in the |
75 |
directory tree, plus some meta information. |
76 |
|
77 |
The resulting file is an executable that, when run, will unpack all the |
78 |
files and run the embedded program. |
79 |
|
80 |
=head1 CONCEPTS |
81 |
|
82 |
=over 4 |
83 |
|
84 |
=item urlader |
85 |
|
86 |
A small (hopefully) and relatively portable (hopefully) binary that is |
87 |
prepended to a pack file to make it executable. |
88 |
|
89 |
You can build it yourself from sources (see F<prebuilt/Makefile> in the |
90 |
distribution) or use one of the precompiled ones at: |
91 |
|
92 |
http://urlader.schmorp.de/prebuilt/1.0/ |
93 |
|
94 |
The F<README> there has further information on the binaries provided. |
95 |
|
96 |
=item exe_id |
97 |
|
98 |
A string that uniquely identifies your program - all branches of it. It |
99 |
must consist of the characters C<A-Za-z0-9_-> only and should be a valid |
100 |
directory name on all systems you want to deploy on. |
101 |
|
102 |
=item exe_ver |
103 |
|
104 |
A string the uniquely identifies the contents of the archive, i.e. the |
105 |
version. It has the same restrictions as the C<exe_id>, and should be |
106 |
fixed-length, as Urlader assumes lexicographically higher versions are |
107 |
newer, and thus preferable. |
108 |
|
109 |
=item pack file (archive) |
110 |
|
111 |
This contains the C<exe_id>, the C<exe_ver>, a number of environment |
112 |
variable assignments, the program name to execute, the initial arguments |
113 |
it receives, and finally, a list of files (with contents :) and |
114 |
directories. |
115 |
|
116 |
=item override |
117 |
|
118 |
When the urlader starts, it first finds out what C<exe_id> is |
119 |
embedded in it. It then looks for an override file for this id |
120 |
(F<$URLADER_EXE_DIR/override>) and verifies that it is for the same |
121 |
C<exe_id>, and the version is newer. If this is the case, then it will |
122 |
unpack and run the override file instead of unpacking the files attched to |
123 |
itself. |
124 |
|
125 |
This way one can implement software upgrades - download a new executable, |
126 |
write it safely to disk and move it to the override path. |
127 |
|
128 |
=back |
129 |
|
130 |
=head1 ENVIRONMENT VARIABLES |
131 |
|
132 |
The urlader sets and maintains the following environment variables, in |
133 |
addition to any variables specified on the commandline. The values in |
134 |
parentheses are typical (but not gauranteed) values for unix - on windows, |
135 |
F<~/.urlader> is replaced by F<%AppData%/urlader>. |
136 |
|
137 |
=over 4 |
138 |
|
139 |
=item URLADER_VERSION (C<1.0>) |
140 |
|
141 |
Set to the version of the urlader binary itself. All versions with the |
142 |
same major number should be compatible to older versions with the same |
143 |
major number. |
144 |
|
145 |
=item URLADER_DATADIR (F<~/.urlader>) |
146 |
|
147 |
The data directory used to store whatever urlader needs to store. |
148 |
|
149 |
=item URLADER_CURRDIR |
150 |
|
151 |
This is set to the full path of the current working directory where |
152 |
the urlader was started. Atfer unpacking, the urlader changes to the |
153 |
C<URLADER_EXECDIR>, so any relative paths should be resolved via this |
154 |
path. |
155 |
|
156 |
=item URLADER_EXEPATH |
157 |
|
158 |
This is set to the path of the urlader executable itself, usually relative |
159 |
to F<$URLADER_CURRDIR>. |
160 |
|
161 |
=item URLADER_EXE_ID |
162 |
|
163 |
This stores the executable id of the pack file attached to the urlader. |
164 |
|
165 |
=item URLADER_EXE_VER |
166 |
|
167 |
This is the executable version of the pack file attached to the urlader, |
168 |
or the override, whichever was newer. Or in other words, this is the |
169 |
version of the application running at the moment. |
170 |
|
171 |
=item URLADER_EXE_DIR (F<~/.urlader/$URLADER_EXE_ID>> |
172 |
|
173 |
The directory where urlader stores files related to the executable with |
174 |
the given id. |
175 |
|
176 |
=item URLADER_EXECDIR (F<~/.urlader/$URLADER_EXE_ID/i-$URLADER_EXE_VER>) |
177 |
|
178 |
The directory where the files from the pack file are unpacked and the |
179 |
program is being run. Also the working directory of the program when it is |
180 |
run. |
181 |
|
182 |
=item URLADER_OVERRIDE (empty or F<override>) |
183 |
|
184 |
The override file used, if any, relative to F<$URLADER_EXECDIR>. This is |
185 |
either missing, when no override was used, or the string F<override>, as |
186 |
thta is currently the only override file urlader is looking for. |
187 |
|
188 |
=back |
189 |
|
190 |
=head1 FUNCTIONS AND VARIABLES IN THIS MODULE |
191 |
|
192 |
=over 4 |
193 |
|
194 |
=cut |
195 |
|
196 |
package Urlader; |
197 |
|
198 |
use common::sense; |
199 |
|
200 |
BEGIN { |
201 |
our $VERSION = '1.0'; |
202 |
|
203 |
use XSLoader; |
204 |
XSLoader::load __PACKAGE__, $VERSION; |
205 |
} |
206 |
|
207 |
=item $Urlader::URLADER_VERSION |
208 |
|
209 |
Set to the urlader version (C<URLADER_VERSION>) when the program is |
210 |
running form within urlader, undef otherwise. |
211 |
|
212 |
=item $Urlader::DATADIR, $Urlader::EXE_ID, $Urlader::EXE_VER, $Urlader::EXE_DIR, $Urlader::EXECDIR |
213 |
|
214 |
Contain the same value as the environment variable of the (almost) same |
215 |
name. You should prefer these, though, as these might even be set to |
216 |
correct values when not running form within an urlader environment. |
217 |
|
218 |
=cut |
219 |
|
220 |
our $URLADER_VERSION; # only set when running under urlader |
221 |
our $DATADIR; |
222 |
our $EXE_ID; |
223 |
our $EXE_VER; |
224 |
our $EXE_DIR; # %AppData%/urlader/EXE_ID |
225 |
our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER |
226 |
|
227 |
sub _get_env { |
228 |
$URLADER_VERSION = getenv "URLADER_VERSION"; |
229 |
$DATADIR = getenv "URLADER_DATADIR"; |
230 |
$EXE_ID = getenv "URLADER_EXE_ID"; |
231 |
$EXE_VER = getenv "URLADER_EXE_VER"; |
232 |
$EXE_DIR = getenv "URLADER_EXE_DIR"; # %AppData%/urlader/EXE_ID |
233 |
$EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER |
234 |
} |
235 |
|
236 |
_get_env; |
237 |
|
238 |
=item Urlader::set_exe_info $exe_id, $exe_ver |
239 |
|
240 |
Sets up the paths and variables as if running the given executable and |
241 |
version from within urlader. |
242 |
|
243 |
=cut |
244 |
|
245 |
sub set_exe_info($$) { |
246 |
_set_datadir unless defined getenv "URLADER_DATADIR"; |
247 |
&_set_exe_info; |
248 |
_get_env; |
249 |
} |
250 |
|
251 |
=item $lock = Urlader::lock $path, $exclusive, $wait |
252 |
|
253 |
Tries to acquire a lock on the given path (which must specify a file which |
254 |
will be created if necessary). If C<$exclusive> is true, then it tries to |
255 |
acquire an exclusive lock, otherwise the lock will be shared. If C<$wait> |
256 |
is true, then it will wait until the lock can be acquired, otherwise it |
257 |
only attempts to acquire it and returns immediately if it can't. |
258 |
|
259 |
If successful it returns a lock object - the lock will be given up when |
260 |
the lock object is destroyed or when the process exits (even on a crash) |
261 |
and has a good chance of working on network drives as well. |
262 |
|
263 |
If the lock could not be acquired, C<undef> is returned. |
264 |
|
265 |
This function is provided to assist applications that want to clean up old |
266 |
versions, see "TIPS AND TRICKS", below. |
267 |
|
268 |
=cut |
269 |
|
270 |
1; |
271 |
|
272 |
=back |
273 |
|
274 |
=head1 TIPS AND TRICKS |
275 |
|
276 |
=over 4 |
277 |
|
278 |
=item Gathering files |
279 |
|
280 |
Gathering all the files needed for distribution cna be a big |
281 |
problem. Right now, Urlader does not assist you in this task in any way, |
282 |
however, just like perl source stripping, it is planned to unbundle the |
283 |
relevant technology from B<staticperl> (L<http://staticperl.plan9.de>) for |
284 |
use with this module. |
285 |
|
286 |
You could always use par to find all library files, unpack the bundle and |
287 |
add F<perl>, F<libperl> and other support libraries (e.g. F<libgcc_s>). |
288 |
|
289 |
=item Software update |
290 |
|
291 |
Updating the software can be done by downloading a new packfile (with the |
292 |
same C<exe_id> but a higher C<exe_ver> - this can simply be the executable |
293 |
you create when making a release) and replacing the F<override> file in |
294 |
the F<$URLADER_EXE_DIR>. |
295 |
|
296 |
When looking for updates, you should include C<$URLADER_VERSION>, |
297 |
C<$URLADER_EXE_ID> and C<$URLADER_EXE_VER> - the first two must be |
298 |
identical for update and currently running program, while the last one |
299 |
should be lexicographically higher. |
300 |
|
301 |
Replacing the override file can be done like this: |
302 |
|
303 |
rename "new-override.tmp", "$Urlader::EXE_DIR/override" |
304 |
or die "could not replace override"; |
305 |
|
306 |
This can fail on windows when another urlader currently reads it, but |
307 |
should work on all platforms even when other urlader programs execute |
308 |
concurrently. |
309 |
|
310 |
=item Cleaning up old directories |
311 |
|
312 |
Urlader only packs executables once and then caches them in the |
313 |
F<$URLADER_EXECDIR>. After upgrades there will be old versions in there |
314 |
that are not being used anymore. Or are they? |
315 |
|
316 |
Each instance directory (F<i-*>) in the F<$URLADER_EXE_DIR>) has an |
317 |
associated lock file (F<i-*.lck>) - while urlader executes an app it keeps |
318 |
a shared lock on this file. |
319 |
|
320 |
To detect whether a version is in use or not, you must try to acquire an |
321 |
exclusive lock, i.e.: |
322 |
|
323 |
my $lock = Urlader::lock "~/.urlader/myexe/i-ver01.lck", 1, 0; |
324 |
if (!$lock) { |
325 |
# instance dir is not in use and can be safely deleted |
326 |
} |
327 |
|
328 |
If an older urlader wants to use an instance that was deleted or is |
329 |
currently being deleted it will wait until it's gone and simply recreate |
330 |
it, so while less efficient, deleting instance directories should always |
331 |
be safe. |
332 |
|
333 |
The lockfile itself can be deleted as long as you have an exclusive lock |
334 |
on it (if your platform allows this). |
335 |
|
336 |
=item A real world project |
337 |
|
338 |
The only real world project using this that I know of at the moment is the |
339 |
deliantra client (http://www.deliantra.net for more info). |
340 |
|
341 |
It uses some scary scripts to build the client and some dependnet modules |
342 |
(F<build.*>), to gather perl source files into a distribution tree, shared |
343 |
objects and system shared libraries (some of which have to be patched or, |
344 |
due to the horrible dll hell on OS X, even renamed), called C<gatherer>, |
345 |
and a script called C<gendist> to build executable distributions. |
346 |
|
347 |
These can be found at |
348 |
L<http://cvs.schmorp.de/deliantra/Deliantra-Client/util/>, but looking at |
349 |
them can lead to premature blindless. |
350 |
|
351 |
=item Shared Libraries |
352 |
|
353 |
It is often desirable to package shared libraries - for example the |
354 |
Deliantra client packages SD>, Berkely DB, Pango and amny other libraries |
355 |
that are unlikely to be available on the target system. |
356 |
|
357 |
This usually requires some fiddling (see below), and additionally some |
358 |
environment variables to be set. |
359 |
|
360 |
For example, on ELF systems you usually want F<LD_LIBRARY_PATH=.> and on |
361 |
OS X, you want F<DYLD_LIBRARY_PATH=.> (these are effectively the default |
362 |
on windows). |
363 |
|
364 |
These can most easily be specified when building the packfile: |
365 |
|
366 |
urlader-util ... LD_LIBRARY_PATH=. ./perl run |
367 |
|
368 |
=item Portability: RPATH |
369 |
|
370 |
Often F<perl> is linked against a shared F<libperl.so> - and might be so |
371 |
using an rpath. Perl extensikns likewise might use an rpath, which means |
372 |
the binary will mostly ignore LD_LIBRARY_PATH, which leads to trouble. |
373 |
|
374 |
There is an utility called F<chrpath>, whose F<-d> option can remove the |
375 |
rpath from binaries, shared library and shared objects. |
376 |
|
377 |
=item Portability: OS X DLL HELL |
378 |
|
379 |
OS X has the most severe form of DLL hell I have seen - if you link |
380 |
against system libraries, which is practically unavoidable, you get |
381 |
libraries of well-known names (e.g. libjpeg) that have nothing to do with |
382 |
what you normally expect libjpeg to be, and there is no way to get your |
383 |
version of libjpeg into your program. |
384 |
|
385 |
Moreover, even if apple ships well-known libraries (e.g. libiconv), they |
386 |
often ship patched versions which have a different ABI or even API then |
387 |
the real releases. |
388 |
|
389 |
The only way aorund this I found was to change all library names |
390 |
in my releases (libjpeg.dylib becomes libdeliantra-jpeg.dylin and |
391 |
so on), by patching the paths in the share dlibraries and shared |
392 |
objects. F<install-name-tool> (with F<-id> and F<-change>) works in many |
393 |
cases, but often paths are embedded indirectly, so you might have to use a |
394 |
I<dirty> string replacement. |
395 |
|
396 |
=back |
397 |
|
398 |
=head1 SECURITY CONSIDERATIONS |
399 |
|
400 |
The urlader executable itself does not support setuig/setgid operation, or |
401 |
running with elevated privileges - it does no input sanitisation, and is |
402 |
trivially exploitable. |
403 |
|
404 |
=head1 AUTHOR |
405 |
|
406 |
Marc Lehmann <schmorp@schmorp.de> |
407 |
http://home.schmorp.de/ |
408 |
|
409 |
=cut |
410 |
|