1 |
root |
1.1 |
=head1 NAME |
2 |
|
|
|
3 |
|
|
Urlader - installer-less single-file independent executables |
4 |
|
|
|
5 |
|
|
=head1 SYNOPSIS |
6 |
|
|
|
7 |
|
|
use Urlader; |
8 |
|
|
|
9 |
|
|
=head1 DESCRIPTION |
10 |
|
|
|
11 |
root |
1.4 |
Urlader (that's german for "bootloader" btw.) was created out of |
12 |
root |
1.5 |
frustration over PAR always being horribly slow, again not working, again |
13 |
|
|
not being flexible enough for simple things such as software upgrades, and |
14 |
|
|
again causing mysterious missing file issues on various platforms. |
15 |
root |
1.4 |
|
16 |
|
|
That doesn't mean this module replaces PAR, in fact, you should stay with |
17 |
root |
1.5 |
PAR for many reasons right now, user-friendlyness is one of them. |
18 |
root |
1.4 |
|
19 |
|
|
However, if you want to make single-file distributions out of your perl |
20 |
root |
1.5 |
programs (or python, or C or whatever), and you are prepared to fiddle |
21 |
|
|
a LOT, this module might provide a faster and more versatile deployment |
22 |
|
|
technique then PAR. Well, if it ever gets finished. |
23 |
root |
1.4 |
|
24 |
|
|
Also, I<nothing in this module is considered very stable yet>, and it's |
25 |
|
|
far from feature-complete. |
26 |
|
|
|
27 |
|
|
Having said all that, Urlader basically provides three services: |
28 |
|
|
|
29 |
|
|
=over 4 |
30 |
|
|
|
31 |
|
|
=item A simple archiver that packs a directory tree into a single file. |
32 |
|
|
|
33 |
|
|
=item A small C program that works on windows and unix, which unpacks an attached |
34 |
root |
1.5 |
archive and runs a program (perl, python, whatever...). |
35 |
root |
1.4 |
|
36 |
|
|
=item A perl module support module (I<this one>), that can be used to query |
37 |
|
|
the runtime environment, find out where to install updates and so on. |
38 |
|
|
|
39 |
|
|
=back |
40 |
|
|
|
41 |
|
|
=head1 EXAMPLE |
42 |
|
|
|
43 |
|
|
How can it be used to provide single-file executables? |
44 |
|
|
|
45 |
|
|
So simple, create a directory with everything that's needed, e.g.: |
46 |
|
|
|
47 |
|
|
# find bintree |
48 |
|
|
bintree/perl |
49 |
|
|
bintree/libperl.so.5.10 |
50 |
|
|
bintree/run |
51 |
|
|
bintree/pm/Guard.pm |
52 |
|
|
bintree/pm/auto/Guard/Guard.so |
53 |
root |
1.5 |
bintree/pm/XSLoader.pm |
54 |
|
|
bintree/pm/DynaLoader.pm |
55 |
|
|
bintree/pm/Config.pm |
56 |
|
|
bintree/pm/strict.pm |
57 |
|
|
bintree/pm/vars.pm |
58 |
|
|
bintree/pm/warnings.pm |
59 |
root |
1.4 |
|
60 |
|
|
# cat bintree/run |
61 |
|
|
@INC = ("pm", "."); # "." works around buggy AutoLoader |
62 |
|
|
use Guard; |
63 |
|
|
guard { warn "hello, world!\n" }; # just to show off |
64 |
|
|
exit 0; # tell the urlader that everything was fine |
65 |
|
|
|
66 |
|
|
Then pack it: |
67 |
|
|
|
68 |
|
|
# wget http://urlader.schmorp.de/prebuilt/1.0/linux-x86 |
69 |
|
|
# urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \ |
70 |
|
|
LD_LIBRARY_PATH=. ./perl run \ |
71 |
|
|
>myprog |
72 |
|
|
# chmod 755 myprog |
73 |
|
|
|
74 |
root |
1.5 |
The packing step takes a binary loader and appends all the files in the |
75 |
|
|
directory tree, plus some meta information. |
76 |
|
|
|
77 |
|
|
The resulting file is an executable that, when run, will unpack all the |
78 |
|
|
files and run the embedded program. |
79 |
|
|
|
80 |
root |
1.4 |
=head1 CONCEPTS |
81 |
|
|
|
82 |
|
|
=over 4 |
83 |
|
|
|
84 |
|
|
=item urlader |
85 |
|
|
|
86 |
|
|
A small (hopefully) and relatively portable (hopefully) binary that is |
87 |
|
|
prepended to a pack file to make it executable. |
88 |
|
|
|
89 |
|
|
You can build it yourself from sources (see F<prebuilt/Makefile> in the |
90 |
|
|
distribution) or use one of the precompiled ones at: |
91 |
|
|
|
92 |
|
|
http://urlader.schmorp.de/prebuilt/1.0/ |
93 |
|
|
|
94 |
|
|
The F<README> there has further information on the binaries provided. |
95 |
|
|
|
96 |
|
|
=item exe_id |
97 |
|
|
|
98 |
|
|
A string that uniquely identifies your program - all branches of it. It |
99 |
|
|
must consist of the characters C<A-Za-z0-9_-> only and should be a valid |
100 |
|
|
directory name on all systems you want to deploy on. |
101 |
|
|
|
102 |
|
|
=item exe_ver |
103 |
|
|
|
104 |
|
|
A string the uniquely identifies the contents of the archive, i.e. the |
105 |
|
|
version. It has the same restrictions as the C<exe_id>, and should be |
106 |
|
|
fixed-length, as Urlader assumes lexicographically higher versions are |
107 |
|
|
newer, and thus preferable. |
108 |
|
|
|
109 |
|
|
=item pack file (archive) |
110 |
|
|
|
111 |
|
|
This contains the C<exe_id>, the C<exe_ver>, a number of environment |
112 |
|
|
variable assignments, the program name to execute, the initial arguments |
113 |
|
|
it receives, and finally, a list of files (with contents :) and |
114 |
|
|
directories. |
115 |
|
|
|
116 |
|
|
=item override |
117 |
|
|
|
118 |
root |
1.5 |
When the urlader starts, it first finds out what C<exe_id> is |
119 |
|
|
embedded in it. It then looks for an override file for this id |
120 |
|
|
(F<$URLADER_EXE_DIR/override>) and verifies that it is for the same |
121 |
|
|
C<exe_id>, and the version is newer. If this is the case, then it will |
122 |
|
|
unpack and run the override file instead of unpacking the files attched to |
123 |
|
|
itself. |
124 |
|
|
|
125 |
|
|
This way one can implement software upgrades - download a new executable, |
126 |
|
|
write it safely to disk and move it to the override path. |
127 |
|
|
|
128 |
|
|
=back |
129 |
|
|
|
130 |
|
|
=head1 ENVIRONMENT VARIABLES |
131 |
|
|
|
132 |
|
|
The urlader sets and maintains the following environment variables, in |
133 |
|
|
addition to any variables specified on the commandline. The values in |
134 |
|
|
parentheses are typical (but not gauranteed) values for unix - on windows, |
135 |
|
|
F<~/.urlader> is replaced by F<%AppData%/urlader>. |
136 |
|
|
|
137 |
|
|
=over 4 |
138 |
|
|
|
139 |
|
|
=item URLADER_VERSION (C<1.0>) |
140 |
|
|
|
141 |
|
|
Set to the version of the urlader binary itself. All versions with the |
142 |
|
|
same major number should be compatible to older versions with the same |
143 |
|
|
major number. |
144 |
|
|
|
145 |
|
|
=item URLADER_DATADIR (F<~/.urlader>) |
146 |
|
|
|
147 |
|
|
The data directory used to store whatever urlader needs to store. |
148 |
|
|
|
149 |
|
|
=item URLADER_CURRDIR |
150 |
|
|
|
151 |
|
|
This is set to the full path of the current working directory where |
152 |
|
|
the urlader was started. Atfer unpacking, the urlader changes to the |
153 |
|
|
C<URLADER_EXECDIR>, so any relative paths should be resolved via this |
154 |
|
|
path. |
155 |
|
|
|
156 |
|
|
=item URLADER_EXEPATH |
157 |
|
|
|
158 |
|
|
This is set to the path of the urlader executable itself, usually relative |
159 |
|
|
to F<$URLADER_CURRDIR>. |
160 |
|
|
|
161 |
|
|
=item URLADER_EXE_ID |
162 |
|
|
|
163 |
|
|
This stores the executable id of the pack file attached to the urlader. |
164 |
|
|
|
165 |
|
|
=item URLADER_EXE_VER |
166 |
|
|
|
167 |
|
|
This is the executable version of the pack file attached to the urlader, |
168 |
|
|
or the override, whichever was newer. Or in other words, this is the |
169 |
|
|
version of the application running at the moment. |
170 |
|
|
|
171 |
|
|
=item URLADER_EXE_DIR (F<~/.urlader/$URLADER_EXE_ID>> |
172 |
|
|
|
173 |
|
|
The directory where urlader stores files related to the executable with |
174 |
|
|
the given id. |
175 |
|
|
|
176 |
|
|
=item URLADER_EXECDIR (F<~/.urlader/$URLADER_EXE_ID/i-$URLADER_EXE_VER>) |
177 |
|
|
|
178 |
|
|
The directory where the files from the pack file are unpacked and the |
179 |
|
|
program is being run. Also the working directory of the program when it is |
180 |
|
|
run. |
181 |
|
|
|
182 |
|
|
=item URLADER_OVERRIDE (empty or F<override>) |
183 |
|
|
|
184 |
|
|
The override file used, if any, relative to F<$URLADER_EXECDIR>. This is |
185 |
|
|
either missing, when no override was used, or the string F<override>, as |
186 |
|
|
thta is currently the only override file urlader is looking for. |
187 |
|
|
|
188 |
root |
1.4 |
=back |
189 |
|
|
|
190 |
|
|
=head1 FUNCTIONS AND VARIABLES IN THIS MODULE |
191 |
|
|
|
192 |
root |
1.1 |
=over 4 |
193 |
|
|
|
194 |
|
|
=cut |
195 |
|
|
|
196 |
|
|
package Urlader; |
197 |
|
|
|
198 |
|
|
use common::sense; |
199 |
|
|
|
200 |
|
|
BEGIN { |
201 |
root |
1.11 |
our $VERSION = '1.01'; |
202 |
root |
1.1 |
|
203 |
|
|
use XSLoader; |
204 |
|
|
XSLoader::load __PACKAGE__, $VERSION; |
205 |
|
|
} |
206 |
|
|
|
207 |
root |
1.5 |
=item $Urlader::URLADER_VERSION |
208 |
|
|
|
209 |
|
|
Set to the urlader version (C<URLADER_VERSION>) when the program is |
210 |
|
|
running form within urlader, undef otherwise. |
211 |
|
|
|
212 |
|
|
=item $Urlader::DATADIR, $Urlader::EXE_ID, $Urlader::EXE_VER, $Urlader::EXE_DIR, $Urlader::EXECDIR |
213 |
|
|
|
214 |
|
|
Contain the same value as the environment variable of the (almost) same |
215 |
|
|
name. You should prefer these, though, as these might even be set to |
216 |
|
|
correct values when not running form within an urlader environment. |
217 |
|
|
|
218 |
|
|
=cut |
219 |
|
|
|
220 |
|
|
our $URLADER_VERSION; # only set when running under urlader |
221 |
root |
1.1 |
our $DATADIR; |
222 |
|
|
our $EXE_ID; |
223 |
|
|
our $EXE_VER; |
224 |
|
|
our $EXE_DIR; # %AppData%/urlader/EXE_ID |
225 |
|
|
our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER |
226 |
|
|
|
227 |
|
|
sub _get_env { |
228 |
root |
1.3 |
$URLADER_VERSION = getenv "URLADER_VERSION"; |
229 |
|
|
$DATADIR = getenv "URLADER_DATADIR"; |
230 |
|
|
$EXE_ID = getenv "URLADER_EXE_ID"; |
231 |
|
|
$EXE_VER = getenv "URLADER_EXE_VER"; |
232 |
|
|
$EXE_DIR = getenv "URLADER_EXE_DIR"; # %AppData%/urlader/EXE_ID |
233 |
|
|
$EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER |
234 |
root |
1.1 |
} |
235 |
|
|
|
236 |
|
|
_get_env; |
237 |
|
|
|
238 |
root |
1.5 |
=item Urlader::set_exe_info $exe_id, $exe_ver |
239 |
|
|
|
240 |
|
|
Sets up the paths and variables as if running the given executable and |
241 |
|
|
version from within urlader. |
242 |
|
|
|
243 |
|
|
=cut |
244 |
|
|
|
245 |
root |
1.1 |
sub set_exe_info($$) { |
246 |
root |
1.2 |
_set_datadir unless defined getenv "URLADER_DATADIR"; |
247 |
root |
1.1 |
&_set_exe_info; |
248 |
|
|
_get_env; |
249 |
|
|
} |
250 |
|
|
|
251 |
root |
1.5 |
=item $lock = Urlader::lock $path, $exclusive, $wait |
252 |
|
|
|
253 |
|
|
Tries to acquire a lock on the given path (which must specify a file which |
254 |
|
|
will be created if necessary). If C<$exclusive> is true, then it tries to |
255 |
|
|
acquire an exclusive lock, otherwise the lock will be shared. If C<$wait> |
256 |
|
|
is true, then it will wait until the lock can be acquired, otherwise it |
257 |
|
|
only attempts to acquire it and returns immediately if it can't. |
258 |
|
|
|
259 |
root |
1.9 |
If successful it returns a lock object - the lock will be given up when |
260 |
|
|
the lock object is destroyed or when the process exits (even on a crash) |
261 |
|
|
and has a good chance of working on network drives as well. |
262 |
root |
1.5 |
|
263 |
|
|
If the lock could not be acquired, C<undef> is returned. |
264 |
|
|
|
265 |
root |
1.9 |
This function is provided to assist applications that want to clean up old |
266 |
|
|
versions, see "TIPS AND TRICKS", below. |
267 |
root |
1.5 |
|
268 |
|
|
=cut |
269 |
root |
1.1 |
|
270 |
|
|
1; |
271 |
|
|
|
272 |
|
|
=back |
273 |
|
|
|
274 |
root |
1.5 |
=head1 TIPS AND TRICKS |
275 |
|
|
|
276 |
|
|
=over 4 |
277 |
|
|
|
278 |
|
|
=item Gathering files |
279 |
|
|
|
280 |
root |
1.10 |
Gathering all the files needed for distribution can be a big |
281 |
root |
1.5 |
problem. Right now, Urlader does not assist you in this task in any way, |
282 |
|
|
however, just like perl source stripping, it is planned to unbundle the |
283 |
|
|
relevant technology from B<staticperl> (L<http://staticperl.plan9.de>) for |
284 |
|
|
use with this module. |
285 |
|
|
|
286 |
|
|
You could always use par to find all library files, unpack the bundle and |
287 |
|
|
add F<perl>, F<libperl> and other support libraries (e.g. F<libgcc_s>). |
288 |
|
|
|
289 |
|
|
=item Software update |
290 |
|
|
|
291 |
|
|
Updating the software can be done by downloading a new packfile (with the |
292 |
|
|
same C<exe_id> but a higher C<exe_ver> - this can simply be the executable |
293 |
|
|
you create when making a release) and replacing the F<override> file in |
294 |
|
|
the F<$URLADER_EXE_DIR>. |
295 |
|
|
|
296 |
|
|
When looking for updates, you should include C<$URLADER_VERSION>, |
297 |
|
|
C<$URLADER_EXE_ID> and C<$URLADER_EXE_VER> - the first two must be |
298 |
|
|
identical for update and currently running program, while the last one |
299 |
|
|
should be lexicographically higher. |
300 |
|
|
|
301 |
|
|
Replacing the override file can be done like this: |
302 |
|
|
|
303 |
|
|
rename "new-override.tmp", "$Urlader::EXE_DIR/override" |
304 |
|
|
or die "could not replace override"; |
305 |
|
|
|
306 |
|
|
This can fail on windows when another urlader currently reads it, but |
307 |
|
|
should work on all platforms even when other urlader programs execute |
308 |
|
|
concurrently. |
309 |
|
|
|
310 |
|
|
=item Cleaning up old directories |
311 |
|
|
|
312 |
|
|
Urlader only packs executables once and then caches them in the |
313 |
|
|
F<$URLADER_EXECDIR>. After upgrades there will be old versions in there |
314 |
|
|
that are not being used anymore. Or are they? |
315 |
|
|
|
316 |
|
|
Each instance directory (F<i-*>) in the F<$URLADER_EXE_DIR>) has an |
317 |
|
|
associated lock file (F<i-*.lck>) - while urlader executes an app it keeps |
318 |
|
|
a shared lock on this file. |
319 |
|
|
|
320 |
|
|
To detect whether a version is in use or not, you must try to acquire an |
321 |
|
|
exclusive lock, i.e.: |
322 |
|
|
|
323 |
|
|
my $lock = Urlader::lock "~/.urlader/myexe/i-ver01.lck", 1, 0; |
324 |
|
|
if (!$lock) { |
325 |
|
|
# instance dir is not in use and can be safely deleted |
326 |
|
|
} |
327 |
|
|
|
328 |
|
|
If an older urlader wants to use an instance that was deleted or is |
329 |
|
|
currently being deleted it will wait until it's gone and simply recreate |
330 |
|
|
it, so while less efficient, deleting instance directories should always |
331 |
|
|
be safe. |
332 |
|
|
|
333 |
|
|
The lockfile itself can be deleted as long as you have an exclusive lock |
334 |
|
|
on it (if your platform allows this). |
335 |
|
|
|
336 |
|
|
=item A real world project |
337 |
|
|
|
338 |
|
|
The only real world project using this that I know of at the moment is the |
339 |
|
|
deliantra client (http://www.deliantra.net for more info). |
340 |
|
|
|
341 |
|
|
It uses some scary scripts to build the client and some dependnet modules |
342 |
|
|
(F<build.*>), to gather perl source files into a distribution tree, shared |
343 |
|
|
objects and system shared libraries (some of which have to be patched or, |
344 |
|
|
due to the horrible dll hell on OS X, even renamed), called C<gatherer>, |
345 |
|
|
and a script called C<gendist> to build executable distributions. |
346 |
|
|
|
347 |
|
|
These can be found at |
348 |
|
|
L<http://cvs.schmorp.de/deliantra/Deliantra-Client/util/>, but looking at |
349 |
|
|
them can lead to premature blindless. |
350 |
|
|
|
351 |
root |
1.6 |
=item Shared Libraries |
352 |
|
|
|
353 |
|
|
It is often desirable to package shared libraries - for example the |
354 |
|
|
Deliantra client packages SD>, Berkely DB, Pango and amny other libraries |
355 |
|
|
that are unlikely to be available on the target system. |
356 |
|
|
|
357 |
|
|
This usually requires some fiddling (see below), and additionally some |
358 |
|
|
environment variables to be set. |
359 |
|
|
|
360 |
|
|
For example, on ELF systems you usually want F<LD_LIBRARY_PATH=.> and on |
361 |
|
|
OS X, you want F<DYLD_LIBRARY_PATH=.> (these are effectively the default |
362 |
|
|
on windows). |
363 |
|
|
|
364 |
|
|
These can most easily be specified when building the packfile: |
365 |
|
|
|
366 |
|
|
urlader-util ... LD_LIBRARY_PATH=. ./perl run |
367 |
|
|
|
368 |
|
|
=item Portability: RPATH |
369 |
|
|
|
370 |
|
|
Often F<perl> is linked against a shared F<libperl.so> - and might be so |
371 |
|
|
using an rpath. Perl extensikns likewise might use an rpath, which means |
372 |
|
|
the binary will mostly ignore LD_LIBRARY_PATH, which leads to trouble. |
373 |
|
|
|
374 |
|
|
There is an utility called F<chrpath>, whose F<-d> option can remove the |
375 |
|
|
rpath from binaries, shared library and shared objects. |
376 |
|
|
|
377 |
|
|
=item Portability: OS X DLL HELL |
378 |
|
|
|
379 |
|
|
OS X has the most severe form of DLL hell I have seen - if you link |
380 |
|
|
against system libraries, which is practically unavoidable, you get |
381 |
|
|
libraries of well-known names (e.g. libjpeg) that have nothing to do with |
382 |
|
|
what you normally expect libjpeg to be, and there is no way to get your |
383 |
|
|
version of libjpeg into your program. |
384 |
|
|
|
385 |
|
|
Moreover, even if apple ships well-known libraries (e.g. libiconv), they |
386 |
|
|
often ship patched versions which have a different ABI or even API then |
387 |
|
|
the real releases. |
388 |
|
|
|
389 |
|
|
The only way aorund this I found was to change all library names |
390 |
|
|
in my releases (libjpeg.dylib becomes libdeliantra-jpeg.dylin and |
391 |
|
|
so on), by patching the paths in the share dlibraries and shared |
392 |
|
|
objects. F<install-name-tool> (with F<-id> and F<-change>) works in many |
393 |
|
|
cases, but often paths are embedded indirectly, so you might have to use a |
394 |
|
|
I<dirty> string replacement. |
395 |
|
|
|
396 |
root |
1.5 |
=back |
397 |
|
|
|
398 |
root |
1.7 |
=head1 SECURITY CONSIDERATIONS |
399 |
|
|
|
400 |
|
|
The urlader executable itself does not support setuig/setgid operation, or |
401 |
|
|
running with elevated privileges - it does no input sanitisation, and is |
402 |
|
|
trivially exploitable. |
403 |
|
|
|
404 |
root |
1.1 |
=head1 AUTHOR |
405 |
|
|
|
406 |
|
|
Marc Lehmann <schmorp@schmorp.de> |
407 |
|
|
http://home.schmorp.de/ |
408 |
|
|
|
409 |
|
|
=cut |
410 |
|
|
|