… | |
… | |
6 | |
6 | |
7 | use Urlader; |
7 | use Urlader; |
8 | |
8 | |
9 | =head1 DESCRIPTION |
9 | =head1 DESCRIPTION |
10 | |
10 | |
|
|
11 | Urlader (that's german for "bootloader" btw.) was created out of |
|
|
12 | frustration over PAR always being horribly slow, again not working, again |
|
|
13 | not being flexible enough for simple things such as software upgrades, and |
|
|
14 | again causing mysterious missing file issues on various platforms. |
|
|
15 | |
|
|
16 | That doesn't mean this module replaces PAR, in fact, you should stay with |
|
|
17 | PAR for many reasons right now, user-friendlyness is one of them. |
|
|
18 | |
|
|
19 | However, if you want to make single-file distributions out of your perl |
|
|
20 | programs (or python, or C or whatever), and you are prepared to fiddle |
|
|
21 | a LOT, this module might provide a faster and more versatile deployment |
|
|
22 | technique then PAR. Well, if it ever gets finished. |
|
|
23 | |
|
|
24 | Also, I<nothing in this module is considered very stable yet>, and it's |
|
|
25 | far from feature-complete. |
|
|
26 | |
|
|
27 | Having said all that, Urlader basically provides three services: |
|
|
28 | |
11 | =over 4 |
29 | =over 4 |
12 | |
30 | |
|
|
31 | =item A simple archiver that packs a directory tree into a single file. |
|
|
32 | |
|
|
33 | =item A small C program that works on windows and unix, which unpacks an attached |
|
|
34 | archive and runs a program (perl, python, whatever...). |
|
|
35 | |
|
|
36 | =item A perl module support module (I<this one>), that can be used to query |
|
|
37 | the runtime environment, find out where to install updates and so on. |
|
|
38 | |
|
|
39 | =back |
|
|
40 | |
|
|
41 | =head1 EXAMPLE |
|
|
42 | |
|
|
43 | How can it be used to provide single-file executables? |
|
|
44 | |
|
|
45 | So simple, create a directory with everything that's needed, e.g.: |
|
|
46 | |
|
|
47 | # find bintree |
|
|
48 | bintree/perl |
|
|
49 | bintree/libperl.so.5.10 |
|
|
50 | bintree/run |
|
|
51 | bintree/pm/Guard.pm |
|
|
52 | bintree/pm/auto/Guard/Guard.so |
|
|
53 | bintree/pm/XSLoader.pm |
|
|
54 | bintree/pm/DynaLoader.pm |
|
|
55 | bintree/pm/Config.pm |
|
|
56 | bintree/pm/strict.pm |
|
|
57 | bintree/pm/vars.pm |
|
|
58 | bintree/pm/warnings.pm |
|
|
59 | |
|
|
60 | # cat bintree/run |
|
|
61 | @INC = ("pm", "."); # "." works around buggy AutoLoader |
|
|
62 | use Guard; |
|
|
63 | guard { warn "hello, world!\n" }; # just to show off |
|
|
64 | exit 0; # tell the urlader that everything was fine |
|
|
65 | |
|
|
66 | Then pack it: |
|
|
67 | |
|
|
68 | # wget http://urlader.schmorp.de/prebuilt/1.0/linux-x86 |
|
|
69 | # urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \ |
|
|
70 | LD_LIBRARY_PATH=. ./perl run \ |
|
|
71 | >myprog |
|
|
72 | # chmod 755 myprog |
|
|
73 | |
|
|
74 | The packing step takes a binary loader and appends all the files in the |
|
|
75 | directory tree, plus some meta information. |
|
|
76 | |
|
|
77 | The resulting file is an executable that, when run, will unpack all the |
|
|
78 | files and run the embedded program. |
|
|
79 | |
|
|
80 | =head1 CONCEPTS |
|
|
81 | |
|
|
82 | =over 4 |
|
|
83 | |
|
|
84 | =item urlader |
|
|
85 | |
|
|
86 | A small (hopefully) and relatively portable (hopefully) binary that is |
|
|
87 | prepended to a pack file to make it executable. |
|
|
88 | |
|
|
89 | You can build it yourself from sources (see F<prebuilt/Makefile> in the |
|
|
90 | distribution) or use one of the precompiled ones at: |
|
|
91 | |
|
|
92 | http://urlader.schmorp.de/prebuilt/1.0/ |
|
|
93 | |
|
|
94 | The F<README> there has further information on the binaries provided. |
|
|
95 | |
|
|
96 | =item exe_id |
|
|
97 | |
|
|
98 | A string that uniquely identifies your program - all branches of it. It |
|
|
99 | must consist of the characters C<A-Za-z0-9_-> only and should be a valid |
|
|
100 | directory name on all systems you want to deploy on. |
|
|
101 | |
|
|
102 | =item exe_ver |
|
|
103 | |
|
|
104 | A string the uniquely identifies the contents of the archive, i.e. the |
|
|
105 | version. It has the same restrictions as the C<exe_id>, and should be |
|
|
106 | fixed-length, as Urlader assumes lexicographically higher versions are |
|
|
107 | newer, and thus preferable. |
|
|
108 | |
|
|
109 | =item pack file (archive) |
|
|
110 | |
|
|
111 | This contains the C<exe_id>, the C<exe_ver>, a number of environment |
|
|
112 | variable assignments, the program name to execute, the initial arguments |
|
|
113 | it receives, and finally, a list of files (with contents :) and |
|
|
114 | directories. |
|
|
115 | |
|
|
116 | =item override |
|
|
117 | |
|
|
118 | When the urlader starts, it first finds out what C<exe_id> is |
|
|
119 | embedded in it. It then looks for an override file for this id |
|
|
120 | (F<$URLADER_EXE_DIR/override>) and verifies that it is for the same |
|
|
121 | C<exe_id>, and the version is newer. If this is the case, then it will |
|
|
122 | unpack and run the override file instead of unpacking the files attched to |
|
|
123 | itself. |
|
|
124 | |
|
|
125 | This way one can implement software upgrades - download a new executable, |
|
|
126 | write it safely to disk and move it to the override path. |
|
|
127 | |
|
|
128 | =back |
|
|
129 | |
|
|
130 | =head1 ENVIRONMENT VARIABLES |
|
|
131 | |
|
|
132 | The urlader sets and maintains the following environment variables, in |
|
|
133 | addition to any variables specified on the commandline. The values in |
|
|
134 | parentheses are typical (but not gauranteed) values for unix - on windows, |
|
|
135 | F<~/.urlader> is replaced by F<%AppData%/urlader>. |
|
|
136 | |
|
|
137 | =over 4 |
|
|
138 | |
|
|
139 | =item URLADER_VERSION (C<1.0>) |
|
|
140 | |
|
|
141 | Set to the version of the urlader binary itself. All versions with the |
|
|
142 | same major number should be compatible to older versions with the same |
|
|
143 | major number. |
|
|
144 | |
|
|
145 | =item URLADER_DATADIR (F<~/.urlader>) |
|
|
146 | |
|
|
147 | The data directory used to store whatever urlader needs to store. |
|
|
148 | |
|
|
149 | =item URLADER_CURRDIR |
|
|
150 | |
|
|
151 | This is set to the full path of the current working directory where |
|
|
152 | the urlader was started. Atfer unpacking, the urlader changes to the |
|
|
153 | C<URLADER_EXECDIR>, so any relative paths should be resolved via this |
|
|
154 | path. |
|
|
155 | |
|
|
156 | =item URLADER_EXEPATH |
|
|
157 | |
|
|
158 | This is set to the path of the urlader executable itself, usually relative |
|
|
159 | to F<$URLADER_CURRDIR>. |
|
|
160 | |
|
|
161 | =item URLADER_EXE_ID |
|
|
162 | |
|
|
163 | This stores the executable id of the pack file attached to the urlader. |
|
|
164 | |
|
|
165 | =item URLADER_EXE_VER |
|
|
166 | |
|
|
167 | This is the executable version of the pack file attached to the urlader, |
|
|
168 | or the override, whichever was newer. Or in other words, this is the |
|
|
169 | version of the application running at the moment. |
|
|
170 | |
|
|
171 | =item URLADER_EXE_DIR (F<~/.urlader/$URLADER_EXE_ID>> |
|
|
172 | |
|
|
173 | The directory where urlader stores files related to the executable with |
|
|
174 | the given id. |
|
|
175 | |
|
|
176 | =item URLADER_EXECDIR (F<~/.urlader/$URLADER_EXE_ID/i-$URLADER_EXE_VER>) |
|
|
177 | |
|
|
178 | The directory where the files from the pack file are unpacked and the |
|
|
179 | program is being run. Also the working directory of the program when it is |
|
|
180 | run. |
|
|
181 | |
|
|
182 | =item URLADER_OVERRIDE (empty or F<override>) |
|
|
183 | |
|
|
184 | The override file used, if any, relative to F<$URLADER_EXECDIR>. This is |
|
|
185 | either missing, when no override was used, or the string F<override>, as |
|
|
186 | thta is currently the only override file urlader is looking for. |
|
|
187 | |
|
|
188 | =back |
|
|
189 | |
|
|
190 | =head1 FUNCTIONS AND VARIABLES IN THIS MODULE |
|
|
191 | |
|
|
192 | =over 4 |
|
|
193 | |
13 | =cut |
194 | =cut |
14 | |
195 | |
15 | package Urlader; |
196 | package Urlader; |
16 | |
197 | |
17 | use common::sense; |
198 | use common::sense; |
18 | |
199 | |
19 | BEGIN { |
200 | BEGIN { |
20 | our $VERSION = '0.2'; |
201 | our $VERSION = '1.0'; |
21 | |
202 | |
22 | use XSLoader; |
203 | use XSLoader; |
23 | XSLoader::load __PACKAGE__, $VERSION; |
204 | XSLoader::load __PACKAGE__, $VERSION; |
24 | } |
205 | } |
25 | |
206 | |
|
|
207 | =item $Urlader::URLADER_VERSION |
|
|
208 | |
|
|
209 | Set to the urlader version (C<URLADER_VERSION>) when the program is |
|
|
210 | running form within urlader, undef otherwise. |
|
|
211 | |
|
|
212 | =item $Urlader::DATADIR, $Urlader::EXE_ID, $Urlader::EXE_VER, $Urlader::EXE_DIR, $Urlader::EXECDIR |
|
|
213 | |
|
|
214 | Contain the same value as the environment variable of the (almost) same |
|
|
215 | name. You should prefer these, though, as these might even be set to |
|
|
216 | correct values when not running form within an urlader environment. |
|
|
217 | |
|
|
218 | =cut |
|
|
219 | |
|
|
220 | our $URLADER_VERSION; # only set when running under urlader |
26 | our $DATADIR; |
221 | our $DATADIR; |
27 | our $EXE_ID; |
222 | our $EXE_ID; |
28 | our $EXE_VER; |
223 | our $EXE_VER; |
29 | our $EXE_DIR; # %AppData%/urlader/EXE_ID |
224 | our $EXE_DIR; # %AppData%/urlader/EXE_ID |
30 | our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER |
225 | our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER |
31 | |
226 | |
32 | sub _get_env { |
227 | sub _get_env { |
|
|
228 | $URLADER_VERSION = getenv "URLADER_VERSION"; |
|
|
229 | $DATADIR = getenv "URLADER_DATADIR"; |
33 | $EXE_ID = getenv "URLADER_EXE_ID"; |
230 | $EXE_ID = getenv "URLADER_EXE_ID"; |
34 | $EXE_VER = getenv "URLADER_EXE_VER"; |
231 | $EXE_VER = getenv "URLADER_EXE_VER"; |
35 | $EXE_DIR = getenv "URLADER_EXE_DIR"; # %AppData%/urlader/EXE_ID |
232 | $EXE_DIR = getenv "URLADER_EXE_DIR"; # %AppData%/urlader/EXE_ID |
36 | $EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER |
233 | $EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER |
37 | } |
234 | } |
38 | |
235 | |
39 | _set_datadir unless defined getenv "URLADER_DATADIR"; |
|
|
40 | $DATADIR = getenv "URLADER_DATADIR"; |
|
|
41 | |
|
|
42 | _get_env; |
236 | _get_env; |
43 | |
237 | |
|
|
238 | =item Urlader::set_exe_info $exe_id, $exe_ver |
|
|
239 | |
|
|
240 | Sets up the paths and variables as if running the given executable and |
|
|
241 | version from within urlader. |
|
|
242 | |
|
|
243 | =cut |
|
|
244 | |
44 | sub set_exe_info($$) { |
245 | sub set_exe_info($$) { |
|
|
246 | _set_datadir unless defined getenv "URLADER_DATADIR"; |
45 | &_set_exe_info; |
247 | &_set_exe_info; |
46 | _get_env; |
248 | _get_env; |
47 | } |
249 | } |
48 | |
250 | |
|
|
251 | =item $lock = Urlader::lock $path, $exclusive, $wait |
|
|
252 | |
|
|
253 | Tries to acquire a lock on the given path (which must specify a file which |
|
|
254 | will be created if necessary). If C<$exclusive> is true, then it tries to |
|
|
255 | acquire an exclusive lock, otherwise the lock will be shared. If C<$wait> |
|
|
256 | is true, then it will wait until the lock can be acquired, otherwise it |
|
|
257 | only attempts to acquire it and returns immediately if it can't. |
|
|
258 | |
|
|
259 | If successful it returns a lock object - the lock will be given up |
|
|
260 | when the lock object is destroyed. |
|
|
261 | |
|
|
262 | If the lock could not be acquired, C<undef> is returned. |
|
|
263 | |
|
|
264 | This function will probably go awway in the future, but is provided to |
|
|
265 | assist applications that want to clean up old versions, see "TIPS AND |
|
|
266 | TRICKS", below. |
|
|
267 | |
|
|
268 | =cut |
49 | |
269 | |
50 | 1; |
270 | 1; |
51 | |
271 | |
52 | =back |
272 | =back |
|
|
273 | |
|
|
274 | =head1 TIPS AND TRICKS |
|
|
275 | |
|
|
276 | =over 4 |
|
|
277 | |
|
|
278 | =item Gathering files |
|
|
279 | |
|
|
280 | Gathering all the files needed for distribution cna be a big |
|
|
281 | problem. Right now, Urlader does not assist you in this task in any way, |
|
|
282 | however, just like perl source stripping, it is planned to unbundle the |
|
|
283 | relevant technology from B<staticperl> (L<http://staticperl.plan9.de>) for |
|
|
284 | use with this module. |
|
|
285 | |
|
|
286 | You could always use par to find all library files, unpack the bundle and |
|
|
287 | add F<perl>, F<libperl> and other support libraries (e.g. F<libgcc_s>). |
|
|
288 | |
|
|
289 | =item Software update |
|
|
290 | |
|
|
291 | Updating the software can be done by downloading a new packfile (with the |
|
|
292 | same C<exe_id> but a higher C<exe_ver> - this can simply be the executable |
|
|
293 | you create when making a release) and replacing the F<override> file in |
|
|
294 | the F<$URLADER_EXE_DIR>. |
|
|
295 | |
|
|
296 | When looking for updates, you should include C<$URLADER_VERSION>, |
|
|
297 | C<$URLADER_EXE_ID> and C<$URLADER_EXE_VER> - the first two must be |
|
|
298 | identical for update and currently running program, while the last one |
|
|
299 | should be lexicographically higher. |
|
|
300 | |
|
|
301 | Replacing the override file can be done like this: |
|
|
302 | |
|
|
303 | rename "new-override.tmp", "$Urlader::EXE_DIR/override" |
|
|
304 | or die "could not replace override"; |
|
|
305 | |
|
|
306 | This can fail on windows when another urlader currently reads it, but |
|
|
307 | should work on all platforms even when other urlader programs execute |
|
|
308 | concurrently. |
|
|
309 | |
|
|
310 | =item Cleaning up old directories |
|
|
311 | |
|
|
312 | Urlader only packs executables once and then caches them in the |
|
|
313 | F<$URLADER_EXECDIR>. After upgrades there will be old versions in there |
|
|
314 | that are not being used anymore. Or are they? |
|
|
315 | |
|
|
316 | Each instance directory (F<i-*>) in the F<$URLADER_EXE_DIR>) has an |
|
|
317 | associated lock file (F<i-*.lck>) - while urlader executes an app it keeps |
|
|
318 | a shared lock on this file. |
|
|
319 | |
|
|
320 | To detect whether a version is in use or not, you must try to acquire an |
|
|
321 | exclusive lock, i.e.: |
|
|
322 | |
|
|
323 | my $lock = Urlader::lock "~/.urlader/myexe/i-ver01.lck", 1, 0; |
|
|
324 | if (!$lock) { |
|
|
325 | # instance dir is not in use and can be safely deleted |
|
|
326 | } |
|
|
327 | |
|
|
328 | If an older urlader wants to use an instance that was deleted or is |
|
|
329 | currently being deleted it will wait until it's gone and simply recreate |
|
|
330 | it, so while less efficient, deleting instance directories should always |
|
|
331 | be safe. |
|
|
332 | |
|
|
333 | The lockfile itself can be deleted as long as you have an exclusive lock |
|
|
334 | on it (if your platform allows this). |
|
|
335 | |
|
|
336 | =item A real world project |
|
|
337 | |
|
|
338 | The only real world project using this that I know of at the moment is the |
|
|
339 | deliantra client (http://www.deliantra.net for more info). |
|
|
340 | |
|
|
341 | It uses some scary scripts to build the client and some dependnet modules |
|
|
342 | (F<build.*>), to gather perl source files into a distribution tree, shared |
|
|
343 | objects and system shared libraries (some of which have to be patched or, |
|
|
344 | due to the horrible dll hell on OS X, even renamed), called C<gatherer>, |
|
|
345 | and a script called C<gendist> to build executable distributions. |
|
|
346 | |
|
|
347 | These can be found at |
|
|
348 | L<http://cvs.schmorp.de/deliantra/Deliantra-Client/util/>, but looking at |
|
|
349 | them can lead to premature blindless. |
|
|
350 | |
|
|
351 | =item Shared Libraries |
|
|
352 | |
|
|
353 | It is often desirable to package shared libraries - for example the |
|
|
354 | Deliantra client packages SD>, Berkely DB, Pango and amny other libraries |
|
|
355 | that are unlikely to be available on the target system. |
|
|
356 | |
|
|
357 | This usually requires some fiddling (see below), and additionally some |
|
|
358 | environment variables to be set. |
|
|
359 | |
|
|
360 | For example, on ELF systems you usually want F<LD_LIBRARY_PATH=.> and on |
|
|
361 | OS X, you want F<DYLD_LIBRARY_PATH=.> (these are effectively the default |
|
|
362 | on windows). |
|
|
363 | |
|
|
364 | These can most easily be specified when building the packfile: |
|
|
365 | |
|
|
366 | urlader-util ... LD_LIBRARY_PATH=. ./perl run |
|
|
367 | |
|
|
368 | =item Portability: RPATH |
|
|
369 | |
|
|
370 | Often F<perl> is linked against a shared F<libperl.so> - and might be so |
|
|
371 | using an rpath. Perl extensikns likewise might use an rpath, which means |
|
|
372 | the binary will mostly ignore LD_LIBRARY_PATH, which leads to trouble. |
|
|
373 | |
|
|
374 | There is an utility called F<chrpath>, whose F<-d> option can remove the |
|
|
375 | rpath from binaries, shared library and shared objects. |
|
|
376 | |
|
|
377 | =item Portability: OS X DLL HELL |
|
|
378 | |
|
|
379 | OS X has the most severe form of DLL hell I have seen - if you link |
|
|
380 | against system libraries, which is practically unavoidable, you get |
|
|
381 | libraries of well-known names (e.g. libjpeg) that have nothing to do with |
|
|
382 | what you normally expect libjpeg to be, and there is no way to get your |
|
|
383 | version of libjpeg into your program. |
|
|
384 | |
|
|
385 | Moreover, even if apple ships well-known libraries (e.g. libiconv), they |
|
|
386 | often ship patched versions which have a different ABI or even API then |
|
|
387 | the real releases. |
|
|
388 | |
|
|
389 | The only way aorund this I found was to change all library names |
|
|
390 | in my releases (libjpeg.dylib becomes libdeliantra-jpeg.dylin and |
|
|
391 | so on), by patching the paths in the share dlibraries and shared |
|
|
392 | objects. F<install-name-tool> (with F<-id> and F<-change>) works in many |
|
|
393 | cases, but often paths are embedded indirectly, so you might have to use a |
|
|
394 | I<dirty> string replacement. |
|
|
395 | |
|
|
396 | =back |
|
|
397 | |
|
|
398 | =head1 SECURITY CONSIDERATIONS |
|
|
399 | |
|
|
400 | The urlader executable itself does not support setuig/setgid operation, or |
|
|
401 | running with elevated privileges - it does no input sanitisation, and is |
|
|
402 | trivially exploitable. |
53 | |
403 | |
54 | =head1 AUTHOR |
404 | =head1 AUTHOR |
55 | |
405 | |
56 | Marc Lehmann <schmorp@schmorp.de> |
406 | Marc Lehmann <schmorp@schmorp.de> |
57 | http://home.schmorp.de/ |
407 | http://home.schmorp.de/ |