[ViewVC] Diff of: cvs/Urlader/Urlader.pm

Comparing Urlader/Urlader.pm (file contents):
Revision 1.4 by root, Wed Jan 4 14:19:05 2012 UTC vs.
Revision 1.9 by root, Sun Jun 17 12:56:16 2012 UTC

…		…
7	use Urlader;	7	use Urlader;
8		8
9	=head1 DESCRIPTION	9	=head1 DESCRIPTION
10		10
11	Urlader (that's german for "bootloader" btw.) was created out of	11	Urlader (that's german for "bootloader" btw.) was created out of
12	frustration over PAR again not working, again not being flexible enough	12	frustration over PAR always being horribly slow, again not working, again
13	for simple things, and again causing mysterious missing files issues on	13	not being flexible enough for simple things such as software upgrades, and
14	various platforms.	14	again causing mysterious missing file issues on various platforms.
15		15
16	That doesn't mean this module replaces PAR, in fact, you should stay with	16	That doesn't mean this module replaces PAR, in fact, you should stay with
17	it for many reasons, user-friendlyness is one of them.	17	PAR for many reasons right now, user-friendlyness is one of them.
18		18
19	However, if you want to make single-file distributions out of your perl	19	However, if you want to make single-file distributions out of your perl
20	programs (or python, or C or whatever), and you are prepared to fiddle a	20	programs (or python, or C or whatever), and you are prepared to fiddle
21	LOT, this module might provide a tiny step towards your goal. Well, if it	21	a LOT, this module might provide a faster and more versatile deployment
22	ever gets finished.	22	technique then PAR. Well, if it ever gets finished.
23		23
24	Also, I<nothing in this module is considered very stable yet>, and it's	24	Also, I<nothing in this module is considered very stable yet>, and it's
25	far from feature-complete.	25	far from feature-complete.
26		26
27	Having said all that, Urlader basically provides three services:	27	Having said all that, Urlader basically provides three services:
…		…
29	=over 4	29	=over 4
30		30
31	=item A simple archiver that packs a directory tree into a single file.	31	=item A simple archiver that packs a directory tree into a single file.
32		32
33	=item A small C program that works on windows and unix, which unpacks an attached	33	=item A small C program that works on windows and unix, which unpacks an attached
34	archive and runs a program.	34	archive and runs a program (perl, python, whatever...).
35		35
36	=item A perl module support module (I<this one>), that can be used to query	36	=item A perl module support module (I<this one>), that can be used to query
37	the runtime environment, find out where to install updates and so on.	37	the runtime environment, find out where to install updates and so on.
38		38
39	=back	39	=back
…		…
48	bintree/perl	48	bintree/perl
49	bintree/libperl.so.5.10	49	bintree/libperl.so.5.10
50	bintree/run	50	bintree/run
51	bintree/pm/Guard.pm	51	bintree/pm/Guard.pm
52	bintree/pm/auto/Guard/Guard.so	52	bintree/pm/auto/Guard/Guard.so
		53	bintree/pm/XSLoader.pm
		54	bintree/pm/DynaLoader.pm
		55	bintree/pm/Config.pm
		56	bintree/pm/strict.pm
		57	bintree/pm/vars.pm
		58	bintree/pm/warnings.pm
53		59
54	# cat bintree/run	60	# cat bintree/run
55	@INC = ("pm", "."); # "." works around buggy AutoLoader	61	@INC = ("pm", "."); # "." works around buggy AutoLoader
56	use Guard;	62	use Guard;
57	guard { warn "hello, world!\n" }; # just to show off	63	guard { warn "hello, world!\n" }; # just to show off
…		…
63	# urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \	69	# urlader-util --urlader linux-x86 --pack myprog ver1_000 bintree \
64	LD_LIBRARY_PATH=. ./perl run \	70	LD_LIBRARY_PATH=. ./perl run \
65	>myprog	71	>myprog
66	# chmod 755 myprog	72	# chmod 755 myprog
67		73
		74	The packing step takes a binary loader and appends all the files in the
		75	directory tree, plus some meta information.
		76
		77	The resulting file is an executable that, when run, will unpack all the
		78	files and run the embedded program.
		79
68	=head1 CONCEPTS	80	=head1 CONCEPTS
69		81
70	=over 4	82	=over 4
71		83
72	=item urlader	84	=item urlader
…		…
101	it receives, and finally, a list of files (with contents :) and	113	it receives, and finally, a list of files (with contents :) and
102	directories.	114	directories.
103		115
104	=item override	116	=item override
105		117
		118	When the urlader starts, it first finds out what C<exe_id> is
		119	embedded in it. It then looks for an override file for this id
		120	(F<$URLADER_EXE_DIR/override>) and verifies that it is for the same
		121	C<exe_id>, and the version is newer. If this is the case, then it will
		122	unpack and run the override file instead of unpacking the files attched to
		123	itself.
		124
		125	This way one can implement software upgrades - download a new executable,
		126	write it safely to disk and move it to the override path.
		127
106	=back	128	=back
107		129
		130	=head1 ENVIRONMENT VARIABLES
		131
		132	The urlader sets and maintains the following environment variables, in
		133	addition to any variables specified on the commandline. The values in
		134	parentheses are typical (but not gauranteed) values for unix - on windows,
		135	F<~/.urlader> is replaced by F<%AppData%/urlader>.
		136
		137	=over 4
		138
		139	=item URLADER_VERSION (C<1.0>)
		140
		141	Set to the version of the urlader binary itself. All versions with the
		142	same major number should be compatible to older versions with the same
		143	major number.
		144
		145	=item URLADER_DATADIR (F<~/.urlader>)
		146
		147	The data directory used to store whatever urlader needs to store.
		148
		149	=item URLADER_CURRDIR
		150
		151	This is set to the full path of the current working directory where
		152	the urlader was started. Atfer unpacking, the urlader changes to the
		153	C<URLADER_EXECDIR>, so any relative paths should be resolved via this
		154	path.
		155
		156	=item URLADER_EXEPATH
		157
		158	This is set to the path of the urlader executable itself, usually relative
		159	to F<$URLADER_CURRDIR>.
		160
		161	=item URLADER_EXE_ID
		162
		163	This stores the executable id of the pack file attached to the urlader.
		164
		165	=item URLADER_EXE_VER
		166
		167	This is the executable version of the pack file attached to the urlader,
		168	or the override, whichever was newer. Or in other words, this is the
		169	version of the application running at the moment.
		170
		171	=item URLADER_EXE_DIR (F<~/.urlader/$URLADER_EXE_ID>>
		172
		173	The directory where urlader stores files related to the executable with
		174	the given id.
		175
		176	=item URLADER_EXECDIR (F<~/.urlader/$URLADER_EXE_ID/i-$URLADER_EXE_VER>)
		177
		178	The directory where the files from the pack file are unpacked and the
		179	program is being run. Also the working directory of the program when it is
		180	run.
		181
		182	=item URLADER_OVERRIDE (empty or F<override>)
		183
		184	The override file used, if any, relative to F<$URLADER_EXECDIR>. This is
		185	either missing, when no override was used, or the string F<override>, as
		186	thta is currently the only override file urlader is looking for.
		187
		188	=back
		189
108	=head1 FUNCTIONS AND VARIABLES IN THIS MODULE	190	=head1 FUNCTIONS AND VARIABLES IN THIS MODULE
109		191
110	=over 4	192	=over 4
111		193
112	=cut	194	=cut
…		…
114	package Urlader;	196	package Urlader;
115		197
116	use common::sense;	198	use common::sense;
117		199
118	BEGIN {	200	BEGIN {
119	our $VERSION = '0.2';	201	our $VERSION = '1.0';
120		202
121	use XSLoader;	203	use XSLoader;
122	XSLoader::load __PACKAGE__, $VERSION;	204	XSLoader::load __PACKAGE__, $VERSION;
123	}	205	}
124		206
		207	=item $Urlader::URLADER_VERSION
		208
		209	Set to the urlader version (C<URLADER_VERSION>) when the program is
		210	running form within urlader, undef otherwise.
		211
		212	=item $Urlader::DATADIR, $Urlader::EXE_ID, $Urlader::EXE_VER, $Urlader::EXE_DIR, $Urlader::EXECDIR
		213
		214	Contain the same value as the environment variable of the (almost) same
		215	name. You should prefer these, though, as these might even be set to
		216	correct values when not running form within an urlader environment.
		217
		218	=cut
		219
125	our $URLADER_VERSION; # only ste when running under urlader	220	our $URLADER_VERSION; # only set when running under urlader
126	our $DATADIR;	221	our $DATADIR;
127	our $EXE_ID;	222	our $EXE_ID;
128	our $EXE_VER;	223	our $EXE_VER;
129	our $EXE_DIR; # %AppData%/urlader/EXE_ID	224	our $EXE_DIR; # %AppData%/urlader/EXE_ID
130	our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER	225	our $EXECDIR; # %AppData%/urlader/EXE_ID/i-EXE_VER
…		…
138	$EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER	233	$EXECDIR = getenv "URLADER_EXECDIR"; # %AppData%/urlader/EXE_ID/i-EXE_VER
139	}	234	}
140		235
141	_get_env;	236	_get_env;
142		237
		238	=item Urlader::set_exe_info $exe_id, $exe_ver
		239
		240	Sets up the paths and variables as if running the given executable and
		241	version from within urlader.
		242
		243	=cut
		244
143	sub set_exe_info($$) {	245	sub set_exe_info($$) {
144	_set_datadir unless defined getenv "URLADER_DATADIR";	246	_set_datadir unless defined getenv "URLADER_DATADIR";
145	&_set_exe_info;	247	&_set_exe_info;
146	_get_env;	248	_get_env;
147	}	249	}
148		250
		251	=item $lock = Urlader::lock $path, $exclusive, $wait
		252
		253	Tries to acquire a lock on the given path (which must specify a file which
		254	will be created if necessary). If C<$exclusive> is true, then it tries to
		255	acquire an exclusive lock, otherwise the lock will be shared. If C<$wait>
		256	is true, then it will wait until the lock can be acquired, otherwise it
		257	only attempts to acquire it and returns immediately if it can't.
		258
		259	If successful it returns a lock object - the lock will be given up when
		260	the lock object is destroyed or when the process exits (even on a crash)
		261	and has a good chance of working on network drives as well.
		262
		263	If the lock could not be acquired, C<undef> is returned.
		264
		265	This function is provided to assist applications that want to clean up old
		266	versions, see "TIPS AND TRICKS", below.
		267
		268	=cut
149		269
150	1;	270	1;
151		271
152	=back	272	=back
		273
		274	=head1 TIPS AND TRICKS
		275
		276	=over 4
		277
		278	=item Gathering files
		279
		280	Gathering all the files needed for distribution cna be a big
		281	problem. Right now, Urlader does not assist you in this task in any way,
		282	however, just like perl source stripping, it is planned to unbundle the
		283	relevant technology from B<staticperl> (L<http://staticperl.plan9.de>) for
		284	use with this module.
		285
		286	You could always use par to find all library files, unpack the bundle and
		287	add F<perl>, F<libperl> and other support libraries (e.g. F<libgcc_s>).
		288
		289	=item Software update
		290
		291	Updating the software can be done by downloading a new packfile (with the
		292	same C<exe_id> but a higher C<exe_ver> - this can simply be the executable
		293	you create when making a release) and replacing the F<override> file in
		294	the F<$URLADER_EXE_DIR>.
		295
		296	When looking for updates, you should include C<$URLADER_VERSION>,
		297	C<$URLADER_EXE_ID> and C<$URLADER_EXE_VER> - the first two must be
		298	identical for update and currently running program, while the last one
		299	should be lexicographically higher.
		300
		301	Replacing the override file can be done like this:
		302
		303	rename "new-override.tmp", "$Urlader::EXE_DIR/override"
		304	or die "could not replace override";
		305
		306	This can fail on windows when another urlader currently reads it, but
		307	should work on all platforms even when other urlader programs execute
		308	concurrently.
		309
		310	=item Cleaning up old directories
		311
		312	Urlader only packs executables once and then caches them in the
		313	F<$URLADER_EXECDIR>. After upgrades there will be old versions in there
		314	that are not being used anymore. Or are they?
		315
		316	Each instance directory (F<i-*>) in the F<$URLADER_EXE_DIR>) has an
		317	associated lock file (F<i-*.lck>) - while urlader executes an app it keeps
		318	a shared lock on this file.
		319
		320	To detect whether a version is in use or not, you must try to acquire an
		321	exclusive lock, i.e.:
		322
		323	my $lock = Urlader::lock "~/.urlader/myexe/i-ver01.lck", 1, 0;
		324	if (!$lock) {
		325	# instance dir is not in use and can be safely deleted
		326	}
		327
		328	If an older urlader wants to use an instance that was deleted or is
		329	currently being deleted it will wait until it's gone and simply recreate
		330	it, so while less efficient, deleting instance directories should always
		331	be safe.
		332
		333	The lockfile itself can be deleted as long as you have an exclusive lock
		334	on it (if your platform allows this).
		335
		336	=item A real world project
		337
		338	The only real world project using this that I know of at the moment is the
		339	deliantra client (http://www.deliantra.net for more info).
		340
		341	It uses some scary scripts to build the client and some dependnet modules
		342	(F<build.*>), to gather perl source files into a distribution tree, shared
		343	objects and system shared libraries (some of which have to be patched or,
		344	due to the horrible dll hell on OS X, even renamed), called C<gatherer>,
		345	and a script called C<gendist> to build executable distributions.
		346
		347	These can be found at
		348	L<http://cvs.schmorp.de/deliantra/Deliantra-Client/util/>, but looking at
		349	them can lead to premature blindless.
		350
		351	=item Shared Libraries
		352
		353	It is often desirable to package shared libraries - for example the
		354	Deliantra client packages SD>, Berkely DB, Pango and amny other libraries
		355	that are unlikely to be available on the target system.
		356
		357	This usually requires some fiddling (see below), and additionally some
		358	environment variables to be set.
		359
		360	For example, on ELF systems you usually want F<LD_LIBRARY_PATH=.> and on
		361	OS X, you want F<DYLD_LIBRARY_PATH=.> (these are effectively the default
		362	on windows).
		363
		364	These can most easily be specified when building the packfile:
		365
		366	urlader-util ... LD_LIBRARY_PATH=. ./perl run
		367
		368	=item Portability: RPATH
		369
		370	Often F<perl> is linked against a shared F<libperl.so> - and might be so
		371	using an rpath. Perl extensikns likewise might use an rpath, which means
		372	the binary will mostly ignore LD_LIBRARY_PATH, which leads to trouble.
		373
		374	There is an utility called F<chrpath>, whose F<-d> option can remove the
		375	rpath from binaries, shared library and shared objects.
		376
		377	=item Portability: OS X DLL HELL
		378
		379	OS X has the most severe form of DLL hell I have seen - if you link
		380	against system libraries, which is practically unavoidable, you get
		381	libraries of well-known names (e.g. libjpeg) that have nothing to do with
		382	what you normally expect libjpeg to be, and there is no way to get your
		383	version of libjpeg into your program.
		384
		385	Moreover, even if apple ships well-known libraries (e.g. libiconv), they
		386	often ship patched versions which have a different ABI or even API then
		387	the real releases.
		388
		389	The only way aorund this I found was to change all library names
		390	in my releases (libjpeg.dylib becomes libdeliantra-jpeg.dylin and
		391	so on), by patching the paths in the share dlibraries and shared
		392	objects. F<install-name-tool> (with F<-id> and F<-change>) works in many
		393	cases, but often paths are embedded indirectly, so you might have to use a
		394	I<dirty> string replacement.
		395
		396	=back
		397
		398	=head1 SECURITY CONSIDERATIONS
		399
		400	The urlader executable itself does not support setuig/setgid operation, or
		401	running with elevated privileges - it does no input sanitisation, and is
		402	trivially exploitable.
153		403
154	=head1 AUTHOR	404	=head1 AUTHOR
155		405
156	Marc Lehmann <schmorp@schmorp.de>	406	Marc Lehmann <schmorp@schmorp.de>
157	http://home.schmorp.de/	407	http://home.schmorp.de/

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing Urlader/Urlader.pm (file contents): Revision 1.4 by root, Wed Jan 4 14:19:05 2012 UTC vs. Revision 1.9 by root, Sun Jun 17 12:56:16 2012 UTC

Diff Legend

Comparing Urlader/Urlader.pm (file contents):
Revision 1.4 by root, Wed Jan 4 14:19:05 2012 UTC vs.
Revision 1.9 by root, Sun Jun 17 12:56:16 2012 UTC