[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.17 by root, Mon Jun 26 14:53:55 2006 UTC vs.
Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC

…		…
13		13
14	aio_read $fh, 30000, 1024, $buffer, 0, sub {	14	aio_read $fh, 30000, 1024, $buffer, 0, sub {
15	$_[0] > 0 or die "read error: $!";	15	$_[0] > 0 or die "read error: $!";
16	};	16	};
17		17
18	# AnyEvent	18	# version 2+ has request and group objects
		19	use IO::AIO 2;
		20
		21	aioreq_pri 4; # give next request a very high priority
		22	my $req = aio_unlink "/tmp/file", sub { };
		23	$req->cancel; # cancel request if still in queue
		24
		25	my $grp = aio_group sub { print "all stats done\n" };
		26	add $grp aio_stat "..." for ...;
		27
		28	# AnyEvent integration
19	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";	29	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
20	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });	30	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
21		31
22	# Event	32	# Event integration
23	Event->io (fd => IO::AIO::poll_fileno,	33	Event->io (fd => IO::AIO::poll_fileno,
24	poll => 'r',	34	poll => 'r',
25	cb => \&IO::AIO::poll_cb);	35	cb => \&IO::AIO::poll_cb);
26		36
27	# Glib/Gtk2	37	# Glib/Gtk2 integration
28	add_watch Glib::IO IO::AIO::poll_fileno,	38	add_watch Glib::IO IO::AIO::poll_fileno,
29	in => sub { IO::AIO::poll_cb; 1 };	39	in => sub { IO::AIO::poll_cb; 1 };
30		40
31	# Tk	41	# Tk integration
32	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",	42	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
33	readable => \&IO::AIO::poll_cb);	43	readable => \&IO::AIO::poll_cb);
34		44
35	# Danga::Socket	45	# Danga::Socket integration
36	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>	46	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
37	\&IO::AIO::poll_cb);	47	\&IO::AIO::poll_cb);
38		48
39	DESCRIPTION	49	DESCRIPTION
40	This module implements asynchronous I/O using whatever means your	50	This module implements asynchronous I/O using whatever means your
41	operating system supports.	51	operating system supports.
42		52
43	Currently, a number of threads are started that execute your read/writes	53	In this version, a number of threads are started that execute your
44	and signal their completion. You don't need thread support in your libc	54	requests and signal their completion. You don't need thread support in
45	or perl, and the threads created by this module will not be visible to	55	perl, and the threads created by this module will not be visible to
46	the pthreads library. In the future, this module might make use of the	56	perl. In the future, this module might make use of the native aio
47	native aio functions available on many operating systems. However, they	57	functions available on many operating systems. However, they are often
48	are often not well-supported (Linux doesn't allow them on normal files	58	not well-supported or restricted (Linux doesn't allow them on normal
49	currently, for example), and they would only support aio_read and	59	files currently, for example), and they would only support aio_read and
50	aio_write, so the remaining functionality would have to be implemented	60	aio_write, so the remaining functionality would have to be implemented
51	using threads anyway.	61	using threads anyway.
52		62
53	Although the module will work with in the presence of other threads, it	63	Although the module will work with in the presence of other (Perl-)
54	is currently not reentrant, so use appropriate locking yourself, always	64	threads, it is currently not reentrant in any way, so use appropriate
55	call "poll_cb" from within the same thread, or never call "poll_cb" (or	65	locking yourself, always call "poll_cb" from within the same thread, or
56	other "aio_" functions) recursively.	66	never call "poll_cb" (or other "aio_" functions) recursively.
		67
		68	REQUEST ANATOMY AND LIFETIME
		69	Every "aio_*" function creates a request. which is a C data structure
		70	not directly visible to Perl.
		71
		72	If called in non-void context, every request function returns a Perl
		73	object representing the request. In void context, nothing is returned,
		74	which saves a bit of memory.
		75
		76	The perl object is a fairly standard ref-to-hash object. The hash
		77	contents are not used by IO::AIO so you are free to store anything you
		78	like in it.
		79
		80	During their existance, aio requests travel through the following
		81	states, in order:
		82
		83	ready
		84	Immediately after a request is created it is put into the ready
		85	state, waiting for a thread to execute it.
		86
		87	execute
		88	A thread has accepted the request for processing and is currently
		89	executing it (e.g. blocking in read).
		90
		91	pending
		92	The request has been executed and is waiting for result processing.
		93
		94	While request submission and execution is fully asynchronous, result
		95	processing is not and relies on the perl interpreter calling
		96	"poll_cb" (or another function with the same effect).
		97
		98	result
		99	The request results are processed synchronously by "poll_cb".
		100
		101	The "poll_cb" function will process all outstanding aio requests by
		102	calling their callbacks, freeing memory associated with them and
		103	managing any groups they are contained in.
		104
		105	done
		106	Request has reached the end of its lifetime and holds no resources
		107	anymore (except possibly for the Perl object, but its connection to
		108	the actual aio request is severed and calling its methods will
		109	either do nothing or result in a runtime error).
57		110
58	FUNCTIONS	111	FUNCTIONS
59	AIO FUNCTIONS	112	AIO FUNCTIONS
60	All the "aio_*" calls are more or less thin wrappers around the syscall	113	All the "aio_*" calls are more or less thin wrappers around the
61	with the same name (sans "aio_"). The arguments are similar or	114	syscall with the same name (sans "aio_"). The arguments are similar
62	identical, and they all accept an additional (and optional) $callback	115	or identical, and they all accept an additional (and optional)
63	argument which must be a code reference. This code reference will get	116	$callback argument which must be a code reference. This code
64	called with the syscall return code (e.g. most syscalls return -1 on	117	reference will get called with the syscall return code (e.g. most
65	error, unlike perl, which usually delivers "false") as it's sole	118	syscalls return -1 on error, unlike perl, which usually delivers
66	argument when the given syscall has been executed asynchronously.	119	"false") as it's sole argument when the given syscall has been
		120	executed asynchronously.
67		121
68	All functions expecting a filehandle keep a copy of the filehandle	122	All functions expecting a filehandle keep a copy of the filehandle
69	internally until the request has finished.	123	internally until the request has finished.
70		124
		125	All requests return objects of type IO::AIO::REQ that allow further
		126	manipulation of those requests while they are in-flight.
		127
71	The pathnames you pass to these routines must be absolute and encoded	128	The pathnames you pass to these routines must be absolute and
72	in byte form. The reason for the former is that at the time the request	129	encoded in byte form. The reason for the former is that at the time
73	is being executed, the current working directory could have changed.	130	the request is being executed, the current working directory could
74	Alternatively, you can make sure that you never change the current	131	have changed. Alternatively, you can make sure that you never change
75	working directory.	132	the current working directory.
76		133
77	To encode pathnames to byte form, either make sure you either: a) always	134	To encode pathnames to byte form, either make sure you either: a)
78	pass in filenames you got from outside (command line, readdir etc.), b)	135	always pass in filenames you got from outside (command line, readdir
79	are ASCII or ISO 8859-1, c) use the Encode module and encode your	136	etc.), b) are ASCII or ISO 8859-1, c) use the Encode module and
80	pathnames to the locale (or other) encoding in effect in the user	137	encode your pathnames to the locale (or other) encoding in effect in
81	environment, d) use Glib::filename_from_unicode on unicode filenames or	138	the user environment, d) use Glib::filename_from_unicode on unicode
82	e) use something else.	139	filenames or e) use something else.
83		140
		141	$prev_pri = aioreq_pri [$pri]
		142	Returns the priority value that would be used for the next
		143	request and, if $pri is given, sets the priority for the next
		144	aio request.
		145
		146	The default priority is 0, the minimum and maximum priorities
		147	are -4 and 4, respectively. Requests with higher priority will
		148	be serviced first.
		149
		150	The priority will be reset to 0 after each call to one of the
		151	"aio_*" functions.
		152
		153	Example: open a file with low priority, then read something from
		154	it with higher priority so the read request is serviced before
		155	other low priority open requests (potentially spamming the
		156	cache):
		157
		158	aioreq_pri -3;
		159	aio_open ..., sub {
		160	return unless $_[0];
		161
		162	aioreq_pri -2;
		163	aio_read $_[0], ..., sub {
		164	...
		165	};
		166	};
		167
		168	aioreq_nice $pri_adjust
		169	Similar to "aioreq_pri", but subtracts the given value from the
		170	current priority, so effects are cumulative.
		171
84	aio_open $pathname, $flags, $mode, $callback->($fh)	172	aio_open $pathname, $flags, $mode, $callback->($fh)
85	Asynchronously open or create a file and call the callback with a	173	Asynchronously open or create a file and call the callback with
86	newly created filehandle for the file.	174	a newly created filehandle for the file.
87		175
88	The pathname passed to "aio_open" must be absolute. See API NOTES,	176	The pathname passed to "aio_open" must be absolute. See API
89	above, for an explanation.	177	NOTES, above, for an explanation.
90		178
91	The $flags argument is a bitmask. See the "Fcntl" module for a list.	179	The $flags argument is a bitmask. See the "Fcntl" module for a
92	They are the same as used by "sysopen".	180	list. They are the same as used by "sysopen".
93		181
94	Likewise, $mode specifies the mode of the newly created file, if it	182	Likewise, $mode specifies the mode of the newly created file, if
95	didn't exist and "O_CREAT" has been given, just like perl's	183	it didn't exist and "O_CREAT" has been given, just like perl's
96	"sysopen", except that it is mandatory (i.e. use 0 if you don't	184	"sysopen", except that it is mandatory (i.e. use 0 if you don't
97	create new files, and 0666 or 0777 if you do).	185	create new files, and 0666 or 0777 if you do).
98		186
99	Example:	187	Example:
100		188
101	aio_open "/etc/passwd", O_RDONLY, 0, sub {	189	aio_open "/etc/passwd", O_RDONLY, 0, sub {
102	if ($_[0]) {	190	if ($_[0]) {
103	print "open successful, fh is $_[0]\n";	191	print "open successful, fh is $_[0]\n";
104	...	192	...
105	} else {	193	} else {
106	die "open failed: $!\n";	194	die "open failed: $!\n";
		195	}
107	}	196	};
		197
		198	aio_close $fh, $callback->($status)
		199	Asynchronously close a file and call the callback with the
		200	result code. WARNING: although accepted, you should not pass
		201	in a perl filehandle here, as perl will likely close the file
		202	descriptor another time when the filehandle is destroyed.
		203	Normally, you can safely call perls "close" or just let
		204	filehandles go out of scope.
		205
		206	This is supposed to be a bug in the API, so that might change.
		207	It's therefore best to avoid this function.
		208
		209	aio_read $fh,$offset,$length, $data,$dataoffset,
		210	$callback->($retval)
		211	aio_write $fh,$offset,$length, $data,$dataoffset,
		212	$callback->($retval)
		213	Reads or writes "length" bytes from the specified "fh" and
		214	"offset" into the scalar given by "data" and offset "dataoffset"
		215	and calls the callback without the actual number of bytes read
		216	(or -1 on error, just like the syscall).
		217
		218	The $data scalar MUST NOT be modified in any way while the
		219	request is outstanding. Modifying it can result in segfaults or
		220	WW3 (if the necessary/optional hardware is installed).
		221
		222	Example: Read 15 bytes at offset 7 into scalar $buffer, starting
		223	at offset 0 within the scalar:
		224
		225	aio_read $fh, 7, 15, $buffer, 0, sub {
		226	$_[0] > 0 or die "read error: $!";
		227	print "read $_[0] bytes: <$buffer>\n";
		228	};
		229
		230	aio_move $srcpath, $dstpath, $callback->($status)
		231	Try to move the file (directories not supported as either
		232	source or destination) from $srcpath to $dstpath and call the
		233	callback with the 0 (error) or -1 ok.
		234
		235	This is a composite request that tries to rename(2) the file
		236	first. If rename files with "EXDEV", it creates the destination
		237	file with mode 0200 and copies the contents of the source file
		238	into it using "aio_sendfile", followed by restoring atime,
		239	mtime, access mode and uid/gid, in that order, and unlinking the
		240	$srcpath.
		241
		242	If an error occurs, the partial destination file will be
		243	unlinked, if possible, except when setting atime, mtime, access
		244	mode and uid/gid, where errors are being ignored.
		245
		246	aio_sendfile $out_fh, $in_fh, $in_offset, $length,
		247	$callback->($retval)
		248	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		249	reading at byte offset $in_offset, and starts writing at the
		250	current file offset of $out_fh. Because of that, it is not safe
		251	to issue more than one "aio_sendfile" per $out_fh, as they will
		252	interfere with each other.
		253
		254	This call tries to make use of a native "sendfile" syscall to
		255	provide zero-copy operation. For this to work, $out_fh should
		256	refer to a socket, and $in_fh should refer to mmap'able file.
		257
		258	If the native sendfile call fails or is not implemented, it will
		259	be emulated, so you can call "aio_sendfile" on any type of
		260	filehandle regardless of the limitations of the operating
		261	system.
		262
		263	Please note, however, that "aio_sendfile" can read more bytes
		264	from $in_fh than are written, and there is no way to find out
		265	how many bytes have been read from "aio_sendfile" alone, as
		266	"aio_sendfile" only provides the number of bytes written to
		267	$out_fh. Only if the result value equals $length one can assume
		268	that $length bytes have been read.
		269
		270	aio_readahead $fh,$offset,$length, $callback->($retval)
		271	"aio_readahead" populates the page cache with data from a file
		272	so that subsequent reads from that file will not block on disk
		273	I/O. The $offset argument specifies the starting point from
		274	which data is to be read and $length specifies the number of
		275	bytes to be read. I/O is performed in whole pages, so that
		276	offset is effectively rounded down to a page boundary and bytes
		277	are read up to the next page boundary greater than or equal to
		278	(off-set+length). "aio_readahead" does not read beyond the end
		279	of the file. The current file offset of the file is left
		280	unchanged.
		281
		282	If that syscall doesn't exist (likely if your OS isn't Linux) it
		283	will be emulated by simply reading the data, which would have a
		284	similar effect.
		285
		286	aio_stat $fh_or_path, $callback->($status)
		287	aio_lstat $fh, $callback->($status)
		288	Works like perl's "stat" or "lstat" in void context. The
		289	callback will be called after the stat and the results will be
		290	available using "stat _" or "-s _" etc...
		291
		292	The pathname passed to "aio_stat" must be absolute. See API
		293	NOTES, above, for an explanation.
		294
		295	Currently, the stats are always 64-bit-stats, i.e. instead of
		296	returning an error when stat'ing a large file, the results will
		297	be silently truncated unless perl itself is compiled with large
		298	file support.
		299
		300	Example: Print the length of /etc/passwd:
		301
		302	aio_stat "/etc/passwd", sub {
		303	$_[0] and die "stat failed: $!";
		304	print "size is ", -s _, "\n";
		305	};
		306
		307	aio_unlink $pathname, $callback->($status)
		308	Asynchronously unlink (delete) a file and call the callback with
		309	the result code.
		310
		311	aio_link $srcpath, $dstpath, $callback->($status)
		312	Asynchronously create a new link to the existing object at
		313	$srcpath at the path $dstpath and call the callback with the
		314	result code.
		315
		316	aio_symlink $srcpath, $dstpath, $callback->($status)
		317	Asynchronously create a new symbolic link to the existing object
		318	at $srcpath at the path $dstpath and call the callback with the
		319	result code.
		320
		321	aio_rename $srcpath, $dstpath, $callback->($status)
		322	Asynchronously rename the object at $srcpath to $dstpath, just
		323	as rename(2) and call the callback with the result code.
		324
		325	aio_rmdir $pathname, $callback->($status)
		326	Asynchronously rmdir (delete) a directory and call the callback
		327	with the result code.
		328
		329	aio_readdir $pathname, $callback->($entries)
		330	Unlike the POSIX call of the same name, "aio_readdir" reads an
		331	entire directory (i.e. opendir + readdir + closedir). The
		332	entries will not be sorted, and will NOT include the "." and
		333	".." entries.
		334
		335	The callback a single argument which is either "undef" or an
		336	array-ref with the filenames.
		337
		338	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		339	Scans a directory (similar to "aio_readdir") but additionally
		340	tries to efficiently separate the entries of directory $path
		341	into two sets of names, directories you can recurse into
		342	(directories), and ones you cannot recurse into (everything
		343	else, including symlinks to directories).
		344
		345	"aio_scandir" is a composite request that creates of many sub
		346	requests_ $maxreq specifies the maximum number of outstanding
		347	aio requests that this function generates. If it is "<= 0", then
		348	a suitable default will be chosen (currently 6).
		349
		350	On error, the callback is called without arguments, otherwise it
		351	receives two array-refs with path-relative entry names.
		352
		353	Example:
		354
		355	aio_scandir $dir, 0, sub {
		356	my ($dirs, $nondirs) = @_;
		357	print "real directories: @$dirs\n";
		358	print "everything else: @$nondirs\n";
		359	};
		360
		361	Implementation notes.
		362
		363	The "aio_readdir" cannot be avoided, but "stat()"'ing every
		364	entry can.
		365
		366	After reading the directory, the modification time, size etc. of
		367	the directory before and after the readdir is checked, and if
		368	they match (and isn't the current time), the link count will be
		369	used to decide how many entries are directories (if >= 2).
		370	Otherwise, no knowledge of the number of subdirectories will be
		371	assumed.
		372
		373	Then entries will be sorted into likely directories (everything
		374	without a non-initial dot currently) and likely non-directories
		375	(everything else). Then every entry plus an appended "/." will
		376	be "stat"'ed, likely directories first. If that succeeds, it
		377	assumes that the entry is a directory or a symlink to directory
		378	(which will be checked seperately). This is often faster than
		379	stat'ing the entry itself because filesystems might detect the
		380	type of the entry without reading the inode data (e.g. ext2fs
		381	filetype feature).
		382
		383	If the known number of directories (link count - 2) has been
		384	reached, the rest of the entries is assumed to be
		385	non-directories.
		386
		387	This only works with certainty on POSIX (= UNIX) filesystems,
		388	which fortunately are the vast majority of filesystems around.
		389
		390	It will also likely work on non-POSIX filesystems with reduced
		391	efficiency as those tend to return 0 or 1 as link counts, which
		392	disables the directory counting heuristic.
		393
		394	aio_fsync $fh, $callback->($status)
		395	Asynchronously call fsync on the given filehandle and call the
		396	callback with the fsync result code.
		397
		398	aio_fdatasync $fh, $callback->($status)
		399	Asynchronously call fdatasync on the given filehandle and call
		400	the callback with the fdatasync result code.
		401
		402	If this call isn't available because your OS lacks it or it
		403	couldn't be detected, it will be emulated by calling "fsync"
		404	instead.
		405
		406	aio_group $callback->(...)
		407	This is a very special aio request: Instead of doing something,
		408	it is a container for other aio requests, which is useful if you
		409	want to bundle many requests into a single, composite, request
		410	with a definite callback and the ability to cancel the whole
		411	request with its subrequests.
		412
		413	Returns an object of class IO::AIO::GRP. See its documentation
		414	below for more info.
		415
		416	Example:
		417
		418	my $grp = aio_group sub {
		419	print "all stats done\n";
		420	};
		421
		422	add $grp
		423	(aio_stat ...),
		424	(aio_stat ...),
		425	...;
		426
		427	aio_nop $callback->()
		428	This is a special request - it does nothing in itself and is
		429	only used for side effects, such as when you want to add a dummy
		430	request to a group so that finishing the requests in the group
		431	depends on executing the given code.
		432
		433	While this request does nothing, it still goes through the
		434	execution phase and still requires a worker thread. Thus, the
		435	callback will not be executed immediately but only after other
		436	requests in the queue have entered their execution phase. This
		437	can be used to measure request latency.
		438
		439	IO::AIO::aio_busy $fractional_seconds, $callback->() NOT EXPORTED
		440	Mainly used for debugging and benchmarking, this aio request
		441	puts one of the request workers to sleep for the given time.
		442
		443	While it is theoretically handy to have simple I/O scheduling
		444	requests like sleep and file handle readable/writable, the
		445	overhead this creates is immense (it blocks a thread for a long
		446	time) so do not use this function except to put your application
		447	under artificial I/O pressure.
		448
		449	IO::AIO::REQ CLASS
		450	All non-aggregate "aio_*" functions return an object of this class
		451	when called in non-void context.
		452
		453	cancel $req
		454	Cancels the request, if possible. Has the effect of skipping
		455	execution when entering the execute state and skipping calling
		456	the callback when entering the the result state, but will leave
		457	the request otherwise untouched. That means that requests that
		458	currently execute will not be stopped and resources held by the
		459	request will not be freed prematurely.
		460
		461	cb $req $callback->(...)
		462	Replace (or simply set) the callback registered to the request.
		463
		464	IO::AIO::GRP CLASS
		465	This class is a subclass of IO::AIO::REQ, so all its methods apply
		466	to objects of this class, too.
		467
		468	A IO::AIO::GRP object is a special request that can contain multiple
		469	other aio requests.
		470
		471	You create one by calling the "aio_group" constructing function with
		472	a callback that will be called when all contained requests have
		473	entered the "done" state:
		474
		475	my $grp = aio_group sub {
		476	print "all requests are done\n";
108	};	477	};
109		478
110	aio_close $fh, $callback->($status)	479	You add requests by calling the "add" method with one or more
111	Asynchronously close a file and call the callback with the result	480	"IO::AIO::REQ" objects:
112	code. WARNING: although accepted, you should not pass in a perl
113	filehandle here, as perl will likely close the file descriptor
114	another time when the filehandle is destroyed. Normally, you can
115	safely call perls "close" or just let filehandles go out of scope.
116		481
117	This is supposed to be a bug in the API, so that might change. It's	482	$grp->add (aio_unlink "...");
118	therefore best to avoid this function.
119		483
120	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)	484	add $grp aio_stat "...", sub {
121	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)	485	$_[0] or return $grp->result ("error");
122	Reads or writes "length" bytes from the specified "fh" and "offset"
123	into the scalar given by "data" and offset "dataoffset" and calls
124	the callback without the actual number of bytes read (or -1 on
125	error, just like the syscall).
126		486
127	The $data scalar MUST NOT be modified in any way while the request	487	# add another request dynamically, if first succeeded
128	is outstanding. Modifying it can result in segfaults or WW3 (if the	488	add $grp aio_open "...", sub {
129	necessary/optional hardware is installed).	489	$grp->result ("ok");
130		490	};
131	Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
132	offset 0 within the scalar:
133
134	aio_read $fh, 7, 15, $buffer, 0, sub {
135	$_[0] > 0 or die "read error: $!";
136	print "read $_[0] bytes: <$buffer>\n";
137	};	491	};
138		492
139	aio_move $srcpath, $dstpath, $callback->($status)	493	This makes it very easy to create composite requests (see the source
140	[EXPERIMENTAL]	494	of "aio_move" for an application) that work and feel like simple
		495	requests.
141		496
142	Try to move the file (directories not supported as either source	497	* The IO::AIO::GRP objects will be cleaned up during calls to
143	or destination) from $srcpath to $dstpath and call the callback with	498	"IO::AIO::poll_cb", just like any other request.
144	the 0 (error) or -1 ok.	499	* They can be canceled like any other request. Canceling will cancel
		500	not only the request itself, but also all requests it contains.
		501	* They can also can also be added to other IO::AIO::GRP objects.
		502	* You must not add requests to a group from within the group
		503	callback (or any later time).
145		504
146	This is a composite request that tries to rename(2) the file first.	505	Their lifetime, simplified, looks like this: when they are empty,
147	If rename files with "EXDEV", it creates the destination file with	506	they will finish very quickly. If they contain only requests that
148	mode 0200 and copies the contents of the source file into it using	507	are in the "done" state, they will also finish. Otherwise they will
149	"aio_sendfile", followed by restoring atime, mtime, access mode and	508	continue to exist.
150	uid/gid, in that order, and unlinking the $srcpath.
151		509
152	If an error occurs, the partial destination file will be unlinked,	510	That means after creating a group you have some time to add
153	if possible, except when setting atime, mtime, access mode and	511	requests. And in the callbacks of those requests, you can add
154	uid/gid, where errors are being ignored.	512	further requests to the group. And only when all those requests have
		513	finished will the the group itself finish.
155		514
156	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)	515	add $grp ...
157	Tries to copy $length bytes from $in_fh to $out_fh. It starts	516	$grp->add (...)
158	reading at byte offset $in_offset, and starts writing at the current	517	Add one or more requests to the group. Any type of IO::AIO::REQ
159	file offset of $out_fh. Because of that, it is not safe to issue	518	can be added, including other groups, as long as you do not
160	more than one "aio_sendfile" per $out_fh, as they will interfere	519	create circular dependencies.
161	with each other.
162		520
163	This call tries to make use of a native "sendfile" syscall to	521	Returns all its arguments.
164	provide zero-copy operation. For this to work, $out_fh should refer
165	to a socket, and $in_fh should refer to mmap'able file.
166		522
167	If the native sendfile call fails or is not implemented, it will be	523	$grp->cancel_subs
168	emulated, so you can call "aio_sendfile" on any type of filehandle	524	Cancel all subrequests and clears any feeder, but not the group
169	regardless of the limitations of the operating system.	525	request itself. Useful when you queued a lot of events but got a
170
171	Please note, however, that "aio_sendfile" can read more bytes from
172	$in_fh than are written, and there is no way to find out how many
173	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
174	only provides the number of bytes written to $out_fh. Only if the
175	result value equals $length one can assume that $length bytes have
176	been read.
177
178	aio_readahead $fh,$offset,$length, $callback->($retval)
179	"aio_readahead" populates the page cache with data from a file so
180	that subsequent reads from that file will not block on disk I/O. The
181	$offset argument specifies the starting point from which data is to
182	be read and $length specifies the number of bytes to be read. I/O is
183	performed in whole pages, so that offset is effectively rounded down
184	to a page boundary and bytes are read up to the next page boundary
185	greater than or equal to (off-set+length). "aio_readahead" does not
186	read beyond the end of the file. The current file offset of the file
187	is left unchanged.
188
189	If that syscall doesn't exist (likely if your OS isn't Linux) it
190	will be emulated by simply reading the data, which would have a
191	similar effect.
192
193	aio_stat $fh_or_path, $callback->($status)
194	aio_lstat $fh, $callback->($status)
195	Works like perl's "stat" or "lstat" in void context. The callback
196	will be called after the stat and the results will be available
197	using "stat _" or "-s _" etc...
198
199	The pathname passed to "aio_stat" must be absolute. See API NOTES,
200	above, for an explanation.
201
202	Currently, the stats are always 64-bit-stats, i.e. instead of
203	returning an error when stat'ing a large file, the results will be
204	silently truncated unless perl itself is compiled with large file
205	support.
206
207	Example: Print the length of /etc/passwd:
208
209	aio_stat "/etc/passwd", sub {
210	$_[0] and die "stat failed: $!";
211	print "size is ", -s _, "\n";
212	};
213
214	aio_unlink $pathname, $callback->($status)
215	Asynchronously unlink (delete) a file and call the callback with the
216	result code.	526	result early.
217		527
218	aio_link $srcpath, $dstpath, $callback->($status)	528	$grp->result (...)
219	Asynchronously create a new link to the existing object at $srcpath	529	Set the result value(s) that will be passed to the group
220	at the path $dstpath and call the callback with the result code.	530	callback when all subrequests have finished and set thre groups
		531	errno to the current value of errno (just like calling "errno"
		532	without an error number). By default, no argument will be passed
		533	and errno is zero.
221		534
222	aio_symlink $srcpath, $dstpath, $callback->($status)	535	$grp->errno ([$errno])
223	Asynchronously create a new symbolic link to the existing object at	536	Sets the group errno value to $errno, or the current value of
224	$srcpath at the path $dstpath and call the callback with the result	537	errno when the argument is missing.
225	code.
226		538
227	aio_rename $srcpath, $dstpath, $callback->($status)	539	Every aio request has an associated errno value that is restored
228	Asynchronously rename the object at $srcpath to $dstpath, just as	540	when the callback is invoked. This method lets you change this
229	rename(2) and call the callback with the result code.	541	value from its default (0).
230		542
231	aio_rmdir $pathname, $callback->($status)	543	Calling "result" will also set errno, so make sure you either
232	Asynchronously rmdir (delete) a directory and call the callback with	544	set $! before the call to "result", or call c<errno> after it.
233	the result code.
234		545
235	aio_readdir $pathname, $callback->($entries)	546	feed $grp $callback->($grp)
236	Unlike the POSIX call of the same name, "aio_readdir" reads an	547	Sets a feeder/generator on this group: every group can have an
237	entire directory (i.e. opendir + readdir + closedir). The entries	548	attached generator that generates requests if idle. The idea
238	will not be sorted, and will NOT include the "." and ".." entries.	549	behind this is that, although you could just queue as many
		550	requests as you want in a group, this might starve other
		551	requests for a potentially long time. For example, "aio_scandir"
		552	might generate hundreds of thousands "aio_stat" requests,
		553	delaying any later requests for a long time.
239		554
240	The callback a single argument which is either "undef" or an	555	To avoid this, and allow incremental generation of requests, you
241	array-ref with the filenames.	556	can instead a group and set a feeder on it that generates those
		557	requests. The feed callback will be called whenever there are
		558	few enough (see "limit", below) requests active in the group
		559	itself and is expected to queue more requests.
242		560
243	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)	561	The feed callback can queue as many requests as it likes (i.e.
244	Scans a directory (similar to "aio_readdir") and tries to separate	562	"add" does not impose any limits).
245	the entries of directory $path into two sets of names, ones you can
246	recurse into (directories), and ones you cannot recurse into
247	(everything else).
248		563
249	"aio_scandir" is a composite request that consists of many	564	If the feed does not queue more requests when called, it will be
250	aio-primitives. $maxreq specifies the maximum number of outstanding	565	automatically removed from the group.
251	aio requests that this function generates. If it is "<= 0", then a
252	suitable default will be chosen (currently 8).
253		566
254	On error, the callback is called without arguments, otherwise it	567	If the feed limit is 0, it will be set to 2 automatically.
255	receives two array-refs with path-relative entry names.
256		568
257	Example:	569	Example:
258		570
259	aio_scandir $dir, 0, sub {	571	# stat all files in @files, but only ever use four aio requests concurrently:
260	my ($dirs, $nondirs) = @_;	572
261	print "real directories: @$dirs\n";	573	my $grp = aio_group sub { print "finished\n" };
262	print "everything else: @$nondirs\n";	574	limit $grp 4;
		575	feed $grp sub {
		576	my $file = pop @files
		577	or return;
		578
		579	add $grp aio_stat $file, sub { ... };
263	};	580	};
264		581
265	Implementation notes.	582	limit $grp $num
		583	Sets the feeder limit for the group: The feeder will be called
		584	whenever the group contains less than this many requests.
266		585
267	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry	586	Setting the limit to 0 will pause the feeding process.
268	can.
269
270	After reading the directory, the modification time, size etc. of the
271	directory before and after the readdir is checked, and if they
272	match, the link count will be used to decide how many entries are
273	directories (if >= 2). Otherwise, no knowledge of the number of
274	subdirectories will be assumed.
275
276	Then entires will be sorted into likely directories (everything
277	without a non-initial dot) and likely non-directories (everything
278	else). Then every entry + "/." will be "stat"'ed, likely directories
279	first. This is often faster because filesystems might detect the
280	type of the entry without reading the inode data (e.g. ext2fs
281	filetype feature). If that succeeds, it assumes that the entry is a
282	directory or a symlink to directory (which will be checked
283	seperately).
284
285	If the known number of directories has been reached, the rest of the
286	entries is assumed to be non-directories.
287
288	aio_fsync $fh, $callback->($status)
289	Asynchronously call fsync on the given filehandle and call the
290	callback with the fsync result code.
291
292	aio_fdatasync $fh, $callback->($status)
293	Asynchronously call fdatasync on the given filehandle and call the
294	callback with the fdatasync result code.
295
296	If this call isn't available because your OS lacks it or it couldn't
297	be detected, it will be emulated by calling "fsync" instead.
298		587
299	SUPPORT FUNCTIONS	588	SUPPORT FUNCTIONS
300	$fileno = IO::AIO::poll_fileno	589	$fileno = IO::AIO::poll_fileno
301	Return the request result pipe file descriptor. This filehandle	590	Return the request result pipe file descriptor. This
302	must be polled for reading by some mechanism outside this module	591	filehandle must be polled for reading by some mechanism outside
303	(e.g. Event or select, see below or the SYNOPSIS). If the pipe	592	this module (e.g. Event or select, see below or the SYNOPSIS).
304	becomes readable you have to call "poll_cb" to check the results.	593	If the pipe becomes readable you have to call "poll_cb" to check
		594	the results.
305		595
306	See "poll_cb" for an example.	596	See "poll_cb" for an example.
307		597
308	IO::AIO::poll_cb	598	IO::AIO::poll_cb
309	Process all outstanding events on the result pipe. You have to call	599	Process all outstanding events on the result pipe. You have to
310	this regularly. Returns the number of events processed. Returns	600	call this regularly. Returns the number of events processed.
311	immediately when no events are outstanding.	601	Returns immediately when no events are outstanding.
312		602
		603	If not all requests were processed for whatever reason, the
		604	filehandle will still be ready when "poll_cb" returns.
		605
313	Example: Install an Event watcher that automatically calls	606	Example: Install an Event watcher that automatically calls
314	IO::AIO::poll_cb with high priority:	607	IO::AIO::poll_cb with high priority:
315		608
316	Event->io (fd => IO::AIO::poll_fileno,	609	Event->io (fd => IO::AIO::poll_fileno,
317	poll => 'r', async => 1,	610	poll => 'r', async => 1,
318	cb => \&IO::AIO::poll_cb);	611	cb => \&IO::AIO::poll_cb);
319		612
		613	IO::AIO::poll_some $max_requests
		614	Similar to "poll_cb", but only processes up to $max_requests
		615	requests at a time.
		616
		617	Useful if you want to ensure some level of interactiveness when
		618	perl is not fast enough to process all requests in time.
		619
		620	Example: Install an Event watcher that automatically calls
		621	IO::AIO::poll_some with low priority, to ensure that other parts
		622	of the program get the CPU sometimes even under high AIO load.
		623
		624	Event->io (fd => IO::AIO::poll_fileno,
		625	poll => 'r', nice => 1,
		626	cb => sub { IO::AIO::poll_some 256 });
		627
320	IO::AIO::poll_wait	628	IO::AIO::poll_wait
321	Wait till the result filehandle becomes ready for reading (simply	629	Wait till the result filehandle becomes ready for reading
322	does a "select" on the filehandle. This is useful if you want to	630	(simply does a "select" on the filehandle. This is useful if you
323	synchronously wait for some requests to finish).	631	want to synchronously wait for some requests to finish).
324		632
325	See "nreqs" for an example.	633	See "nreqs" for an example.
326		634
327	IO::AIO::nreqs	635	IO::AIO::nreqs
328	Returns the number of requests currently outstanding (i.e. for which	636	Returns the number of requests currently in the ready, execute
329	their callback has not been invoked yet).	637	or pending states (i.e. for which their callback has not been
		638	invoked yet).
330		639
331	Example: wait till there are no outstanding requests anymore:	640	Example: wait till there are no outstanding requests anymore:
332		641
333	IO::AIO::poll_wait, IO::AIO::poll_cb	642	IO::AIO::poll_wait, IO::AIO::poll_cb
334	while IO::AIO::nreqs;	643	while IO::AIO::nreqs;
335		644
		645	IO::AIO::nready
		646	Returns the number of requests currently in the ready state (not
		647	yet executed).
		648
		649	IO::AIO::npending
		650	Returns the number of requests currently in the pending state
		651	(executed, but not yet processed by poll_cb).
		652
336	IO::AIO::flush	653	IO::AIO::flush
337	Wait till all outstanding AIO requests have been handled.	654	Wait till all outstanding AIO requests have been handled.
338		655
339	Strictly equivalent to:	656	Strictly equivalent to:
340		657
341	IO::AIO::poll_wait, IO::AIO::poll_cb	658	IO::AIO::poll_wait, IO::AIO::poll_cb
342	while IO::AIO::nreqs;	659	while IO::AIO::nreqs;
343		660
344	IO::AIO::poll	661	IO::AIO::poll
345	Waits until some requests have been handled.	662	Waits until some requests have been handled.
346		663
347	Strictly equivalent to:	664	Strictly equivalent to:
348		665
349	IO::AIO::poll_wait, IO::AIO::poll_cb	666	IO::AIO::poll_wait, IO::AIO::poll_cb
350	if IO::AIO::nreqs;	667	if IO::AIO::nreqs;
351		668
352	IO::AIO::min_parallel $nthreads	669	IO::AIO::min_parallel $nthreads
353	Set the minimum number of AIO threads to $nthreads. The current	670	Set the minimum number of AIO threads to $nthreads. The current
354	default is 4, which means four asynchronous operations can be done	671	default is 8, which means eight asynchronous operations can
355	at one time (the number of outstanding operations, however, is	672	execute concurrently at any one time (the number of outstanding
356	unlimited).	673	requests, however, is unlimited).
357		674
358	IO::AIO starts threads only on demand, when an AIO request is queued	675	IO::AIO starts threads only on demand, when an AIO request is
359	and no free thread exists.	676	queued and no free thread exists.
360		677
361	It is recommended to keep the number of threads low, as some Linux	678	It is recommended to keep the number of threads relatively low,
362	kernel versions will scale negatively with the number of threads	679	as some Linux kernel versions will scale negatively with the
363	(higher parallelity => MUCH higher latency). With current Linux 2.6	680	number of threads (higher parallelity => MUCH higher latency).
364	versions, 4-32 threads should be fine.	681	With current Linux 2.6 versions, 4-32 threads should be fine.
365		682
366	Under most circumstances you don't need to call this function, as	683	Under most circumstances you don't need to call this function,
367	the module selects a default that is suitable for low to moderate	684	as the module selects a default that is suitable for low to
368	load.	685	moderate load.
369		686
370	IO::AIO::max_parallel $nthreads	687	IO::AIO::max_parallel $nthreads
371	Sets the maximum number of AIO threads to $nthreads. If more than	688	Sets the maximum number of AIO threads to $nthreads. If more
372	the specified number of threads are currently running, this function	689	than the specified number of threads are currently running, this
373	kills them. This function blocks until the limit is reached.	690	function kills them. This function blocks until the limit is
		691	reached.
374		692
375	While $nthreads are zero, aio requests get queued but not executed	693	While $nthreads are zero, aio requests get queued but not
376	until the number of threads has been increased again.	694	executed until the number of threads has been increased again.
377		695
378	This module automatically runs "max_parallel 0" at program end, to	696	This module automatically runs "max_parallel 0" at program end,
379	ensure that all threads are killed and that there are no outstanding	697	to ensure that all threads are killed and that there are no
380	requests.	698	outstanding requests.
381		699
382	Under normal circumstances you don't need to call this function.	700	Under normal circumstances you don't need to call this function.
383		701
384	$oldnreqs = IO::AIO::max_outstanding $nreqs	702	$oldmaxreqs = IO::AIO::max_outstanding $maxreqs
		703	This is a very bad function to use in interactive programs
		704	because it blocks, and a bad way to reduce concurrency because
		705	it is inexact: Better use an "aio_group" together with a feed
		706	callback.
		707
385	Sets the maximum number of outstanding requests to $nreqs. If you	708	Sets the maximum number of outstanding requests to $nreqs. If
386	try to queue up more than this number of requests, the caller will	709	you to queue up more than this number of requests, the next call
387	block until some requests have been handled.	710	to the "poll_cb" (and "poll_some" and other functions calling
		711	"poll_cb") function will block until the limit is no longer
		712	exceeded.
388		713
389	The default is very large, so normally there is no practical limit.	714	The default value is very large, so there is no practical limit
390	If you queue up many requests in a loop it often improves speed if	715	on the number of outstanding requests.
391	you set this to a relatively low number, such as 100.
392		716
393	Under normal circumstances you don't need to call this function.	717	You can still queue as many requests as you want. Therefore,
		718	"max_oustsanding" is mainly useful in simple scripts (with low
		719	values) or as a stop gap to shield against fatal memory overflow
		720	(with large values).
394		721
395	FORK BEHAVIOUR	722	FORK BEHAVIOUR
		723	This module should do "the right thing" when the process using it
		724	forks:
		725
396	Before the fork, IO::AIO enters a quiescent state where no requests can	726	Before the fork, IO::AIO enters a quiescent state where no requests
397	be added in other threads and no results will be processed. After the	727	can be added in other threads and no results will be processed.
398	fork the parent simply leaves the quiescent state and continues	728	After the fork the parent simply leaves the quiescent state and
399	request/result processing, while the child clears the request/result	729	continues request/result processing, while the child frees the
400	queue (so the requests started before the fork will only be handled in	730	request/result queue (so that the requests started before the fork
401	the parent). Threats will be started on demand until the limit ste in	731	will only be handled in the parent). Threads will be started on
402	the parent process has been reached again.	732	demand until the limit set in the parent process has been reached
		733	again.
		734
		735	In short: the parent will, after a short pause, continue as if fork
		736	had not been called, while the child will act as if IO::AIO has not
		737	been used yet.
		738
		739	MEMORY USAGE
		740	Per-request usage:
		741
		742	Each aio request uses - depending on your architecture - around
		743	100-200 bytes of memory. In addition, stat requests need a stat
		744	buffer (possibly a few hundred bytes), readdir requires a result
		745	buffer and so on. Perl scalars and other data passed into aio
		746	requests will also be locked and will consume memory till the
		747	request has entered the done state.
		748
		749	This is now awfully much, so queuing lots of requests is not usually
		750	a problem.
		751
		752	Per-thread usage:
		753
		754	In the execution phase, some aio requests require more memory for
		755	temporary buffers, and each thread requires a stack and other data
		756	structures (usually around 16k-128k, depending on the OS).
		757
		758	KNOWN BUGS
		759	Known bugs will be fixed in the next release.
403		760
404	SEE ALSO	761	SEE ALSO
405	Coro, Linux::AIO.	762	Coro::AIO.
406		763
407	AUTHOR	764	AUTHOR
408	Marc Lehmann <schmorp@schmorp.de>	765	Marc Lehmann <schmorp@schmorp.de>
409	http://home.schmorp.de/	766	http://home.schmorp.de/
410		767

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.17 by root, Mon Jun 26 14:53:55 2006 UTC vs. Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.17 by root, Mon Jun 26 14:53:55 2006 UTC vs.
Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC