[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.13 by root, Tue Aug 30 15:45:10 2005 UTC vs.
Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC

…		…
13		13
14	aio_read $fh, 30000, 1024, $buffer, 0, sub {	14	aio_read $fh, 30000, 1024, $buffer, 0, sub {
15	$_[0] > 0 or die "read error: $!";	15	$_[0] > 0 or die "read error: $!";
16	};	16	};
17		17
18	# Event	18	# version 2+ has request and group objects
		19	use IO::AIO 2;
		20
		21	aioreq_pri 4; # give next request a very high priority
		22	my $req = aio_unlink "/tmp/file", sub { };
		23	$req->cancel; # cancel request if still in queue
		24
		25	my $grp = aio_group sub { print "all stats done\n" };
		26	add $grp aio_stat "..." for ...;
		27
		28	# AnyEvent integration
		29	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
		30	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
		31
		32	# Event integration
19	Event->io (fd => IO::AIO::poll_fileno,	33	Event->io (fd => IO::AIO::poll_fileno,
20	poll => 'r',	34	poll => 'r',
21	cb => \&IO::AIO::poll_cb);	35	cb => \&IO::AIO::poll_cb);
22		36
23	# Glib/Gtk2	37	# Glib/Gtk2 integration
24	add_watch Glib::IO IO::AIO::poll_fileno,	38	add_watch Glib::IO IO::AIO::poll_fileno,
25	in => sub { IO::AIO::poll_cb; 1 };	39	in => sub { IO::AIO::poll_cb; 1 };
26		40
27	# Tk	41	# Tk integration
28	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",	42	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
29	readable => \&IO::AIO::poll_cb);	43	readable => \&IO::AIO::poll_cb);
30		44
31	# Danga::Socket	45	# Danga::Socket integration
32	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>	46	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
33	\&IO::AIO::poll_cb);	47	\&IO::AIO::poll_cb);
34		48
35	DESCRIPTION	49	DESCRIPTION
36	This module implements asynchronous I/O using whatever means your	50	This module implements asynchronous I/O using whatever means your
37	operating system supports.	51	operating system supports.
38		52
39	Currently, a number of threads are started that execute your read/writes	53	In this version, a number of threads are started that execute your
40	and signal their completion. You don't need thread support in your libc	54	requests and signal their completion. You don't need thread support in
41	or perl, and the threads created by this module will not be visible to	55	perl, and the threads created by this module will not be visible to
42	the pthreads library. In the future, this module might make use of the	56	perl. In the future, this module might make use of the native aio
43	native aio functions available on many operating systems. However, they	57	functions available on many operating systems. However, they are often
44	are often not well-supported (Linux doesn't allow them on normal files	58	not well-supported or restricted (Linux doesn't allow them on normal
45	currently, for example), and they would only support aio_read and	59	files currently, for example), and they would only support aio_read and
46	aio_write, so the remaining functionality would have to be implemented	60	aio_write, so the remaining functionality would have to be implemented
47	using threads anyway.	61	using threads anyway.
48		62
49	Although the module will work with in the presence of other threads, it	63	Although the module will work with in the presence of other (Perl-)
50	is currently not reentrant, so use appropriate locking yourself, always	64	threads, it is currently not reentrant in any way, so use appropriate
51	call "poll_cb" from within the same thread, or never call "poll_cb" (or	65	locking yourself, always call "poll_cb" from within the same thread, or
52	other "aio_" functions) recursively.	66	never call "poll_cb" (or other "aio_" functions) recursively.
		67
		68	REQUEST ANATOMY AND LIFETIME
		69	Every "aio_*" function creates a request. which is a C data structure
		70	not directly visible to Perl.
		71
		72	If called in non-void context, every request function returns a Perl
		73	object representing the request. In void context, nothing is returned,
		74	which saves a bit of memory.
		75
		76	The perl object is a fairly standard ref-to-hash object. The hash
		77	contents are not used by IO::AIO so you are free to store anything you
		78	like in it.
		79
		80	During their existance, aio requests travel through the following
		81	states, in order:
		82
		83	ready
		84	Immediately after a request is created it is put into the ready
		85	state, waiting for a thread to execute it.
		86
		87	execute
		88	A thread has accepted the request for processing and is currently
		89	executing it (e.g. blocking in read).
		90
		91	pending
		92	The request has been executed and is waiting for result processing.
		93
		94	While request submission and execution is fully asynchronous, result
		95	processing is not and relies on the perl interpreter calling
		96	"poll_cb" (or another function with the same effect).
		97
		98	result
		99	The request results are processed synchronously by "poll_cb".
		100
		101	The "poll_cb" function will process all outstanding aio requests by
		102	calling their callbacks, freeing memory associated with them and
		103	managing any groups they are contained in.
		104
		105	done
		106	Request has reached the end of its lifetime and holds no resources
		107	anymore (except possibly for the Perl object, but its connection to
		108	the actual aio request is severed and calling its methods will
		109	either do nothing or result in a runtime error).
53		110
54	FUNCTIONS	111	FUNCTIONS
55	AIO FUNCTIONS	112	AIO FUNCTIONS
56	All the "aio_*" calls are more or less thin wrappers around the syscall	113	All the "aio_*" calls are more or less thin wrappers around the
57	with the same name (sans "aio_"). The arguments are similar or	114	syscall with the same name (sans "aio_"). The arguments are similar
58	identical, and they all accept an additional (and optional) $callback	115	or identical, and they all accept an additional (and optional)
59	argument which must be a code reference. This code reference will get	116	$callback argument which must be a code reference. This code
60	called with the syscall return code (e.g. most syscalls return -1 on	117	reference will get called with the syscall return code (e.g. most
61	error, unlike perl, which usually delivers "false") as it's sole	118	syscalls return -1 on error, unlike perl, which usually delivers
62	argument when the given syscall has been executed asynchronously.	119	"false") as it's sole argument when the given syscall has been
		120	executed asynchronously.
63		121
64	All functions expecting a filehandle keep a copy of the filehandle	122	All functions expecting a filehandle keep a copy of the filehandle
65	internally until the request has finished.	123	internally until the request has finished.
66		124
		125	All requests return objects of type IO::AIO::REQ that allow further
		126	manipulation of those requests while they are in-flight.
		127
67	The pathnames you pass to these routines must be absolute and encoded	128	The pathnames you pass to these routines must be absolute and
68	in byte form. The reason for the former is that at the time the request	129	encoded in byte form. The reason for the former is that at the time
69	is being executed, the current working directory could have changed.	130	the request is being executed, the current working directory could
70	Alternatively, you can make sure that you never change the current	131	have changed. Alternatively, you can make sure that you never change
71	working directory.	132	the current working directory.
72		133
73	To encode pathnames to byte form, either make sure you either: a) always	134	To encode pathnames to byte form, either make sure you either: a)
74	pass in filenames you got from outside (command line, readdir etc.), b)	135	always pass in filenames you got from outside (command line, readdir
75	are ASCII or ISO 8859-1, c) use the Encode module and encode your	136	etc.), b) are ASCII or ISO 8859-1, c) use the Encode module and
76	pathnames to the locale (or other) encoding in effect in the user	137	encode your pathnames to the locale (or other) encoding in effect in
77	environment, d) use Glib::filename_from_unicode on unicode filenames or	138	the user environment, d) use Glib::filename_from_unicode on unicode
78	e) use something else.	139	filenames or e) use something else.
79		140
		141	$prev_pri = aioreq_pri [$pri]
		142	Returns the priority value that would be used for the next
		143	request and, if $pri is given, sets the priority for the next
		144	aio request.
		145
		146	The default priority is 0, the minimum and maximum priorities
		147	are -4 and 4, respectively. Requests with higher priority will
		148	be serviced first.
		149
		150	The priority will be reset to 0 after each call to one of the
		151	"aio_*" functions.
		152
		153	Example: open a file with low priority, then read something from
		154	it with higher priority so the read request is serviced before
		155	other low priority open requests (potentially spamming the
		156	cache):
		157
		158	aioreq_pri -3;
		159	aio_open ..., sub {
		160	return unless $_[0];
		161
		162	aioreq_pri -2;
		163	aio_read $_[0], ..., sub {
		164	...
		165	};
		166	};
		167
		168	aioreq_nice $pri_adjust
		169	Similar to "aioreq_pri", but subtracts the given value from the
		170	current priority, so effects are cumulative.
		171
80	aio_open $pathname, $flags, $mode, $callback->($fh)	172	aio_open $pathname, $flags, $mode, $callback->($fh)
81	Asynchronously open or create a file and call the callback with a	173	Asynchronously open or create a file and call the callback with
82	newly created filehandle for the file.	174	a newly created filehandle for the file.
83		175
84	The pathname passed to "aio_open" must be absolute. See API NOTES,	176	The pathname passed to "aio_open" must be absolute. See API
85	above, for an explanation.	177	NOTES, above, for an explanation.
86		178
87	The $flags argument is a bitmask. See the "Fcntl" module for a list.	179	The $flags argument is a bitmask. See the "Fcntl" module for a
88	They are the same as used by "sysopen".	180	list. They are the same as used by "sysopen".
89		181
90	Likewise, $mode specifies the mode of the newly created file, if it	182	Likewise, $mode specifies the mode of the newly created file, if
91	didn't exist and "O_CREAT" has been given, just like perl's	183	it didn't exist and "O_CREAT" has been given, just like perl's
92	"sysopen", except that it is mandatory (i.e. use 0 if you don't	184	"sysopen", except that it is mandatory (i.e. use 0 if you don't
93	create new files, and 0666 or 0777 if you do).	185	create new files, and 0666 or 0777 if you do).
94		186
95	Example:	187	Example:
96		188
97	aio_open "/etc/passwd", O_RDONLY, 0, sub {	189	aio_open "/etc/passwd", O_RDONLY, 0, sub {
98	if ($_[0]) {	190	if ($_[0]) {
99	print "open successful, fh is $_[0]\n";	191	print "open successful, fh is $_[0]\n";
100	...	192	...
101	} else {	193	} else {
102	die "open failed: $!\n";	194	die "open failed: $!\n";
		195	}
103	}	196	};
		197
		198	aio_close $fh, $callback->($status)
		199	Asynchronously close a file and call the callback with the
		200	result code. WARNING: although accepted, you should not pass
		201	in a perl filehandle here, as perl will likely close the file
		202	descriptor another time when the filehandle is destroyed.
		203	Normally, you can safely call perls "close" or just let
		204	filehandles go out of scope.
		205
		206	This is supposed to be a bug in the API, so that might change.
		207	It's therefore best to avoid this function.
		208
		209	aio_read $fh,$offset,$length, $data,$dataoffset,
		210	$callback->($retval)
		211	aio_write $fh,$offset,$length, $data,$dataoffset,
		212	$callback->($retval)
		213	Reads or writes "length" bytes from the specified "fh" and
		214	"offset" into the scalar given by "data" and offset "dataoffset"
		215	and calls the callback without the actual number of bytes read
		216	(or -1 on error, just like the syscall).
		217
		218	The $data scalar MUST NOT be modified in any way while the
		219	request is outstanding. Modifying it can result in segfaults or
		220	WW3 (if the necessary/optional hardware is installed).
		221
		222	Example: Read 15 bytes at offset 7 into scalar $buffer, starting
		223	at offset 0 within the scalar:
		224
		225	aio_read $fh, 7, 15, $buffer, 0, sub {
		226	$_[0] > 0 or die "read error: $!";
		227	print "read $_[0] bytes: <$buffer>\n";
		228	};
		229
		230	aio_move $srcpath, $dstpath, $callback->($status)
		231	Try to move the file (directories not supported as either
		232	source or destination) from $srcpath to $dstpath and call the
		233	callback with the 0 (error) or -1 ok.
		234
		235	This is a composite request that tries to rename(2) the file
		236	first. If rename files with "EXDEV", it creates the destination
		237	file with mode 0200 and copies the contents of the source file
		238	into it using "aio_sendfile", followed by restoring atime,
		239	mtime, access mode and uid/gid, in that order, and unlinking the
		240	$srcpath.
		241
		242	If an error occurs, the partial destination file will be
		243	unlinked, if possible, except when setting atime, mtime, access
		244	mode and uid/gid, where errors are being ignored.
		245
		246	aio_sendfile $out_fh, $in_fh, $in_offset, $length,
		247	$callback->($retval)
		248	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		249	reading at byte offset $in_offset, and starts writing at the
		250	current file offset of $out_fh. Because of that, it is not safe
		251	to issue more than one "aio_sendfile" per $out_fh, as they will
		252	interfere with each other.
		253
		254	This call tries to make use of a native "sendfile" syscall to
		255	provide zero-copy operation. For this to work, $out_fh should
		256	refer to a socket, and $in_fh should refer to mmap'able file.
		257
		258	If the native sendfile call fails or is not implemented, it will
		259	be emulated, so you can call "aio_sendfile" on any type of
		260	filehandle regardless of the limitations of the operating
		261	system.
		262
		263	Please note, however, that "aio_sendfile" can read more bytes
		264	from $in_fh than are written, and there is no way to find out
		265	how many bytes have been read from "aio_sendfile" alone, as
		266	"aio_sendfile" only provides the number of bytes written to
		267	$out_fh. Only if the result value equals $length one can assume
		268	that $length bytes have been read.
		269
		270	aio_readahead $fh,$offset,$length, $callback->($retval)
		271	"aio_readahead" populates the page cache with data from a file
		272	so that subsequent reads from that file will not block on disk
		273	I/O. The $offset argument specifies the starting point from
		274	which data is to be read and $length specifies the number of
		275	bytes to be read. I/O is performed in whole pages, so that
		276	offset is effectively rounded down to a page boundary and bytes
		277	are read up to the next page boundary greater than or equal to
		278	(off-set+length). "aio_readahead" does not read beyond the end
		279	of the file. The current file offset of the file is left
		280	unchanged.
		281
		282	If that syscall doesn't exist (likely if your OS isn't Linux) it
		283	will be emulated by simply reading the data, which would have a
		284	similar effect.
		285
		286	aio_stat $fh_or_path, $callback->($status)
		287	aio_lstat $fh, $callback->($status)
		288	Works like perl's "stat" or "lstat" in void context. The
		289	callback will be called after the stat and the results will be
		290	available using "stat _" or "-s _" etc...
		291
		292	The pathname passed to "aio_stat" must be absolute. See API
		293	NOTES, above, for an explanation.
		294
		295	Currently, the stats are always 64-bit-stats, i.e. instead of
		296	returning an error when stat'ing a large file, the results will
		297	be silently truncated unless perl itself is compiled with large
		298	file support.
		299
		300	Example: Print the length of /etc/passwd:
		301
		302	aio_stat "/etc/passwd", sub {
		303	$_[0] and die "stat failed: $!";
		304	print "size is ", -s _, "\n";
		305	};
		306
		307	aio_unlink $pathname, $callback->($status)
		308	Asynchronously unlink (delete) a file and call the callback with
		309	the result code.
		310
		311	aio_link $srcpath, $dstpath, $callback->($status)
		312	Asynchronously create a new link to the existing object at
		313	$srcpath at the path $dstpath and call the callback with the
		314	result code.
		315
		316	aio_symlink $srcpath, $dstpath, $callback->($status)
		317	Asynchronously create a new symbolic link to the existing object
		318	at $srcpath at the path $dstpath and call the callback with the
		319	result code.
		320
		321	aio_rename $srcpath, $dstpath, $callback->($status)
		322	Asynchronously rename the object at $srcpath to $dstpath, just
		323	as rename(2) and call the callback with the result code.
		324
		325	aio_rmdir $pathname, $callback->($status)
		326	Asynchronously rmdir (delete) a directory and call the callback
		327	with the result code.
		328
		329	aio_readdir $pathname, $callback->($entries)
		330	Unlike the POSIX call of the same name, "aio_readdir" reads an
		331	entire directory (i.e. opendir + readdir + closedir). The
		332	entries will not be sorted, and will NOT include the "." and
		333	".." entries.
		334
		335	The callback a single argument which is either "undef" or an
		336	array-ref with the filenames.
		337
		338	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		339	Scans a directory (similar to "aio_readdir") but additionally
		340	tries to efficiently separate the entries of directory $path
		341	into two sets of names, directories you can recurse into
		342	(directories), and ones you cannot recurse into (everything
		343	else, including symlinks to directories).
		344
		345	"aio_scandir" is a composite request that creates of many sub
		346	requests_ $maxreq specifies the maximum number of outstanding
		347	aio requests that this function generates. If it is "<= 0", then
		348	a suitable default will be chosen (currently 6).
		349
		350	On error, the callback is called without arguments, otherwise it
		351	receives two array-refs with path-relative entry names.
		352
		353	Example:
		354
		355	aio_scandir $dir, 0, sub {
		356	my ($dirs, $nondirs) = @_;
		357	print "real directories: @$dirs\n";
		358	print "everything else: @$nondirs\n";
		359	};
		360
		361	Implementation notes.
		362
		363	The "aio_readdir" cannot be avoided, but "stat()"'ing every
		364	entry can.
		365
		366	After reading the directory, the modification time, size etc. of
		367	the directory before and after the readdir is checked, and if
		368	they match (and isn't the current time), the link count will be
		369	used to decide how many entries are directories (if >= 2).
		370	Otherwise, no knowledge of the number of subdirectories will be
		371	assumed.
		372
		373	Then entries will be sorted into likely directories (everything
		374	without a non-initial dot currently) and likely non-directories
		375	(everything else). Then every entry plus an appended "/." will
		376	be "stat"'ed, likely directories first. If that succeeds, it
		377	assumes that the entry is a directory or a symlink to directory
		378	(which will be checked seperately). This is often faster than
		379	stat'ing the entry itself because filesystems might detect the
		380	type of the entry without reading the inode data (e.g. ext2fs
		381	filetype feature).
		382
		383	If the known number of directories (link count - 2) has been
		384	reached, the rest of the entries is assumed to be
		385	non-directories.
		386
		387	This only works with certainty on POSIX (= UNIX) filesystems,
		388	which fortunately are the vast majority of filesystems around.
		389
		390	It will also likely work on non-POSIX filesystems with reduced
		391	efficiency as those tend to return 0 or 1 as link counts, which
		392	disables the directory counting heuristic.
		393
		394	aio_fsync $fh, $callback->($status)
		395	Asynchronously call fsync on the given filehandle and call the
		396	callback with the fsync result code.
		397
		398	aio_fdatasync $fh, $callback->($status)
		399	Asynchronously call fdatasync on the given filehandle and call
		400	the callback with the fdatasync result code.
		401
		402	If this call isn't available because your OS lacks it or it
		403	couldn't be detected, it will be emulated by calling "fsync"
		404	instead.
		405
		406	aio_group $callback->(...)
		407	This is a very special aio request: Instead of doing something,
		408	it is a container for other aio requests, which is useful if you
		409	want to bundle many requests into a single, composite, request
		410	with a definite callback and the ability to cancel the whole
		411	request with its subrequests.
		412
		413	Returns an object of class IO::AIO::GRP. See its documentation
		414	below for more info.
		415
		416	Example:
		417
		418	my $grp = aio_group sub {
		419	print "all stats done\n";
		420	};
		421
		422	add $grp
		423	(aio_stat ...),
		424	(aio_stat ...),
		425	...;
		426
		427	aio_nop $callback->()
		428	This is a special request - it does nothing in itself and is
		429	only used for side effects, such as when you want to add a dummy
		430	request to a group so that finishing the requests in the group
		431	depends on executing the given code.
		432
		433	While this request does nothing, it still goes through the
		434	execution phase and still requires a worker thread. Thus, the
		435	callback will not be executed immediately but only after other
		436	requests in the queue have entered their execution phase. This
		437	can be used to measure request latency.
		438
		439	IO::AIO::aio_busy $fractional_seconds, $callback->() NOT EXPORTED
		440	Mainly used for debugging and benchmarking, this aio request
		441	puts one of the request workers to sleep for the given time.
		442
		443	While it is theoretically handy to have simple I/O scheduling
		444	requests like sleep and file handle readable/writable, the
		445	overhead this creates is immense (it blocks a thread for a long
		446	time) so do not use this function except to put your application
		447	under artificial I/O pressure.
		448
		449	IO::AIO::REQ CLASS
		450	All non-aggregate "aio_*" functions return an object of this class
		451	when called in non-void context.
		452
		453	cancel $req
		454	Cancels the request, if possible. Has the effect of skipping
		455	execution when entering the execute state and skipping calling
		456	the callback when entering the the result state, but will leave
		457	the request otherwise untouched. That means that requests that
		458	currently execute will not be stopped and resources held by the
		459	request will not be freed prematurely.
		460
		461	cb $req $callback->(...)
		462	Replace (or simply set) the callback registered to the request.
		463
		464	IO::AIO::GRP CLASS
		465	This class is a subclass of IO::AIO::REQ, so all its methods apply
		466	to objects of this class, too.
		467
		468	A IO::AIO::GRP object is a special request that can contain multiple
		469	other aio requests.
		470
		471	You create one by calling the "aio_group" constructing function with
		472	a callback that will be called when all contained requests have
		473	entered the "done" state:
		474
		475	my $grp = aio_group sub {
		476	print "all requests are done\n";
104	};	477	};
105		478
106	aio_close $fh, $callback->($status)	479	You add requests by calling the "add" method with one or more
107	Asynchronously close a file and call the callback with the result	480	"IO::AIO::REQ" objects:
108	code. WARNING: although accepted, you should not pass in a perl
109	filehandle here, as perl will likely close the file descriptor
110	another time when the filehandle is destroyed. Normally, you can
111	safely call perls "close" or just let filehandles go out of scope.
112		481
113	This is supposed to be a bug in the API, so that might change. It's	482	$grp->add (aio_unlink "...");
114	therefore best to avoid this function.
115		483
116	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)	484	add $grp aio_stat "...", sub {
117	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)	485	$_[0] or return $grp->result ("error");
118	Reads or writes "length" bytes from the specified "fh" and "offset"
119	into the scalar given by "data" and offset "dataoffset" and calls
120	the callback without the actual number of bytes read (or -1 on
121	error, just like the syscall).
122		486
123	The $data scalar MUST NOT be modified in any way while the request	487	# add another request dynamically, if first succeeded
124	is outstanding. Modifying it can result in segfaults or WW3 (if the	488	add $grp aio_open "...", sub {
125	necessary/optional hardware is installed).	489	$grp->result ("ok");
126		490	};
127	Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
128	offset 0 within the scalar:
129
130	aio_read $fh, 7, 15, $buffer, 0, sub {
131	$_[0] > 0 or die "read error: $!";
132	print "read $_[0] bytes: <$buffer>\n";
133	};	491	};
134		492
135	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)	493	This makes it very easy to create composite requests (see the source
136	Tries to copy $length bytes from $in_fh to $out_fh. It starts	494	of "aio_move" for an application) that work and feel like simple
137	reading at byte offset $in_offset, and starts writing at the current	495	requests.
138	file offset of $out_fh. Because of that, it is not safe to issue
139	more than one "aio_sendfile" per $out_fh, as they will interfere
140	with each other.
141		496
142	This call tries to make use of a native "sendfile" syscall to	497	* The IO::AIO::GRP objects will be cleaned up during calls to
143	provide zero-copy operation. For this to work, $out_fh should refer	498	"IO::AIO::poll_cb", just like any other request.
144	to a socket, and $in_fh should refer to mmap'able file.	499	* They can be canceled like any other request. Canceling will cancel
		500	not only the request itself, but also all requests it contains.
		501	* They can also can also be added to other IO::AIO::GRP objects.
		502	* You must not add requests to a group from within the group
		503	callback (or any later time).
145		504
146	If the native sendfile call fails or is not implemented, it will be	505	Their lifetime, simplified, looks like this: when they are empty,
147	emulated, so you can call "aio_sendfile" on any type of filehandle	506	they will finish very quickly. If they contain only requests that
148	regardless of the limitations of the operating system.	507	are in the "done" state, they will also finish. Otherwise they will
		508	continue to exist.
149		509
150	Please note, however, that "aio_sendfile" can read more bytes from	510	That means after creating a group you have some time to add
151	$in_fh than are written, and there is no way to find out how many	511	requests. And in the callbacks of those requests, you can add
152	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"	512	further requests to the group. And only when all those requests have
153	only provides the number of bytes written to $out_fh. Only if the	513	finished will the the group itself finish.
154	result value equals $length one can assume that $length bytes have
155	been read.
156		514
157	aio_readahead $fh,$offset,$length, $callback->($retval)	515	add $grp ...
158	"aio_readahead" populates the page cache with data from a file so	516	$grp->add (...)
159	that subsequent reads from that file will not block on disk I/O. The	517	Add one or more requests to the group. Any type of IO::AIO::REQ
160	$offset argument specifies the starting point from which data is to	518	can be added, including other groups, as long as you do not
161	be read and $length specifies the number of bytes to be read. I/O is	519	create circular dependencies.
162	performed in whole pages, so that offset is effectively rounded down
163	to a page boundary and bytes are read up to the next page boundary
164	greater than or equal to (off-set+length). "aio_readahead" does not
165	read beyond the end of the file. The current file offset of the file
166	is left unchanged.
167		520
168	If that syscall doesn't exist (likely if your OS isn't Linux) it	521	Returns all its arguments.
169	will be emulated by simply reading the data, which would have a
170	similar effect.
171		522
172	aio_stat $fh_or_path, $callback->($status)	523	$grp->cancel_subs
173	aio_lstat $fh, $callback->($status)	524	Cancel all subrequests and clears any feeder, but not the group
174	Works like perl's "stat" or "lstat" in void context. The callback	525	request itself. Useful when you queued a lot of events but got a
175	will be called after the stat and the results will be available
176	using "stat _" or "-s _" etc...
177
178	The pathname passed to "aio_stat" must be absolute. See API NOTES,
179	above, for an explanation.
180
181	Currently, the stats are always 64-bit-stats, i.e. instead of
182	returning an error when stat'ing a large file, the results will be
183	silently truncated unless perl itself is compiled with large file
184	support.
185
186	Example: Print the length of /etc/passwd:
187
188	aio_stat "/etc/passwd", sub {
189	$_[0] and die "stat failed: $!";
190	print "size is ", -s _, "\n";
191	};
192
193	aio_unlink $pathname, $callback->($status)
194	Asynchronously unlink (delete) a file and call the callback with the
195	result code.	526	result early.
196		527
197	aio_rmdir $pathname, $callback->($status)	528	$grp->result (...)
198	Asynchronously rmdir (delete) a directory and call the callback with	529	Set the result value(s) that will be passed to the group
199	the result code.	530	callback when all subrequests have finished and set thre groups
		531	errno to the current value of errno (just like calling "errno"
		532	without an error number). By default, no argument will be passed
		533	and errno is zero.
200		534
201	aio_readdir $pathname $callback->($entries)	535	$grp->errno ([$errno])
202	Unlike the POSIX call of the same name, "aio_readdir" reads an	536	Sets the group errno value to $errno, or the current value of
203	entire directory (i.e. opendir + readdir + closedir). The entries	537	errno when the argument is missing.
204	will not be sorted, and will NOT include the "." and ".." entries.
205		538
206	The callback a single argument which is either "undef" or an	539	Every aio request has an associated errno value that is restored
207	array-ref with the filenames.	540	when the callback is invoked. This method lets you change this
		541	value from its default (0).
208		542
209	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)	543	Calling "result" will also set errno, so make sure you either
210	Scans a directory (similar to "aio_readdir") and tries to separate	544	set $! before the call to "result", or call c<errno> after it.
211	the entries of directory $path into two sets of names, ones you can
212	recurse into (directories), and ones you cannot recurse into
213	(everything else).
214		545
215	"aio_scandir" is a composite request that consists of many	546	feed $grp $callback->($grp)
216	aio-primitives. $maxreq specifies the maximum number of outstanding	547	Sets a feeder/generator on this group: every group can have an
217	aio requests that this function generates. If it is "<= 0", then a	548	attached generator that generates requests if idle. The idea
218	suitable default will be chosen (currently 8).	549	behind this is that, although you could just queue as many
		550	requests as you want in a group, this might starve other
		551	requests for a potentially long time. For example, "aio_scandir"
		552	might generate hundreds of thousands "aio_stat" requests,
		553	delaying any later requests for a long time.
219		554
220	On error, the callback is called without arguments, otherwise it	555	To avoid this, and allow incremental generation of requests, you
221	receives two array-refs with path-relative entry names.	556	can instead a group and set a feeder on it that generates those
		557	requests. The feed callback will be called whenever there are
		558	few enough (see "limit", below) requests active in the group
		559	itself and is expected to queue more requests.
222		560
		561	The feed callback can queue as many requests as it likes (i.e.
		562	"add" does not impose any limits).
		563
		564	If the feed does not queue more requests when called, it will be
		565	automatically removed from the group.
		566
		567	If the feed limit is 0, it will be set to 2 automatically.
		568
223	Example:	569	Example:
224		570
225	aio_scandir $dir, 0, sub {	571	# stat all files in @files, but only ever use four aio requests concurrently:
226	my ($dirs, $nondirs) = @_;	572
227	print "real directories: @$dirs\n";	573	my $grp = aio_group sub { print "finished\n" };
228	print "everything else: @$nondirs\n";	574	limit $grp 4;
		575	feed $grp sub {
		576	my $file = pop @files
		577	or return;
		578
		579	add $grp aio_stat $file, sub { ... };
229	};	580	};
230		581
231	Implementation notes.	582	limit $grp $num
		583	Sets the feeder limit for the group: The feeder will be called
		584	whenever the group contains less than this many requests.
232		585
233	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry	586	Setting the limit to 0 will pause the feeding process.
234	can.
235
236	After reading the directory, the modification time, size etc. of the
237	directory before and after the readdir is checked, and if they
238	match, the link count will be used to decide how many entries are
239	directories (if >= 2). Otherwise, no knowledge of the number of
240	subdirectories will be assumed.
241
242	Then entires will be sorted into likely directories (everything
243	without a non-initial dot) and likely non-directories (everything
244	else). Then every entry + "/." will be "stat"'ed, likely directories
245	first. This is often faster because filesystems might detect the
246	type of the entry without reading the inode data (e.g. ext2s
247	filetype feature). If that succeeds, it assumes that the entry is a
248	directory or a symlink to directory (which will be checked
249	seperately).
250
251	If the known number of directories has been reached, the rest of the
252	entries is assumed to be non-directories.
253
254	aio_fsync $fh, $callback->($status)
255	Asynchronously call fsync on the given filehandle and call the
256	callback with the fsync result code.
257
258	aio_fdatasync $fh, $callback->($status)
259	Asynchronously call fdatasync on the given filehandle and call the
260	callback with the fdatasync result code.
261
262	If this call isn't available because your OS lacks it or it couldn't
263	be detected, it will be emulated by calling "fsync" instead.
264		587
265	SUPPORT FUNCTIONS	588	SUPPORT FUNCTIONS
266	$fileno = IO::AIO::poll_fileno	589	$fileno = IO::AIO::poll_fileno
267	Return the request result pipe file descriptor. This filehandle	590	Return the request result pipe file descriptor. This
268	must be polled for reading by some mechanism outside this module	591	filehandle must be polled for reading by some mechanism outside
269	(e.g. Event or select, see below or the SYNOPSIS). If the pipe	592	this module (e.g. Event or select, see below or the SYNOPSIS).
270	becomes readable you have to call "poll_cb" to check the results.	593	If the pipe becomes readable you have to call "poll_cb" to check
		594	the results.
271		595
272	See "poll_cb" for an example.	596	See "poll_cb" for an example.
273		597
274	IO::AIO::poll_cb	598	IO::AIO::poll_cb
275	Process all outstanding events on the result pipe. You have to call	599	Process all outstanding events on the result pipe. You have to
276	this regularly. Returns the number of events processed. Returns	600	call this regularly. Returns the number of events processed.
277	immediately when no events are outstanding.	601	Returns immediately when no events are outstanding.
278		602
		603	If not all requests were processed for whatever reason, the
		604	filehandle will still be ready when "poll_cb" returns.
		605
279	Example: Install an Event watcher that automatically calls	606	Example: Install an Event watcher that automatically calls
280	IO::AIO::poll_cb with high priority:	607	IO::AIO::poll_cb with high priority:
281		608
282	Event->io (fd => IO::AIO::poll_fileno,	609	Event->io (fd => IO::AIO::poll_fileno,
283	poll => 'r', async => 1,	610	poll => 'r', async => 1,
284	cb => \&IO::AIO::poll_cb);	611	cb => \&IO::AIO::poll_cb);
285		612
		613	IO::AIO::poll_some $max_requests
		614	Similar to "poll_cb", but only processes up to $max_requests
		615	requests at a time.
		616
		617	Useful if you want to ensure some level of interactiveness when
		618	perl is not fast enough to process all requests in time.
		619
		620	Example: Install an Event watcher that automatically calls
		621	IO::AIO::poll_some with low priority, to ensure that other parts
		622	of the program get the CPU sometimes even under high AIO load.
		623
		624	Event->io (fd => IO::AIO::poll_fileno,
		625	poll => 'r', nice => 1,
		626	cb => sub { IO::AIO::poll_some 256 });
		627
286	IO::AIO::poll_wait	628	IO::AIO::poll_wait
287	Wait till the result filehandle becomes ready for reading (simply	629	Wait till the result filehandle becomes ready for reading
288	does a "select" on the filehandle. This is useful if you want to	630	(simply does a "select" on the filehandle. This is useful if you
289	synchronously wait for some requests to finish).	631	want to synchronously wait for some requests to finish).
290		632
291	See "nreqs" for an example.	633	See "nreqs" for an example.
292		634
293	IO::AIO::nreqs	635	IO::AIO::nreqs
294	Returns the number of requests currently outstanding (i.e. for which	636	Returns the number of requests currently in the ready, execute
295	their callback has not been invoked yet).	637	or pending states (i.e. for which their callback has not been
		638	invoked yet).
296		639
297	Example: wait till there are no outstanding requests anymore:	640	Example: wait till there are no outstanding requests anymore:
298		641
299	IO::AIO::poll_wait, IO::AIO::poll_cb	642	IO::AIO::poll_wait, IO::AIO::poll_cb
300	while IO::AIO::nreqs;	643	while IO::AIO::nreqs;
301		644
		645	IO::AIO::nready
		646	Returns the number of requests currently in the ready state (not
		647	yet executed).
		648
		649	IO::AIO::npending
		650	Returns the number of requests currently in the pending state
		651	(executed, but not yet processed by poll_cb).
		652
302	IO::AIO::flush	653	IO::AIO::flush
303	Wait till all outstanding AIO requests have been handled.	654	Wait till all outstanding AIO requests have been handled.
304		655
305	Strictly equivalent to:	656	Strictly equivalent to:
306		657
307	IO::AIO::poll_wait, IO::AIO::poll_cb	658	IO::AIO::poll_wait, IO::AIO::poll_cb
308	while IO::AIO::nreqs;	659	while IO::AIO::nreqs;
309		660
310	IO::AIO::poll	661	IO::AIO::poll
311	Waits until some requests have been handled.	662	Waits until some requests have been handled.
312		663
313	Strictly equivalent to:	664	Strictly equivalent to:
314		665
315	IO::AIO::poll_wait, IO::AIO::poll_cb	666	IO::AIO::poll_wait, IO::AIO::poll_cb
316	if IO::AIO::nreqs;	667	if IO::AIO::nreqs;
317		668
318	IO::AIO::min_parallel $nthreads	669	IO::AIO::min_parallel $nthreads
319	Set the minimum number of AIO threads to $nthreads. The current	670	Set the minimum number of AIO threads to $nthreads. The current
320	default is 4, which means four asynchronous operations can be done	671	default is 8, which means eight asynchronous operations can
321	at one time (the number of outstanding operations, however, is	672	execute concurrently at any one time (the number of outstanding
322	unlimited).	673	requests, however, is unlimited).
323		674
324	IO::AIO starts threads only on demand, when an AIO request is queued	675	IO::AIO starts threads only on demand, when an AIO request is
325	and no free thread exists.	676	queued and no free thread exists.
326		677
327	It is recommended to keep the number of threads low, as some Linux	678	It is recommended to keep the number of threads relatively low,
328	kernel versions will scale negatively with the number of threads	679	as some Linux kernel versions will scale negatively with the
329	(higher parallelity => MUCH higher latency). With current Linux 2.6	680	number of threads (higher parallelity => MUCH higher latency).
330	versions, 4-32 threads should be fine.	681	With current Linux 2.6 versions, 4-32 threads should be fine.
331		682
332	Under most circumstances you don't need to call this function, as	683	Under most circumstances you don't need to call this function,
333	the module selects a default that is suitable for low to moderate	684	as the module selects a default that is suitable for low to
334	load.	685	moderate load.
335		686
336	IO::AIO::max_parallel $nthreads	687	IO::AIO::max_parallel $nthreads
337	Sets the maximum number of AIO threads to $nthreads. If more than	688	Sets the maximum number of AIO threads to $nthreads. If more
338	the specified number of threads are currently running, this function	689	than the specified number of threads are currently running, this
339	kills them. This function blocks until the limit is reached.	690	function kills them. This function blocks until the limit is
		691	reached.
340		692
341	While $nthreads are zero, aio requests get queued but not executed	693	While $nthreads are zero, aio requests get queued but not
342	until the number of threads has been increased again.	694	executed until the number of threads has been increased again.
343		695
344	This module automatically runs "max_parallel 0" at program end, to	696	This module automatically runs "max_parallel 0" at program end,
345	ensure that all threads are killed and that there are no outstanding	697	to ensure that all threads are killed and that there are no
346	requests.	698	outstanding requests.
347		699
348	Under normal circumstances you don't need to call this function.	700	Under normal circumstances you don't need to call this function.
349		701
350	$oldnreqs = IO::AIO::max_outstanding $nreqs	702	$oldmaxreqs = IO::AIO::max_outstanding $maxreqs
		703	This is a very bad function to use in interactive programs
		704	because it blocks, and a bad way to reduce concurrency because
		705	it is inexact: Better use an "aio_group" together with a feed
		706	callback.
		707
351	Sets the maximum number of outstanding requests to $nreqs. If you	708	Sets the maximum number of outstanding requests to $nreqs. If
352	try to queue up more than this number of requests, the caller will	709	you to queue up more than this number of requests, the next call
353	block until some requests have been handled.	710	to the "poll_cb" (and "poll_some" and other functions calling
		711	"poll_cb") function will block until the limit is no longer
		712	exceeded.
354		713
355	The default is very large, so normally there is no practical limit.	714	The default value is very large, so there is no practical limit
356	If you queue up many requests in a loop it often improves speed if	715	on the number of outstanding requests.
357	you set this to a relatively low number, such as 100.
358		716
359	Under normal circumstances you don't need to call this function.	717	You can still queue as many requests as you want. Therefore,
		718	"max_oustsanding" is mainly useful in simple scripts (with low
		719	values) or as a stop gap to shield against fatal memory overflow
		720	(with large values).
360		721
361	FORK BEHAVIOUR	722	FORK BEHAVIOUR
		723	This module should do "the right thing" when the process using it
		724	forks:
		725
362	Before the fork, IO::AIO enters a quiescent state where no requests can	726	Before the fork, IO::AIO enters a quiescent state where no requests
363	be added in other threads and no results will be processed. After the	727	can be added in other threads and no results will be processed.
364	fork the parent simply leaves the quiescent state and continues	728	After the fork the parent simply leaves the quiescent state and
365	request/result processing, while the child clears the request/result	729	continues request/result processing, while the child frees the
366	queue (so the requests started before the fork will only be handled in	730	request/result queue (so that the requests started before the fork
367	the parent). Threats will be started on demand until the limit ste in	731	will only be handled in the parent). Threads will be started on
368	the parent process has been reached again.	732	demand until the limit set in the parent process has been reached
		733	again.
		734
		735	In short: the parent will, after a short pause, continue as if fork
		736	had not been called, while the child will act as if IO::AIO has not
		737	been used yet.
		738
		739	MEMORY USAGE
		740	Per-request usage:
		741
		742	Each aio request uses - depending on your architecture - around
		743	100-200 bytes of memory. In addition, stat requests need a stat
		744	buffer (possibly a few hundred bytes), readdir requires a result
		745	buffer and so on. Perl scalars and other data passed into aio
		746	requests will also be locked and will consume memory till the
		747	request has entered the done state.
		748
		749	This is now awfully much, so queuing lots of requests is not usually
		750	a problem.
		751
		752	Per-thread usage:
		753
		754	In the execution phase, some aio requests require more memory for
		755	temporary buffers, and each thread requires a stack and other data
		756	structures (usually around 16k-128k, depending on the OS).
		757
		758	KNOWN BUGS
		759	Known bugs will be fixed in the next release.
369		760
370	SEE ALSO	761	SEE ALSO
371	Coro, Linux::AIO.	762	Coro::AIO.
372		763
373	AUTHOR	764	AUTHOR
374	Marc Lehmann <schmorp@schmorp.de>	765	Marc Lehmann <schmorp@schmorp.de>
375	http://home.schmorp.de/	766	http://home.schmorp.de/
376		767

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.13 by root, Tue Aug 30 15:45:10 2005 UTC vs. Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.13 by root, Tue Aug 30 15:45:10 2005 UTC vs.
Revision 1.18 by root, Thu Oct 26 16:28:33 2006 UTC