[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.4 by root, Sun Jul 10 21:04:46 2005 UTC vs.
Revision 1.28 by root, Sun Mar 30 06:31:49 2008 UTC

…		…
1	NAME	1	NAME
2	IO::AIO - Asynchronous Input/Output	2	IO::AIO - Asynchronous Input/Output
3		3
4	SYNOPSIS	4	SYNOPSIS
5	use IO::AIO;	5	use IO::AIO;
		6
		7	aio_open "/etc/passwd", O_RDONLY, 0, sub {
		8	my $fh = shift
		9	or die "/etc/passwd: $!";
		10	...
		11	};
		12
		13	aio_unlink "/tmp/file", sub { };
		14
		15	aio_read $fh, 30000, 1024, $buffer, 0, sub {
		16	$_[0] > 0 or die "read error: $!";
		17	};
		18
		19	# version 2+ has request and group objects
		20	use IO::AIO 2;
		21
		22	aioreq_pri 4; # give next request a very high priority
		23	my $req = aio_unlink "/tmp/file", sub { };
		24	$req->cancel; # cancel request if still in queue
		25
		26	my $grp = aio_group sub { print "all stats done\n" };
		27	add $grp aio_stat "..." for ...;
		28
		29	# AnyEvent integration (EV, Event, Glib, Tk, urxvt, pureperl...)
		30	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
		31	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
		32
		33	# EV integration
		34	my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
		35
		36	# Event integration
		37	Event->io (fd => IO::AIO::poll_fileno,
		38	poll => 'r',
		39	cb => \&IO::AIO::poll_cb);
		40
		41	# Glib/Gtk2 integration
		42	add_watch Glib::IO IO::AIO::poll_fileno,
		43	in => sub { IO::AIO::poll_cb; 1 };
		44
		45	# Tk integration
		46	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
		47	readable => \&IO::AIO::poll_cb);
		48
		49	# Danga::Socket integration
		50	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
		51	\&IO::AIO::poll_cb);
6		52
7	DESCRIPTION	53	DESCRIPTION
8	This module implements asynchronous I/O using whatever means your	54	This module implements asynchronous I/O using whatever means your
9	operating system supports.	55	operating system supports.
10		56
		57	Asynchronous means that operations that can normally block your program
		58	(e.g. reading from disk) will be done asynchronously: the operation will
		59	still block, but you can do something else in the meantime. This is
		60	extremely useful for programs that need to stay interactive even when
		61	doing heavy I/O (GUI programs, high performance network servers etc.),
		62	but can also be used to easily do operations in parallel that are
		63	normally done sequentially, e.g. stat'ing many files, which is much
		64	faster on a RAID volume or over NFS when you do a number of stat
		65	operations concurrently.
		66
		67	While most of this works on all types of file descriptors (for example
		68	sockets), using these functions on file descriptors that support
		69	nonblocking operation (again, sockets, pipes etc.) is very inefficient.
		70	Use an event loop for that (such as the Event module): IO::AIO will
		71	naturally fit into such an event loop itself.
		72
11	Currently, a number of threads are started that execute your read/writes	73	In this version, a number of threads are started that execute your
12	and signal their completion. You don't need thread support in your libc	74	requests and signal their completion. You don't need thread support in
13	or perl, and the threads created by this module will not be visible to	75	perl, and the threads created by this module will not be visible to
14	the pthreads library. In the future, this module might make use of the	76	perl. In the future, this module might make use of the native aio
15	native aio functions available on many operating systems. However, they	77	functions available on many operating systems. However, they are often
16	are often not well-supported (Linux doesn't allow them on normal files	78	not well-supported or restricted (GNU/Linux doesn't allow them on normal
17	currently, for example), and they would only support aio_read and	79	files currently, for example), and they would only support aio_read and
18	aio_write, so the remaining functionality would have to be implemented	80	aio_write, so the remaining functionality would have to be implemented
19	using threads anyway.	81	using threads anyway.
20		82
21	Although the module will work with in the presence of other threads, it	83	Although the module will work in the presence of other (Perl-) threads,
22	is currently not reentrant, so use appropriate locking yourself.	84	it is currently not reentrant in any way, so use appropriate locking
		85	yourself, always call "poll_cb" from within the same thread, or never
		86	call "poll_cb" (or other "aio_" functions) recursively.
		87
		88	EXAMPLE
		89	This is a simple example that uses the Event module and loads
		90	/etc/passwd asynchronously:
		91
		92	use Fcntl;
		93	use Event;
		94	use IO::AIO;
		95
		96	# register the IO::AIO callback with Event
		97	Event->io (fd => IO::AIO::poll_fileno,
		98	poll => 'r',
		99	cb => \&IO::AIO::poll_cb);
		100
		101	# queue the request to open /etc/passwd
		102	aio_open "/etc/passwd", O_RDONLY, 0, sub {
		103	my $fh = shift
		104	or die "error while opening: $!";
		105
		106	# stat'ing filehandles is generally non-blocking
		107	my $size = -s $fh;
		108
		109	# queue a request to read the file
		110	my $contents;
		111	aio_read $fh, 0, $size, $contents, 0, sub {
		112	$_[0] == $size
		113	or die "short read: $!";
		114
		115	close $fh;
		116
		117	# file contents now in $contents
		118	print $contents;
		119
		120	# exit event loop and program
		121	Event::unloop;
		122	};
		123	};
		124
		125	# possibly queue up other requests, or open GUI windows,
		126	# check for sockets etc. etc.
		127
		128	# process events as long as there are some:
		129	Event::loop;
		130
		131	REQUEST ANATOMY AND LIFETIME
		132	Every "aio_*" function creates a request. which is a C data structure
		133	not directly visible to Perl.
		134
		135	If called in non-void context, every request function returns a Perl
		136	object representing the request. In void context, nothing is returned,
		137	which saves a bit of memory.
		138
		139	The perl object is a fairly standard ref-to-hash object. The hash
		140	contents are not used by IO::AIO so you are free to store anything you
		141	like in it.
		142
		143	During their existance, aio requests travel through the following
		144	states, in order:
		145
		146	ready
		147	Immediately after a request is created it is put into the ready
		148	state, waiting for a thread to execute it.
		149
		150	execute
		151	A thread has accepted the request for processing and is currently
		152	executing it (e.g. blocking in read).
		153
		154	pending
		155	The request has been executed and is waiting for result processing.
		156
		157	While request submission and execution is fully asynchronous, result
		158	processing is not and relies on the perl interpreter calling
		159	"poll_cb" (or another function with the same effect).
		160
		161	result
		162	The request results are processed synchronously by "poll_cb".
		163
		164	The "poll_cb" function will process all outstanding aio requests by
		165	calling their callbacks, freeing memory associated with them and
		166	managing any groups they are contained in.
		167
		168	done
		169	Request has reached the end of its lifetime and holds no resources
		170	anymore (except possibly for the Perl object, but its connection to
		171	the actual aio request is severed and calling its methods will
		172	either do nothing or result in a runtime error).
23		173
24	FUNCTIONS	174	FUNCTIONS
25	AIO FUNCTIONS	175	AIO REQUEST FUNCTIONS
26	All the "aio_*" calls are more or less thin wrappers around the syscall	176	All the "aio_*" calls are more or less thin wrappers around the syscall
27	with the same name (sans "aio_"). The arguments are similar or	177	with the same name (sans "aio_"). The arguments are similar or
28	identical, and they all accept an additional $callback argument which	178	identical, and they all accept an additional (and optional) $callback
29	must be a code reference. This code reference will get called with the	179	argument which must be a code reference. This code reference will get
30	syscall return code (e.g. most syscalls return -1 on error, unlike perl,	180	called with the syscall return code (e.g. most syscalls return -1 on
31	which usually delivers "false") as it's sole argument when the given	181	error, unlike perl, which usually delivers "false") as it's sole
32	syscall has been executed asynchronously.	182	argument when the given syscall has been executed asynchronously.
33		183
34	All functions that expect a filehandle will also accept a file	184	All functions expecting a filehandle keep a copy of the filehandle
35	descriptor.	185	internally until the request has finished.
36		186
		187	All functions return request objects of type IO::AIO::REQ that allow
		188	further manipulation of those requests while they are in-flight.
		189
37	The filenames you pass to these routines must be absolute. The reason	190	The pathnames you pass to these routines must be absolute and encoded
38	is that at the time the request is being executed, the current working	191	as octets. The reason for the former is that at the time the request is
39	directory could have changed. Alternatively, you can make sure that you	192	being executed, the current working directory could have changed.
40	never change the current working directory.	193	Alternatively, you can make sure that you never change the current
		194	working directory anywhere in the program and then use relative paths.
41		195
		196	To encode pathnames as octets, either make sure you either: a) always
		197	pass in filenames you got from outside (command line, readdir etc.)
		198	without tinkering, b) are ASCII or ISO 8859-1, c) use the Encode module
		199	and encode your pathnames to the locale (or other) encoding in effect in
		200	the user environment, d) use Glib::filename_from_unicode on unicode
		201	filenames or e) use something else to ensure your scalar has the correct
		202	contents.
		203
		204	This works, btw. independent of the internal UTF-8 bit, which IO::AIO
		205	handles correctly wether it is set or not.
		206
		207	$prev_pri = aioreq_pri [$pri]
		208	Returns the priority value that would be used for the next request
		209	and, if $pri is given, sets the priority for the next aio request.
		210
		211	The default priority is 0, the minimum and maximum priorities are -4
		212	and 4, respectively. Requests with higher priority will be serviced
		213	first.
		214
		215	The priority will be reset to 0 after each call to one of the
		216	"aio_*" functions.
		217
		218	Example: open a file with low priority, then read something from it
		219	with higher priority so the read request is serviced before other
		220	low priority open requests (potentially spamming the cache):
		221
		222	aioreq_pri -3;
		223	aio_open ..., sub {
		224	return unless $_[0];
		225
		226	aioreq_pri -2;
		227	aio_read $_[0], ..., sub {
		228	...
		229	};
		230	};
		231
		232	aioreq_nice $pri_adjust
		233	Similar to "aioreq_pri", but subtracts the given value from the
		234	current priority, so the effect is cumulative.
		235
42	aio_open $pathname, $flags, $mode, $callback	236	aio_open $pathname, $flags, $mode, $callback->($fh)
43	Asynchronously open or create a file and call the callback with a	237	Asynchronously open or create a file and call the callback with a
44	newly created filehandle for the file.	238	newly created filehandle for the file.
45		239
46	The pathname passed to "aio_open" must be absolute. See API NOTES,	240	The pathname passed to "aio_open" must be absolute. See API NOTES,
47	above, for an explanation.	241	above, for an explanation.
48		242
49	The $mode argument is a bitmask. See the "Fcntl" module for a list.	243	The $flags argument is a bitmask. See the "Fcntl" module for a list.
50	They are the same as used in "sysopen".	244	They are the same as used by "sysopen".
		245
		246	Likewise, $mode specifies the mode of the newly created file, if it
		247	didn't exist and "O_CREAT" has been given, just like perl's
		248	"sysopen", except that it is mandatory (i.e. use 0 if you don't
		249	create new files, and 0666 or 0777 if you do). Note that the $mode
		250	will be modified by the umask in effect then the request is being
		251	executed, so better never change the umask.
51		252
52	Example:	253	Example:
53		254
54	aio_open "/etc/passwd", O_RDONLY, 0, sub {	255	aio_open "/etc/passwd", O_RDONLY, 0, sub {
55	if ($_[0]) {	256	if ($_[0]) {
58	} else {	259	} else {
59	die "open failed: $!\n";	260	die "open failed: $!\n";
60	}	261	}
61	};	262	};
62		263
63	aio_close $fh, $callback	264	aio_close $fh, $callback->($status)
64	Asynchronously close a file and call the callback with the result	265	Asynchronously close a file and call the callback with the result
65	code. WARNING: although accepted, you should not pass in a perl	266	code.
66	filehandle here, as perl will likely close the file descriptor
67	itself when the filehandle is destroyed. Normally, you can safely
68	call perls "close" or just let filehandles go out of scope.
69		267
		268	Unfortunately, you can't do this to perl. Perl insists very
		269	strongly on closing the file descriptor associated with the
		270	filehandle itself. Here is what aio_close will try:
		271
		272	1. dup()licate the fd
		273	2. asynchronously close() the duplicated fd
		274	3. dup()licate the fd once more
		275	4. let perl close() the filehandle
		276	5. asynchronously close the duplicated fd
		277
		278	The idea is that the first close() flushes stuff to disk that
		279	closing an fd will flush, so when perl closes the fd, nothing much
		280	will need to be flushed. The second async. close() will then flush
		281	stuff to disk that closing the last fd to the file will flush.
		282
		283	Just FYI, SuSv3 has this to say on close:
		284
		285	All outstanding record locks owned by the process on the file
		286	associated with the file descriptor shall be removed.
		287
		288	If fildes refers to a socket, close() shall cause the socket to be
		289	destroyed. ... close() shall block for up to the current linger
		290	interval until all data is transmitted.
		291	[this actually sounds like a specification bug, but who knows]
		292
		293	And at least Linux additionally actually flushes stuff on every
		294	close, even when the file itself is still open.
		295
		296	Sounds enourmously inefficient and complicated? Yes... please show
		297	me how to nuke perl's fd out of existence...
		298
70	aio_read $fh,$offset,$length, $data,$dataoffset,$callback	299	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
71	aio_write $fh,$offset,$length, $data,$dataoffset,$callback	300	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
72	Reads or writes "length" bytes from the specified "fh" and "offset"	301	Reads or writes $length bytes from the specified $fh and $offset
73	into the scalar given by "data" and offset "dataoffset" and calls	302	into the scalar given by $data and offset $dataoffset and calls the
74	the callback without the actual number of bytes read (or -1 on	303	callback without the actual number of bytes read (or -1 on error,
75	error, just like the syscall).	304	just like the syscall).
76		305
		306	If $offset is undefined, then the current file descriptor offset
		307	will be used (and updated), otherwise the file descriptor offset
		308	will not be changed by these calls.
		309
		310	If $length is undefined in "aio_write", use the remaining length of
		311	$data.
		312
		313	If $dataoffset is less than zero, it will be counted from the end of
		314	$data.
		315
		316	The $data scalar MUST NOT be modified in any way while the request
		317	is outstanding. Modifying it can result in segfaults or World War
		318	III (if the necessary/optional hardware is installed).
		319
77	Example: Read 15 bytes at offset 7 into scalar $buffer, strating at	320	Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
78	offset 0 within the scalar:	321	offset 0 within the scalar:
79		322
80	aio_read $fh, 7, 15, $buffer, 0, sub {	323	aio_read $fh, 7, 15, $buffer, 0, sub {
81	$_[0] >= 0 or die "read error: $!";	324	$_[0] > 0 or die "read error: $!";
82	print "read <$buffer>\n";	325	print "read $_[0] bytes: <$buffer>\n";
83	};	326	};
84		327
		328	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
		329	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		330	reading at byte offset $in_offset, and starts writing at the current
		331	file offset of $out_fh. Because of that, it is not safe to issue
		332	more than one "aio_sendfile" per $out_fh, as they will interfere
		333	with each other.
		334
		335	This call tries to make use of a native "sendfile" syscall to
		336	provide zero-copy operation. For this to work, $out_fh should refer
		337	to a socket, and $in_fh should refer to mmap'able file.
		338
		339	If the native sendfile call fails or is not implemented, it will be
		340	emulated, so you can call "aio_sendfile" on any type of filehandle
		341	regardless of the limitations of the operating system.
		342
		343	Please note, however, that "aio_sendfile" can read more bytes from
		344	$in_fh than are written, and there is no way to find out how many
		345	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
		346	only provides the number of bytes written to $out_fh. Only if the
		347	result value equals $length one can assume that $length bytes have
		348	been read.
		349
85	aio_readahead $fh,$offset,$length, $callback	350	aio_readahead $fh,$offset,$length, $callback->($retval)
86	Asynchronously reads the specified byte range into the page cache,
87	using the "readahead" syscall. If that syscall doesn't exist the
88	status will be -1 and $! is set to ENOSYS.
89
90	readahead() populates the page cache with data from a file so that	351	"aio_readahead" populates the page cache with data from a file so
91	subsequent reads from that file will not block on disk I/O. The	352	that subsequent reads from that file will not block on disk I/O. The
92	$offset argument specifies the starting point from which data is to	353	$offset argument specifies the starting point from which data is to
93	be read and $length specifies the number of bytes to be read. I/O is	354	be read and $length specifies the number of bytes to be read. I/O is
94	performed in whole pages, so that offset is effectively rounded down	355	performed in whole pages, so that offset is effectively rounded down
95	to a page boundary and bytes are read up to the next page boundary	356	to a page boundary and bytes are read up to the next page boundary
96	greater than or equal to (off-set+length). aio_readahead() does not	357	greater than or equal to (off-set+length). "aio_readahead" does not
97	read beyond the end of the file. The current file offset of the file	358	read beyond the end of the file. The current file offset of the file
98	is left unchanged.	359	is left unchanged.
99		360
		361	If that syscall doesn't exist (likely if your OS isn't Linux) it
		362	will be emulated by simply reading the data, which would have a
		363	similar effect.
		364
100	aio_stat $fh_or_path, $callback	365	aio_stat $fh_or_path, $callback->($status)
101	aio_lstat $fh, $callback	366	aio_lstat $fh, $callback->($status)
102	Works like perl's "stat" or "lstat" in void context. The callback	367	Works like perl's "stat" or "lstat" in void context. The callback
103	will be called after the stat and the results will be available	368	will be called after the stat and the results will be available
104	using "stat _" or "-s _" etc...	369	using "stat _" or "-s _" etc...
105		370
106	The pathname passed to "aio_stat" must be absolute. See API NOTES,	371	The pathname passed to "aio_stat" must be absolute. See API NOTES,
…		…
116	aio_stat "/etc/passwd", sub {	381	aio_stat "/etc/passwd", sub {
117	$_[0] and die "stat failed: $!";	382	$_[0] and die "stat failed: $!";
118	print "size is ", -s _, "\n";	383	print "size is ", -s _, "\n";
119	};	384	};
120		385
		386	aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
		387	Works like perl's "utime" function (including the special case of
		388	$atime and $mtime being undef). Fractional times are supported if
		389	the underlying syscalls support them.
		390
		391	When called with a pathname, uses utimes(2) if available, otherwise
		392	utime(2). If called on a file descriptor, uses futimes(2) if
		393	available, otherwise returns ENOSYS, so this is not portable.
		394
		395	Examples:
		396
		397	# set atime and mtime to current time (basically touch(1)):
		398	aio_utime "path", undef, undef;
		399	# set atime to current time and mtime to beginning of the epoch:
		400	aio_utime "path", time, undef; # undef==0
		401
		402	aio_chown $fh_or_path, $uid, $gid, $callback->($status)
		403	Works like perl's "chown" function, except that "undef" for either
		404	$uid or $gid is being interpreted as "do not change" (but -1 can
		405	also be used).
		406
		407	Examples:
		408
		409	# same as "chown root path" in the shell:
		410	aio_chown "path", 0, -1;
		411	# same as above:
		412	aio_chown "path", 0, undef;
		413
		414	aio_truncate $fh_or_path, $offset, $callback->($status)
		415	Works like truncate(2) or ftruncate(2).
		416
		417	aio_chmod $fh_or_path, $mode, $callback->($status)
		418	Works like perl's "chmod" function.
		419
121	aio_unlink $pathname, $callback	420	aio_unlink $pathname, $callback->($status)
122	Asynchronously unlink (delete) a file and call the callback with the	421	Asynchronously unlink (delete) a file and call the callback with the
123	result code.	422	result code.
124		423
		424	aio_mknod $path, $mode, $dev, $callback->($status)
		425	[EXPERIMENTAL]
		426
		427	Asynchronously create a device node (or fifo). See mknod(2).
		428
		429	The only (POSIX-) portable way of calling this function is:
		430
		431	aio_mknod $path, IO::AIO::S_IFIFO \| $mode, 0, sub { ...
		432
		433	aio_link $srcpath, $dstpath, $callback->($status)
		434	Asynchronously create a new link to the existing object at $srcpath
		435	at the path $dstpath and call the callback with the result code.
		436
		437	aio_symlink $srcpath, $dstpath, $callback->($status)
		438	Asynchronously create a new symbolic link to the existing object at
		439	$srcpath at the path $dstpath and call the callback with the result
		440	code.
		441
		442	aio_readlink $path, $callback->($link)
		443	Asynchronously read the symlink specified by $path and pass it to
		444	the callback. If an error occurs, nothing or undef gets passed to
		445	the callback.
		446
		447	aio_rename $srcpath, $dstpath, $callback->($status)
		448	Asynchronously rename the object at $srcpath to $dstpath, just as
		449	rename(2) and call the callback with the result code.
		450
		451	aio_mkdir $pathname, $mode, $callback->($status)
		452	Asynchronously mkdir (create) a directory and call the callback with
		453	the result code. $mode will be modified by the umask at the time the
		454	request is executed, so do not change your umask.
		455
		456	aio_rmdir $pathname, $callback->($status)
		457	Asynchronously rmdir (delete) a directory and call the callback with
		458	the result code.
		459
		460	aio_readdir $pathname, $callback->($entries)
		461	Unlike the POSIX call of the same name, "aio_readdir" reads an
		462	entire directory (i.e. opendir + readdir + closedir). The entries
		463	will not be sorted, and will NOT include the "." and ".." entries.
		464
		465	The callback a single argument which is either "undef" or an
		466	array-ref with the filenames.
		467
		468	aio_load $path, $data, $callback->($status)
		469	This is a composite request that tries to fully load the given file
		470	into memory. Status is the same as with aio_read.
		471
		472	aio_copy $srcpath, $dstpath, $callback->($status)
		473	Try to copy the file (directories not supported as either source
		474	or destination) from $srcpath to $dstpath and call the callback with
		475	the 0 (error) or -1 ok.
		476
		477	This is a composite request that it creates the destination file
		478	with mode 0200 and copies the contents of the source file into it
		479	using "aio_sendfile", followed by restoring atime, mtime, access
		480	mode and uid/gid, in that order.
		481
		482	If an error occurs, the partial destination file will be unlinked,
		483	if possible, except when setting atime, mtime, access mode and
		484	uid/gid, where errors are being ignored.
		485
		486	aio_move $srcpath, $dstpath, $callback->($status)
		487	Try to move the file (directories not supported as either source
		488	or destination) from $srcpath to $dstpath and call the callback with
		489	the 0 (error) or -1 ok.
		490
		491	This is a composite request that tries to rename(2) the file first.
		492	If rename files with "EXDEV", it copies the file with "aio_copy"
		493	and, if that is successful, unlinking the $srcpath.
		494
		495	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		496	Scans a directory (similar to "aio_readdir") but additionally tries
		497	to efficiently separate the entries of directory $path into two sets
		498	of names, directories you can recurse into (directories), and ones
		499	you cannot recurse into (everything else, including symlinks to
		500	directories).
		501
		502	"aio_scandir" is a composite request that creates of many sub
		503	requests_ $maxreq specifies the maximum number of outstanding aio
		504	requests that this function generates. If it is "<= 0", then a
		505	suitable default will be chosen (currently 4).
		506
		507	On error, the callback is called without arguments, otherwise it
		508	receives two array-refs with path-relative entry names.
		509
		510	Example:
		511
		512	aio_scandir $dir, 0, sub {
		513	my ($dirs, $nondirs) = @_;
		514	print "real directories: @$dirs\n";
		515	print "everything else: @$nondirs\n";
		516	};
		517
		518	Implementation notes.
		519
		520	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
		521	can.
		522
		523	After reading the directory, the modification time, size etc. of the
		524	directory before and after the readdir is checked, and if they match
		525	(and isn't the current time), the link count will be used to decide
		526	how many entries are directories (if >= 2). Otherwise, no knowledge
		527	of the number of subdirectories will be assumed.
		528
		529	Then entries will be sorted into likely directories (everything
		530	without a non-initial dot currently) and likely non-directories
		531	(everything else). Then every entry plus an appended "/." will be
		532	"stat"'ed, likely directories first. If that succeeds, it assumes
		533	that the entry is a directory or a symlink to directory (which will
		534	be checked seperately). This is often faster than stat'ing the entry
		535	itself because filesystems might detect the type of the entry
		536	without reading the inode data (e.g. ext2fs filetype feature).
		537
		538	If the known number of directories (link count - 2) has been
		539	reached, the rest of the entries is assumed to be non-directories.
		540
		541	This only works with certainty on POSIX (= UNIX) filesystems, which
		542	fortunately are the vast majority of filesystems around.
		543
		544	It will also likely work on non-POSIX filesystems with reduced
		545	efficiency as those tend to return 0 or 1 as link counts, which
		546	disables the directory counting heuristic.
		547
		548	aio_rmtree $path, $callback->($status)
		549	Delete a directory tree starting (and including) $path, return the
		550	status of the final "rmdir" only. This is a composite request that
		551	uses "aio_scandir" to recurse into and rmdir directories, and unlink
		552	everything else.
		553
		554	aio_sync $callback->($status)
		555	Asynchronously call sync and call the callback when finished.
		556
125	aio_fsync $fh, $callback	557	aio_fsync $fh, $callback->($status)
126	Asynchronously call fsync on the given filehandle and call the	558	Asynchronously call fsync on the given filehandle and call the
127	callback with the fsync result code.	559	callback with the fsync result code.
128		560
129	aio_fdatasync $fh, $callback	561	aio_fdatasync $fh, $callback->($status)
130	Asynchronously call fdatasync on the given filehandle and call the	562	Asynchronously call fdatasync on the given filehandle and call the
131	callback with the fdatasync result code.	563	callback with the fdatasync result code.
132		564
		565	If this call isn't available because your OS lacks it or it couldn't
		566	be detected, it will be emulated by calling "fsync" instead.
		567
		568	aio_pathsync $path, $callback->($status)
		569	This request tries to open, fsync and close the given path. This is
		570	a composite request intended tosync directories after directory
		571	operations (E.g. rename). This might not work on all operating
		572	systems or have any specific effect, but usually it makes sure that
		573	directory changes get written to disc. It works for anything that
		574	can be opened for read-only, not just directories.
		575
		576	Passes 0 when everything went ok, and -1 on error.
		577
		578	aio_group $callback->(...)
		579	This is a very special aio request: Instead of doing something, it
		580	is a container for other aio requests, which is useful if you want
		581	to bundle many requests into a single, composite, request with a
		582	definite callback and the ability to cancel the whole request with
		583	its subrequests.
		584
		585	Returns an object of class IO::AIO::GRP. See its documentation below
		586	for more info.
		587
		588	Example:
		589
		590	my $grp = aio_group sub {
		591	print "all stats done\n";
		592	};
		593
		594	add $grp
		595	(aio_stat ...),
		596	(aio_stat ...),
		597	...;
		598
		599	aio_nop $callback->()
		600	This is a special request - it does nothing in itself and is only
		601	used for side effects, such as when you want to add a dummy request
		602	to a group so that finishing the requests in the group depends on
		603	executing the given code.
		604
		605	While this request does nothing, it still goes through the execution
		606	phase and still requires a worker thread. Thus, the callback will
		607	not be executed immediately but only after other requests in the
		608	queue have entered their execution phase. This can be used to
		609	measure request latency.
		610
		611	IO::AIO::aio_busy $fractional_seconds, $callback->() NOT EXPORTED
		612	Mainly used for debugging and benchmarking, this aio request puts
		613	one of the request workers to sleep for the given time.
		614
		615	While it is theoretically handy to have simple I/O scheduling
		616	requests like sleep and file handle readable/writable, the overhead
		617	this creates is immense (it blocks a thread for a long time) so do
		618	not use this function except to put your application under
		619	artificial I/O pressure.
		620
		621	IO::AIO::REQ CLASS
		622	All non-aggregate "aio_*" functions return an object of this class when
		623	called in non-void context.
		624
		625	cancel $req
		626	Cancels the request, if possible. Has the effect of skipping
		627	execution when entering the execute state and skipping calling the
		628	callback when entering the the result state, but will leave the
		629	request otherwise untouched. That means that requests that currently
		630	execute will not be stopped and resources held by the request will
		631	not be freed prematurely.
		632
		633	cb $req $callback->(...)
		634	Replace (or simply set) the callback registered to the request.
		635
		636	IO::AIO::GRP CLASS
		637	This class is a subclass of IO::AIO::REQ, so all its methods apply to
		638	objects of this class, too.
		639
		640	A IO::AIO::GRP object is a special request that can contain multiple
		641	other aio requests.
		642
		643	You create one by calling the "aio_group" constructing function with a
		644	callback that will be called when all contained requests have entered
		645	the "done" state:
		646
		647	my $grp = aio_group sub {
		648	print "all requests are done\n";
		649	};
		650
		651	You add requests by calling the "add" method with one or more
		652	"IO::AIO::REQ" objects:
		653
		654	$grp->add (aio_unlink "...");
		655
		656	add $grp aio_stat "...", sub {
		657	$_[0] or return $grp->result ("error");
		658
		659	# add another request dynamically, if first succeeded
		660	add $grp aio_open "...", sub {
		661	$grp->result ("ok");
		662	};
		663	};
		664
		665	This makes it very easy to create composite requests (see the source of
		666	"aio_move" for an application) that work and feel like simple requests.
		667
		668	* The IO::AIO::GRP objects will be cleaned up during calls to
		669	"IO::AIO::poll_cb", just like any other request.
		670
		671	* They can be canceled like any other request. Canceling will cancel
		672	not only the request itself, but also all requests it contains.
		673
		674	* They can also can also be added to other IO::AIO::GRP objects.
		675
		676	* You must not add requests to a group from within the group callback
		677	(or any later time).
		678
		679	Their lifetime, simplified, looks like this: when they are empty, they
		680	will finish very quickly. If they contain only requests that are in the
		681	"done" state, they will also finish. Otherwise they will continue to
		682	exist.
		683
		684	That means after creating a group you have some time to add requests.
		685	And in the callbacks of those requests, you can add further requests to
		686	the group. And only when all those requests have finished will the the
		687	group itself finish.
		688
		689	add $grp ...
		690	$grp->add (...)
		691	Add one or more requests to the group. Any type of IO::AIO::REQ can
		692	be added, including other groups, as long as you do not create
		693	circular dependencies.
		694
		695	Returns all its arguments.
		696
		697	$grp->cancel_subs
		698	Cancel all subrequests and clears any feeder, but not the group
		699	request itself. Useful when you queued a lot of events but got a
		700	result early.
		701
		702	$grp->result (...)
		703	Set the result value(s) that will be passed to the group callback
		704	when all subrequests have finished and set the groups errno to the
		705	current value of errno (just like calling "errno" without an error
		706	number). By default, no argument will be passed and errno is zero.
		707
		708	$grp->errno ([$errno])
		709	Sets the group errno value to $errno, or the current value of errno
		710	when the argument is missing.
		711
		712	Every aio request has an associated errno value that is restored
		713	when the callback is invoked. This method lets you change this value
		714	from its default (0).
		715
		716	Calling "result" will also set errno, so make sure you either set $!
		717	before the call to "result", or call c<errno> after it.
		718
		719	feed $grp $callback->($grp)
		720	Sets a feeder/generator on this group: every group can have an
		721	attached generator that generates requests if idle. The idea behind
		722	this is that, although you could just queue as many requests as you
		723	want in a group, this might starve other requests for a potentially
		724	long time. For example, "aio_scandir" might generate hundreds of
		725	thousands "aio_stat" requests, delaying any later requests for a
		726	long time.
		727
		728	To avoid this, and allow incremental generation of requests, you can
		729	instead a group and set a feeder on it that generates those
		730	requests. The feed callback will be called whenever there are few
		731	enough (see "limit", below) requests active in the group itself and
		732	is expected to queue more requests.
		733
		734	The feed callback can queue as many requests as it likes (i.e. "add"
		735	does not impose any limits).
		736
		737	If the feed does not queue more requests when called, it will be
		738	automatically removed from the group.
		739
		740	If the feed limit is 0, it will be set to 2 automatically.
		741
		742	Example:
		743
		744	# stat all files in @files, but only ever use four aio requests concurrently:
		745
		746	my $grp = aio_group sub { print "finished\n" };
		747	limit $grp 4;
		748	feed $grp sub {
		749	my $file = pop @files
		750	or return;
		751
		752	add $grp aio_stat $file, sub { ... };
		753	};
		754
		755	limit $grp $num
		756	Sets the feeder limit for the group: The feeder will be called
		757	whenever the group contains less than this many requests.
		758
		759	Setting the limit to 0 will pause the feeding process.
		760
133	SUPPORT FUNCTIONS	761	SUPPORT FUNCTIONS
		762	EVENT PROCESSING AND EVENT LOOP INTEGRATION
134	$fileno = IO::AIO::poll_fileno	763	$fileno = IO::AIO::poll_fileno
135	Return the request result pipe filehandle. This filehandle must be	764	Return the request result pipe file descriptor. This filehandle
136	polled for reading by some mechanism outside this module (e.g. Event	765	must be polled for reading by some mechanism outside this module
137	or select, see below). If the pipe becomes readable you have to call	766	(e.g. Event or select, see below or the SYNOPSIS). If the pipe
138	"poll_cb" to check the results.	767	becomes readable you have to call "poll_cb" to check the results.
139		768
140	See "poll_cb" for an example.	769	See "poll_cb" for an example.
141		770
142	IO::AIO::poll_cb	771	IO::AIO::poll_cb
143	Process all outstanding events on the result pipe. You have to call	772	Process some outstanding events on the result pipe. You have to call
144	this regularly. Returns the number of events processed. Returns	773	this regularly. Returns the number of events processed. Returns
145	immediately when no events are outstanding.	774	immediately when no events are outstanding. The amount of events
		775	processed depends on the settings of "IO::AIO::max_poll_req" and
		776	"IO::AIO::max_poll_time".
146		777
147	You can use Event to multiplex, e.g.:	778	If not all requests were processed for whatever reason, the
		779	filehandle will still be ready when "poll_cb" returns.
		780
		781	Example: Install an Event watcher that automatically calls
		782	IO::AIO::poll_cb with high priority:
148		783
149	Event->io (fd => IO::AIO::poll_fileno,	784	Event->io (fd => IO::AIO::poll_fileno,
150	poll => 'r', async => 1,	785	poll => 'r', async => 1,
151	cb => \&IO::AIO::poll_cb);	786	cb => \&IO::AIO::poll_cb);
152		787
		788	IO::AIO::max_poll_reqs $nreqs
		789	IO::AIO::max_poll_time $seconds
		790	These set the maximum number of requests (default 0, meaning
		791	infinity) that are being processed by "IO::AIO::poll_cb" in one
		792	call, respectively the maximum amount of time (default 0, meaning
		793	infinity) spent in "IO::AIO::poll_cb" to process requests (more
		794	correctly the mininum amount of time "poll_cb" is allowed to use).
		795
		796	Setting "max_poll_time" to a non-zero value creates an overhead of
		797	one syscall per request processed, which is not normally a problem
		798	unless your callbacks are really really fast or your OS is really
		799	really slow (I am not mentioning Solaris here). Using
		800	"max_poll_reqs" incurs no overhead.
		801
		802	Setting these is useful if you want to ensure some level of
		803	interactiveness when perl is not fast enough to process all requests
		804	in time.
		805
		806	For interactive programs, values such as 0.01 to 0.1 should be fine.
		807
		808	Example: Install an Event watcher that automatically calls
		809	IO::AIO::poll_cb with low priority, to ensure that other parts of
		810	the program get the CPU sometimes even under high AIO load.
		811
		812	# try not to spend much more than 0.1s in poll_cb
		813	IO::AIO::max_poll_time 0.1;
		814
		815	# use a low priority so other tasks have priority
		816	Event->io (fd => IO::AIO::poll_fileno,
		817	poll => 'r', nice => 1,
		818	cb => &IO::AIO::poll_cb);
		819
153	IO::AIO::poll_wait	820	IO::AIO::poll_wait
		821	If there are any outstanding requests and none of them in the result
154	Wait till the result filehandle becomes ready for reading (simply	822	phase, wait till the result filehandle becomes ready for reading
155	does a select on the filehandle. This is useful if you want to	823	(simply does a "select" on the filehandle. This is useful if you
156	synchronously wait for some requests to finish).	824	want to synchronously wait for some requests to finish).
157		825
158	See "nreqs" for an example.	826	See "nreqs" for an example.
159		827
		828	IO::AIO::poll
		829	Waits until some requests have been handled.
		830
		831	Returns the number of requests processed, but is otherwise strictly
		832	equivalent to:
		833
		834	IO::AIO::poll_wait, IO::AIO::poll_cb
		835
160	IO::AIO::nreqs	836	IO::AIO::flush
161	Returns the number of requests currently outstanding.	837	Wait till all outstanding AIO requests have been handled.
162		838
163	Example: wait till there are no outstanding requests anymore:	839	Strictly equivalent to:
164		840
165	IO::AIO::poll_wait, IO::AIO::poll_cb	841	IO::AIO::poll_wait, IO::AIO::poll_cb
166	while IO::AIO::nreqs;	842	while IO::AIO::nreqs;
167		843
		844	CONTROLLING THE NUMBER OF THREADS
168	IO::AIO::min_parallel $nthreads	845	IO::AIO::min_parallel $nthreads
169	Set the minimum number of AIO threads to $nthreads. The default is	846	Set the minimum number of AIO threads to $nthreads. The current
170	1, which means a single asynchronous operation can be done at one	847	default is 8, which means eight asynchronous operations can execute
171	time (the number of outstanding operations, however, is unlimited).	848	concurrently at any one time (the number of outstanding requests,
		849	however, is unlimited).
172		850
		851	IO::AIO starts threads only on demand, when an AIO request is queued
		852	and no free thread exists. Please note that queueing up a hundred
		853	requests can create demand for a hundred threads, even if it turns
		854	out that everything is in the cache and could have been processed
		855	faster by a single thread.
		856
173	It is recommended to keep the number of threads low, as some Linux	857	It is recommended to keep the number of threads relatively low, as
174	kernel versions will scale negatively with the number of threads	858	some Linux kernel versions will scale negatively with the number of
175	(higher parallelity => MUCH higher latency). With current Linux 2.6	859	threads (higher parallelity => MUCH higher latency). With current
176	versions, 4-32 threads should be fine.	860	Linux 2.6 versions, 4-32 threads should be fine.
177		861
178	Under normal circumstances you don't need to call this function, as	862	Under most circumstances you don't need to call this function, as
179	this module automatically starts some threads (the exact number	863	the module selects a default that is suitable for low to moderate
180	might change, and is currently 4).	864	load.
181		865
182	IO::AIO::max_parallel $nthreads	866	IO::AIO::max_parallel $nthreads
183	Sets the maximum number of AIO threads to $nthreads. If more than	867	Sets the maximum number of AIO threads to $nthreads. If more than
184	the specified number of threads are currently running, kill them.	868	the specified number of threads are currently running, this function
185	This function blocks until the limit is reached.	869	kills them. This function blocks until the limit is reached.
		870
		871	While $nthreads are zero, aio requests get queued but not executed
		872	until the number of threads has been increased again.
186		873
187	This module automatically runs "max_parallel 0" at program end, to	874	This module automatically runs "max_parallel 0" at program end, to
188	ensure that all threads are killed and that there are no outstanding	875	ensure that all threads are killed and that there are no outstanding
189	requests.	876	requests.
190		877
191	Under normal circumstances you don't need to call this function.	878	Under normal circumstances you don't need to call this function.
192		879
		880	IO::AIO::max_idle $nthreads
		881	Limit the number of threads (default: 4) that are allowed to idle
		882	(i.e., threads that did not get a request to process within 10
		883	seconds). That means if a thread becomes idle while $nthreads other
		884	threads are also idle, it will free its resources and exit.
		885
		886	This is useful when you allow a large number of threads (e.g. 100 or
		887	1000) to allow for extremely high load situations, but want to free
		888	resources under normal circumstances (1000 threads can easily
		889	consume 30MB of RAM).
		890
		891	The default is probably ok in most situations, especially if thread
		892	creation is fast. If thread creation is very slow on your system you
		893	might want to use larger values.
		894
193	$oldnreqs = IO::AIO::max_outstanding $nreqs	895	$oldmaxreqs = IO::AIO::max_outstanding $maxreqs
		896	This is a very bad function to use in interactive programs because
		897	it blocks, and a bad way to reduce concurrency because it is
		898	inexact: Better use an "aio_group" together with a feed callback.
		899
194	Sets the maximum number of outstanding requests to $nreqs. If you	900	Sets the maximum number of outstanding requests to $nreqs. If you do
195	try to queue up more than this number of requests, the caller will	901	queue up more than this number of requests, the next call to the
196	block until some requests have been handled.	902	"poll_cb" (and "poll_some" and other functions calling "poll_cb")
		903	function will block until the limit is no longer exceeded.
197		904
198	The default is very large, so normally there is no practical limit.	905	The default value is very large, so there is no practical limit on
199	If you queue up many requests in a loop it it often improves speed	906	the number of outstanding requests.
200	if you set this to a relatively low number, such as 100.
201		907
202	Under normal circumstances you don't need to call this function.	908	You can still queue as many requests as you want. Therefore,
		909	"max_oustsanding" is mainly useful in simple scripts (with low
		910	values) or as a stop gap to shield against fatal memory overflow
		911	(with large values).
		912
		913	STATISTICAL INFORMATION
		914	IO::AIO::nreqs
		915	Returns the number of requests currently in the ready, execute or
		916	pending states (i.e. for which their callback has not been invoked
		917	yet).
		918
		919	Example: wait till there are no outstanding requests anymore:
		920
		921	IO::AIO::poll_wait, IO::AIO::poll_cb
		922	while IO::AIO::nreqs;
		923
		924	IO::AIO::nready
		925	Returns the number of requests currently in the ready state (not yet
		926	executed).
		927
		928	IO::AIO::npending
		929	Returns the number of requests currently in the pending state
		930	(executed, but not yet processed by poll_cb).
		931
		932	FORK BEHAVIOUR
		933	This module should do "the right thing" when the process using it forks:
		934
		935	Before the fork, IO::AIO enters a quiescent state where no requests can
		936	be added in other threads and no results will be processed. After the
		937	fork the parent simply leaves the quiescent state and continues
		938	request/result processing, while the child frees the request/result
		939	queue (so that the requests started before the fork will only be handled
		940	in the parent). Threads will be started on demand until the limit set in
		941	the parent process has been reached again.
		942
		943	In short: the parent will, after a short pause, continue as if fork had
		944	not been called, while the child will act as if IO::AIO has not been
		945	used yet.
		946
		947	MEMORY USAGE
		948	Per-request usage:
		949
		950	Each aio request uses - depending on your architecture - around 100-200
		951	bytes of memory. In addition, stat requests need a stat buffer (possibly
		952	a few hundred bytes), readdir requires a result buffer and so on. Perl
		953	scalars and other data passed into aio requests will also be locked and
		954	will consume memory till the request has entered the done state.
		955
		956	This is not awfully much, so queuing lots of requests is not usually a
		957	problem.
		958
		959	Per-thread usage:
		960
		961	In the execution phase, some aio requests require more memory for
		962	temporary buffers, and each thread requires a stack and other data
		963	structures (usually around 16k-128k, depending on the OS).
		964
		965	KNOWN BUGS
		966	Known bugs will be fixed in the next release.
203		967
204	SEE ALSO	968	SEE ALSO
205	Coro, Linux::AIO.	969	Coro::AIO.
206		970
207	AUTHOR	971	AUTHOR
208	Marc Lehmann <schmorp@schmorp.de>	972	Marc Lehmann <schmorp@schmorp.de>
209	http://home.schmorp.de/	973	http://home.schmorp.de/
210		974

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.4 by root, Sun Jul 10 21:04:46 2005 UTC vs. Revision 1.28 by root, Sun Mar 30 06:31:49 2008 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.4 by root, Sun Jul 10 21:04:46 2005 UTC vs.
Revision 1.28 by root, Sun Mar 30 06:31:49 2008 UTC