[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.10 by root, Wed Aug 17 05:26:20 2005 UTC vs.
Revision 1.20 by root, Tue Oct 31 00:45:41 2006 UTC

…		…
13		13
14	aio_read $fh, 30000, 1024, $buffer, 0, sub {	14	aio_read $fh, 30000, 1024, $buffer, 0, sub {
15	$_[0] > 0 or die "read error: $!";	15	$_[0] > 0 or die "read error: $!";
16	};	16	};
17		17
18	# Event	18	# version 2+ has request and group objects
		19	use IO::AIO 2;
		20
		21	aioreq_pri 4; # give next request a very high priority
		22	my $req = aio_unlink "/tmp/file", sub { };
		23	$req->cancel; # cancel request if still in queue
		24
		25	my $grp = aio_group sub { print "all stats done\n" };
		26	add $grp aio_stat "..." for ...;
		27
		28	# AnyEvent integration
		29	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
		30	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
		31
		32	# Event integration
19	Event->io (fd => IO::AIO::poll_fileno,	33	Event->io (fd => IO::AIO::poll_fileno,
20	poll => 'r',	34	poll => 'r',
21	cb => \&IO::AIO::poll_cb);	35	cb => \&IO::AIO::poll_cb);
22		36
23	# Glib/Gtk2	37	# Glib/Gtk2 integration
24	add_watch Glib::IO IO::AIO::poll_fileno,	38	add_watch Glib::IO IO::AIO::poll_fileno,
25	in => sub { IO::AIO::poll_cb; 1 };	39	in => sub { IO::AIO::poll_cb; 1 };
26		40
27	# Tk	41	# Tk integration
28	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",	42	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
29	readable => \&IO::AIO::poll_cb);	43	readable => \&IO::AIO::poll_cb);
30		44
31	# Danga::Socket	45	# Danga::Socket integration
32	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>	46	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
33	\&IO::AIO::poll_cb);	47	\&IO::AIO::poll_cb);
34		48
35	DESCRIPTION	49	DESCRIPTION
36	This module implements asynchronous I/O using whatever means your	50	This module implements asynchronous I/O using whatever means your
37	operating system supports.	51	operating system supports.
38		52
		53	Asynchronous means that operations that can normally block your program
		54	(e.g. reading from disk) will be done asynchronously: the operation will
		55	still block, but you can do something else in the meantime. This is
		56	extremely useful for programs that need to stay interactive even when
		57	doing heavy I/O (GUI programs, high performance network servers etc.),
		58	but can also be used to easily do operations in parallel that are
		59	normally done sequentially, e.g. stat'ing many files, which is much
		60	faster on a RAID volume or over NFS when you do a number of stat
		61	operations concurrently.
		62
		63	While most of this works on all types of file descriptors (for example
		64	sockets), using these functions on file descriptors that support
		65	nonblocking operation (again, sockets, pipes etc.) is very inefficient
		66	or might not work (aio_read fails on sockets/pipes/fifos). Use an event
		67	loop for that (such as the Event module): IO::AIO will naturally fit
		68	into such an event loop itself.
		69
39	Currently, a number of threads are started that execute your read/writes	70	In this version, a number of threads are started that execute your
40	and signal their completion. You don't need thread support in your libc	71	requests and signal their completion. You don't need thread support in
41	or perl, and the threads created by this module will not be visible to	72	perl, and the threads created by this module will not be visible to
42	the pthreads library. In the future, this module might make use of the	73	perl. In the future, this module might make use of the native aio
43	native aio functions available on many operating systems. However, they	74	functions available on many operating systems. However, they are often
44	are often not well-supported (Linux doesn't allow them on normal files	75	not well-supported or restricted (GNU/Linux doesn't allow them on normal
45	currently, for example), and they would only support aio_read and	76	files currently, for example), and they would only support aio_read and
46	aio_write, so the remaining functionality would have to be implemented	77	aio_write, so the remaining functionality would have to be implemented
47	using threads anyway.	78	using threads anyway.
48		79
49	Although the module will work with in the presence of other threads, it	80	Although the module will work with in the presence of other (Perl-)
50	is currently not reentrant, so use appropriate locking yourself, always	81	threads, it is currently not reentrant in any way, so use appropriate
51	call "poll_cb" from within the same thread, or never call "poll_cb" (or	82	locking yourself, always call "poll_cb" from within the same thread, or
52	other "aio_" functions) recursively.	83	never call "poll_cb" (or other "aio_" functions) recursively.
		84
		85	EXAMPLE
		86	This is a simple example that uses the Event module and loads
		87	/etc/passwd asynchronously:
		88
		89	use Fcntl;
		90	use Event;
		91	use IO::AIO;
		92
		93	# register the IO::AIO callback with Event
		94	Event->io (fd => IO::AIO::poll_fileno,
		95	poll => 'r',
		96	cb => \&IO::AIO::poll_cb);
		97
		98	# queue the request to open /etc/passwd
		99	aio_open "/etc/passwd", O_RDONLY, 0, sub {
		100	my $fh = $_[0]
		101	or die "error while opening: $!";
		102
		103	# stat'ing filehandles is generally non-blocking
		104	my $size = -s $fh;
		105
		106	# queue a request to read the file
		107	my $contents;
		108	aio_read $fh, 0, $size, $contents, 0, sub {
		109	$_[0] == $size
		110	or die "short read: $!";
		111
		112	close $fh;
		113
		114	# file contents now in $contents
		115	print $contents;
		116
		117	# exit event loop and program
		118	Event::unloop;
		119	};
		120	};
		121
		122	# possibly queue up other requests, or open GUI windows,
		123	# check for sockets etc. etc.
		124
		125	# process events as long as there are some:
		126	Event::loop;
		127
		128	REQUEST ANATOMY AND LIFETIME
		129	Every "aio_*" function creates a request. which is a C data structure
		130	not directly visible to Perl.
		131
		132	If called in non-void context, every request function returns a Perl
		133	object representing the request. In void context, nothing is returned,
		134	which saves a bit of memory.
		135
		136	The perl object is a fairly standard ref-to-hash object. The hash
		137	contents are not used by IO::AIO so you are free to store anything you
		138	like in it.
		139
		140	During their existance, aio requests travel through the following
		141	states, in order:
		142
		143	ready
		144	Immediately after a request is created it is put into the ready
		145	state, waiting for a thread to execute it.
		146
		147	execute
		148	A thread has accepted the request for processing and is currently
		149	executing it (e.g. blocking in read).
		150
		151	pending
		152	The request has been executed and is waiting for result processing.
		153
		154	While request submission and execution is fully asynchronous, result
		155	processing is not and relies on the perl interpreter calling
		156	"poll_cb" (or another function with the same effect).
		157
		158	result
		159	The request results are processed synchronously by "poll_cb".
		160
		161	The "poll_cb" function will process all outstanding aio requests by
		162	calling their callbacks, freeing memory associated with them and
		163	managing any groups they are contained in.
		164
		165	done
		166	Request has reached the end of its lifetime and holds no resources
		167	anymore (except possibly for the Perl object, but its connection to
		168	the actual aio request is severed and calling its methods will
		169	either do nothing or result in a runtime error).
53		170
54	FUNCTIONS	171	FUNCTIONS
55	AIO FUNCTIONS	172	AIO REQUEST FUNCTIONS
56	All the "aio_*" calls are more or less thin wrappers around the syscall	173	All the "aio_*" calls are more or less thin wrappers around the syscall
57	with the same name (sans "aio_"). The arguments are similar or	174	with the same name (sans "aio_"). The arguments are similar or
58	identical, and they all accept an additional (and optional) $callback	175	identical, and they all accept an additional (and optional) $callback
59	argument which must be a code reference. This code reference will get	176	argument which must be a code reference. This code reference will get
60	called with the syscall return code (e.g. most syscalls return -1 on	177	called with the syscall return code (e.g. most syscalls return -1 on
…		…
62	argument when the given syscall has been executed asynchronously.	179	argument when the given syscall has been executed asynchronously.
63		180
64	All functions expecting a filehandle keep a copy of the filehandle	181	All functions expecting a filehandle keep a copy of the filehandle
65	internally until the request has finished.	182	internally until the request has finished.
66		183
		184	All functions return request objects of type IO::AIO::REQ that allow
		185	further manipulation of those requests while they are in-flight.
		186
67	The pathnames you pass to these routines must be absolute and encoded	187	The pathnames you pass to these routines must be absolute and encoded
68	in byte form. The reason for the former is that at the time the request	188	as octets. The reason for the former is that at the time the request is
69	is being executed, the current working directory could have changed.	189	being executed, the current working directory could have changed.
70	Alternatively, you can make sure that you never change the current	190	Alternatively, you can make sure that you never change the current
71	working directory.	191	working directory anywhere in the program and then use relative paths.
72		192
73	To encode pathnames to byte form, either make sure you either: a) always	193	To encode pathnames as octets, either make sure you either: a) always
74	pass in filenames you got from outside (command line, readdir etc.), b)	194	pass in filenames you got from outside (command line, readdir etc.)
75	are ASCII or ISO 8859-1, c) use the Encode module and encode your	195	without tinkering, b) are ASCII or ISO 8859-1, c) use the Encode module
76	pathnames to the locale (or other) encoding in effect in the user	196	and encode your pathnames to the locale (or other) encoding in effect in
77	environment, d) use Glib::filename_from_unicode on unicode filenames or	197	the user environment, d) use Glib::filename_from_unicode on unicode
78	e) use something else.	198	filenames or e) use something else to ensure your scalar has the correct
		199	contents.
79		200
		201	This works, btw. independent of the internal UTF-8 bit, which IO::AIO
		202	handles correctly wether it is set or not.
		203
		204	$prev_pri = aioreq_pri [$pri]
		205	Returns the priority value that would be used for the next request
		206	and, if $pri is given, sets the priority for the next aio request.
		207
		208	The default priority is 0, the minimum and maximum priorities are -4
		209	and 4, respectively. Requests with higher priority will be serviced
		210	first.
		211
		212	The priority will be reset to 0 after each call to one of the
		213	"aio_*" functions.
		214
		215	Example: open a file with low priority, then read something from it
		216	with higher priority so the read request is serviced before other
		217	low priority open requests (potentially spamming the cache):
		218
		219	aioreq_pri -3;
		220	aio_open ..., sub {
		221	return unless $_[0];
		222
		223	aioreq_pri -2;
		224	aio_read $_[0], ..., sub {
		225	...
		226	};
		227	};
		228
		229	aioreq_nice $pri_adjust
		230	Similar to "aioreq_pri", but subtracts the given value from the
		231	current priority, so the effect is cumulative.
		232
80	aio_open $pathname, $flags, $mode, $callback	233	aio_open $pathname, $flags, $mode, $callback->($fh)
81	Asynchronously open or create a file and call the callback with a	234	Asynchronously open or create a file and call the callback with a
82	newly created filehandle for the file.	235	newly created filehandle for the file.
83		236
84	The pathname passed to "aio_open" must be absolute. See API NOTES,	237	The pathname passed to "aio_open" must be absolute. See API NOTES,
85	above, for an explanation.	238	above, for an explanation.
…		…
101	} else {	254	} else {
102	die "open failed: $!\n";	255	die "open failed: $!\n";
103	}	256	}
104	};	257	};
105		258
106	aio_close $fh, $callback	259	aio_close $fh, $callback->($status)
107	Asynchronously close a file and call the callback with the result	260	Asynchronously close a file and call the callback with the result
108	code. WARNING: although accepted, you should not pass in a perl	261	code. WARNING: although accepted, you should not pass in a perl
109	filehandle here, as perl will likely close the file descriptor	262	filehandle here, as perl will likely close the file descriptor
110	another time when the filehandle is destroyed. Normally, you can	263	another time when the filehandle is destroyed. Normally, you can
111	safely call perls "close" or just let filehandles go out of scope.	264	safely call perls "close" or just let filehandles go out of scope.
112		265
113	This is supposed to be a bug in the API, so that might change. It's	266	This is supposed to be a bug in the API, so that might change. It's
114	therefore best to avoid this function.	267	therefore best to avoid this function.
115		268
116	aio_read $fh,$offset,$length, $data,$dataoffset,$callback	269	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
117	aio_write $fh,$offset,$length, $data,$dataoffset,$callback	270	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
118	Reads or writes "length" bytes from the specified "fh" and "offset"	271	Reads or writes "length" bytes from the specified "fh" and "offset"
119	into the scalar given by "data" and offset "dataoffset" and calls	272	into the scalar given by "data" and offset "dataoffset" and calls
120	the callback without the actual number of bytes read (or -1 on	273	the callback without the actual number of bytes read (or -1 on
121	error, just like the syscall).	274	error, just like the syscall).
122		275
…		…
130	aio_read $fh, 7, 15, $buffer, 0, sub {	283	aio_read $fh, 7, 15, $buffer, 0, sub {
131	$_[0] > 0 or die "read error: $!";	284	$_[0] > 0 or die "read error: $!";
132	print "read $_[0] bytes: <$buffer>\n";	285	print "read $_[0] bytes: <$buffer>\n";
133	};	286	};
134		287
		288	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
		289	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		290	reading at byte offset $in_offset, and starts writing at the current
		291	file offset of $out_fh. Because of that, it is not safe to issue
		292	more than one "aio_sendfile" per $out_fh, as they will interfere
		293	with each other.
		294
		295	This call tries to make use of a native "sendfile" syscall to
		296	provide zero-copy operation. For this to work, $out_fh should refer
		297	to a socket, and $in_fh should refer to mmap'able file.
		298
		299	If the native sendfile call fails or is not implemented, it will be
		300	emulated, so you can call "aio_sendfile" on any type of filehandle
		301	regardless of the limitations of the operating system.
		302
		303	Please note, however, that "aio_sendfile" can read more bytes from
		304	$in_fh than are written, and there is no way to find out how many
		305	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
		306	only provides the number of bytes written to $out_fh. Only if the
		307	result value equals $length one can assume that $length bytes have
		308	been read.
		309
135	aio_readahead $fh,$offset,$length, $callback	310	aio_readahead $fh,$offset,$length, $callback->($retval)
136	"aio_readahead" populates the page cache with data from a file so	311	"aio_readahead" populates the page cache with data from a file so
137	that subsequent reads from that file will not block on disk I/O. The	312	that subsequent reads from that file will not block on disk I/O. The
138	$offset argument specifies the starting point from which data is to	313	$offset argument specifies the starting point from which data is to
139	be read and $length specifies the number of bytes to be read. I/O is	314	be read and $length specifies the number of bytes to be read. I/O is
140	performed in whole pages, so that offset is effectively rounded down	315	performed in whole pages, so that offset is effectively rounded down
…		…
145		320
146	If that syscall doesn't exist (likely if your OS isn't Linux) it	321	If that syscall doesn't exist (likely if your OS isn't Linux) it
147	will be emulated by simply reading the data, which would have a	322	will be emulated by simply reading the data, which would have a
148	similar effect.	323	similar effect.
149		324
150	aio_stat $fh_or_path, $callback	325	aio_stat $fh_or_path, $callback->($status)
151	aio_lstat $fh, $callback	326	aio_lstat $fh, $callback->($status)
152	Works like perl's "stat" or "lstat" in void context. The callback	327	Works like perl's "stat" or "lstat" in void context. The callback
153	will be called after the stat and the results will be available	328	will be called after the stat and the results will be available
154	using "stat _" or "-s _" etc...	329	using "stat _" or "-s _" etc...
155		330
156	The pathname passed to "aio_stat" must be absolute. See API NOTES,	331	The pathname passed to "aio_stat" must be absolute. See API NOTES,
…		…
166	aio_stat "/etc/passwd", sub {	341	aio_stat "/etc/passwd", sub {
167	$_[0] and die "stat failed: $!";	342	$_[0] and die "stat failed: $!";
168	print "size is ", -s _, "\n";	343	print "size is ", -s _, "\n";
169	};	344	};
170		345
171	aio_unlink $pathname, $callback	346	aio_unlink $pathname, $callback->($status)
172	Asynchronously unlink (delete) a file and call the callback with the	347	Asynchronously unlink (delete) a file and call the callback with the
173	result code.	348	result code.
174		349
		350	aio_mknod $path, $mode, $dev, $callback->($status)
		351	[EXPERIMENTAL]
		352
		353	Asynchronously create a device node (or fifo). See mknod(2).
		354
		355	The only (POSIX-) portable way of calling this function is:
		356
		357	aio_mknod $path, IO::AIO::S_IFIFO \| $mode, 0, sub { ...
		358
		359	aio_link $srcpath, $dstpath, $callback->($status)
		360	Asynchronously create a new link to the existing object at $srcpath
		361	at the path $dstpath and call the callback with the result code.
		362
		363	aio_symlink $srcpath, $dstpath, $callback->($status)
		364	Asynchronously create a new symbolic link to the existing object at
		365	$srcpath at the path $dstpath and call the callback with the result
		366	code.
		367
		368	aio_readlink $path, $callback->($link)
		369	Asynchronously read the symlink specified by $path and pass it to
		370	the callback. If an error occurs, nothing or undef gets passed to
		371	the callback.
		372
		373	aio_rename $srcpath, $dstpath, $callback->($status)
		374	Asynchronously rename the object at $srcpath to $dstpath, just as
		375	rename(2) and call the callback with the result code.
		376
175	aio_rmdir $pathname, $callback	377	aio_rmdir $pathname, $callback->($status)
176	Asynchronously rmdir (delete) a directory and call the callback with	378	Asynchronously rmdir (delete) a directory and call the callback with
177	the result code.	379	the result code.
178		380
		381	aio_readdir $pathname, $callback->($entries)
		382	Unlike the POSIX call of the same name, "aio_readdir" reads an
		383	entire directory (i.e. opendir + readdir + closedir). The entries
		384	will not be sorted, and will NOT include the "." and ".." entries.
		385
		386	The callback a single argument which is either "undef" or an
		387	array-ref with the filenames.
		388
		389	aio_copy $srcpath, $dstpath, $callback->($status)
		390	Try to copy the file (directories not supported as either source
		391	or destination) from $srcpath to $dstpath and call the callback with
		392	the 0 (error) or -1 ok.
		393
		394	This is a composite request that it creates the destination file
		395	with mode 0200 and copies the contents of the source file into it
		396	using "aio_sendfile", followed by restoring atime, mtime, access
		397	mode and uid/gid, in that order.
		398
		399	If an error occurs, the partial destination file will be unlinked,
		400	if possible, except when setting atime, mtime, access mode and
		401	uid/gid, where errors are being ignored.
		402
		403	aio_move $srcpath, $dstpath, $callback->($status)
		404	Try to move the file (directories not supported as either source
		405	or destination) from $srcpath to $dstpath and call the callback with
		406	the 0 (error) or -1 ok.
		407
		408	This is a composite request that tries to rename(2) the file first.
		409	If rename files with "EXDEV", it copies the file with "aio_copy"
		410	and, if that is successful, unlinking the $srcpath.
		411
		412	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		413	Scans a directory (similar to "aio_readdir") but additionally tries
		414	to efficiently separate the entries of directory $path into two sets
		415	of names, directories you can recurse into (directories), and ones
		416	you cannot recurse into (everything else, including symlinks to
		417	directories).
		418
		419	"aio_scandir" is a composite request that creates of many sub
		420	requests_ $maxreq specifies the maximum number of outstanding aio
		421	requests that this function generates. If it is "<= 0", then a
		422	suitable default will be chosen (currently 4).
		423
		424	On error, the callback is called without arguments, otherwise it
		425	receives two array-refs with path-relative entry names.
		426
		427	Example:
		428
		429	aio_scandir $dir, 0, sub {
		430	my ($dirs, $nondirs) = @_;
		431	print "real directories: @$dirs\n";
		432	print "everything else: @$nondirs\n";
		433	};
		434
		435	Implementation notes.
		436
		437	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
		438	can.
		439
		440	After reading the directory, the modification time, size etc. of the
		441	directory before and after the readdir is checked, and if they match
		442	(and isn't the current time), the link count will be used to decide
		443	how many entries are directories (if >= 2). Otherwise, no knowledge
		444	of the number of subdirectories will be assumed.
		445
		446	Then entries will be sorted into likely directories (everything
		447	without a non-initial dot currently) and likely non-directories
		448	(everything else). Then every entry plus an appended "/." will be
		449	"stat"'ed, likely directories first. If that succeeds, it assumes
		450	that the entry is a directory or a symlink to directory (which will
		451	be checked seperately). This is often faster than stat'ing the entry
		452	itself because filesystems might detect the type of the entry
		453	without reading the inode data (e.g. ext2fs filetype feature).
		454
		455	If the known number of directories (link count - 2) has been
		456	reached, the rest of the entries is assumed to be non-directories.
		457
		458	This only works with certainty on POSIX (= UNIX) filesystems, which
		459	fortunately are the vast majority of filesystems around.
		460
		461	It will also likely work on non-POSIX filesystems with reduced
		462	efficiency as those tend to return 0 or 1 as link counts, which
		463	disables the directory counting heuristic.
		464
179	aio_fsync $fh, $callback	465	aio_fsync $fh, $callback->($status)
180	Asynchronously call fsync on the given filehandle and call the	466	Asynchronously call fsync on the given filehandle and call the
181	callback with the fsync result code.	467	callback with the fsync result code.
182		468
183	aio_fdatasync $fh, $callback	469	aio_fdatasync $fh, $callback->($status)
184	Asynchronously call fdatasync on the given filehandle and call the	470	Asynchronously call fdatasync on the given filehandle and call the
185	callback with the fdatasync result code.	471	callback with the fdatasync result code.
186		472
187	If this call isn't available because your OS lacks it or it couldn't	473	If this call isn't available because your OS lacks it or it couldn't
188	be detected, it will be emulated by calling "fsync" instead.	474	be detected, it will be emulated by calling "fsync" instead.
189		475
		476	aio_group $callback->(...)
		477	This is a very special aio request: Instead of doing something, it
		478	is a container for other aio requests, which is useful if you want
		479	to bundle many requests into a single, composite, request with a
		480	definite callback and the ability to cancel the whole request with
		481	its subrequests.
		482
		483	Returns an object of class IO::AIO::GRP. See its documentation below
		484	for more info.
		485
		486	Example:
		487
		488	my $grp = aio_group sub {
		489	print "all stats done\n";
		490	};
		491
		492	add $grp
		493	(aio_stat ...),
		494	(aio_stat ...),
		495	...;
		496
		497	aio_nop $callback->()
		498	This is a special request - it does nothing in itself and is only
		499	used for side effects, such as when you want to add a dummy request
		500	to a group so that finishing the requests in the group depends on
		501	executing the given code.
		502
		503	While this request does nothing, it still goes through the execution
		504	phase and still requires a worker thread. Thus, the callback will
		505	not be executed immediately but only after other requests in the
		506	queue have entered their execution phase. This can be used to
		507	measure request latency.
		508
		509	IO::AIO::aio_busy $fractional_seconds, $callback->() NOT EXPORTED
		510	Mainly used for debugging and benchmarking, this aio request puts
		511	one of the request workers to sleep for the given time.
		512
		513	While it is theoretically handy to have simple I/O scheduling
		514	requests like sleep and file handle readable/writable, the overhead
		515	this creates is immense (it blocks a thread for a long time) so do
		516	not use this function except to put your application under
		517	artificial I/O pressure.
		518
		519	IO::AIO::REQ CLASS
		520	All non-aggregate "aio_*" functions return an object of this class when
		521	called in non-void context.
		522
		523	cancel $req
		524	Cancels the request, if possible. Has the effect of skipping
		525	execution when entering the execute state and skipping calling the
		526	callback when entering the the result state, but will leave the
		527	request otherwise untouched. That means that requests that currently
		528	execute will not be stopped and resources held by the request will
		529	not be freed prematurely.
		530
		531	cb $req $callback->(...)
		532	Replace (or simply set) the callback registered to the request.
		533
		534	IO::AIO::GRP CLASS
		535	This class is a subclass of IO::AIO::REQ, so all its methods apply to
		536	objects of this class, too.
		537
		538	A IO::AIO::GRP object is a special request that can contain multiple
		539	other aio requests.
		540
		541	You create one by calling the "aio_group" constructing function with a
		542	callback that will be called when all contained requests have entered
		543	the "done" state:
		544
		545	my $grp = aio_group sub {
		546	print "all requests are done\n";
		547	};
		548
		549	You add requests by calling the "add" method with one or more
		550	"IO::AIO::REQ" objects:
		551
		552	$grp->add (aio_unlink "...");
		553
		554	add $grp aio_stat "...", sub {
		555	$_[0] or return $grp->result ("error");
		556
		557	# add another request dynamically, if first succeeded
		558	add $grp aio_open "...", sub {
		559	$grp->result ("ok");
		560	};
		561	};
		562
		563	This makes it very easy to create composite requests (see the source of
		564	"aio_move" for an application) that work and feel like simple requests.
		565
		566	* The IO::AIO::GRP objects will be cleaned up during calls to
		567	"IO::AIO::poll_cb", just like any other request.
		568	* They can be canceled like any other request. Canceling will cancel not
		569	only the request itself, but also all requests it contains.
		570	* They can also can also be added to other IO::AIO::GRP objects.
		571	* You must not add requests to a group from within the group callback
		572	(or any later time).
		573
		574	Their lifetime, simplified, looks like this: when they are empty, they
		575	will finish very quickly. If they contain only requests that are in the
		576	"done" state, they will also finish. Otherwise they will continue to
		577	exist.
		578
		579	That means after creating a group you have some time to add requests.
		580	And in the callbacks of those requests, you can add further requests to
		581	the group. And only when all those requests have finished will the the
		582	group itself finish.
		583
		584	add $grp ...
		585	$grp->add (...)
		586	Add one or more requests to the group. Any type of IO::AIO::REQ can
		587	be added, including other groups, as long as you do not create
		588	circular dependencies.
		589
		590	Returns all its arguments.
		591
		592	$grp->cancel_subs
		593	Cancel all subrequests and clears any feeder, but not the group
		594	request itself. Useful when you queued a lot of events but got a
		595	result early.
		596
		597	$grp->result (...)
		598	Set the result value(s) that will be passed to the group callback
		599	when all subrequests have finished and set thre groups errno to the
		600	current value of errno (just like calling "errno" without an error
		601	number). By default, no argument will be passed and errno is zero.
		602
		603	$grp->errno ([$errno])
		604	Sets the group errno value to $errno, or the current value of errno
		605	when the argument is missing.
		606
		607	Every aio request has an associated errno value that is restored
		608	when the callback is invoked. This method lets you change this value
		609	from its default (0).
		610
		611	Calling "result" will also set errno, so make sure you either set $!
		612	before the call to "result", or call c<errno> after it.
		613
		614	feed $grp $callback->($grp)
		615	Sets a feeder/generator on this group: every group can have an
		616	attached generator that generates requests if idle. The idea behind
		617	this is that, although you could just queue as many requests as you
		618	want in a group, this might starve other requests for a potentially
		619	long time. For example, "aio_scandir" might generate hundreds of
		620	thousands "aio_stat" requests, delaying any later requests for a
		621	long time.
		622
		623	To avoid this, and allow incremental generation of requests, you can
		624	instead a group and set a feeder on it that generates those
		625	requests. The feed callback will be called whenever there are few
		626	enough (see "limit", below) requests active in the group itself and
		627	is expected to queue more requests.
		628
		629	The feed callback can queue as many requests as it likes (i.e. "add"
		630	does not impose any limits).
		631
		632	If the feed does not queue more requests when called, it will be
		633	automatically removed from the group.
		634
		635	If the feed limit is 0, it will be set to 2 automatically.
		636
		637	Example:
		638
		639	# stat all files in @files, but only ever use four aio requests concurrently:
		640
		641	my $grp = aio_group sub { print "finished\n" };
		642	limit $grp 4;
		643	feed $grp sub {
		644	my $file = pop @files
		645	or return;
		646
		647	add $grp aio_stat $file, sub { ... };
		648	};
		649
		650	limit $grp $num
		651	Sets the feeder limit for the group: The feeder will be called
		652	whenever the group contains less than this many requests.
		653
		654	Setting the limit to 0 will pause the feeding process.
		655
190	SUPPORT FUNCTIONS	656	SUPPORT FUNCTIONS
		657	EVENT PROCESSING AND EVENT LOOP INTEGRATION
191	$fileno = IO::AIO::poll_fileno	658	$fileno = IO::AIO::poll_fileno
192	Return the request result pipe file descriptor. This filehandle	659	Return the request result pipe file descriptor. This filehandle
193	must be polled for reading by some mechanism outside this module	660	must be polled for reading by some mechanism outside this module
194	(e.g. Event or select, see below or the SYNOPSIS). If the pipe	661	(e.g. Event or select, see below or the SYNOPSIS). If the pipe
195	becomes readable you have to call "poll_cb" to check the results.	662	becomes readable you have to call "poll_cb" to check the results.
196		663
197	See "poll_cb" for an example.	664	See "poll_cb" for an example.
198		665
199	IO::AIO::poll_cb	666	IO::AIO::poll_cb
200	Process all outstanding events on the result pipe. You have to call	667	Process some outstanding events on the result pipe. You have to call
201	this regularly. Returns the number of events processed. Returns	668	this regularly. Returns the number of events processed. Returns
202	immediately when no events are outstanding.	669	immediately when no events are outstanding. The amount of events
		670	processed depends on the settings of "IO::AIO::max_poll_req" and
		671	"IO::AIO::max_poll_time".
		672
		673	If not all requests were processed for whatever reason, the
		674	filehandle will still be ready when "poll_cb" returns.
203		675
204	Example: Install an Event watcher that automatically calls	676	Example: Install an Event watcher that automatically calls
205	IO::AIO::poll_cb with high priority:	677	IO::AIO::poll_cb with high priority:
206		678
207	Event->io (fd => IO::AIO::poll_fileno,	679	Event->io (fd => IO::AIO::poll_fileno,
208	poll => 'r', async => 1,	680	poll => 'r', async => 1,
209	cb => \&IO::AIO::poll_cb);	681	cb => \&IO::AIO::poll_cb);
		682
		683	IO::AIO::max_poll_reqs $nreqs
		684	IO::AIO::max_poll_time $seconds
		685	These set the maximum number of requests (default 0, meaning
		686	infinity) that are being processed by "IO::AIO::poll_cb" in one
		687	call, respectively the maximum amount of time (default 0, meaning
		688	infinity) spent in "IO::AIO::poll_cb" to process requests (more
		689	correctly the mininum amount of time "poll_cb" is allowed to use).
		690
		691	Setting "max_poll_time" to a non-zero value creates an overhead of
		692	one syscall per request processed, which is not normally a problem
		693	unless your callbacks are really really fast or your OS is really
		694	really slow (I am not mentioning Solaris here). Using
		695	"max_poll_reqs" incurs no overhead.
		696
		697	Setting these is useful if you want to ensure some level of
		698	interactiveness when perl is not fast enough to process all requests
		699	in time.
		700
		701	For interactive programs, values such as 0.01 to 0.1 should be fine.
		702
		703	Example: Install an Event watcher that automatically calls
		704	IO::AIO::poll_cb with low priority, to ensure that other parts of
		705	the program get the CPU sometimes even under high AIO load.
		706
		707	# try not to spend much more than 0.1s in poll_cb
		708	IO::AIO::max_poll_time 0.1;
		709
		710	# use a low priority so other tasks have priority
		711	Event->io (fd => IO::AIO::poll_fileno,
		712	poll => 'r', nice => 1,
		713	cb => &IO::AIO::poll_cb);
210		714
211	IO::AIO::poll_wait	715	IO::AIO::poll_wait
212	Wait till the result filehandle becomes ready for reading (simply	716	Wait till the result filehandle becomes ready for reading (simply
213	does a "select" on the filehandle. This is useful if you want to	717	does a "select" on the filehandle. This is useful if you want to
214	synchronously wait for some requests to finish).	718	synchronously wait for some requests to finish).
215		719
216	See "nreqs" for an example.	720	See "nreqs" for an example.
217		721
		722	IO::AIO::poll
		723	Waits until some requests have been handled.
		724
		725	Strictly equivalent to:
		726
		727	IO::AIO::poll_wait, IO::AIO::poll_cb
		728	if IO::AIO::nreqs;
		729
218	IO::AIO::nreqs	730	IO::AIO::flush
219	Returns the number of requests currently outstanding (i.e. for which	731	Wait till all outstanding AIO requests have been handled.
220	their callback has not been invoked yet).
221		732
222	Example: wait till there are no outstanding requests anymore:	733	Strictly equivalent to:
223		734
224	IO::AIO::poll_wait, IO::AIO::poll_cb	735	IO::AIO::poll_wait, IO::AIO::poll_cb
225	while IO::AIO::nreqs;	736	while IO::AIO::nreqs;
226		737
227	IO::AIO::flush	738	CONTROLLING THE NUMBER OF THREADS
228	Wait till all outstanding AIO requests have been handled.
229
230	Strictly equivalent to:
231
232	IO::AIO::poll_wait, IO::AIO::poll_cb
233	while IO::AIO::nreqs;
234
235	IO::AIO::poll
236	Waits until some requests have been handled.
237
238	Strictly equivalent to:
239
240	IO::AIO::poll_wait, IO::AIO::poll_cb
241	if IO::AIO::nreqs;
242
243	IO::AIO::min_parallel $nthreads	739	IO::AIO::min_parallel $nthreads
244	Set the minimum number of AIO threads to $nthreads. The default is	740	Set the minimum number of AIO threads to $nthreads. The current
245	1, which means a single asynchronous operation can be done at one	741	default is 8, which means eight asynchronous operations can execute
246	time (the number of outstanding operations, however, is unlimited).	742	concurrently at any one time (the number of outstanding requests,
		743	however, is unlimited).
247		744
		745	IO::AIO starts threads only on demand, when an AIO request is queued
		746	and no free thread exists. Please note that queueing up a hundred
		747	requests can create demand for a hundred threads, even if it turns
		748	out that everything is in the cache and could have been processed
		749	faster by a single thread.
		750
248	It is recommended to keep the number of threads low, as some Linux	751	It is recommended to keep the number of threads relatively low, as
249	kernel versions will scale negatively with the number of threads	752	some Linux kernel versions will scale negatively with the number of
250	(higher parallelity => MUCH higher latency). With current Linux 2.6	753	threads (higher parallelity => MUCH higher latency). With current
251	versions, 4-32 threads should be fine.	754	Linux 2.6 versions, 4-32 threads should be fine.
252		755
253	Under normal circumstances you don't need to call this function, as	756	Under most circumstances you don't need to call this function, as
254	this module automatically starts some threads (the exact number	757	the module selects a default that is suitable for low to moderate
255	might change, and is currently 4).	758	load.
256		759
257	IO::AIO::max_parallel $nthreads	760	IO::AIO::max_parallel $nthreads
258	Sets the maximum number of AIO threads to $nthreads. If more than	761	Sets the maximum number of AIO threads to $nthreads. If more than
259	the specified number of threads are currently running, kill them.	762	the specified number of threads are currently running, this function
260	This function blocks until the limit is reached.	763	kills them. This function blocks until the limit is reached.
		764
		765	While $nthreads are zero, aio requests get queued but not executed
		766	until the number of threads has been increased again.
261		767
262	This module automatically runs "max_parallel 0" at program end, to	768	This module automatically runs "max_parallel 0" at program end, to
263	ensure that all threads are killed and that there are no outstanding	769	ensure that all threads are killed and that there are no outstanding
264	requests.	770	requests.
265		771
266	Under normal circumstances you don't need to call this function.	772	Under normal circumstances you don't need to call this function.
267		773
		774	IO::AIO::max_idle $nthreads
		775	Limit the number of threads (default: 4) that are allowed to idle
		776	(i.e., threads that did not get a request to process within 10
		777	seconds). That means if a thread becomes idle while $nthreads other
		778	threads are also idle, it will free its resources and exit.
		779
		780	This is useful when you allow a large number of threads (e.g. 100 or
		781	1000) to allow for extremely high load situations, but want to free
		782	resources under normal circumstances (1000 threads can easily
		783	consume 30MB of RAM).
		784
		785	The default is probably ok in most situations, especially if thread
		786	creation is fast. If thread creation is very slow on your system you
		787	might want to use larger values.
		788
268	$oldnreqs = IO::AIO::max_outstanding $nreqs	789	$oldmaxreqs = IO::AIO::max_outstanding $maxreqs
		790	This is a very bad function to use in interactive programs because
		791	it blocks, and a bad way to reduce concurrency because it is
		792	inexact: Better use an "aio_group" together with a feed callback.
		793
269	Sets the maximum number of outstanding requests to $nreqs. If you	794	Sets the maximum number of outstanding requests to $nreqs. If you to
270	try to queue up more than this number of requests, the caller will	795	queue up more than this number of requests, the next call to the
271	block until some requests have been handled.	796	"poll_cb" (and "poll_some" and other functions calling "poll_cb")
		797	function will block until the limit is no longer exceeded.
272		798
273	The default is very large, so normally there is no practical limit.	799	The default value is very large, so there is no practical limit on
274	If you queue up many requests in a loop it it often improves speed	800	the number of outstanding requests.
275	if you set this to a relatively low number, such as 100.
276		801
277	Under normal circumstances you don't need to call this function.	802	You can still queue as many requests as you want. Therefore,
		803	"max_oustsanding" is mainly useful in simple scripts (with low
		804	values) or as a stop gap to shield against fatal memory overflow
		805	(with large values).
		806
		807	STATISTICAL INFORMATION
		808	IO::AIO::nreqs
		809	Returns the number of requests currently in the ready, execute or
		810	pending states (i.e. for which their callback has not been invoked
		811	yet).
		812
		813	Example: wait till there are no outstanding requests anymore:
		814
		815	IO::AIO::poll_wait, IO::AIO::poll_cb
		816	while IO::AIO::nreqs;
		817
		818	IO::AIO::nready
		819	Returns the number of requests currently in the ready state (not yet
		820	executed).
		821
		822	IO::AIO::npending
		823	Returns the number of requests currently in the pending state
		824	(executed, but not yet processed by poll_cb).
278		825
279	FORK BEHAVIOUR	826	FORK BEHAVIOUR
		827	This module should do "the right thing" when the process using it forks:
		828
280	Before the fork IO::AIO enters a quiescent state where no requests can	829	Before the fork, IO::AIO enters a quiescent state where no requests can
281	be added in other threads and no results will be processed. After the	830	be added in other threads and no results will be processed. After the
282	fork the parent simply leaves the quiescent state and continues	831	fork the parent simply leaves the quiescent state and continues
283	request/result processing, while the child clears the request/result	832	request/result processing, while the child frees the request/result
284	queue and starts the same number of threads as were in use by the	833	queue (so that the requests started before the fork will only be handled
285	parent.	834	in the parent). Threads will be started on demand until the limit set in
		835	the parent process has been reached again.
		836
		837	In short: the parent will, after a short pause, continue as if fork had
		838	not been called, while the child will act as if IO::AIO has not been
		839	used yet.
		840
		841	MEMORY USAGE
		842	Per-request usage:
		843
		844	Each aio request uses - depending on your architecture - around 100-200
		845	bytes of memory. In addition, stat requests need a stat buffer (possibly
		846	a few hundred bytes), readdir requires a result buffer and so on. Perl
		847	scalars and other data passed into aio requests will also be locked and
		848	will consume memory till the request has entered the done state.
		849
		850	This is now awfully much, so queuing lots of requests is not usually a
		851	problem.
		852
		853	Per-thread usage:
		854
		855	In the execution phase, some aio requests require more memory for
		856	temporary buffers, and each thread requires a stack and other data
		857	structures (usually around 16k-128k, depending on the OS).
		858
		859	KNOWN BUGS
		860	Known bugs will be fixed in the next release.
286		861
287	SEE ALSO	862	SEE ALSO
288	Coro, Linux::AIO.	863	Coro::AIO.
289		864
290	AUTHOR	865	AUTHOR
291	Marc Lehmann <schmorp@schmorp.de>	866	Marc Lehmann <schmorp@schmorp.de>
292	http://home.schmorp.de/	867	http://home.schmorp.de/
293		868

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.10 by root, Wed Aug 17 05:26:20 2005 UTC vs. Revision 1.20 by root, Tue Oct 31 00:45:41 2006 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.10 by root, Wed Aug 17 05:26:20 2005 UTC vs.
Revision 1.20 by root, Tue Oct 31 00:45:41 2006 UTC