[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.1 by root, Sun Jul 10 17:07:44 2005 UTC vs.
Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC

…		…
2	IO::AIO - Asynchronous Input/Output	2	IO::AIO - Asynchronous Input/Output
3		3
4	SYNOPSIS	4	SYNOPSIS
5	use IO::AIO;	5	use IO::AIO;
6		6
		7	aio_open "/etc/passwd", O_RDONLY, 0, sub {
		8	my ($fh) = @_;
		9	...
		10	};
		11
		12	aio_unlink "/tmp/file", sub { };
		13
		14	aio_read $fh, 30000, 1024, $buffer, 0, sub {
		15	$_[0] > 0 or die "read error: $!";
		16	};
		17
		18	# AnyEvent
		19	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
		20	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
		21
		22	# Event
		23	Event->io (fd => IO::AIO::poll_fileno,
		24	poll => 'r',
		25	cb => \&IO::AIO::poll_cb);
		26
		27	# Glib/Gtk2
		28	add_watch Glib::IO IO::AIO::poll_fileno,
		29	in => sub { IO::AIO::poll_cb; 1 };
		30
		31	# Tk
		32	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
		33	readable => \&IO::AIO::poll_cb);
		34
		35	# Danga::Socket
		36	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
		37	\&IO::AIO::poll_cb);
		38
7	DESCRIPTION	39	DESCRIPTION
8	This module implements asynchronous I/O using whatever means your	40	This module implements asynchronous I/O using whatever means your
9	operating system supports. Currently, it falls back to Linux::AIO if	41	operating system supports.
10	that module is available, or uses pthreads to emulato aio functionality.
11		42
12	Currently, in this module a number of threads are started that execute	43	Currently, a number of threads are started that execute your read/writes
13	your read/writes and signal their completion. You don't need thread	44	and signal their completion. You don't need thread support in your libc
14	support in your libc or perl, and the threads created by this module	45	or perl, and the threads created by this module will not be visible to
15	will not be visible to the pthreads library.	46	the pthreads library. In the future, this module might make use of the
		47	native aio functions available on many operating systems. However, they
		48	are often not well-supported (Linux doesn't allow them on normal files
		49	currently, for example), and they would only support aio_read and
		50	aio_write, so the remaining functionality would have to be implemented
		51	using threads anyway.
16		52
17	Although the module will work with in the presence of other threads, it	53	Although the module will work with in the presence of other threads, it
18	is not reentrant, so use appropriate locking yourself.	54	is currently not reentrant, so use appropriate locking yourself, always
		55	call "poll_cb" from within the same thread, or never call "poll_cb" (or
		56	other "aio_" functions) recursively.
19		57
20	API NOTES	58	FUNCTIONS
		59	AIO FUNCTIONS
21	All the "aio_*" calls are more or less thin wrappers around the syscall	60	All the "aio_*" calls are more or less thin wrappers around the syscall
22	with the same name (sans "aio_"). The arguments are similar or	61	with the same name (sans "aio_"). The arguments are similar or
23	identical, and they all accept an additional $callback argument which	62	identical, and they all accept an additional (and optional) $callback
24	must be a code reference. This code reference will get called with the	63	argument which must be a code reference. This code reference will get
25	syscall return code (e.g. most syscalls return -1 on error, unlike perl,	64	called with the syscall return code (e.g. most syscalls return -1 on
26	which usually delivers "false") as it's sole argument when the given	65	error, unlike perl, which usually delivers "false") as it's sole
27	syscall has been executed asynchronously.	66	argument when the given syscall has been executed asynchronously.
28		67
29	All functions that expect a filehandle will also accept a file	68	All functions expecting a filehandle keep a copy of the filehandle
30	descriptor.	69	internally until the request has finished.
31		70
32	The filenames you pass to these routines must be absolute. The reason	71	The pathnames you pass to these routines must be absolute and encoded
33	is that at the time the request is being executed, the current working	72	in byte form. The reason for the former is that at the time the request
34	directory could have changed. Alternatively, you can make sure that you	73	is being executed, the current working directory could have changed.
35	never change the current working directory.	74	Alternatively, you can make sure that you never change the current
		75	working directory.
36		76
37	IO::AIO::min_parallel $nthreads	77	To encode pathnames to byte form, either make sure you either: a) always
38	Set the minimum number of AIO threads to $nthreads. The default is	78	pass in filenames you got from outside (command line, readdir etc.), b)
39	1, which means a single asynchronous operation can be done at one	79	are ASCII or ISO 8859-1, c) use the Encode module and encode your
40	time (the number of outstanding operations, however, is unlimited).	80	pathnames to the locale (or other) encoding in effect in the user
		81	environment, d) use Glib::filename_from_unicode on unicode filenames or
		82	e) use something else.
41		83
42	It is recommended to keep the number of threads low, as some linux
43	kernel versions will scale negatively with the number of threads
44	(higher parallelity => MUCH higher latency).
45
46	Under normal circumstances you don't need to call this function, as
47	this module automatically starts a single async thread.
48
49	IO::AIO::max_parallel $nthreads
50	Sets the maximum number of AIO threads to $nthreads. If more than
51	the specified number of threads are currently running, kill them.
52	This function blocks until the limit is reached.
53
54	This module automatically runs "max_parallel 0" at program end, to
55	ensure that all threads are killed and that there are no outstanding
56	requests.
57
58	Under normal circumstances you don't need to call this function.
59
60	$fileno = IO::AIO::poll_fileno
61	Return the request result pipe filehandle. This filehandle must be
62	polled for reading by some mechanism outside this module (e.g. Event
63	or select, see below). If the pipe becomes readable you have to call
64	"poll_cb" to check the results.
65
66	See "poll_cb" for an example.
67
68	IO::AIO::poll_cb
69	Process all outstanding events on the result pipe. You have to call
70	this regularly. Returns the number of events processed. Returns
71	immediately when no events are outstanding.
72
73	You can use Event to multiplex, e.g.:
74
75	Event->io (fd => IO::AIO::poll_fileno,
76	poll => 'r', async => 1,
77	cb => \&IO::AIO::poll_cb);
78
79	IO::AIO::poll_wait
80	Wait till the result filehandle becomes ready for reading (simply
81	does a select on the filehandle. This is useful if you want to
82	synchronously wait for some requests to finish).
83
84	See "nreqs" for an example.
85
86	IO::AIO::nreqs
87	Returns the number of requests currently outstanding.
88
89	Example: wait till there are no outstanding requests anymore:
90
91	IO::AIO::poll_wait, IO::AIO::poll_cb
92	while IO::AIO::nreqs;
93
94	aio_open $pathname, $flags, $mode, $callback	84	aio_open $pathname, $flags, $mode, $callback->($fh)
95	Asynchronously open or create a file and call the callback with the	85	Asynchronously open or create a file and call the callback with a
96	filedescriptor (NOT a perl filehandle, sorry for that, but watch	86	newly created filehandle for the file.
97	out, this might change in the future).
98		87
		88	The pathname passed to "aio_open" must be absolute. See API NOTES,
		89	above, for an explanation.
		90
99	The $mode argument is a bitmask. See the "Fcntl" module for a list.	91	The $flags argument is a bitmask. See the "Fcntl" module for a list.
100	They are the same as used in "sysopen".	92	They are the same as used by "sysopen".
		93
		94	Likewise, $mode specifies the mode of the newly created file, if it
		95	didn't exist and "O_CREAT" has been given, just like perl's
		96	"sysopen", except that it is mandatory (i.e. use 0 if you don't
		97	create new files, and 0666 or 0777 if you do).
101		98
102	Example:	99	Example:
103		100
104	aio_open "/etc/passwd", O_RDONLY, 0, sub {	101	aio_open "/etc/passwd", O_RDONLY, 0, sub {
105	if ($_[0] >= 0) {	102	if ($_[0]) {
106	open my $fh, "<&$_[0]"; # create a copy for perl
107	aio_close $_[0], sub { }; # close the aio handle
108	print "open successful, fh is $fh\n";	103	print "open successful, fh is $_[0]\n";
109	...	104	...
110	} else {	105	} else {
111	die "open failed: $!\n";	106	die "open failed: $!\n";
112	}	107	}
113	};	108	};
114		109
115	aio_close $fh, $callback	110	aio_close $fh, $callback->($status)
116	Asynchronously close a file and call the callback with the result	111	Asynchronously close a file and call the callback with the result
117	code.	112	code. WARNING: although accepted, you should not pass in a perl
		113	filehandle here, as perl will likely close the file descriptor
		114	another time when the filehandle is destroyed. Normally, you can
		115	safely call perls "close" or just let filehandles go out of scope.
118		116
		117	This is supposed to be a bug in the API, so that might change. It's
		118	therefore best to avoid this function.
		119
119	aio_read $fh,$offset,$length, $data,$dataoffset,$callback	120	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
120	aio_write $fh,$offset,$length, $data,$dataoffset,$callback	121	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
121	Reads or writes "length" bytes from the specified "fh" and "offset"	122	Reads or writes "length" bytes from the specified "fh" and "offset"
122	into the scalar given by "data" and offset "dataoffset" and calls	123	into the scalar given by "data" and offset "dataoffset" and calls
123	the callback without the actual number of bytes read (or -1 on	124	the callback without the actual number of bytes read (or -1 on
124	error, just like the syscall).	125	error, just like the syscall).
125		126
		127	The $data scalar MUST NOT be modified in any way while the request
		128	is outstanding. Modifying it can result in segfaults or WW3 (if the
		129	necessary/optional hardware is installed).
		130
126	Example: Read 15 bytes at offset 7 into scalar $buffer, strating at	131	Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
127	offset 0 within the scalar:	132	offset 0 within the scalar:
128		133
129	aio_read $fh, 7, 15, $buffer, 0, sub {	134	aio_read $fh, 7, 15, $buffer, 0, sub {
130	$_[0] >= 0 or die "read error: $!";	135	$_[0] > 0 or die "read error: $!";
131	print "read <$buffer>\n";	136	print "read $_[0] bytes: <$buffer>\n";
132	};	137	};
133		138
		139	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
		140	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		141	reading at byte offset $in_offset, and starts writing at the current
		142	file offset of $out_fh. Because of that, it is not safe to issue
		143	more than one "aio_sendfile" per $out_fh, as they will interfere
		144	with each other.
		145
		146	This call tries to make use of a native "sendfile" syscall to
		147	provide zero-copy operation. For this to work, $out_fh should refer
		148	to a socket, and $in_fh should refer to mmap'able file.
		149
		150	If the native sendfile call fails or is not implemented, it will be
		151	emulated, so you can call "aio_sendfile" on any type of filehandle
		152	regardless of the limitations of the operating system.
		153
		154	Please note, however, that "aio_sendfile" can read more bytes from
		155	$in_fh than are written, and there is no way to find out how many
		156	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
		157	only provides the number of bytes written to $out_fh. Only if the
		158	result value equals $length one can assume that $length bytes have
		159	been read.
		160
134	aio_readahead $fh,$offset,$length, $callback	161	aio_readahead $fh,$offset,$length, $callback->($retval)
135	Asynchronously reads the specified byte range into the page cache,
136	using the "readahead" syscall.
137
138	readahead() populates the page cache with data from a file so that	162	"aio_readahead" populates the page cache with data from a file so
139	subsequent reads from that file will not block on disk I/O. The	163	that subsequent reads from that file will not block on disk I/O. The
140	$offset argument specifies the starting point from which data is to	164	$offset argument specifies the starting point from which data is to
141	be read and $length specifies the number of bytes to be read. I/O is	165	be read and $length specifies the number of bytes to be read. I/O is
142	performed in whole pages, so that offset is effectively rounded down	166	performed in whole pages, so that offset is effectively rounded down
143	to a page boundary and bytes are read up to the next page boundary	167	to a page boundary and bytes are read up to the next page boundary
144	greater than or equal to (off-set+length). aio_readahead() does not	168	greater than or equal to (off-set+length). "aio_readahead" does not
145	read beyond the end of the file. The current file offset of the file	169	read beyond the end of the file. The current file offset of the file
146	is left unchanged.	170	is left unchanged.
147		171
		172	If that syscall doesn't exist (likely if your OS isn't Linux) it
		173	will be emulated by simply reading the data, which would have a
		174	similar effect.
		175
148	aio_stat $fh_or_path, $callback	176	aio_stat $fh_or_path, $callback->($status)
149	aio_lstat $fh, $callback	177	aio_lstat $fh, $callback->($status)
150	Works like perl's "stat" or "lstat" in void context. The callback	178	Works like perl's "stat" or "lstat" in void context. The callback
151	will be called after the stat and the results will be available	179	will be called after the stat and the results will be available
152	using "stat _" or "-s _" etc...	180	using "stat _" or "-s _" etc...
153		181
154	The pathname passed to "aio_stat" must be absolute. See API NOTES,	182	The pathname passed to "aio_stat" must be absolute. See API NOTES,
…		…
164	aio_stat "/etc/passwd", sub {	192	aio_stat "/etc/passwd", sub {
165	$_[0] and die "stat failed: $!";	193	$_[0] and die "stat failed: $!";
166	print "size is ", -s _, "\n";	194	print "size is ", -s _, "\n";
167	};	195	};
168		196
169	aio_unlink $pathname, $callback	197	aio_unlink $pathname, $callback->($status)
170	Asynchronously unlink (delete) a file and call the callback with the	198	Asynchronously unlink (delete) a file and call the callback with the
171	result code.	199	result code.
172		200
		201	aio_rmdir $pathname, $callback->($status)
		202	Asynchronously rmdir (delete) a directory and call the callback with
		203	the result code.
		204
		205	aio_readdir $pathname $callback->($entries)
		206	Unlike the POSIX call of the same name, "aio_readdir" reads an
		207	entire directory (i.e. opendir + readdir + closedir). The entries
		208	will not be sorted, and will NOT include the "." and ".." entries.
		209
		210	The callback a single argument which is either "undef" or an
		211	array-ref with the filenames.
		212
		213	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		214	Scans a directory (similar to "aio_readdir") and tries to separate
		215	the entries of directory $path into two sets of names, ones you can
		216	recurse into (directories), and ones you cannot recurse into
		217	(everything else).
		218
		219	"aio_scandir" is a composite request that consists of many
		220	aio-primitives. $maxreq specifies the maximum number of outstanding
		221	aio requests that this function generates. If it is "<= 0", then a
		222	suitable default will be chosen (currently 8).
		223
		224	On error, the callback is called without arguments, otherwise it
		225	receives two array-refs with path-relative entry names.
		226
		227	Example:
		228
		229	aio_scandir $dir, 0, sub {
		230	my ($dirs, $nondirs) = @_;
		231	print "real directories: @$dirs\n";
		232	print "everything else: @$nondirs\n";
		233	};
		234
		235	Implementation notes.
		236
		237	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
		238	can.
		239
		240	After reading the directory, the modification time, size etc. of the
		241	directory before and after the readdir is checked, and if they
		242	match, the link count will be used to decide how many entries are
		243	directories (if >= 2). Otherwise, no knowledge of the number of
		244	subdirectories will be assumed.
		245
		246	Then entires will be sorted into likely directories (everything
		247	without a non-initial dot) and likely non-directories (everything
		248	else). Then every entry + "/." will be "stat"'ed, likely directories
		249	first. This is often faster because filesystems might detect the
		250	type of the entry without reading the inode data (e.g. ext2s
		251	filetype feature). If that succeeds, it assumes that the entry is a
		252	directory or a symlink to directory (which will be checked
		253	seperately).
		254
		255	If the known number of directories has been reached, the rest of the
		256	entries is assumed to be non-directories.
		257
173	aio_fsync $fh, $callback	258	aio_fsync $fh, $callback->($status)
174	Asynchronously call fsync on the given filehandle and call the	259	Asynchronously call fsync on the given filehandle and call the
175	callback with the fsync result code.	260	callback with the fsync result code.
176		261
177	aio_fdatasync $fh, $callback	262	aio_fdatasync $fh, $callback->($status)
178	Asynchronously call fdatasync on the given filehandle and call the	263	Asynchronously call fdatasync on the given filehandle and call the
179	callback with the fdatasync result code.	264	callback with the fdatasync result code.
180		265
181	BUGS	266	If this call isn't available because your OS lacks it or it couldn't
182	This module has been extensively tested in a large and very busy	267	be detected, it will be emulated by calling "fsync" instead.
183	webserver for many years now.
184		268
185	- aio_open gives a fd, but all other functions expect a perl filehandle.	269	SUPPORT FUNCTIONS
		270	$fileno = IO::AIO::poll_fileno
		271	Return the request result pipe file descriptor. This filehandle
		272	must be polled for reading by some mechanism outside this module
		273	(e.g. Event or select, see below or the SYNOPSIS). If the pipe
		274	becomes readable you have to call "poll_cb" to check the results.
		275
		276	See "poll_cb" for an example.
		277
		278	IO::AIO::poll_cb
		279	Process all outstanding events on the result pipe. You have to call
		280	this regularly. Returns the number of events processed. Returns
		281	immediately when no events are outstanding.
		282
		283	Example: Install an Event watcher that automatically calls
		284	IO::AIO::poll_cb with high priority:
		285
		286	Event->io (fd => IO::AIO::poll_fileno,
		287	poll => 'r', async => 1,
		288	cb => \&IO::AIO::poll_cb);
		289
		290	IO::AIO::poll_wait
		291	Wait till the result filehandle becomes ready for reading (simply
		292	does a "select" on the filehandle. This is useful if you want to
		293	synchronously wait for some requests to finish).
		294
		295	See "nreqs" for an example.
		296
		297	IO::AIO::nreqs
		298	Returns the number of requests currently outstanding (i.e. for which
		299	their callback has not been invoked yet).
		300
		301	Example: wait till there are no outstanding requests anymore:
		302
		303	IO::AIO::poll_wait, IO::AIO::poll_cb
		304	while IO::AIO::nreqs;
		305
		306	IO::AIO::flush
		307	Wait till all outstanding AIO requests have been handled.
		308
		309	Strictly equivalent to:
		310
		311	IO::AIO::poll_wait, IO::AIO::poll_cb
		312	while IO::AIO::nreqs;
		313
		314	IO::AIO::poll
		315	Waits until some requests have been handled.
		316
		317	Strictly equivalent to:
		318
		319	IO::AIO::poll_wait, IO::AIO::poll_cb
		320	if IO::AIO::nreqs;
		321
		322	IO::AIO::min_parallel $nthreads
		323	Set the minimum number of AIO threads to $nthreads. The current
		324	default is 4, which means four asynchronous operations can be done
		325	at one time (the number of outstanding operations, however, is
		326	unlimited).
		327
		328	IO::AIO starts threads only on demand, when an AIO request is queued
		329	and no free thread exists.
		330
		331	It is recommended to keep the number of threads low, as some Linux
		332	kernel versions will scale negatively with the number of threads
		333	(higher parallelity => MUCH higher latency). With current Linux 2.6
		334	versions, 4-32 threads should be fine.
		335
		336	Under most circumstances you don't need to call this function, as
		337	the module selects a default that is suitable for low to moderate
		338	load.
		339
		340	IO::AIO::max_parallel $nthreads
		341	Sets the maximum number of AIO threads to $nthreads. If more than
		342	the specified number of threads are currently running, this function
		343	kills them. This function blocks until the limit is reached.
		344
		345	While $nthreads are zero, aio requests get queued but not executed
		346	until the number of threads has been increased again.
		347
		348	This module automatically runs "max_parallel 0" at program end, to
		349	ensure that all threads are killed and that there are no outstanding
		350	requests.
		351
		352	Under normal circumstances you don't need to call this function.
		353
		354	$oldnreqs = IO::AIO::max_outstanding $nreqs
		355	Sets the maximum number of outstanding requests to $nreqs. If you
		356	try to queue up more than this number of requests, the caller will
		357	block until some requests have been handled.
		358
		359	The default is very large, so normally there is no practical limit.
		360	If you queue up many requests in a loop it often improves speed if
		361	you set this to a relatively low number, such as 100.
		362
		363	Under normal circumstances you don't need to call this function.
		364
		365	FORK BEHAVIOUR
		366	Before the fork, IO::AIO enters a quiescent state where no requests can
		367	be added in other threads and no results will be processed. After the
		368	fork the parent simply leaves the quiescent state and continues
		369	request/result processing, while the child clears the request/result
		370	queue (so the requests started before the fork will only be handled in
		371	the parent). Threats will be started on demand until the limit ste in
		372	the parent process has been reached again.
186		373
187	SEE ALSO	374	SEE ALSO
188	Coro.	375	Coro, Linux::AIO.
189		376
190	AUTHOR	377	AUTHOR
191	Marc Lehmann <schmorp@schmorp.de>	378	Marc Lehmann <schmorp@schmorp.de>
192	http://home.schmorp.de/	379	http://home.schmorp.de/
193		380

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.1 by root, Sun Jul 10 17:07:44 2005 UTC vs. Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.1 by root, Sun Jul 10 17:07:44 2005 UTC vs.
Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC