[ViewVC] Diff of: cvs/IO-AIO/README

Comparing IO-AIO/README (file contents):
Revision 1.2 by root, Sun Jul 10 18:16:49 2005 UTC vs.
Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC

…		…
1	NAME	1	NAME
2	IO::AIO - Asynchronous Input/Output	2	IO::AIO - Asynchronous Input/Output
3		3
4	SYNOPSIS	4	SYNOPSIS
5	use IO::AIO;	5	use IO::AIO;
		6
		7	aio_open "/etc/passwd", O_RDONLY, 0, sub {
		8	my ($fh) = @_;
		9	...
		10	};
		11
		12	aio_unlink "/tmp/file", sub { };
		13
		14	aio_read $fh, 30000, 1024, $buffer, 0, sub {
		15	$_[0] > 0 or die "read error: $!";
		16	};
		17
		18	# AnyEvent
		19	open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!";
		20	my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
		21
		22	# Event
		23	Event->io (fd => IO::AIO::poll_fileno,
		24	poll => 'r',
		25	cb => \&IO::AIO::poll_cb);
		26
		27	# Glib/Gtk2
		28	add_watch Glib::IO IO::AIO::poll_fileno,
		29	in => sub { IO::AIO::poll_cb; 1 };
		30
		31	# Tk
		32	Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
		33	readable => \&IO::AIO::poll_cb);
		34
		35	# Danga::Socket
		36	Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
		37	\&IO::AIO::poll_cb);
6		38
7	DESCRIPTION	39	DESCRIPTION
8	This module implements asynchronous I/O using whatever means your	40	This module implements asynchronous I/O using whatever means your
9	operating system supports.	41	operating system supports.
10		42
17	currently, for example), and they would only support aio_read and	49	currently, for example), and they would only support aio_read and
18	aio_write, so the remaining functionality would have to be implemented	50	aio_write, so the remaining functionality would have to be implemented
19	using threads anyway.	51	using threads anyway.
20		52
21	Although the module will work with in the presence of other threads, it	53	Although the module will work with in the presence of other threads, it
22	is currently not reentrant, so use appropriate locking yourself.	54	is currently not reentrant, so use appropriate locking yourself, always
		55	call "poll_cb" from within the same thread, or never call "poll_cb" (or
		56	other "aio_" functions) recursively.
23		57
24	API NOTES	58	FUNCTIONS
		59	AIO FUNCTIONS
25	All the "aio_*" calls are more or less thin wrappers around the syscall	60	All the "aio_*" calls are more or less thin wrappers around the syscall
26	with the same name (sans "aio_"). The arguments are similar or	61	with the same name (sans "aio_"). The arguments are similar or
27	identical, and they all accept an additional $callback argument which	62	identical, and they all accept an additional (and optional) $callback
28	must be a code reference. This code reference will get called with the	63	argument which must be a code reference. This code reference will get
29	syscall return code (e.g. most syscalls return -1 on error, unlike perl,	64	called with the syscall return code (e.g. most syscalls return -1 on
30	which usually delivers "false") as it's sole argument when the given	65	error, unlike perl, which usually delivers "false") as it's sole
31	syscall has been executed asynchronously.	66	argument when the given syscall has been executed asynchronously.
32		67
33	All functions that expect a filehandle will also accept a file	68	All functions expecting a filehandle keep a copy of the filehandle
34	descriptor.	69	internally until the request has finished.
35		70
36	The filenames you pass to these routines must be absolute. The reason	71	The pathnames you pass to these routines must be absolute and encoded
37	is that at the time the request is being executed, the current working	72	in byte form. The reason for the former is that at the time the request
38	directory could have changed. Alternatively, you can make sure that you	73	is being executed, the current working directory could have changed.
39	never change the current working directory.	74	Alternatively, you can make sure that you never change the current
		75	working directory.
40		76
41	IO::AIO::min_parallel $nthreads	77	To encode pathnames to byte form, either make sure you either: a) always
42	Set the minimum number of AIO threads to $nthreads. The default is	78	pass in filenames you got from outside (command line, readdir etc.), b)
43	1, which means a single asynchronous operation can be done at one	79	are ASCII or ISO 8859-1, c) use the Encode module and encode your
44	time (the number of outstanding operations, however, is unlimited).	80	pathnames to the locale (or other) encoding in effect in the user
		81	environment, d) use Glib::filename_from_unicode on unicode filenames or
		82	e) use something else.
45		83
46	It is recommended to keep the number of threads low, as some linux
47	kernel versions will scale negatively with the number of threads
48	(higher parallelity => MUCH higher latency).
49
50	Under normal circumstances you don't need to call this function, as
51	this module automatically starts a single async thread.
52
53	IO::AIO::max_parallel $nthreads
54	Sets the maximum number of AIO threads to $nthreads. If more than
55	the specified number of threads are currently running, kill them.
56	This function blocks until the limit is reached.
57
58	This module automatically runs "max_parallel 0" at program end, to
59	ensure that all threads are killed and that there are no outstanding
60	requests.
61
62	Under normal circumstances you don't need to call this function.
63
64	$fileno = IO::AIO::poll_fileno
65	Return the request result pipe filehandle. This filehandle must be
66	polled for reading by some mechanism outside this module (e.g. Event
67	or select, see below). If the pipe becomes readable you have to call
68	"poll_cb" to check the results.
69
70	See "poll_cb" for an example.
71
72	IO::AIO::poll_cb
73	Process all outstanding events on the result pipe. You have to call
74	this regularly. Returns the number of events processed. Returns
75	immediately when no events are outstanding.
76
77	You can use Event to multiplex, e.g.:
78
79	Event->io (fd => IO::AIO::poll_fileno,
80	poll => 'r', async => 1,
81	cb => \&IO::AIO::poll_cb);
82
83	IO::AIO::poll_wait
84	Wait till the result filehandle becomes ready for reading (simply
85	does a select on the filehandle. This is useful if you want to
86	synchronously wait for some requests to finish).
87
88	See "nreqs" for an example.
89
90	IO::AIO::nreqs
91	Returns the number of requests currently outstanding.
92
93	Example: wait till there are no outstanding requests anymore:
94
95	IO::AIO::poll_wait, IO::AIO::poll_cb
96	while IO::AIO::nreqs;
97
98	aio_open $pathname, $flags, $mode, $callback	84	aio_open $pathname, $flags, $mode, $callback->($fh)
99	Asynchronously open or create a file and call the callback with a	85	Asynchronously open or create a file and call the callback with a
100	newly created filehandle for the file.	86	newly created filehandle for the file.
101		87
102	The pathname passed to "aio_open" must be absolute. See API NOTES,	88	The pathname passed to "aio_open" must be absolute. See API NOTES,
103	above, for an explanation.	89	above, for an explanation.
104		90
105	The $mode argument is a bitmask. See the "Fcntl" module for a list.	91	The $flags argument is a bitmask. See the "Fcntl" module for a list.
106	They are the same as used in "sysopen".	92	They are the same as used by "sysopen".
		93
		94	Likewise, $mode specifies the mode of the newly created file, if it
		95	didn't exist and "O_CREAT" has been given, just like perl's
		96	"sysopen", except that it is mandatory (i.e. use 0 if you don't
		97	create new files, and 0666 or 0777 if you do).
107		98
108	Example:	99	Example:
109		100
110	aio_open "/etc/passwd", O_RDONLY, 0, sub {	101	aio_open "/etc/passwd", O_RDONLY, 0, sub {
111	if ($_[0]) {	102	if ($_[0]) {
…		…
114	} else {	105	} else {
115	die "open failed: $!\n";	106	die "open failed: $!\n";
116	}	107	}
117	};	108	};
118		109
119	aio_close $fh, $callback	110	aio_close $fh, $callback->($status)
120	Asynchronously close a file and call the callback with the result	111	Asynchronously close a file and call the callback with the result
121	code. WARNING: although accepted, you should not pass in a perl	112	code. WARNING: although accepted, you should not pass in a perl
122	filehandle here, as perl will likely close the file descriptor	113	filehandle here, as perl will likely close the file descriptor
123	itself when the filehandle is destroyed. Normally, you can safely	114	another time when the filehandle is destroyed. Normally, you can
124	call perls "close" or just let filehandles go out of scope.	115	safely call perls "close" or just let filehandles go out of scope.
125		116
		117	This is supposed to be a bug in the API, so that might change. It's
		118	therefore best to avoid this function.
		119
126	aio_read $fh,$offset,$length, $data,$dataoffset,$callback	120	aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
127	aio_write $fh,$offset,$length, $data,$dataoffset,$callback	121	aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
128	Reads or writes "length" bytes from the specified "fh" and "offset"	122	Reads or writes "length" bytes from the specified "fh" and "offset"
129	into the scalar given by "data" and offset "dataoffset" and calls	123	into the scalar given by "data" and offset "dataoffset" and calls
130	the callback without the actual number of bytes read (or -1 on	124	the callback without the actual number of bytes read (or -1 on
131	error, just like the syscall).	125	error, just like the syscall).
132		126
		127	The $data scalar MUST NOT be modified in any way while the request
		128	is outstanding. Modifying it can result in segfaults or WW3 (if the
		129	necessary/optional hardware is installed).
		130
133	Example: Read 15 bytes at offset 7 into scalar $buffer, strating at	131	Example: Read 15 bytes at offset 7 into scalar $buffer, starting at
134	offset 0 within the scalar:	132	offset 0 within the scalar:
135		133
136	aio_read $fh, 7, 15, $buffer, 0, sub {	134	aio_read $fh, 7, 15, $buffer, 0, sub {
137	$_[0] >= 0 or die "read error: $!";	135	$_[0] > 0 or die "read error: $!";
138	print "read <$buffer>\n";	136	print "read $_[0] bytes: <$buffer>\n";
139	};	137	};
140		138
		139	aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
		140	Tries to copy $length bytes from $in_fh to $out_fh. It starts
		141	reading at byte offset $in_offset, and starts writing at the current
		142	file offset of $out_fh. Because of that, it is not safe to issue
		143	more than one "aio_sendfile" per $out_fh, as they will interfere
		144	with each other.
		145
		146	This call tries to make use of a native "sendfile" syscall to
		147	provide zero-copy operation. For this to work, $out_fh should refer
		148	to a socket, and $in_fh should refer to mmap'able file.
		149
		150	If the native sendfile call fails or is not implemented, it will be
		151	emulated, so you can call "aio_sendfile" on any type of filehandle
		152	regardless of the limitations of the operating system.
		153
		154	Please note, however, that "aio_sendfile" can read more bytes from
		155	$in_fh than are written, and there is no way to find out how many
		156	bytes have been read from "aio_sendfile" alone, as "aio_sendfile"
		157	only provides the number of bytes written to $out_fh. Only if the
		158	result value equals $length one can assume that $length bytes have
		159	been read.
		160
141	aio_readahead $fh,$offset,$length, $callback	161	aio_readahead $fh,$offset,$length, $callback->($retval)
142	Asynchronously reads the specified byte range into the page cache,
143	using the "readahead" syscall. If that syscall doesn't exist the
144	status will be -1 and $! is set to ENOSYS.
145
146	readahead() populates the page cache with data from a file so that	162	"aio_readahead" populates the page cache with data from a file so
147	subsequent reads from that file will not block on disk I/O. The	163	that subsequent reads from that file will not block on disk I/O. The
148	$offset argument specifies the starting point from which data is to	164	$offset argument specifies the starting point from which data is to
149	be read and $length specifies the number of bytes to be read. I/O is	165	be read and $length specifies the number of bytes to be read. I/O is
150	performed in whole pages, so that offset is effectively rounded down	166	performed in whole pages, so that offset is effectively rounded down
151	to a page boundary and bytes are read up to the next page boundary	167	to a page boundary and bytes are read up to the next page boundary
152	greater than or equal to (off-set+length). aio_readahead() does not	168	greater than or equal to (off-set+length). "aio_readahead" does not
153	read beyond the end of the file. The current file offset of the file	169	read beyond the end of the file. The current file offset of the file
154	is left unchanged.	170	is left unchanged.
155		171
		172	If that syscall doesn't exist (likely if your OS isn't Linux) it
		173	will be emulated by simply reading the data, which would have a
		174	similar effect.
		175
156	aio_stat $fh_or_path, $callback	176	aio_stat $fh_or_path, $callback->($status)
157	aio_lstat $fh, $callback	177	aio_lstat $fh, $callback->($status)
158	Works like perl's "stat" or "lstat" in void context. The callback	178	Works like perl's "stat" or "lstat" in void context. The callback
159	will be called after the stat and the results will be available	179	will be called after the stat and the results will be available
160	using "stat _" or "-s _" etc...	180	using "stat _" or "-s _" etc...
161		181
162	The pathname passed to "aio_stat" must be absolute. See API NOTES,	182	The pathname passed to "aio_stat" must be absolute. See API NOTES,
…		…
172	aio_stat "/etc/passwd", sub {	192	aio_stat "/etc/passwd", sub {
173	$_[0] and die "stat failed: $!";	193	$_[0] and die "stat failed: $!";
174	print "size is ", -s _, "\n";	194	print "size is ", -s _, "\n";
175	};	195	};
176		196
177	aio_unlink $pathname, $callback	197	aio_unlink $pathname, $callback->($status)
178	Asynchronously unlink (delete) a file and call the callback with the	198	Asynchronously unlink (delete) a file and call the callback with the
179	result code.	199	result code.
180		200
		201	aio_rmdir $pathname, $callback->($status)
		202	Asynchronously rmdir (delete) a directory and call the callback with
		203	the result code.
		204
		205	aio_readdir $pathname $callback->($entries)
		206	Unlike the POSIX call of the same name, "aio_readdir" reads an
		207	entire directory (i.e. opendir + readdir + closedir). The entries
		208	will not be sorted, and will NOT include the "." and ".." entries.
		209
		210	The callback a single argument which is either "undef" or an
		211	array-ref with the filenames.
		212
		213	aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
		214	Scans a directory (similar to "aio_readdir") and tries to separate
		215	the entries of directory $path into two sets of names, ones you can
		216	recurse into (directories), and ones you cannot recurse into
		217	(everything else).
		218
		219	"aio_scandir" is a composite request that consists of many
		220	aio-primitives. $maxreq specifies the maximum number of outstanding
		221	aio requests that this function generates. If it is "<= 0", then a
		222	suitable default will be chosen (currently 8).
		223
		224	On error, the callback is called without arguments, otherwise it
		225	receives two array-refs with path-relative entry names.
		226
		227	Example:
		228
		229	aio_scandir $dir, 0, sub {
		230	my ($dirs, $nondirs) = @_;
		231	print "real directories: @$dirs\n";
		232	print "everything else: @$nondirs\n";
		233	};
		234
		235	Implementation notes.
		236
		237	The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
		238	can.
		239
		240	After reading the directory, the modification time, size etc. of the
		241	directory before and after the readdir is checked, and if they
		242	match, the link count will be used to decide how many entries are
		243	directories (if >= 2). Otherwise, no knowledge of the number of
		244	subdirectories will be assumed.
		245
		246	Then entires will be sorted into likely directories (everything
		247	without a non-initial dot) and likely non-directories (everything
		248	else). Then every entry + "/." will be "stat"'ed, likely directories
		249	first. This is often faster because filesystems might detect the
		250	type of the entry without reading the inode data (e.g. ext2s
		251	filetype feature). If that succeeds, it assumes that the entry is a
		252	directory or a symlink to directory (which will be checked
		253	seperately).
		254
		255	If the known number of directories has been reached, the rest of the
		256	entries is assumed to be non-directories.
		257
181	aio_fsync $fh, $callback	258	aio_fsync $fh, $callback->($status)
182	Asynchronously call fsync on the given filehandle and call the	259	Asynchronously call fsync on the given filehandle and call the
183	callback with the fsync result code.	260	callback with the fsync result code.
184		261
185	aio_fdatasync $fh, $callback	262	aio_fdatasync $fh, $callback->($status)
186	Asynchronously call fdatasync on the given filehandle and call the	263	Asynchronously call fdatasync on the given filehandle and call the
187	callback with the fdatasync result code.	264	callback with the fdatasync result code.
188		265
189	BUGS	266	If this call isn't available because your OS lacks it or it couldn't
190	- could be optimized to use more semaphores instead of filehandles.	267	be detected, it will be emulated by calling "fsync" instead.
		268
		269	SUPPORT FUNCTIONS
		270	$fileno = IO::AIO::poll_fileno
		271	Return the request result pipe file descriptor. This filehandle
		272	must be polled for reading by some mechanism outside this module
		273	(e.g. Event or select, see below or the SYNOPSIS). If the pipe
		274	becomes readable you have to call "poll_cb" to check the results.
		275
		276	See "poll_cb" for an example.
		277
		278	IO::AIO::poll_cb
		279	Process all outstanding events on the result pipe. You have to call
		280	this regularly. Returns the number of events processed. Returns
		281	immediately when no events are outstanding.
		282
		283	Example: Install an Event watcher that automatically calls
		284	IO::AIO::poll_cb with high priority:
		285
		286	Event->io (fd => IO::AIO::poll_fileno,
		287	poll => 'r', async => 1,
		288	cb => \&IO::AIO::poll_cb);
		289
		290	IO::AIO::poll_wait
		291	Wait till the result filehandle becomes ready for reading (simply
		292	does a "select" on the filehandle. This is useful if you want to
		293	synchronously wait for some requests to finish).
		294
		295	See "nreqs" for an example.
		296
		297	IO::AIO::nreqs
		298	Returns the number of requests currently outstanding (i.e. for which
		299	their callback has not been invoked yet).
		300
		301	Example: wait till there are no outstanding requests anymore:
		302
		303	IO::AIO::poll_wait, IO::AIO::poll_cb
		304	while IO::AIO::nreqs;
		305
		306	IO::AIO::flush
		307	Wait till all outstanding AIO requests have been handled.
		308
		309	Strictly equivalent to:
		310
		311	IO::AIO::poll_wait, IO::AIO::poll_cb
		312	while IO::AIO::nreqs;
		313
		314	IO::AIO::poll
		315	Waits until some requests have been handled.
		316
		317	Strictly equivalent to:
		318
		319	IO::AIO::poll_wait, IO::AIO::poll_cb
		320	if IO::AIO::nreqs;
		321
		322	IO::AIO::min_parallel $nthreads
		323	Set the minimum number of AIO threads to $nthreads. The current
		324	default is 4, which means four asynchronous operations can be done
		325	at one time (the number of outstanding operations, however, is
		326	unlimited).
		327
		328	IO::AIO starts threads only on demand, when an AIO request is queued
		329	and no free thread exists.
		330
		331	It is recommended to keep the number of threads low, as some Linux
		332	kernel versions will scale negatively with the number of threads
		333	(higher parallelity => MUCH higher latency). With current Linux 2.6
		334	versions, 4-32 threads should be fine.
		335
		336	Under most circumstances you don't need to call this function, as
		337	the module selects a default that is suitable for low to moderate
		338	load.
		339
		340	IO::AIO::max_parallel $nthreads
		341	Sets the maximum number of AIO threads to $nthreads. If more than
		342	the specified number of threads are currently running, this function
		343	kills them. This function blocks until the limit is reached.
		344
		345	While $nthreads are zero, aio requests get queued but not executed
		346	until the number of threads has been increased again.
		347
		348	This module automatically runs "max_parallel 0" at program end, to
		349	ensure that all threads are killed and that there are no outstanding
		350	requests.
		351
		352	Under normal circumstances you don't need to call this function.
		353
		354	$oldnreqs = IO::AIO::max_outstanding $nreqs
		355	Sets the maximum number of outstanding requests to $nreqs. If you
		356	try to queue up more than this number of requests, the caller will
		357	block until some requests have been handled.
		358
		359	The default is very large, so normally there is no practical limit.
		360	If you queue up many requests in a loop it often improves speed if
		361	you set this to a relatively low number, such as 100.
		362
		363	Under normal circumstances you don't need to call this function.
		364
		365	FORK BEHAVIOUR
		366	Before the fork, IO::AIO enters a quiescent state where no requests can
		367	be added in other threads and no results will be processed. After the
		368	fork the parent simply leaves the quiescent state and continues
		369	request/result processing, while the child clears the request/result
		370	queue (so the requests started before the fork will only be handled in
		371	the parent). Threats will be started on demand until the limit ste in
		372	the parent process has been reached again.
191		373
192	SEE ALSO	374	SEE ALSO
193	Coro, Linux::AIO.	375	Coro, Linux::AIO.
194		376
195	AUTHOR	377	AUTHOR

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/README (file contents): Revision 1.2 by root, Sun Jul 10 18:16:49 2005 UTC vs. Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC

Diff Legend

Comparing IO-AIO/README (file contents):
Revision 1.2 by root, Sun Jul 10 18:16:49 2005 UTC vs.
Revision 1.14 by root, Mon Dec 26 18:31:12 2005 UTC