[ViewVC] Diff of: cvs/libeio/eio.pod

Comparing libeio/eio.pod (file contents):
Revision 1.12 by root, Wed Jun 29 10:32:55 2011 UTC vs.
Revision 1.24 by sf-exg, Thu Jul 14 08:51:29 2011 UTC

…		…
45	Unlike the name component C<stamp> might indicate, it is also used for	45	Unlike the name component C<stamp> might indicate, it is also used for
46	time differences throughout libeio.	46	time differences throughout libeio.
47		47
48	=head2 FORK SUPPORT	48	=head2 FORK SUPPORT
49		49
50	Calling C<fork ()> is fully supported by this module. It is implemented in these steps:	50	Calling C<fork ()> is fully supported by this module - but you must not
		51	rely on this. It is currently implemented in these steps:
51		52
52	1. wait till all requests in "execute" state have been handled	53	1. wait till all requests in "execute" state have been handled
53	(basically requests that are already handed over to the kernel).	54	(basically requests that are already handed over to the kernel).
54	2. fork	55	2. fork
55	3. in the parent, continue business as usual, done	56	3. in the parent, continue business as usual, done
56	4. in the child, destroy all ready and pending requests and free the	57	4. in the child, destroy all ready and pending requests and free the
57	memory used by the worker threads. This gives you a fully empty	58	memory used by the worker threads. This gives you a fully empty
58	libeio queue.	59	libeio queue.
59		60
60	Note, however, since libeio does use threads, thr above guarantee doesn't	61	Note, however, since libeio does use threads, the above guarantee doesn't
61	cover your libc, for example, malloc and other libc functions are not	62	cover your libc, for example, malloc and other libc functions are not
62	fork-safe, so there is very little you can do after a fork, and in fatc,	63	fork-safe, so there is very little you can do after a fork, and in fact,
63	the above might crash, and thus change.	64	the above might crash, and thus change.
64		65
65	=head1 INITIALISATION/INTEGRATION	66	=head1 INITIALISATION/INTEGRATION
66		67
67	Before you can call any eio functions you first have to initialise the	68	Before you can call any eio functions you first have to initialise the
…		…
124	=back	125	=back
125		126
126	For libev, you would typically use an C<ev_async> watcher: the	127	For libev, you would typically use an C<ev_async> watcher: the
127	C<want_poll> callback would invoke C<ev_async_send> to wake up the event	128	C<want_poll> callback would invoke C<ev_async_send> to wake up the event
128	loop. Inside the callback set for the watcher, one would call C<eio_poll	129	loop. Inside the callback set for the watcher, one would call C<eio_poll
129	()> (followed by C<ev_async_send> again if C<eio_poll> indicates that not	130	()>.
130	all requests have been handled yet). The race is taken care of because	131
131	libev resets/rearms the async watcher before calling your callback,	132	If C<eio_poll ()> is configured to not handle all results in one go
132	and therefore, before calling C<eio_poll>. This might result in (some)	133	(i.e. it returns C<-1>) then you should start an idle watcher that calls
133	spurious wake-ups, but is generally harmless.	134	C<eio_poll> until it returns something C<!= -1>.
		135
		136	A full-featured connector between libeio and libev would look as follows
		137	(if C<eio_poll> is handling all requests, it can of course be simplified a
		138	lot by removing the idle watcher logic):
		139
		140	static struct ev_loop *loop;
		141	static ev_idle repeat_watcher;
		142	static ev_async ready_watcher;
		143
		144	/* idle watcher callback, only used when eio_poll */
		145	/* didn't handle all results in one call */
		146	static void
		147	repeat (EV_P_ ev_idle *w, int revents)
		148	{
		149	if (eio_poll () != -1)
		150	ev_idle_stop (EV_A_ w);
		151	}
		152
		153	/* eio has some results, process them */
		154	static void
		155	ready (EV_P_ ev_async *w, int revents)
		156	{
		157	if (eio_poll () == -1)
		158	ev_idle_start (EV_A_ &repeat_watcher);
		159	}
		160
		161	/* wake up the event loop */
		162	static void
		163	want_poll (void)
		164	{
		165	ev_async_send (loop, &ready_watcher)
		166	}
		167
		168	void
		169	my_init_eio ()
		170	{
		171	loop = EV_DEFAULT;
		172
		173	ev_idle_init (&repeat_watcher, repeat);
		174	ev_async_init (&ready_watcher, ready);
		175	ev_async_start (loop &watcher);
		176
		177	eio_init (want_poll, 0);
		178	}
134		179
135	For most other event loops, you would typically use a pipe - the event	180	For most other event loops, you would typically use a pipe - the event
136	loop should be told to wait for read readiness on the read end. In	181	loop should be told to wait for read readiness on the read end. In
137	C<want_poll> you would write a single byte, in C<done_poll> you would try	182	C<want_poll> you would write a single byte, in C<done_poll> you would try
138	to read that byte, and in the callback for the read end, you would call	183	to read that byte, and in the callback for the read end, you would call
139	C<eio_poll>. The race is avoided here because the event loop should invoke	184	C<eio_poll>.
140	your callback again and again until the byte has been read (as the pipe	185
141	read callback does not read it, only C<done_poll>).	186	You don't have to take special care in the case C<eio_poll> doesn't handle
		187	all requests, as the done callback will not be invoked, so the event loop
		188	will still signal readiness for the pipe until I<all> results have been
		189	processed.
142		190
143		191
144	=head1 HIGH LEVEL REQUEST API	192	=head1 HIGH LEVEL REQUEST API
145		193
146	Libeio has both a high-level API, which consists of calling a request	194	Libeio has both a high-level API, which consists of calling a request
…		…
213	}	261	}
214		262
215	/* the first three arguments are passed to open(2) */	263	/* the first three arguments are passed to open(2) */
216	/* the remaining are priority, callback and data */	264	/* the remaining are priority, callback and data */
217	if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0))	265	if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0))
218	abort (); /* something ent wrong, we will all die!!! */	266	abort (); /* something went wrong, we will all die!!! */
219		267
220	Note that you additionally need to call C<eio_poll> when the C<want_cb>	268	Note that you additionally need to call C<eio_poll> when the C<want_cb>
221	indicates that requests are ready to be processed.	269	indicates that requests are ready to be processed.
		270
		271	=head2 CANCELLING REQUESTS
		272
		273	Sometimes the need for a request goes away before the request is
		274	finished. In that case, one can cancel the request by a call to
		275	C<eio_cancel>:
		276
		277	=over 4
		278
		279	=item eio_cancel (eio_req *req)
		280
		281	Cancel the request (and all its subrequests). If the request is currently
		282	executing it might still continue to execute, and in other cases it might
		283	still take a while till the request is cancelled.
		284
		285	Even if cancelled, the finish callback will still be invoked - the
		286	callbacks of all cancellable requests need to check whether the request
		287	has been cancelled by calling C<EIO_CANCELLED (req)>:
		288
		289	static int
		290	my_eio_cb (eio_req *req)
		291	{
		292	if (EIO_CANCELLED (req))
		293	return 0;
		294	}
		295
		296	In addition, cancelled requests will I<either> have C<< req->result >>
		297	set to C<-1> and C<errno> to C<ECANCELED>, or I<otherwise> they were
		298	successfully executed, despite being cancelled (e.g. when they have
		299	already been executed at the time they were cancelled).
		300
		301	C<EIO_CANCELLED> is still true for requests that have successfully
		302	executed, as long as C<eio_cancel> was called on them at some point.
		303
		304	=back
222		305
223	=head2 AVAILABLE REQUESTS	306	=head2 AVAILABLE REQUESTS
224		307
225	The following request functions are available. I<All> of them return the	308	The following request functions are available. I<All> of them return the
226	C<eio_req *> on success and C<0> on failure, and I<all> of them have the	309	C<eio_req *> on success and C<0> on failure, and I<all> of them have the
…		…
320	char target = strndup ((char )req->ptr2, req->result);	403	char target = strndup ((char )req->ptr2, req->result);
321		404
322	free (target);	405	free (target);
323	}	406	}
324		407
		408	=item eio_realpath (const char path, int pri, eio_cb cb, void data)
		409
		410	Similar to the realpath libc function, but unlike that one, C<<
		411	req->result >> is C<-1> on failure. On success, the result is the length
		412	of the returned path in C<ptr2> (which is I<NOT> 0-terminated) - this is
		413	similar to readlink.
		414
325	=item eio_stat (const char path, int pri, eio_cb cb, void data)	415	=item eio_stat (const char path, int pri, eio_cb cb, void data)
326		416
327	=item eio_lstat (const char path, int pri, eio_cb cb, void data)	417	=item eio_lstat (const char path, int pri, eio_cb cb, void data)
328		418
329	=item eio_fstat (int fd, int pri, eio_cb cb, void *data)	419	=item eio_fstat (int fd, int pri, eio_cb cb, void *data)
330		420
331	Stats a file - if C<< req->result >> indicates success, then you can	421	Stats a file - if C<< req->result >> indicates success, then you can
332	access the C<struct stat>-like structure via C<< req->ptr2 >>:	422	access the C<struct stat>-like structure via C<< req->ptr2 >>:
333		423
334	EIO_STRUCT_STAT statdata = (EIO_STRUCT_STAT )req->ptr2;	424	EIO_STRUCT_STAT statdata = (EIO_STRUCT_STAT )req->ptr2;
335		425
336	=item eio_statvfs (const char path, int pri, eio_cb cb, void data)	426	=item eio_statvfs (const char path, int pri, eio_cb cb, void data)
337		427
338	=item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data)	428	=item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data)
339		429
340	Stats a filesystem - if C<< req->result >> indicates success, then you can	430	Stats a filesystem - if C<< req->result >> indicates success, then you can
341	access the C<struct statvfs>-like structure via C<< req->ptr2 >>:	431	access the C<struct statvfs>-like structure via C<< req->ptr2 >>:
342		432
343	EIO_STRUCT_STATVFS statdata = (EIO_STRUCT_STATVFS )req->ptr2;	433	EIO_STRUCT_STATVFS statdata = (EIO_STRUCT_STATVFS )req->ptr2;
344		434
345	=back	435	=back
346		436
347	=head3 READING DIRECTORIES	437	=head3 READING DIRECTORIES
348		438
349	Reading directories sounds simple, but can be rather demanding, especially	439	Reading directories sounds simple, but can be rather demanding, especially
350	if you want to do stuff such as traversing a diretcory hierarchy or	440	if you want to do stuff such as traversing a directory hierarchy or
351	processing all files in a directory. Libeio can assist thess complex tasks	441	processing all files in a directory. Libeio can assist these complex tasks
352	with it's C<eio_readdir> call.	442	with it's C<eio_readdir> call.
353		443
354	=over 4	444	=over 4
355		445
356	=item eio_readdir (const char path, int flags, int pri, eio_cb cb, void data)	446	=item eio_readdir (const char path, int flags, int pri, eio_cb cb, void data)
…		…
388		478
389	If this flag is specified, then, in addition to the names in C<ptr2>,	479	If this flag is specified, then, in addition to the names in C<ptr2>,
390	also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct	480	also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct
391	eio_dirent> looks like this:	481	eio_dirent> looks like this:
392		482
393	struct eio_dirent	483	struct eio_dirent
394	{	484	{
395	int nameofs; /* offset of null-terminated name string in (char )req->ptr2 /	485	int nameofs; /* offset of null-terminated name string in (char )req->ptr2 /
396	unsigned short namelen; /* size of filename without trailing 0 */	486	unsigned short namelen; /* size of filename without trailing 0 */
397	unsigned char type; /* one of EIO_DT_* */	487	unsigned char type; /* one of EIO_DT_* */
398	signed char score; /* internal use */	488	signed char score; /* internal use */
399	ino_t inode; /* the inode number, if available, otherwise unspecified */	489	ino_t inode; /* the inode number, if available, otherwise unspecified */
400	};	490	};
401		491
402	The only members you normally would access are C<nameofs>, which is the	492	The only members you normally would access are C<nameofs>, which is the
403	byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>.	493	byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>.
404		494
405	C<type> can be one of:	495	C<type> can be one of:
…		…
448	When this flag is specified, then the names will be returned in an order	538	When this flag is specified, then the names will be returned in an order
449	suitable for stat()'ing each one. That is, when you plan to stat()	539	suitable for stat()'ing each one. That is, when you plan to stat()
450	all files in the given directory, then the returned order will likely	540	all files in the given directory, then the returned order will likely
451	be fastest.	541	be fastest.
452		542
453	If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then	543	If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then the
454	the likely dirs come first, resulting in a less optimal stat order.	544	likely directories come first, resulting in a less optimal stat order.
455		545
456	=item EIO_READDIR_FOUND_UNKNOWN	546	=item EIO_READDIR_FOUND_UNKNOWN
457		547
458	This flag should not be specified when calling C<eio_readdir>. Instead,	548	This flag should not be specified when calling C<eio_readdir>. Instead,
459	it is being set by C<eio_readdir> (you can access the C<flags> via C<<	549	it is being set by C<eio_readdir> (you can access the C<flags> via C<<
460	req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The	550	req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The
461	absense of this flag therefore indicates that all C<type>'s are known,	551	absence of this flag therefore indicates that all C<type>'s are known,
462	which can be used to speed up some algorithms.	552	which can be used to speed up some algorithms.
463		553
464	A typical use case would be to identify all subdirectories within a	554	A typical use case would be to identify all subdirectories within a
465	directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If	555	directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If
466	then this flag is I<NOT> set, then all the entries at the beginning of the	556	then this flag is I<NOT> set, then all the entries at the beginning of the
…		…
504	as calling C<fdatasync>.	594	as calling C<fdatasync>.
505		595
506	Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>,	596	Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>,
507	C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>.	597	C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>.
508		598
		599	=item eio_fallocate (int fd, int mode, off_t offset, off_t len, int pri, eio_cb cb, void *data)
		600
		601	Calls C<fallocate> (note: I<NOT> C<posix_fallocate>!). If the syscall is
		602	missing, then it returns failure and sets C<errno> to C<ENOSYS>.
		603
		604	The C<mode> argument can be C<0> (for behaviour similar to
		605	C<posix_fallocate>), or C<EIO_FALLOC_FL_KEEP_SIZE>, which keeps the size
		606	of the file unchanged (but still preallocates space beyond end of file).
		607
509	=back	608	=back
510		609
511	=head3 LIBEIO-SPECIFIC REQUESTS	610	=head3 LIBEIO-SPECIFIC REQUESTS
512		611
513	These requests are specific to libeio and do not correspond to any OS call.	612	These requests are specific to libeio and do not correspond to any OS call.
…		…
554		653
555	eio_custom (my_open, 0, my_open_done, "/etc/passwd");	654	eio_custom (my_open, 0, my_open_done, "/etc/passwd");
556		655
557	=item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data)	656	=item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data)
558		657
559	This is a a request that takes C<delay> seconds to execute, but otherwise	658	This is a request that takes C<delay> seconds to execute, but otherwise
560	does nothing - it simply puts one of the worker threads to sleep for this	659	does nothing - it simply puts one of the worker threads to sleep for this
561	long.	660	long.
562		661
563	This request can be used to artificially increase load, e.g. for debugging	662	This request can be used to artificially increase load, e.g. for debugging
564	or benchmarking reasons.	663	or benchmarking reasons.
…		…
580	There are two primary use cases for this: a) bundle many requests into a	679	There are two primary use cases for this: a) bundle many requests into a
581	single, composite, request with a definite callback and the ability to	680	single, composite, request with a definite callback and the ability to
582	cancel the whole request with its subrequests and b) limiting the number	681	cancel the whole request with its subrequests and b) limiting the number
583	of "active" requests.	682	of "active" requests.
584		683
585	Further below you will find more dicussion of these topics - first follows	684	Further below you will find more discussion of these topics - first
586	the reference section detailing the request generator and other methods.	685	follows the reference section detailing the request generator and other
		686	methods.
587		687
588	=over 4	688	=over 4
589		689
590	=item eio_grp (eio_cb cb, void *data)	690	=item eio_req grp = eio_grp (eio_cb cb, void data)
591		691
592	Creates and submits a group request.	692	Creates, submits and returns a group request. Note that it doesn't have a
		693	priority, unlike all other requests.
593		694
594	=back	695	=item eio_grp_add (eio_req grp, eio_req req)
595		696
		697	Adds a request to the request group.
		698
		699	=item eio_grp_cancel (eio_req *grp)
		700
		701	Cancels all requests I<in> the group, but I<not> the group request
		702	itself. You can cancel the group request I<and> all subrequests via a
		703	normal C<eio_cancel> call.
		704
		705	=back
		706
		707	=head4 GROUP REQUEST LIFETIME
		708
		709	Left alone, a group request will instantly move to the pending state and
		710	will be finished at the next call of C<eio_poll>.
		711
		712	The usefulness stems from the fact that, if a subrequest is added to a
		713	group I<before> a call to C<eio_poll>, via C<eio_grp_add>, then the group
		714	will not finish until all the subrequests have finished.
		715
		716	So the usage cycle of a group request is like this: after it is created,
		717	you normally instantly add a subrequest. If none is added, the group
		718	request will finish on it's own. As long as subrequests are added before
		719	the group request is finished it will be kept from finishing, that is the
		720	callbacks of any subrequests can, in turn, add more requests to the group,
		721	and as long as any requests are active, the group request itself will not
		722	finish.
		723
		724	=head4 CREATING COMPOSITE REQUESTS
		725
		726	Imagine you wanted to create an C<eio_load> request that opens a file,
		727	reads it and closes it. This means it has to execute at least three eio
		728	requests, but for various reasons it might be nice if that request looked
		729	like any other eio request.
		730
		731	This can be done with groups:
		732
		733	=over 4
		734
		735	=item 1) create the request object
		736
		737	Create a group that contains all further requests. This is the request you
		738	can return as "the load request".
		739
		740	=item 2) open the file, maybe
		741
		742	Next, open the file with C<eio_open> and add the request to the group
		743	request and you are finished setting up the request.
		744
		745	If, for some reason, you cannot C<eio_open> (path is a null ptr?) you
		746	can set C<< grp->result >> to C<-1> to signal an error and let the group
		747	request finish on its own.
		748
		749	=item 3) open callback adds more requests
		750
		751	In the open callback, if the open was not successful, copy C<<
		752	req->errorno >> to C<< grp->errorno >> and set C<< grp->errorno >> to
		753	C<-1> to signal an error.
		754
		755	Otherwise, malloc some memory or so and issue a read request, adding the
		756	read request to the group.
		757
		758	=item 4) continue issuing requests till finished
		759
		760	In the real callback, check for errors and possibly continue with
		761	C<eio_close> or any other eio request in the same way.
		762
		763	As soon as no new requests are added the group request will finish. Make
		764	sure you I<always> set C<< grp->result >> to some sensible value.
		765
		766	=back
		767
		768	=head4 REQUEST LIMITING
596		769
597		770
598	#TODO	771	#TODO
599		772
600	/*****************************************************************************/
601	/* groups */
602
603	eio_req eio_grp (eio_cb cb, void data);
604	void eio_grp_feed (eio_req grp, void (feed)(eio_req *req), int limit);
605	void eio_grp_limit (eio_req *grp, int limit);	773	void eio_grp_limit (eio_req *grp, int limit);
606	void eio_grp_add (eio_req grp, eio_req req);
607	void eio_grp_cancel (eio_req grp); / cancels all sub requests but not the group */
608		774
609		775
610	=back	776	=back
611		777
612		778
…		…
618	=head1 ANATOMY AND LIFETIME OF AN EIO REQUEST	784	=head1 ANATOMY AND LIFETIME OF AN EIO REQUEST
619		785
620	A request is represented by a structure of type C<eio_req>. To initialise	786	A request is represented by a structure of type C<eio_req>. To initialise
621	it, clear it to all zero bytes:	787	it, clear it to all zero bytes:
622		788
623	eio_req req;	789	eio_req req;
624		790
625	memset (&req, 0, sizeof (req));	791	memset (&req, 0, sizeof (req));
626		792
627	A more common way to initialise a new C<eio_req> is to use C<calloc>:	793	A more common way to initialise a new C<eio_req> is to use C<calloc>:
628		794
629	eio_req req = calloc (1, sizeof (req));	795	eio_req req = calloc (1, sizeof (req));
630		796
631	In either case, libeio neither allocates, initialises or frees the	797	In either case, libeio neither allocates, initialises or frees the
632	C<eio_req> structure for you - it merely uses it.	798	C<eio_req> structure for you - it merely uses it.
633		799
634	zero	800	zero
…		…
652	for example, in interactive programs, you might want to limit this time to	818	for example, in interactive programs, you might want to limit this time to
653	C<0.01> seconds or so.	819	C<0.01> seconds or so.
654		820
655	Note that:	821	Note that:
656		822
		823	=over 4
		824
657	a) libeio doesn't know how long your request callbacks take, so the time	825	=item a) libeio doesn't know how long your request callbacks take, so the
658	spent in C<eio_poll> is up to one callback invocation longer then this	826	time spent in C<eio_poll> is up to one callback invocation longer then
659	interval.	827	this interval.
660		828
661	b) this is implemented by calling C<gettimeofday> after each request,	829	=item b) this is implemented by calling C<gettimeofday> after each
662	which can be costly.	830	request, which can be costly.
663		831
664	c) at least one request will be handled.	832	=item c) at least one request will be handled.
		833
		834	=back
665		835
666	=item eio_set_max_poll_reqs (unsigned int nreqs)	836	=item eio_set_max_poll_reqs (unsigned int nreqs)
667		837
668	When C<nreqs> is non-zero, then C<eio_poll> will not handle more than	838	When C<nreqs> is non-zero, then C<eio_poll> will not handle more than
669	C<nreqs> requests per invocation. This is a less costly way to limit the	839	C<nreqs> requests per invocation. This is a less costly way to limit the

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing libeio/eio.pod (file contents): Revision 1.12 by root, Wed Jun 29 10:32:55 2011 UTC vs. Revision 1.24 by sf-exg, Thu Jul 14 08:51:29 2011 UTC

Diff Legend

Comparing libeio/eio.pod (file contents):
Revision 1.12 by root, Wed Jun 29 10:32:55 2011 UTC vs.
Revision 1.24 by sf-exg, Thu Jul 14 08:51:29 2011 UTC