ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/libeio/eio.pod
(Generate patch)

Comparing libeio/eio.pod (file contents):
Revision 1.16 by root, Tue Jul 5 17:05:54 2011 UTC vs.
Revision 1.22 by root, Fri Jul 8 01:12:39 2011 UTC

… …
45Unlike the name component C<stamp> might indicate, it is also used for 45Unlike the name component C<stamp> might indicate, it is also used for
46time differences throughout libeio. 46time differences throughout libeio.
47 47
48=head2 FORK SUPPORT 48=head2 FORK SUPPORT
49 49
50Calling C<fork ()> is fully supported by this module. It is implemented in these steps: 50Calling C<fork ()> is fully supported by this module - but you must not
51rely on this. It is currently implemented in these steps:
51 52
52 1. wait till all requests in "execute" state have been handled 53 1. wait till all requests in "execute" state have been handled
53 (basically requests that are already handed over to the kernel). 54 (basically requests that are already handed over to the kernel).
54 2. fork 55 2. fork
55 3. in the parent, continue business as usual, done 56 3. in the parent, continue business as usual, done
56 4. in the child, destroy all ready and pending requests and free the 57 4. in the child, destroy all ready and pending requests and free the
57 memory used by the worker threads. This gives you a fully empty 58 memory used by the worker threads. This gives you a fully empty
58 libeio queue. 59 libeio queue.
59 60
60Note, however, since libeio does use threads, thr above guarantee doesn't 61Note, however, since libeio does use threads, the above guarantee doesn't
61cover your libc, for example, malloc and other libc functions are not 62cover your libc, for example, malloc and other libc functions are not
62fork-safe, so there is very little you can do after a fork, and in fatc, 63fork-safe, so there is very little you can do after a fork, and in fact,
63the above might crash, and thus change. 64the above might crash, and thus change.
64 65
65=head1 INITIALISATION/INTEGRATION 66=head1 INITIALISATION/INTEGRATION
66 67
67Before you can call any eio functions you first have to initialise the 68Before you can call any eio functions you first have to initialise the
… …
130 131
131If C<eio_poll ()> is configured to not handle all results in one go 132If C<eio_poll ()> is configured to not handle all results in one go
132(i.e. it returns C<-1>) then you should start an idle watcher that calls 133(i.e. it returns C<-1>) then you should start an idle watcher that calls
133C<eio_poll> until it returns something C<!= -1>. 134C<eio_poll> until it returns something C<!= -1>.
134 135
135A full-featured conenctor between libeio and libev would look as follows 136A full-featured connector between libeio and libev would look as follows
136(if C<eio_poll> is handling all requests, it can of course be simplified a 137(if C<eio_poll> is handling all requests, it can of course be simplified a
137lot by removing the idle watcher logic): 138lot by removing the idle watcher logic):
138 139
139 static struct ev_loop *loop; 140 static struct ev_loop *loop;
140 static ev_idle repeat_watcher; 141 static ev_idle repeat_watcher;
141 static ev_async ready_watcher; 142 static ev_async ready_watcher;
142 143
143 /* idle watcher callback, only used when eio_poll */ 144 /* idle watcher callback, only used when eio_poll */
144 /* didn't handle all results in one call */ 145 /* didn't handle all results in one call */
145 static void 146 static void
146 repeat (EV_P_ ev_idle *w, int revents) 147 repeat (EV_P_ ev_idle *w, int revents)
147 { 148 {
148 if (eio_poll () != -1) 149 if (eio_poll () != -1)
149 ev_idle_stop (EV_A_ w); 150 ev_idle_stop (EV_A_ w);
150 } 151 }
151 152
152 /* eio has some results, process them */ 153 /* eio has some results, process them */
153 static void 154 static void
154 ready (EV_P_ ev_async *w, int revents) 155 ready (EV_P_ ev_async *w, int revents)
155 { 156 {
156 if (eio_poll () == -1) 157 if (eio_poll () == -1)
157 ev_idle_start (EV_A_ &repeat_watcher); 158 ev_idle_start (EV_A_ &repeat_watcher);
158 } 159 }
159 160
160 /* wake up the event loop */ 161 /* wake up the event loop */
161 static void 162 static void
162 want_poll (void) 163 want_poll (void)
163 { 164 {
164 ev_async_send (loop, &ready_watcher) 165 ev_async_send (loop, &ready_watcher)
165 } 166 }
166 167
167 void 168 void
168 my_init_eio () 169 my_init_eio ()
169 { 170 {
170 loop = EV_DEFAULT; 171 loop = EV_DEFAULT;
171 172
172 ev_idle_init (&repeat_watcher, repeat); 173 ev_idle_init (&repeat_watcher, repeat);
173 ev_async_init (&ready_watcher, ready); 174 ev_async_init (&ready_watcher, ready);
174 ev_async_start (loop &watcher); 175 ev_async_start (loop &watcher);
175 176
176 eio_init (want_poll, 0); 177 eio_init (want_poll, 0);
177 } 178 }
178 179
179For most other event loops, you would typically use a pipe - the event 180For most other event loops, you would typically use a pipe - the event
180loop should be told to wait for read readiness on the read end. In 181loop should be told to wait for read readiness on the read end. In
181C<want_poll> you would write a single byte, in C<done_poll> you would try 182C<want_poll> you would write a single byte, in C<done_poll> you would try
182to read that byte, and in the callback for the read end, you would call 183to read that byte, and in the callback for the read end, you would call
183C<eio_poll>. 184C<eio_poll>.
184 185
185You don't have to take special care in the case C<eio_poll> doesn't handle 186You don't have to take special care in the case C<eio_poll> doesn't handle
186all requests, as the done callback will not be invoked, so the event loop 187all requests, as the done callback will not be invoked, so the event loop
187will still signal readyness for the pipe until I<all> results have been 188will still signal readiness for the pipe until I<all> results have been
188processed. 189processed.
189 190
190 191
191=head1 HIGH LEVEL REQUEST API 192=head1 HIGH LEVEL REQUEST API
192 193
… …
260 } 261 }
261 262
262 /* the first three arguments are passed to open(2) */ 263 /* the first three arguments are passed to open(2) */
263 /* the remaining are priority, callback and data */ 264 /* the remaining are priority, callback and data */
264 if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0)) 265 if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0))
265 abort (); /* something ent wrong, we will all die!!! */ 266 abort (); /* something went wrong, we will all die!!! */
266 267
267Note that you additionally need to call C<eio_poll> when the C<want_cb> 268Note that you additionally need to call C<eio_poll> when the C<want_cb>
268indicates that requests are ready to be processed. 269indicates that requests are ready to be processed.
270
271=head2 CANCELLING REQUESTS
272
273Sometimes the need for a request goes away before the request is
274finished. In that case, one can cancel the request by a call to
275C<eio_cancel>:
276
277=over 4
278
279=item eio_cancel (eio_req *req)
280
281Cancel the request (and all its subrequests). If the request is currently
282executing it might still continue to execute, and in other cases it might
283still take a while till the request is cancelled.
284
285Even if cancelled, the finish callback will still be invoked - the
286callbacks of all cancellable requests need to check whether the request
287has been cancelled by calling C<EIO_CANCELLED (req)>:
288
289 static int
290 my_eio_cb (eio_req *req)
291 {
292 if (EIO_CANCELLED (req))
293 return 0;
294 }
295
296In addition, cancelled requests will I<either> have C<< req->result >>
297set to C<-1> and C<errno> to C<ECANCELED>, or I<otherwise> they were
298successfully executed, despite being cancelled (e.g. when they have
299already been executed at the time they were cancelled).
300
301C<EIO_CANCELLED> is still true for requests that have successfully
302executed, as long as C<eio_cancel> was called on them at some point.
303
304=back
269 305
270=head2 AVAILABLE REQUESTS 306=head2 AVAILABLE REQUESTS
271 307
272The following request functions are available. I<All> of them return the 308The following request functions are available. I<All> of them return the
273C<eio_req *> on success and C<0> on failure, and I<all> of them have the 309C<eio_req *> on success and C<0> on failure, and I<all> of them have the
… …
369 free (target); 405 free (target);
370 } 406 }
371 407
372=item eio_realpath (const char *path, int pri, eio_cb cb, void *data) 408=item eio_realpath (const char *path, int pri, eio_cb cb, void *data)
373 409
374Similar to the realpath libc function, but unlike that one, result is 410Similar to the realpath libc function, but unlike that one, C<<
375C<-1> on failure and the length of the returned path in C<ptr2> (which is 411req->result >> is C<-1> on failure. On success, the result is the length
376not 0-terminated) - this is similar to readlink. 412of the returned path in C<ptr2> (which is I<NOT> 0-terminated) - this is
413similar to readlink.
377 414
378=item eio_stat (const char *path, int pri, eio_cb cb, void *data) 415=item eio_stat (const char *path, int pri, eio_cb cb, void *data)
379 416
380=item eio_lstat (const char *path, int pri, eio_cb cb, void *data) 417=item eio_lstat (const char *path, int pri, eio_cb cb, void *data)
381 418
382=item eio_fstat (int fd, int pri, eio_cb cb, void *data) 419=item eio_fstat (int fd, int pri, eio_cb cb, void *data)
383 420
384Stats a file - if C<< req->result >> indicates success, then you can 421Stats a file - if C<< req->result >> indicates success, then you can
385access the C<struct stat>-like structure via C<< req->ptr2 >>: 422access the C<struct stat>-like structure via C<< req->ptr2 >>:
386 423
387 EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2; 424 EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2;
388 425
389=item eio_statvfs (const char *path, int pri, eio_cb cb, void *data) 426=item eio_statvfs (const char *path, int pri, eio_cb cb, void *data)
390 427
391=item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data) 428=item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data)
392 429
393Stats a filesystem - if C<< req->result >> indicates success, then you can 430Stats a filesystem - if C<< req->result >> indicates success, then you can
394access the C<struct statvfs>-like structure via C<< req->ptr2 >>: 431access the C<struct statvfs>-like structure via C<< req->ptr2 >>:
395 432
396 EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2; 433 EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2;
397 434
398=back 435=back
399 436
400=head3 READING DIRECTORIES 437=head3 READING DIRECTORIES
401 438
402Reading directories sounds simple, but can be rather demanding, especially 439Reading directories sounds simple, but can be rather demanding, especially
403if you want to do stuff such as traversing a diretcory hierarchy or 440if you want to do stuff such as traversing a directory hierarchy or
404processing all files in a directory. Libeio can assist thess complex tasks 441processing all files in a directory. Libeio can assist these complex tasks
405with it's C<eio_readdir> call. 442with it's C<eio_readdir> call.
406 443
407=over 4 444=over 4
408 445
409=item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data) 446=item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data)
… …
441 478
442If this flag is specified, then, in addition to the names in C<ptr2>, 479If this flag is specified, then, in addition to the names in C<ptr2>,
443also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct 480also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct
444eio_dirent> looks like this: 481eio_dirent> looks like this:
445 482
446 struct eio_dirent 483 struct eio_dirent
447 { 484 {
448 int nameofs; /* offset of null-terminated name string in (char *)req->ptr2 */ 485 int nameofs; /* offset of null-terminated name string in (char *)req->ptr2 */
449 unsigned short namelen; /* size of filename without trailing 0 */ 486 unsigned short namelen; /* size of filename without trailing 0 */
450 unsigned char type; /* one of EIO_DT_* */ 487 unsigned char type; /* one of EIO_DT_* */
451 signed char score; /* internal use */ 488 signed char score; /* internal use */
452 ino_t inode; /* the inode number, if available, otherwise unspecified */ 489 ino_t inode; /* the inode number, if available, otherwise unspecified */
453 }; 490 };
454 491
455The only members you normally would access are C<nameofs>, which is the 492The only members you normally would access are C<nameofs>, which is the
456byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>. 493byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>.
457 494
458C<type> can be one of: 495C<type> can be one of:
… …
501When this flag is specified, then the names will be returned in an order 538When this flag is specified, then the names will be returned in an order
502suitable for stat()'ing each one. That is, when you plan to stat() 539suitable for stat()'ing each one. That is, when you plan to stat()
503all files in the given directory, then the returned order will likely 540all files in the given directory, then the returned order will likely
504be fastest. 541be fastest.
505 542
506If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then 543If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then the
507the likely dirs come first, resulting in a less optimal stat order. 544likely directories come first, resulting in a less optimal stat order.
508 545
509=item EIO_READDIR_FOUND_UNKNOWN 546=item EIO_READDIR_FOUND_UNKNOWN
510 547
511This flag should not be specified when calling C<eio_readdir>. Instead, 548This flag should not be specified when calling C<eio_readdir>. Instead,
512it is being set by C<eio_readdir> (you can access the C<flags> via C<< 549it is being set by C<eio_readdir> (you can access the C<flags> via C<<
513req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The 550req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The
514absense of this flag therefore indicates that all C<type>'s are known, 551absence of this flag therefore indicates that all C<type>'s are known,
515which can be used to speed up some algorithms. 552which can be used to speed up some algorithms.
516 553
517A typical use case would be to identify all subdirectories within a 554A typical use case would be to identify all subdirectories within a
518directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If 555directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If
519then this flag is I<NOT> set, then all the entries at the beginning of the 556then this flag is I<NOT> set, then all the entries at the beginning of the
… …
557as calling C<fdatasync>. 594as calling C<fdatasync>.
558 595
559Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>, 596Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>,
560C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>. 597C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>.
561 598
599=item eio_fallocate (int fd, int mode, off_t offset, off_t len, int pri, eio_cb cb, void *data)
600
601Calls C<fallocate> (note: I<NOT> C<posix_fallocate>!). If the syscall is
602missing, then it returns failure and sets C<errno> to C<ENOSYS>.
603
604The C<mode> argument can be C<0> (for behaviour similar to
605C<posix_fallocate>), or C<EIO_FALLOC_FL_KEEP_SIZE>, which keeps the size
606of the file unchanged (but still preallocates space beyond end of file).
607
562=back 608=back
563 609
564=head3 LIBEIO-SPECIFIC REQUESTS 610=head3 LIBEIO-SPECIFIC REQUESTS
565 611
566These requests are specific to libeio and do not correspond to any OS call. 612These requests are specific to libeio and do not correspond to any OS call.
… …
607 653
608 eio_custom (my_open, 0, my_open_done, "/etc/passwd"); 654 eio_custom (my_open, 0, my_open_done, "/etc/passwd");
609 655
610=item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) 656=item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data)
611 657
612This is a a request that takes C<delay> seconds to execute, but otherwise 658This is a request that takes C<delay> seconds to execute, but otherwise
613does nothing - it simply puts one of the worker threads to sleep for this 659does nothing - it simply puts one of the worker threads to sleep for this
614long. 660long.
615 661
616This request can be used to artificially increase load, e.g. for debugging 662This request can be used to artificially increase load, e.g. for debugging
617or benchmarking reasons. 663or benchmarking reasons.
… …
633There are two primary use cases for this: a) bundle many requests into a 679There are two primary use cases for this: a) bundle many requests into a
634single, composite, request with a definite callback and the ability to 680single, composite, request with a definite callback and the ability to
635cancel the whole request with its subrequests and b) limiting the number 681cancel the whole request with its subrequests and b) limiting the number
636of "active" requests. 682of "active" requests.
637 683
638Further below you will find more dicussion of these topics - first follows 684Further below you will find more discussion of these topics - first
639the reference section detailing the request generator and other methods. 685follows the reference section detailing the request generator and other
686methods.
640 687
641=over 4 688=over 4
642 689
643=item eio_grp (eio_cb cb, void *data) 690=item eio_req *grp = eio_grp (eio_cb cb, void *data)
644 691
645Creates and submits a group request. 692Creates, submits and returns a group request.
693
694=item eio_grp_add (eio_req *grp, eio_req *req)
695
696Adds a request to the request group.
697
698=item eio_grp_cancel (eio_req *grp)
699
700Cancels all requests I<in> the group, but I<not> the group request
701itself. You can cancel the group request via a normal C<eio_cancel> call.
702
703
646 704
647=back 705=back
648 706
649 707
650 708
… …
654/* groups */ 712/* groups */
655 713
656eio_req *eio_grp (eio_cb cb, void *data); 714eio_req *eio_grp (eio_cb cb, void *data);
657void eio_grp_feed (eio_req *grp, void (*feed)(eio_req *req), int limit); 715void eio_grp_feed (eio_req *grp, void (*feed)(eio_req *req), int limit);
658void eio_grp_limit (eio_req *grp, int limit); 716void eio_grp_limit (eio_req *grp, int limit);
659void eio_grp_add (eio_req *grp, eio_req *req);
660void eio_grp_cancel (eio_req *grp); /* cancels all sub requests but not the group */ 717void eio_grp_cancel (eio_req *grp); /* cancels all sub requests but not the group */
661 718
662 719
663=back 720=back
664 721
… …
671=head1 ANATOMY AND LIFETIME OF AN EIO REQUEST 728=head1 ANATOMY AND LIFETIME OF AN EIO REQUEST
672 729
673A request is represented by a structure of type C<eio_req>. To initialise 730A request is represented by a structure of type C<eio_req>. To initialise
674it, clear it to all zero bytes: 731it, clear it to all zero bytes:
675 732
676 eio_req req; 733 eio_req req;
677 734
678 memset (&req, 0, sizeof (req)); 735 memset (&req, 0, sizeof (req));
679 736
680A more common way to initialise a new C<eio_req> is to use C<calloc>: 737A more common way to initialise a new C<eio_req> is to use C<calloc>:
681 738
682 eio_req *req = calloc (1, sizeof (*req)); 739 eio_req *req = calloc (1, sizeof (*req));
683 740
684In either case, libeio neither allocates, initialises or frees the 741In either case, libeio neither allocates, initialises or frees the
685C<eio_req> structure for you - it merely uses it. 742C<eio_req> structure for you - it merely uses it.
686 743
687zero 744zero
… …
705for example, in interactive programs, you might want to limit this time to 762for example, in interactive programs, you might want to limit this time to
706C<0.01> seconds or so. 763C<0.01> seconds or so.
707 764
708Note that: 765Note that:
709 766
767=over 4
768
710a) libeio doesn't know how long your request callbacks take, so the time 769=item a) libeio doesn't know how long your request callbacks take, so the
711spent in C<eio_poll> is up to one callback invocation longer then this 770time spent in C<eio_poll> is up to one callback invocation longer then
712interval. 771this interval.
713 772
714b) this is implemented by calling C<gettimeofday> after each request, 773=item b) this is implemented by calling C<gettimeofday> after each
715which can be costly. 774request, which can be costly.
716 775
717c) at least one request will be handled. 776=item c) at least one request will be handled.
777
778=back
718 779
719=item eio_set_max_poll_reqs (unsigned int nreqs) 780=item eio_set_max_poll_reqs (unsigned int nreqs)
720 781
721When C<nreqs> is non-zero, then C<eio_poll> will not handle more than 782When C<nreqs> is non-zero, then C<eio_poll> will not handle more than
722C<nreqs> requests per invocation. This is a less costly way to limit the 783C<nreqs> requests per invocation. This is a less costly way to limit the

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines