… | |
… | |
131 | |
131 | |
132 | If C<eio_poll ()> is configured to not handle all results in one go |
132 | If C<eio_poll ()> is configured to not handle all results in one go |
133 | (i.e. it returns C<-1>) then you should start an idle watcher that calls |
133 | (i.e. it returns C<-1>) then you should start an idle watcher that calls |
134 | C<eio_poll> until it returns something C<!= -1>. |
134 | C<eio_poll> until it returns something C<!= -1>. |
135 | |
135 | |
136 | A full-featured conenctor between libeio and libev would look as follows |
136 | A full-featured connector between libeio and libev would look as follows |
137 | (if C<eio_poll> is handling all requests, it can of course be simplified a |
137 | (if C<eio_poll> is handling all requests, it can of course be simplified a |
138 | lot by removing the idle watcher logic): |
138 | lot by removing the idle watcher logic): |
139 | |
139 | |
140 | static struct ev_loop *loop; |
140 | static struct ev_loop *loop; |
141 | static ev_idle repeat_watcher; |
141 | static ev_idle repeat_watcher; |
… | |
… | |
183 | to read that byte, and in the callback for the read end, you would call |
183 | to read that byte, and in the callback for the read end, you would call |
184 | C<eio_poll>. |
184 | C<eio_poll>. |
185 | |
185 | |
186 | You don't have to take special care in the case C<eio_poll> doesn't handle |
186 | You don't have to take special care in the case C<eio_poll> doesn't handle |
187 | all requests, as the done callback will not be invoked, so the event loop |
187 | all requests, as the done callback will not be invoked, so the event loop |
188 | will still signal readyness for the pipe until I<all> results have been |
188 | will still signal readiness for the pipe until I<all> results have been |
189 | processed. |
189 | processed. |
190 | |
190 | |
191 | |
191 | |
192 | =head1 HIGH LEVEL REQUEST API |
192 | =head1 HIGH LEVEL REQUEST API |
193 | |
193 | |
… | |
… | |
261 | } |
261 | } |
262 | |
262 | |
263 | /* the first three arguments are passed to open(2) */ |
263 | /* the first three arguments are passed to open(2) */ |
264 | /* the remaining are priority, callback and data */ |
264 | /* the remaining are priority, callback and data */ |
265 | if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0)) |
265 | if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0)) |
266 | abort (); /* something ent wrong, we will all die!!! */ |
266 | abort (); /* something went wrong, we will all die!!! */ |
267 | |
267 | |
268 | Note that you additionally need to call C<eio_poll> when the C<want_cb> |
268 | Note that you additionally need to call C<eio_poll> when the C<want_cb> |
269 | indicates that requests are ready to be processed. |
269 | indicates that requests are ready to be processed. |
270 | |
270 | |
271 | =head2 CANCELLING REQUESTS |
271 | =head2 CANCELLING REQUESTS |
272 | |
272 | |
273 | Sometimes the need for a request goes away before the request is |
273 | Sometimes the need for a request goes away before the request is |
274 | finished. In that case, one can cancel the reqiest by a call to |
274 | finished. In that case, one can cancel the request by a call to |
275 | C<eio_cancel>: |
275 | C<eio_cancel>: |
276 | |
276 | |
277 | =over 4 |
277 | =over 4 |
278 | |
278 | |
279 | =item eio_cancel (eio_req *req) |
279 | =item eio_cancel (eio_req *req) |
280 | |
280 | |
281 | Cancel the request. If the request is currently executing it might still |
281 | Cancel the request (and all its subrequests). If the request is currently |
282 | continue to execute, and in other cases it might still take a while till |
282 | executing it might still continue to execute, and in other cases it might |
283 | the request is cancelled. |
283 | still take a while till the request is cancelled. |
284 | |
284 | |
285 | Even if cancelled, the finish callback will still be invoked - the |
285 | Even if cancelled, the finish callback will still be invoked - the |
286 | callbacks of all cancellable requests need to check whether the request |
286 | callbacks of all cancellable requests need to check whether the request |
287 | has been cancelled by calling C<EIO_CANCELLED (req)>: |
287 | has been cancelled by calling C<EIO_CANCELLED (req)>: |
288 | |
288 | |
… | |
… | |
291 | { |
291 | { |
292 | if (EIO_CANCELLED (req)) |
292 | if (EIO_CANCELLED (req)) |
293 | return 0; |
293 | return 0; |
294 | } |
294 | } |
295 | |
295 | |
296 | In addition, cancelled requests will either have C<< req->result >> set to |
296 | In addition, cancelled requests will I<either> have C<< req->result >> |
297 | C<-1> and C<errno> to C<ECANCELED>, or otherwise they were successfully |
297 | set to C<-1> and C<errno> to C<ECANCELED>, or I<otherwise> they were |
298 | executed despite being cancelled (e.g. when they have already been |
298 | successfully executed, despite being cancelled (e.g. when they have |
299 | executed at the time they were cancelled). |
299 | already been executed at the time they were cancelled). |
|
|
300 | |
|
|
301 | C<EIO_CANCELLED> is still true for requests that have successfully |
|
|
302 | executed, as long as C<eio_cancel> was called on them at some point. |
300 | |
303 | |
301 | =back |
304 | =back |
302 | |
305 | |
303 | =head2 AVAILABLE REQUESTS |
306 | =head2 AVAILABLE REQUESTS |
304 | |
307 | |
… | |
… | |
402 | free (target); |
405 | free (target); |
403 | } |
406 | } |
404 | |
407 | |
405 | =item eio_realpath (const char *path, int pri, eio_cb cb, void *data) |
408 | =item eio_realpath (const char *path, int pri, eio_cb cb, void *data) |
406 | |
409 | |
407 | Similar to the realpath libc function, but unlike that one, result is |
410 | Similar to the realpath libc function, but unlike that one, C<< |
408 | C<-1> on failure and the length of the returned path in C<ptr2> (which is |
411 | req->result >> is C<-1> on failure. On success, the result is the length |
409 | not 0-terminated) - this is similar to readlink. |
412 | of the returned path in C<ptr2> (which is I<NOT> 0-terminated) - this is |
|
|
413 | similar to readlink. |
410 | |
414 | |
411 | =item eio_stat (const char *path, int pri, eio_cb cb, void *data) |
415 | =item eio_stat (const char *path, int pri, eio_cb cb, void *data) |
412 | |
416 | |
413 | =item eio_lstat (const char *path, int pri, eio_cb cb, void *data) |
417 | =item eio_lstat (const char *path, int pri, eio_cb cb, void *data) |
414 | |
418 | |
… | |
… | |
431 | =back |
435 | =back |
432 | |
436 | |
433 | =head3 READING DIRECTORIES |
437 | =head3 READING DIRECTORIES |
434 | |
438 | |
435 | Reading directories sounds simple, but can be rather demanding, especially |
439 | Reading directories sounds simple, but can be rather demanding, especially |
436 | if you want to do stuff such as traversing a diretcory hierarchy or |
440 | if you want to do stuff such as traversing a directory hierarchy or |
437 | processing all files in a directory. Libeio can assist thess complex tasks |
441 | processing all files in a directory. Libeio can assist these complex tasks |
438 | with it's C<eio_readdir> call. |
442 | with it's C<eio_readdir> call. |
439 | |
443 | |
440 | =over 4 |
444 | =over 4 |
441 | |
445 | |
442 | =item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data) |
446 | =item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data) |
… | |
… | |
534 | When this flag is specified, then the names will be returned in an order |
538 | When this flag is specified, then the names will be returned in an order |
535 | suitable for stat()'ing each one. That is, when you plan to stat() |
539 | suitable for stat()'ing each one. That is, when you plan to stat() |
536 | all files in the given directory, then the returned order will likely |
540 | all files in the given directory, then the returned order will likely |
537 | be fastest. |
541 | be fastest. |
538 | |
542 | |
539 | If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then |
543 | If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then the |
540 | the likely dirs come first, resulting in a less optimal stat order. |
544 | likely directories come first, resulting in a less optimal stat order. |
541 | |
545 | |
542 | =item EIO_READDIR_FOUND_UNKNOWN |
546 | =item EIO_READDIR_FOUND_UNKNOWN |
543 | |
547 | |
544 | This flag should not be specified when calling C<eio_readdir>. Instead, |
548 | This flag should not be specified when calling C<eio_readdir>. Instead, |
545 | it is being set by C<eio_readdir> (you can access the C<flags> via C<< |
549 | it is being set by C<eio_readdir> (you can access the C<flags> via C<< |
546 | req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The |
550 | req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The |
547 | absense of this flag therefore indicates that all C<type>'s are known, |
551 | absence of this flag therefore indicates that all C<type>'s are known, |
548 | which can be used to speed up some algorithms. |
552 | which can be used to speed up some algorithms. |
549 | |
553 | |
550 | A typical use case would be to identify all subdirectories within a |
554 | A typical use case would be to identify all subdirectories within a |
551 | directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If |
555 | directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If |
552 | then this flag is I<NOT> set, then all the entries at the beginning of the |
556 | then this flag is I<NOT> set, then all the entries at the beginning of the |
… | |
… | |
590 | as calling C<fdatasync>. |
594 | as calling C<fdatasync>. |
591 | |
595 | |
592 | Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>, |
596 | Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>, |
593 | C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>. |
597 | C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>. |
594 | |
598 | |
|
|
599 | =item eio_fallocate (int fd, int mode, off_t offset, off_t len, int pri, eio_cb cb, void *data) |
|
|
600 | |
|
|
601 | Calls C<fallocate> (note: I<NOT> C<posix_fallocate>!). If the syscall is |
|
|
602 | missing, then it returns failure and sets C<errno> to C<ENOSYS>. |
|
|
603 | |
|
|
604 | The C<mode> argument can be C<0> (for behaviour similar to |
|
|
605 | C<posix_fallocate>), or C<EIO_FALLOC_FL_KEEP_SIZE>, which keeps the size |
|
|
606 | of the file unchanged (but still preallocates space beyond end of file). |
|
|
607 | |
595 | =back |
608 | =back |
596 | |
609 | |
597 | =head3 LIBEIO-SPECIFIC REQUESTS |
610 | =head3 LIBEIO-SPECIFIC REQUESTS |
598 | |
611 | |
599 | These requests are specific to libeio and do not correspond to any OS call. |
612 | These requests are specific to libeio and do not correspond to any OS call. |
… | |
… | |
640 | |
653 | |
641 | eio_custom (my_open, 0, my_open_done, "/etc/passwd"); |
654 | eio_custom (my_open, 0, my_open_done, "/etc/passwd"); |
642 | |
655 | |
643 | =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) |
656 | =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) |
644 | |
657 | |
645 | This is a a request that takes C<delay> seconds to execute, but otherwise |
658 | This is a request that takes C<delay> seconds to execute, but otherwise |
646 | does nothing - it simply puts one of the worker threads to sleep for this |
659 | does nothing - it simply puts one of the worker threads to sleep for this |
647 | long. |
660 | long. |
648 | |
661 | |
649 | This request can be used to artificially increase load, e.g. for debugging |
662 | This request can be used to artificially increase load, e.g. for debugging |
650 | or benchmarking reasons. |
663 | or benchmarking reasons. |
… | |
… | |
666 | There are two primary use cases for this: a) bundle many requests into a |
679 | There are two primary use cases for this: a) bundle many requests into a |
667 | single, composite, request with a definite callback and the ability to |
680 | single, composite, request with a definite callback and the ability to |
668 | cancel the whole request with its subrequests and b) limiting the number |
681 | cancel the whole request with its subrequests and b) limiting the number |
669 | of "active" requests. |
682 | of "active" requests. |
670 | |
683 | |
671 | Further below you will find more dicussion of these topics - first follows |
684 | Further below you will find more discussion of these topics - first |
672 | the reference section detailing the request generator and other methods. |
685 | follows the reference section detailing the request generator and other |
|
|
686 | methods. |
673 | |
687 | |
674 | =over 4 |
688 | =over 4 |
675 | |
689 | |
676 | =item eio_req *grp = eio_grp (eio_cb cb, void *data) |
690 | =item eio_req *grp = eio_grp (eio_cb cb, void *data) |
677 | |
691 | |
678 | Creates, submits and returns a group request. |
692 | Creates, submits and returns a group request. Note that it doesn't have a |
|
|
693 | priority, unlike all other requests. |
679 | |
694 | |
680 | =item eio_grp_add (eio_req *grp, eio_req *req) |
695 | =item eio_grp_add (eio_req *grp, eio_req *req) |
681 | |
696 | |
682 | Adds a request to the request group. |
697 | Adds a request to the request group. |
683 | |
698 | |
684 | =item eio_grp_cancel (eio_req *grp) |
699 | =item eio_grp_cancel (eio_req *grp) |
685 | |
700 | |
686 | Cancels all requests I<in> the group, but I<not> the group request |
701 | Cancels all requests I<in> the group, but I<not> the group request |
687 | itself. You can cancel the group request via a normal C<eio_cancel> call. |
702 | itself. You can cancel the group request I<and> all subrequests via a |
|
|
703 | normal C<eio_cancel> call. |
688 | |
704 | |
689 | |
|
|
690 | |
|
|
691 | =back |
705 | =back |
692 | |
706 | |
|
|
707 | =head4 GROUP REQUEST LIFETIME |
|
|
708 | |
|
|
709 | Left alone, a group request will instantly move to the pending state and |
|
|
710 | will be finished at the next call of C<eio_poll>. |
|
|
711 | |
|
|
712 | The usefulness stems from the fact that, if a subrequest is added to a |
|
|
713 | group I<before> a call to C<eio_poll>, via C<eio_grp_add>, then the group |
|
|
714 | will not finish until all the subrequests have finished. |
|
|
715 | |
|
|
716 | So the usage cycle of a group request is like this: after it is created, |
|
|
717 | you normally instantly add a subrequest. If none is added, the group |
|
|
718 | request will finish on it's own. As long as subrequests are added before |
|
|
719 | the group request is finished it will be kept from finishing, that is the |
|
|
720 | callbacks of any subrequests can, in turn, add more requests to the group, |
|
|
721 | and as long as any requests are active, the group request itself will not |
|
|
722 | finish. |
|
|
723 | |
|
|
724 | =head4 CREATING COMPOSITE REQUESTS |
|
|
725 | |
|
|
726 | Imagine you wanted to create an C<eio_load> request that opens a file, |
|
|
727 | reads it and closes it. This means it has to execute at least three eio |
|
|
728 | requests, but for various reasons it might be nice if that request looked |
|
|
729 | like any other eio request. |
|
|
730 | |
|
|
731 | This can be done with groups: |
|
|
732 | |
|
|
733 | =over 4 |
|
|
734 | |
|
|
735 | =item 1) create the request object |
|
|
736 | |
|
|
737 | Create a group that contains all further requests. This is the request you |
|
|
738 | can return as "the load request". |
|
|
739 | |
|
|
740 | =item 2) open the file, maybe |
|
|
741 | |
|
|
742 | Next, open the file with C<eio_open> and add the request to the group |
|
|
743 | request and you are finished setting up the request. |
|
|
744 | |
|
|
745 | If, for some reason, you cannot C<eio_open> (path is a null ptr?) you |
|
|
746 | can set C<< grp->result >> to C<-1> to signal an error and let the group |
|
|
747 | request finish on its own. |
|
|
748 | |
|
|
749 | =item 3) open callback adds more requests |
|
|
750 | |
|
|
751 | In the open callback, if the open was not successful, copy C<< |
|
|
752 | req->errorno >> to C<< grp->errorno >> and set C<< grp->errorno >> to |
|
|
753 | C<-1> to signal an error. |
|
|
754 | |
|
|
755 | Otherwise, malloc some memory or so and issue a read request, adding the |
|
|
756 | read request to the group. |
|
|
757 | |
|
|
758 | =item 4) continue issuing requests till finished |
|
|
759 | |
|
|
760 | In the real callback, check for errors and possibly continue with |
|
|
761 | C<eio_close> or any other eio request in the same way. |
|
|
762 | |
|
|
763 | As soon as no new requests are added the group request will finish. Make |
|
|
764 | sure you I<always> set C<< grp->result >> to some sensible value. |
|
|
765 | |
|
|
766 | =back |
|
|
767 | |
|
|
768 | =head4 REQUEST LIMITING |
693 | |
769 | |
694 | |
770 | |
695 | #TODO |
771 | #TODO |
696 | |
772 | |
697 | /*****************************************************************************/ |
|
|
698 | /* groups */ |
|
|
699 | |
|
|
700 | eio_req *eio_grp (eio_cb cb, void *data); |
|
|
701 | void eio_grp_feed (eio_req *grp, void (*feed)(eio_req *req), int limit); |
|
|
702 | void eio_grp_limit (eio_req *grp, int limit); |
773 | void eio_grp_limit (eio_req *grp, int limit); |
703 | void eio_grp_cancel (eio_req *grp); /* cancels all sub requests but not the group */ |
|
|
704 | |
774 | |
705 | |
775 | |
706 | =back |
776 | =back |
707 | |
777 | |
708 | |
778 | |
… | |
… | |
748 | for example, in interactive programs, you might want to limit this time to |
818 | for example, in interactive programs, you might want to limit this time to |
749 | C<0.01> seconds or so. |
819 | C<0.01> seconds or so. |
750 | |
820 | |
751 | Note that: |
821 | Note that: |
752 | |
822 | |
|
|
823 | =over 4 |
|
|
824 | |
753 | a) libeio doesn't know how long your request callbacks take, so the time |
825 | =item a) libeio doesn't know how long your request callbacks take, so the |
754 | spent in C<eio_poll> is up to one callback invocation longer then this |
826 | time spent in C<eio_poll> is up to one callback invocation longer then |
755 | interval. |
827 | this interval. |
756 | |
828 | |
757 | b) this is implemented by calling C<gettimeofday> after each request, |
829 | =item b) this is implemented by calling C<gettimeofday> after each |
758 | which can be costly. |
830 | request, which can be costly. |
759 | |
831 | |
760 | c) at least one request will be handled. |
832 | =item c) at least one request will be handled. |
|
|
833 | |
|
|
834 | =back |
761 | |
835 | |
762 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
836 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
763 | |
837 | |
764 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
838 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
765 | C<nreqs> requests per invocation. This is a less costly way to limit the |
839 | C<nreqs> requests per invocation. This is a less costly way to limit the |