[ViewVC] Diff of: cvs/libeio/eio.pod

Comparing libeio/eio.pod (file contents):
Revision 1.17 by root, Tue Jul 5 18:59:28 2011 UTC vs.
Revision 1.36 by root, Sun Jan 24 16:36:20 2016 UTC

 similar functions, as well as less rarely ones such as C<mknod>, C<futime>
 or C<readlink>.
 It also offers wrappers around C<sendfile> (Solaris, Linux, HP-UX and
 FreeBSD, with emulation on other platforms) and C<readahead> (Linux, with
-emulation elsewhere>).
+emulation elsewhere).
 The goal is to enable you to write fully non-blocking programs. For
 example, in a game server, you would not want to freeze for a few seconds
 just because the server is running a backup and you happen to call
 C<readdir>.
 Unlike the name component C<stamp> might indicate, it is also used for
 time differences throughout libeio.
 =head2 FORK SUPPORT
-Calling C<fork ()> is fully supported by this module - but you must not
+Usage of pthreads in a program changes the semantics of fork
-rely on this. It is currently implemented in these steps:
+considerably. Specifically, only async-safe functions can be called after
+fork. Libeio uses pthreads, so this applies, and makes using fork hard for
+anything but relatively fork + exec uses.
-  1. wait till all requests in "execute" state have been handled
+This library only works in the process that initialised it: Forking is
-     (basically requests that are already handed over to the kernel).
+fully supported, but using libeio in any other process than the one that
-  2. fork
+called C<eio_init> is not.
-  3. in the parent, continue business as usual, done
-  4. in the child, destroy all ready and pending requests and free the
-     memory used by the worker threads. This gives you a fully empty
-     libeio queue.
-Note, however, since libeio does use threads, the above guarantee doesn't
+You might get around by not I<using> libeio before (or after) forking in
-cover your libc, for example, malloc and other libc functions are not
+the parent, and using it in the child afterwards. You could also try to
-fork-safe, so there is very little you can do after a fork, and in fact,
+call the L<eio_init> function again in the child, which will brutally
-the above might crash, and thus change.
+reinitialise all data structures, which isn't POSIX conformant, but
+typically works.
+Otherwise, the only recommendation you should follow is: treat fork code
+the same way you treat signal handlers, and only ever call C<eio_init> in
+the process that uses it, and only once ever.
 =head1 INITIALISATION/INTEGRATION
 Before you can call any eio functions you first have to initialise the
 library. The library integrates into any event loop, but can also be used
 This function initialises the library. On success it returns C<0>, on
 failure it returns C<-1> and sets C<errno> appropriately.
 It accepts two function pointers specifying callbacks as argument, both of
 which can be C<0>, in which case the callback isn't called.
+There is currently no way to change these callbacks later, or to
+"uninitialise" the library again.
 =item want_poll callback
 The C<want_poll> callback is invoked whenever libeio wants attention (i.e.
 it wants to be polled by calling C<eio_poll>). It is "edge-triggered",
 This callback is invoked when libeio detects that all pending requests
 have been handled. It is "edge-triggered", that is, it will only be
 called once after C<want_poll>. To put it differently, C<want_poll> and
 C<done_poll> are invoked in pairs: after C<want_poll> you have to call
 C<eio_poll ()> until either C<eio_poll> indicates that everything has been
-handled or C<done_poll> has been called, which signals the same.
+handled or C<done_poll> has been called, which signals the same - only one
+method is needed.
 Note that C<eio_poll> might return after C<done_poll> and C<want_poll>
 have been called again, so watch out for races in your code.
+It is quite common to have an empty C<done_call> callback and only use
+the return value from C<eio_poll>, or, when C<eio_poll> is configured to
+handle all outstanding replies, it's enough to call C<eio_poll> once.
 As with C<want_poll>, this callback is called while locks are being held,
-so you I<must not call any libeio functions form within this callback>.
+so you I<must not call any libeio functions from within this callback>.
 =item int eio_poll ()
 This function has to be called whenever there are pending requests that
 need finishing. You usually call this after C<want_poll> has indicated
 If C<eio_poll ()> is configured to not handle all results in one go
 (i.e. it returns C<-1>) then you should start an idle watcher that calls
 C<eio_poll> until it returns something C<!= -1>.
-A full-featured conenctor between libeio and libev would look as follows
+A full-featured connector between libeio and libev would look as follows
 (if C<eio_poll> is handling all requests, it can of course be simplified a
 lot by removing the idle watcher logic):
   static struct ev_loop *loop;
   static ev_idle repeat_watcher;
   {
     loop = EV_DEFAULT;
     ev_idle_init (&repeat_watcher, repeat);
     ev_async_init (&ready_watcher, ready);
-    ev_async_start (loop &watcher);
+    ev_async_start (loop, &watcher);
     eio_init (want_poll, 0);
   }
 For most other event loops, you would typically use a pipe - the event
 to read that byte, and in the callback for the read end, you would call
 C<eio_poll>.
 You don't have to take special care in the case C<eio_poll> doesn't handle
 all requests, as the done callback will not be invoked, so the event loop
-will still signal readyness for the pipe until I<all> results have been
+will still signal readiness for the pipe until I<all> results have been
 processed.
 =head1 HIGH LEVEL REQUEST API
 The C<void *data> member simply stores the value of the C<data> argument.
 =back
+Members not explicitly described as accessible must not be
+accessed. Specifically, there is no guarantee that any members will still
+have the value they had when the request was submitted.
 The return value of the callback is normally C<0>, which tells libeio to
 continue normally. If a callback returns a nonzero value, libeio will
 stop processing results (in C<eio_poll>) and will return the value to its
 caller.
-Memory areas passed to libeio must stay valid as long as a request
+Memory areas passed to libeio wrappers must stay valid as long as a
-executes, with the exception of paths, which are being copied
+request executes, with the exception of paths, which are being copied
 internally. Any memory libeio itself allocates will be freed after the
 finish callback has been called. If you want to manage all memory passed
 to libeio yourself you can use the low-level API.
 For example, to open a file, you could do this:
   }
   /* the first three arguments are passed to open(2) */
   /* the remaining are priority, callback and data */
   if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0))
-    abort (); /* something ent wrong, we will all die!!! */
+    abort (); /* something went wrong, we will all die!!! */
 Note that you additionally need to call C<eio_poll> when the C<want_cb>
 indicates that requests are ready to be processed.
 =head2 CANCELLING REQUESTS
 Sometimes the need for a request goes away before the request is
-finished. In that case, one can cancel the reqiest by a call to
+finished. In that case, one can cancel the request by a call to
 C<eio_cancel>:
 =over 4
 =item eio_cancel (eio_req *req)
-Cancel the request. If the request is currently executing it might still
+Cancel the request (and all its subrequests). If the request is currently
-continue to execute, and in other cases it might still take a while till
+executing it might still continue to execute, and in other cases it might
-the request is cancelled.
+still take a while till the request is cancelled.
-Even if cancelled, the finish callback will still be invoked - the
+When cancelled, the finish callback will not be invoked.
-callbacks of all cancellable requests need to check whether the request
-has been cancelled by calling C<EIO_CANCELLED (req)>:
-  static int
+C<EIO_CANCELLED> is still true for requests that have successfully
-  my_eio_cb (eio_req *req)
+executed, as long as C<eio_cancel> was called on them at some point.
-  {
-    if (EIO_CANCELLED (req))
-      return 0;
-  }
-In addition, cancelled requests will either have C<< req->result >> set to
-C<-1> and C<errno> to C<ECANCELED>, or otherwise they were successfully
-executed despite being cancelled (e.g. when they have already been
-executed at the time they were cancelled).
 =back
 =head2 AVAILABLE REQUESTS
       free (target);
     }
 =item eio_realpath  (const char *path, int pri, eio_cb cb, void *data)
-Similar to the realpath libc function, but unlike that one, result is
+Similar to the realpath libc function, but unlike that one, C<<
-C<-1> on failure and the length of the returned path in C<ptr2> (which is
+req->result >> is C<-1> on failure. On success, the result is the length
-not 0-terminated) - this is similar to readlink.
+of the returned path in C<ptr2> (which is I<NOT> 0-terminated) - this is
+similar to readlink.
 =item eio_stat      (const char *path, int pri, eio_cb cb, void *data)
 =item eio_lstat     (const char *path, int pri, eio_cb cb, void *data)
 =back
 =head3 READING DIRECTORIES
 Reading directories sounds simple, but can be rather demanding, especially
-if you want to do stuff such as traversing a diretcory hierarchy or
+if you want to do stuff such as traversing a directory hierarchy or
-processing all files in a directory. Libeio can assist thess complex tasks
+processing all files in a directory. Libeio can assist these complex tasks
 with it's C<eio_readdir> call.
 =over 4
 =item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data)
 When this flag is specified, then the names will be returned in an order
 suitable for stat()'ing each one. That is, when you plan to stat()
 all files in the given directory, then the returned order will likely
 be fastest.
-If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then
+If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then the
-the likely dirs come first, resulting in a less optimal stat order.
+likely directories come first, resulting in a less optimal stat order.
 =item EIO_READDIR_FOUND_UNKNOWN
 This flag should not be specified when calling C<eio_readdir>. Instead,
 it is being set by C<eio_readdir> (you can access the C<flags> via C<<
 req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The
-absense of this flag therefore indicates that all C<type>'s are known,
+absence of this flag therefore indicates that all C<type>'s are known,
 which can be used to speed up some algorithms.
 A typical use case would be to identify all subdirectories within a
 directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If
 then this flag is I<NOT> set, then all the entries at the beginning of the
 =item eio_readahead (int fd, off_t offset, size_t length, int pri, eio_cb cb, void *data)
 Calls C<readahead(2)>. If the syscall is missing, then the call is
 emulated by simply reading the data (currently in 64kiB chunks).
+=item eio_syncfs (int fd, int pri, eio_cb cb, void *data)
+Calls Linux' C<syncfs> syscall, if available. Returns C<-1> and sets
+C<errno> to C<ENOSYS> if the call is missing I<but still calls sync()>,
+if the C<fd> is C<< >= 0 >>, so you can probe for the availability of the
+syscall with a negative C<fd> argument and checking for C<-1/ENOSYS>.
 =item eio_sync_file_range (int fd, off_t offset, size_t nbytes, unsigned int flags, int pri, eio_cb cb, void *data)
 Calls C<sync_file_range>. If the syscall is missing, then this is the same
 as calling C<fdatasync>.
 Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>,
 C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>.
+=item eio_fallocate (int fd, int mode, off_t offset, off_t len, int pri, eio_cb cb, void *data)
+Calls C<fallocate> (note: I<NOT> C<posix_fallocate>!). If the syscall is
+missing, then it returns failure and sets C<errno> to C<ENOSYS>.
+The C<mode> argument can be C<0> (for behaviour similar to
+C<posix_fallocate>), or C<EIO_FALLOC_FL_KEEP_SIZE>, which keeps the size
+of the file unchanged (but still preallocates space beyond end of file).
 =back
 =head3 LIBEIO-SPECIFIC REQUESTS
 These requests are specific to libeio and do not correspond to any OS call.
 =over 4
 =item eio_mtouch (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data)
-Reads (C<flags == 0>) or modifies (C<flags == EIO_MT_MODIFY) the given
+Reads (C<flags == 0>) or modifies (C<flags == EIO_MT_MODIFY>) the given
 memory area, page-wise, that is, it reads (or reads and writes back) the
 first octet of every page that spans the memory area.
 This can be used to page in some mmapped file, or dirty some pages. Note
 that dirtying is an unlocked read-write access, so races can ensue when
   eio_custom (my_open, 0, my_open_done, "/etc/passwd");
 =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data)
-This is a a request that takes C<delay> seconds to execute, but otherwise
+This is a request that takes C<delay> seconds to execute, but otherwise
 does nothing - it simply puts one of the worker threads to sleep for this
 long.
 This request can be used to artificially increase load, e.g. for debugging
 or benchmarking reasons.
 There are two primary use cases for this: a) bundle many requests into a
 single, composite, request with a definite callback and the ability to
 cancel the whole request with its subrequests and b) limiting the number
 of "active" requests.
-Further below you will find more dicussion of these topics - first follows
+Further below you will find more discussion of these topics - first
-the reference section detailing the request generator and other methods.
+follows the reference section detailing the request generator and other
+methods.
 =over 4
 =item eio_req *grp = eio_grp (eio_cb cb, void *data)
-Creates, submits and returns a group request.
+Creates, submits and returns a group request. Note that it doesn't have a
+priority, unlike all other requests.
 =item eio_grp_add (eio_req *grp, eio_req *req)
 Adds a request to the request group.
 =item eio_grp_cancel (eio_req *grp)
 Cancels all requests I<in> the group, but I<not> the group request
-itself. You can cancel the group request via a normal C<eio_cancel> call.
+itself. You can cancel the group request I<and> all subrequests via a
+normal C<eio_cancel> call.
 =back
+=head4 GROUP REQUEST LIFETIME
+Left alone, a group request will instantly move to the pending state and
+will be finished at the next call of C<eio_poll>.
+The usefulness stems from the fact that, if a subrequest is added to a
+group I<before> a call to C<eio_poll>, via C<eio_grp_add>, then the group
+will not finish until all the subrequests have finished.
+So the usage cycle of a group request is like this: after it is created,
+you normally instantly add a subrequest. If none is added, the group
+request will finish on it's own. As long as subrequests are added before
+the group request is finished it will be kept from finishing, that is the
+callbacks of any subrequests can, in turn, add more requests to the group,
+and as long as any requests are active, the group request itself will not
+finish.
+=head4 CREATING COMPOSITE REQUESTS
+Imagine you wanted to create an C<eio_load> request that opens a file,
+reads it and closes it. This means it has to execute at least three eio
+requests, but for various reasons it might be nice if that request looked
+like any other eio request.
+This can be done with groups:
+=over 4
+=item 1) create the request object
+Create a group that contains all further requests. This is the request you
+can return as "the load request".
+=item 2) open the file, maybe
+Next, open the file with C<eio_open> and add the request to the group
+request and you are finished setting up the request.
+If, for some reason, you cannot C<eio_open> (path is a null ptr?) you
+can set C<< grp->result >> to C<-1> to signal an error and let the group
+request finish on its own.
+=item 3) open callback adds more requests
+In the open callback, if the open was not successful, copy C<<
+req->errorno >> to C<< grp->errorno >> and set C<< grp->result >> to
+C<-1> to signal an error.
+Otherwise, malloc some memory or so and issue a read request, adding the
+read request to the group.
+=item 4) continue issuing requests till finished
+In the read callback, check for errors and possibly continue with
+C<eio_close> or any other eio request in the same way.
+As soon as no new requests are added, the group request will finish. Make
+sure you I<always> set C<< grp->result >> to some sensible value.
+=back
+=head4 REQUEST LIMITING
 #TODO
-/*****************************************************************************/
-/* groups */
-eio_req *eio_grp       (eio_cb cb, void *data);
-void eio_grp_feed      (eio_req *grp, void (*feed)(eio_req *req), int limit);
 void eio_grp_limit     (eio_req *grp, int limit);
-void eio_grp_cancel    (eio_req *grp); /* cancels all sub requests but not the group */
-=back
 =head1 LOW LEVEL REQUEST API
 #TODO
 for example, in interactive programs, you might want to limit this time to
 C<0.01> seconds or so.
 Note that:
+=over 4
-a) libeio doesn't know how long your request callbacks take, so the time
+=item a) libeio doesn't know how long your request callbacks take, so the
-spent in C<eio_poll> is up to one callback invocation longer then this
+time spent in C<eio_poll> is up to one callback invocation longer then
-interval.
+this interval.
-b) this is implemented by calling C<gettimeofday> after each request,
+=item b) this is implemented by calling C<gettimeofday> after each
-which can be costly.
+request, which can be costly.
-c) at least one request will be handled.
+=item c) at least one request will be handled.
+=back
 =item eio_set_max_poll_reqs (unsigned int nreqs)
 When C<nreqs> is non-zero, then C<eio_poll> will not handle more than
 C<nreqs> requests per invocation. This is a less costly way to limit the
 This symbol governs the stack size for each eio thread. Libeio itself
 was written to use very little stackspace, but when using C<EIO_CUSTOM>
 requests, you might want to increase this.
 If this symbol is undefined (the default) then libeio will use its default
-stack size (C<sizeof (long) * 4096> currently).  If it is defined, but
+stack size (C<sizeof (void *) * 4096> currently). In all other cases, the
-C<0>, then the default operating system stack size will be used. In all
-other cases, the value must be an expression that evaluates to the desired
+value must be an expression that evaluates to the desired stack size.
-stack size.
 =back
 =head1 PORTABILITY REQUIREMENTS

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing libeio/eio.pod (file contents): Revision 1.17 by root, Tue Jul 5 18:59:28 2011 UTC vs. Revision 1.36 by root, Sun Jan 24 16:36:20 2016 UTC

Diff Legend

Comparing libeio/eio.pod (file contents):
Revision 1.17 by root, Tue Jul 5 18:59:28 2011 UTC vs.
Revision 1.36 by root, Sun Jan 24 16:36:20 2016 UTC