ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/libeio/eio.pod
(Generate patch)

Comparing libeio/eio.pod (file contents):
Revision 1.6 by sf-exg, Tue May 31 10:09:38 2011 UTC vs.
Revision 1.7 by root, Sun Jun 5 22:44:30 2011 UTC

… …
54 2. fork 54 2. fork
55 3. in the parent, continue business as usual, done 55 3. in the parent, continue business as usual, done
56 4. in the child, destroy all ready and pending requests and free the 56 4. in the child, destroy all ready and pending requests and free the
57 memory used by the worker threads. This gives you a fully empty 57 memory used by the worker threads. This gives you a fully empty
58 libeio queue. 58 libeio queue.
59
60Note, however, since libeio does use threads, thr above guarantee doesn't
61cover your libc, for example, malloc and other libc functions are not
62fork-safe, so there is very little you can do after a fork, and in fatc,
63the above might crash, and thus change.
59 64
60=head1 INITIALISATION/INTEGRATION 65=head1 INITIALISATION/INTEGRATION
61 66
62Before you can call any eio functions you first have to initialise the 67Before you can call any eio functions you first have to initialise the
63library. The library integrates into any event loop, but can also be used 68library. The library integrates into any event loop, but can also be used
… …
215C<eio_poll>). 220C<eio_poll>).
216 221
217=back 222=back
218 223
219 224
220=head1 ANATOMY OF AN EIO REQUEST 225=head1 HIGH LEVEL REQUEST API
226
227Libeio has both a high-level API, which consists of calling a request
228function with a callback to be called on completion, and a low-level API
229where you fill out request structures and submit them.
230
231This section describes the high-level API.
232
233=head2 REQUEST SUBMISSION AND RESULT PROCESSING
234
235You submit a request by calling the relevant C<eio_TYPE> function with the
236required parameters, a callback of type C<int (*eio_cb)(eio_req *req)>
237(called C<eio_cb> below) and a freely usable C<void *data> argument.
238
239The return value will either be 0
240
241The callback will be called with an C<eio_req *> which contains the
242results of the request. The members you can access inside that structure
243vary from request to request, except for:
244
245=over 4
246
247=item C<ssize_t result>
248
249This contains the result value from the call (usually the same as the
250syscall of the same name).
251
252=item C<int errorno>
253
254This contains the value of C<errno> after the call.
255
256=item C<void *data>
257
258The C<void *data> member simply stores the value of the C<data> argument.
259
260=back
261
262The return value of the callback is normally C<0>, which tells libeio to
263continue normally. If a callback returns a nonzero value, libeio will
264stop processing results (in C<eio_poll>) and will return the value to its
265caller.
266
267Memory areas passed to libeio must stay valid as long as a request
268executes, with the exception of paths, which are being copied
269internally. Any memory libeio itself allocates will be freed after the
270finish callback has been called. If you want to manage all memory passed
271to libeio yourself you can use the low-level API.
272
273For example, to open a file, you could do this:
274
275 static int
276 file_open_done (eio_req *req)
277 {
278 if (req->result < 0)
279 {
280 /* open() returned -1 */
281 errno = req->errorno;
282 perror ("open");
283 }
284 else
285 {
286 int fd = req->result;
287 /* now we have the new fd in fd */
288 }
289
290 return 0;
291 }
292
293 /* the first three arguments are passed to open(2) */
294 /* the remaining are priority, callback and data */
295 if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0))
296 abort (); /* something ent wrong, we will all die!!! */
297
298Note that you additionally need to call C<eio_poll> when the C<want_cb>
299indicates that requests are ready to be processed.
300
301=head2 AVAILABLE REQUESTS
302
303The following request functions are available. I<All> of them return the
304C<eio_req *> on success and C<0> on failure, and I<all> of them have the
305same three trailing arguments: C<pri>, C<cb> and C<data>. The C<cb> is
306mandatory, but in most cases, you pass in C<0> as C<pri> and C<0> or some
307custom data value as C<data>.
308
309=head3 POSIX API WRAPPERS
310
311These requests simply wrap the POSIX call of the same name, with the same
312arguments:
313
314=over 4
315
316=item eio_open (const char *path, int flags, mode_t mode, int pri, eio_cb cb, void *data)
317
318=item eio_utime (const char *path, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data)
319
320=item eio_truncate (const char *path, off_t offset, int pri, eio_cb cb, void *data)
321
322=item eio_chown (const char *path, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data)
323
324=item eio_chmod (const char *path, mode_t mode, int pri, eio_cb cb, void *data)
325
326=item eio_mkdir (const char *path, mode_t mode, int pri, eio_cb cb, void *data)
327
328=item eio_rmdir (const char *path, int pri, eio_cb cb, void *data)
329
330=item eio_unlink (const char *path, int pri, eio_cb cb, void *data)
331
332=item eio_readlink (const char *path, int pri, eio_cb cb, void *data) /* result=ptr2 allocated dynamically */
333
334=item eio_stat (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */
335
336=item eio_lstat (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */
337
338=item eio_statvfs (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */
339
340=item eio_mknod (const char *path, mode_t mode, dev_t dev, int pri, eio_cb cb, void *data)
341
342=item eio_link (const char *path, const char *new_path, int pri, eio_cb cb, void *data)
343
344=item eio_symlink (const char *path, const char *new_path, int pri, eio_cb cb, void *data)
345
346=item eio_rename (const char *path, const char *new_path, int pri, eio_cb cb, void *data)
347
348=item eio_msync (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data)
349
350=item eio_mlock (void *addr, size_t length, int pri, eio_cb cb, void *data)
351
352=item eio_mlockall (int flags, int pri, eio_cb cb, void *data)
353
354=item eio_close (int fd, int pri, eio_cb cb, void *data)
355
356=item eio_sync (int pri, eio_cb cb, void *data)
357
358=item eio_fsync (int fd, int pri, eio_cb cb, void *data)
359
360=item eio_fdatasync (int fd, int pri, eio_cb cb, void *data)
361
362=item eio_futime (int fd, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data)
363
364=item eio_ftruncate (int fd, off_t offset, int pri, eio_cb cb, void *data)
365
366=item eio_fchmod (int fd, mode_t mode, int pri, eio_cb cb, void *data)
367
368=item eio_fchown (int fd, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data)
369
370=item eio_dup2 (int fd, int fd2, int pri, eio_cb cb, void *data)
371
372These have the same semantics as the syscall of the same name, their
373return value is available as C<< req->result >> later.
374
375=item eio_read (int fd, void *buf, size_t length, off_t offset, int pri, eio_cb cb, void *data)
376
377=item eio_write (int fd, void *buf, size_t length, off_t offset, int pri, eio_cb cb, void *data)
378
379These two requests are called C<read> and C<write>, but actually wrap
380C<pread> and C<pwrite>. On systems that lack these calls (such as cygwin),
381libeio uses lseek/read_or_write/lseek and a mutex to serialise the
382requests, so all these requests run serially and do not disturb each
383other. However, they still disturb the file offset while they run, so it's
384not safe to call these functions concurrently with non-libeio functions on
385the same fd on these systems.
386
387Not surprisingly, pread and pwrite are not thread-safe on Darwin (OS/X),
388so it is advised not to submit multiple requests on the same fd on this
389horrible pile of garbage.
390
391=item eio_fstat (int fd, int pri, eio_cb cb, void *data)
392
393Stats a file - if C<< req->result >> indicates success, then you can
394access the C<struct stat>-like structure via C<< req->ptr2 >>:
395
396 EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2;
397
398=item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */
399
400Stats a filesystem - if C<< req->result >> indicates success, then you can
401access the C<struct statvfs>-like structure via C<< req->ptr2 >>:
402
403 EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2;
404
405=back
406
407=head3 READING DIRECTORIES
408
409Reading directories sounds simple, but can be rather demanding, especially
410if you want to do stuff such as traversing a diretcory hierarchy or
411processing all files in a directory. Libeio can assist thess complex tasks
412with it's C<eio_readdir> call.
413
414=over 4
415
416=item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data)
417
418This is a very complex call. It basically reads through a whole directory
419(via the C<opendir>, C<readdir> and C<closedir> calls) and returns either
420the names or an array of C<struct eio_dirent>, depending on the C<flags>
421argument.
422
423The C<< req->result >> indicates either the number of files found, or
424C<-1> on error. On success, zero-terminated names can be found as C<< req->ptr2 >>,
425and C<struct eio_dirents>, if requested by C<flags>, can be found via C<<
426req->ptr1 >>.
427
428Here is an example that prints all the names:
429
430 int i;
431 char *names = (char *)req->ptr2;
432
433 for (i = 0; i < req->result; ++i)
434 {
435 printf ("name #%d: %s\n", i, names);
436
437 /* move to next name */
438 names += strlen (names) + 1;
439 }
440
441Pseudo-entries such as F<.> and F<..> are never returned by C<eio_readdir>.
442
443C<flags> can be any combination of:
444
445=over 4
446
447=item EIO_READDIR_DENTS
448
449If this flag is specified, then, in addition to the names in C<ptr2>,
450also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct
451eio_dirent> looks like this:
452
453 struct eio_dirent
454 {
455 int nameofs; /* offset of null-terminated name string in (char *)req->ptr2 */
456 unsigned short namelen; /* size of filename without trailing 0 */
457 unsigned char type; /* one of EIO_DT_* */
458 signed char score; /* internal use */
459 ino_t inode; /* the inode number, if available, otherwise unspecified */
460 };
461
462The only members you normally would access are C<nameofs>, which is the
463byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>.
464
465C<type> can be one of:
466
467C<EIO_DT_UNKNOWN> - if the type is not known (very common) and you have to C<stat>
468the name yourself if you need to know,
469one of the "standard" POSIX file types (C<EIO_DT_REG>, C<EIO_DT_DIR>, C<EIO_DT_LNK>,
470C<EIO_DT_FIFO>, C<EIO_DT_SOCK>, C<EIO_DT_CHR>, C<EIO_DT_BLK>)
471or some OS-specific type (currently
472C<EIO_DT_MPC> - multiplexed char device (v7+coherent),
473C<EIO_DT_NAM> - xenix special named file,
474C<EIO_DT_MPB> - multiplexed block device (v7+coherent),
475C<EIO_DT_NWK> - HP-UX network special,
476C<EIO_DT_CMP> - VxFS compressed,
477C<EIO_DT_DOOR> - solaris door, or
478C<EIO_DT_WHT>).
479
480This example prints all names and their type:
481
482 int i;
483 struct eio_dirent *ents = (struct eio_dirent *)req->ptr1;
484 char *names = (char *)req->ptr2;
485
486 for (i = 0; i < req->result; ++i)
487 {
488 struct eio_dirent *ent = ents + i;
489 char *name = names + ent->nameofs;
490
491 printf ("name #%d: %s (type %d)\n", i, name, ent->type);
492 }
493
494=item EIO_READDIR_DIRS_FIRST
495
496When this flag is specified, then the names will be returned in an order
497where likely directories come first, in optimal C<stat> order. This is
498useful when you need to quickly find directories, or you want to find all
499directories while avoiding to stat() each entry.
500
501If the system returns type information in readdir, then this is used
502to find directories directly. Otherwise, likely directories are names
503beginning with ".", or otherwise names with no dots, of which names with
504short names are tried first.
505
506=item EIO_READDIR_STAT_ORDER
507
508When this flag is specified, then the names will be returned in an order
509suitable for stat()'ing each one. That is, when you plan to stat()
510all files in the given directory, then the returned order will likely
511be fastest.
512
513If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then
514the likely dirs come first, resulting in a less optimal stat order.
515
516=item EIO_READDIR_FOUND_UNKNOWN
517
518This flag should not be specified when calling C<eio_readdir>. Instead,
519it is being set by C<eio_readdir> (you can access the C<flags> via C<<
520req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The
521absense of this flag therefore indicates that all C<type>'s are known,
522which can be used to speed up some algorithms.
523
524A typical use case would be to identify all subdirectories within a
525directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If
526then this flag is I<NOT> set, then all the entries at the beginning of the
527returned array of type C<EIO_DT_DIR> are the directories. Otherwise, you
528should start C<stat()>'ing the entries starting at the beginning of the
529array, stopping as soon as you found all directories (the count can be
530deduced by the link count of the directory).
531
532=back
533
534=back
535
536=head3 OS-SPECIFIC CALL WRAPPERS
537
538These wrap OS-specific calls (usually Linux ones), and might or might not
539be emulated on other operating systems. Calls that are not emulated will
540return C<-1> and set C<errno> to C<ENOSYS>.
541
542=over 4
543
544=item eio_sendfile (int out_fd, int in_fd, off_t in_offset, size_t length, int pri, eio_cb cb, void *data)
545
546Wraps the C<sendfile> syscall. The arguments follow the Linux version, but
547libeio supports and will use similar calls on FreeBSD, HP/UX, Solaris and
548Darwin.
549
550If the OS doesn't support some sendfile-like call, or the call fails,
551indicating support for the given file descriptor type (for example,
552Linux's sendfile might not support file to file copies), then libeio will
553emulate the call in userspace, so there are almost no limitations on its
554use.
555
556=item eio_readahead (int fd, off_t offset, size_t length, int pri, eio_cb cb, void *data)
557
558Calls C<readahead(2)>. If the syscall is missing, then the call is
559emulated by simply reading the data (currently in 64kiB chunks).
560
561=item eio_sync_file_range (int fd, off_t offset, size_t nbytes, unsigned int flags, int pri, eio_cb cb, void *data)
562
563Calls C<sync_file_range>. If the syscall is missing, then this is the same
564as calling C<fdatasync>.
565
566=back
567
568=head3 LIBEIO-SPECIFIC REQUESTS
569
570These requests are specific to libeio and do not correspond to any OS call.
571
572=over 4
573
574=item eio_mtouch (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data)
575
576=item eio_custom (void (*)(eio_req *) execute, int pri, eio_cb cb, void *data)
577
578Executes a custom request, i.e., a user-specified callback.
579
580The callback gets the C<eio_req *> as parameter and is expected to read
581and modify any request-specific members. Specifically, it should set C<<
582req->result >> to the result value, just like other requests.
583
584Here is an example that simply calls C<open>, like C<eio_open>, but it
585uses the C<data> member as filename and uses a hardcoded C<O_RDONLY>. If
586you want to pass more/other parameters, you either need to pass some
587struct or so via C<data> or provide your own wrapper using the low-level
588API.
589
590 static int
591 my_open_done (eio_req *req)
592 {
593 int fd = req->result;
594
595 return 0;
596 }
597
598 static void
599 my_open (eio_req *req)
600 {
601 req->result = open (req->data, O_RDONLY);
602 }
603
604 eio_custom (my_open, 0, my_open_done, "/etc/passwd");
605
606=item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data)
607
608This is a a request that takes C<delay> seconds to execute, but otherwise
609does nothing - it simply puts one of the worker threads to sleep for this
610long.
611
612This request can be used to artificially increase load, e.g. for debugging
613or benchmarking reasons.
614
615=item eio_nop (int pri, eio_cb cb, void *data)
616
617This request does nothing, except go through the whole request cycle. This
618can be used to measure latency or in some cases to simplify code, but is
619not really of much use.
620
621=back
622
623=head3 GROUPING AND LIMITING REQUESTS
221 624
222#TODO 625#TODO
223 626
627/*****************************************************************************/
628/* groups */
224 629
630eio_req *eio_grp (eio_cb cb, void *data);
631void eio_grp_feed (eio_req *grp, void (*feed)(eio_req *req), int limit);
632void eio_grp_limit (eio_req *grp, int limit);
633void eio_grp_add (eio_req *grp, eio_req *req);
634void eio_grp_cancel (eio_req *grp); /* cancels all sub requests but not the group */
635
636
637=back
638
639
225=head1 HIGH LEVEL REQUEST API 640=head1 LOW LEVEL REQUEST API
226 641
227#TODO 642#TODO
228 643
229=back
230 644
645=head1 ANATOMY AND LIFETIME OF AN EIO REQUEST
231 646
232=head1 LOW LEVEL REQUEST API 647A request is represented by a structure of type C<eio_req>. To initialise
648it, clear it to all zero bytes:
649
650 eio_req req;
651
652 memset (&req, 0, sizeof (req));
653
654A more common way to initialise a new C<eio_req> is to use C<calloc>:
655
656 eio_req *req = calloc (1, sizeof (*req));
657
658In either case, libeio neither allocates, initialises or frees the
659C<eio_req> structure for you - it merely uses it.
660
661zero
233 662
234#TODO 663#TODO
235 664
236=head1 EMBEDDING 665=head1 EMBEDDING
237 666

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines