… | |
… | |
137 | C<want_poll> you would write a single byte, in C<done_poll> you would try |
137 | C<want_poll> you would write a single byte, in C<done_poll> you would try |
138 | to read that byte, and in the callback for the read end, you would call |
138 | to read that byte, and in the callback for the read end, you would call |
139 | C<eio_poll>. The race is avoided here because the event loop should invoke |
139 | C<eio_poll>. The race is avoided here because the event loop should invoke |
140 | your callback again and again until the byte has been read (as the pipe |
140 | your callback again and again until the byte has been read (as the pipe |
141 | read callback does not read it, only C<done_poll>). |
141 | read callback does not read it, only C<done_poll>). |
142 | |
|
|
143 | =head2 CONFIGURATION |
|
|
144 | |
|
|
145 | The functions in this section can sometimes be useful, but the default |
|
|
146 | configuration will do in most case, so you should skip this section on |
|
|
147 | first reading. |
|
|
148 | |
|
|
149 | =over 4 |
|
|
150 | |
|
|
151 | =item eio_set_max_poll_time (eio_tstamp nseconds) |
|
|
152 | |
|
|
153 | This causes C<eio_poll ()> to return after it has detected that it was |
|
|
154 | running for C<nsecond> seconds or longer (this number can be fractional). |
|
|
155 | |
|
|
156 | This can be used to limit the amount of time spent handling eio requests, |
|
|
157 | for example, in interactive programs, you might want to limit this time to |
|
|
158 | C<0.01> seconds or so. |
|
|
159 | |
|
|
160 | Note that: |
|
|
161 | |
|
|
162 | a) libeio doesn't know how long your request callbacks take, so the time |
|
|
163 | spent in C<eio_poll> is up to one callback invocation longer then this |
|
|
164 | interval. |
|
|
165 | |
|
|
166 | b) this is implemented by calling C<gettimeofday> after each request, |
|
|
167 | which can be costly. |
|
|
168 | |
|
|
169 | c) at least one request will be handled. |
|
|
170 | |
|
|
171 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
|
|
172 | |
|
|
173 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
|
|
174 | C<nreqs> requests per invocation. This is a less costly way to limit the |
|
|
175 | amount of work done by C<eio_poll> then setting a time limit. |
|
|
176 | |
|
|
177 | If you know your callbacks are generally fast, you could use this to |
|
|
178 | encourage interactiveness in your programs by setting it to C<10>, C<100> |
|
|
179 | or even C<1000>. |
|
|
180 | |
|
|
181 | =item eio_set_min_parallel (unsigned int nthreads) |
|
|
182 | |
|
|
183 | Make sure libeio can handle at least this many requests in parallel. It |
|
|
184 | might be able handle more. |
|
|
185 | |
|
|
186 | =item eio_set_max_parallel (unsigned int nthreads) |
|
|
187 | |
|
|
188 | Set the maximum number of threads that libeio will spawn. |
|
|
189 | |
|
|
190 | =item eio_set_max_idle (unsigned int nthreads) |
|
|
191 | |
|
|
192 | Libeio uses threads internally to handle most requests, and will start and stop threads on demand. |
|
|
193 | |
|
|
194 | This call can be used to limit the number of idle threads (threads without |
|
|
195 | work to do): libeio will keep some threads idle in preparation for more |
|
|
196 | requests, but never longer than C<nthreads> threads. |
|
|
197 | |
|
|
198 | In addition to this, libeio will also stop threads when they are idle for |
|
|
199 | a few seconds, regardless of this setting. |
|
|
200 | |
|
|
201 | =item unsigned int eio_nthreads () |
|
|
202 | |
|
|
203 | Return the number of worker threads currently running. |
|
|
204 | |
|
|
205 | =item unsigned int eio_nreqs () |
|
|
206 | |
|
|
207 | Return the number of requests currently handled by libeio. This is the |
|
|
208 | total number of requests that have been submitted to libeio, but not yet |
|
|
209 | destroyed. |
|
|
210 | |
|
|
211 | =item unsigned int eio_nready () |
|
|
212 | |
|
|
213 | Returns the number of ready requests, i.e. requests that have been |
|
|
214 | submitted but have not yet entered the execution phase. |
|
|
215 | |
|
|
216 | =item unsigned int eio_npending () |
|
|
217 | |
|
|
218 | Returns the number of pending requests, i.e. requests that have been |
|
|
219 | executed and have results, but have not been finished yet by a call to |
|
|
220 | C<eio_poll>). |
|
|
221 | |
|
|
222 | =back |
|
|
223 | |
142 | |
224 | |
143 | |
225 | =head1 HIGH LEVEL REQUEST API |
144 | =head1 HIGH LEVEL REQUEST API |
226 | |
145 | |
227 | Libeio has both a high-level API, which consists of calling a request |
146 | Libeio has both a high-level API, which consists of calling a request |
… | |
… | |
307 | custom data value as C<data>. |
226 | custom data value as C<data>. |
308 | |
227 | |
309 | =head3 POSIX API WRAPPERS |
228 | =head3 POSIX API WRAPPERS |
310 | |
229 | |
311 | These requests simply wrap the POSIX call of the same name, with the same |
230 | These requests simply wrap the POSIX call of the same name, with the same |
312 | arguments: |
231 | arguments. If a function is not implemented by the OS and cannot be emulated |
|
|
232 | in some way, then all of these return C<-1> and set C<errorno> to C<ENOSYS>. |
313 | |
233 | |
314 | =over 4 |
234 | =over 4 |
315 | |
235 | |
316 | =item eio_open (const char *path, int flags, mode_t mode, int pri, eio_cb cb, void *data) |
236 | =item eio_open (const char *path, int flags, mode_t mode, int pri, eio_cb cb, void *data) |
317 | |
237 | |
|
|
238 | =item eio_truncate (const char *path, off_t offset, int pri, eio_cb cb, void *data) |
|
|
239 | |
|
|
240 | =item eio_chown (const char *path, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data) |
|
|
241 | |
|
|
242 | =item eio_chmod (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
243 | |
|
|
244 | =item eio_mkdir (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
245 | |
|
|
246 | =item eio_rmdir (const char *path, int pri, eio_cb cb, void *data) |
|
|
247 | |
|
|
248 | =item eio_unlink (const char *path, int pri, eio_cb cb, void *data) |
|
|
249 | |
318 | =item eio_utime (const char *path, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data) |
250 | =item eio_utime (const char *path, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data) |
319 | |
251 | |
320 | =item eio_truncate (const char *path, off_t offset, int pri, eio_cb cb, void *data) |
|
|
321 | |
|
|
322 | =item eio_chown (const char *path, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data) |
|
|
323 | |
|
|
324 | =item eio_chmod (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
325 | |
|
|
326 | =item eio_mkdir (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
327 | |
|
|
328 | =item eio_rmdir (const char *path, int pri, eio_cb cb, void *data) |
|
|
329 | |
|
|
330 | =item eio_unlink (const char *path, int pri, eio_cb cb, void *data) |
|
|
331 | |
|
|
332 | =item eio_readlink (const char *path, int pri, eio_cb cb, void *data) /* result=ptr2 allocated dynamically */ |
|
|
333 | |
|
|
334 | =item eio_stat (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */ |
|
|
335 | |
|
|
336 | =item eio_lstat (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */ |
|
|
337 | |
|
|
338 | =item eio_statvfs (const char *path, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */ |
|
|
339 | |
|
|
340 | =item eio_mknod (const char *path, mode_t mode, dev_t dev, int pri, eio_cb cb, void *data) |
252 | =item eio_mknod (const char *path, mode_t mode, dev_t dev, int pri, eio_cb cb, void *data) |
341 | |
253 | |
342 | =item eio_link (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
254 | =item eio_link (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
343 | |
255 | |
344 | =item eio_symlink (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
256 | =item eio_symlink (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
345 | |
257 | |
346 | =item eio_rename (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
258 | =item eio_rename (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
347 | |
259 | |
348 | =item eio_msync (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
|
|
349 | |
|
|
350 | =item eio_mlock (void *addr, size_t length, int pri, eio_cb cb, void *data) |
260 | =item eio_mlock (void *addr, size_t length, int pri, eio_cb cb, void *data) |
351 | |
|
|
352 | =item eio_mlockall (int flags, int pri, eio_cb cb, void *data) |
|
|
353 | |
261 | |
354 | =item eio_close (int fd, int pri, eio_cb cb, void *data) |
262 | =item eio_close (int fd, int pri, eio_cb cb, void *data) |
355 | |
263 | |
356 | =item eio_sync (int pri, eio_cb cb, void *data) |
264 | =item eio_sync (int pri, eio_cb cb, void *data) |
357 | |
265 | |
… | |
… | |
386 | |
294 | |
387 | Not surprisingly, pread and pwrite are not thread-safe on Darwin (OS/X), |
295 | Not surprisingly, pread and pwrite are not thread-safe on Darwin (OS/X), |
388 | so it is advised not to submit multiple requests on the same fd on this |
296 | so it is advised not to submit multiple requests on the same fd on this |
389 | horrible pile of garbage. |
297 | horrible pile of garbage. |
390 | |
298 | |
|
|
299 | =item eio_mlockall (int flags, int pri, eio_cb cb, void *data) |
|
|
300 | |
|
|
301 | Like C<mlockall>, but the flag value constants are called |
|
|
302 | C<EIO_MCL_CURRENT> and C<EIO_MCL_FUTURE>. |
|
|
303 | |
|
|
304 | =item eio_msync (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
|
|
305 | |
|
|
306 | Just like msync, except that the flag values are called C<EIO_MS_ASYNC>, |
|
|
307 | C<EIO_MS_INVALIDATE> and C<EIO_MS_SYNC>. |
|
|
308 | |
|
|
309 | =item eio_readlink (const char *path, int pri, eio_cb cb, void *data) |
|
|
310 | |
|
|
311 | If successful, the path read by C<readlink(2)> can be accessed via C<< |
|
|
312 | req->ptr2 >> and is I<NOT> null-terminated, with the length specified as |
|
|
313 | C<< req->result >>. |
|
|
314 | |
|
|
315 | if (req->result >= 0) |
|
|
316 | { |
|
|
317 | char *target = strndup ((char *)req->ptr2, req->result); |
|
|
318 | |
|
|
319 | free (target); |
|
|
320 | } |
|
|
321 | |
|
|
322 | =item eio_stat (const char *path, int pri, eio_cb cb, void *data) |
|
|
323 | |
|
|
324 | =item eio_lstat (const char *path, int pri, eio_cb cb, void *data) |
|
|
325 | |
391 | =item eio_fstat (int fd, int pri, eio_cb cb, void *data) |
326 | =item eio_fstat (int fd, int pri, eio_cb cb, void *data) |
392 | |
327 | |
393 | Stats a file - if C<< req->result >> indicates success, then you can |
328 | Stats a file - if C<< req->result >> indicates success, then you can |
394 | access the C<struct stat>-like structure via C<< req->ptr2 >>: |
329 | access the C<struct stat>-like structure via C<< req->ptr2 >>: |
395 | |
330 | |
396 | EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2; |
331 | EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2; |
397 | |
332 | |
398 | =item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data) /* stat buffer=ptr2 allocated dynamically */ |
333 | =item eio_statvfs (const char *path, int pri, eio_cb cb, void *data) |
|
|
334 | |
|
|
335 | =item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data) |
399 | |
336 | |
400 | Stats a filesystem - if C<< req->result >> indicates success, then you can |
337 | Stats a filesystem - if C<< req->result >> indicates success, then you can |
401 | access the C<struct statvfs>-like structure via C<< req->ptr2 >>: |
338 | access the C<struct statvfs>-like structure via C<< req->ptr2 >>: |
402 | |
339 | |
403 | EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2; |
340 | EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2; |
… | |
… | |
419 | (via the C<opendir>, C<readdir> and C<closedir> calls) and returns either |
356 | (via the C<opendir>, C<readdir> and C<closedir> calls) and returns either |
420 | the names or an array of C<struct eio_dirent>, depending on the C<flags> |
357 | the names or an array of C<struct eio_dirent>, depending on the C<flags> |
421 | argument. |
358 | argument. |
422 | |
359 | |
423 | The C<< req->result >> indicates either the number of files found, or |
360 | The C<< req->result >> indicates either the number of files found, or |
424 | C<-1> on error. On success, zero-terminated names can be found as C<< req->ptr2 >>, |
361 | C<-1> on error. On success, null-terminated names can be found as C<< req->ptr2 >>, |
425 | and C<struct eio_dirents>, if requested by C<flags>, can be found via C<< |
362 | and C<struct eio_dirents>, if requested by C<flags>, can be found via C<< |
426 | req->ptr1 >>. |
363 | req->ptr1 >>. |
427 | |
364 | |
428 | Here is an example that prints all the names: |
365 | Here is an example that prints all the names: |
429 | |
366 | |
… | |
… | |
561 | =item eio_sync_file_range (int fd, off_t offset, size_t nbytes, unsigned int flags, int pri, eio_cb cb, void *data) |
498 | =item eio_sync_file_range (int fd, off_t offset, size_t nbytes, unsigned int flags, int pri, eio_cb cb, void *data) |
562 | |
499 | |
563 | Calls C<sync_file_range>. If the syscall is missing, then this is the same |
500 | Calls C<sync_file_range>. If the syscall is missing, then this is the same |
564 | as calling C<fdatasync>. |
501 | as calling C<fdatasync>. |
565 | |
502 | |
|
|
503 | Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>, |
|
|
504 | C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>. |
|
|
505 | |
566 | =back |
506 | =back |
567 | |
507 | |
568 | =head3 LIBEIO-SPECIFIC REQUESTS |
508 | =head3 LIBEIO-SPECIFIC REQUESTS |
569 | |
509 | |
570 | These requests are specific to libeio and do not correspond to any OS call. |
510 | These requests are specific to libeio and do not correspond to any OS call. |
571 | |
511 | |
572 | =over 4 |
512 | =over 4 |
573 | |
513 | |
574 | =item eio_mtouch (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
514 | =item eio_mtouch (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
575 | |
515 | |
|
|
516 | Reads (C<flags == 0>) or modifies (C<flags == EIO_MT_MODIFY) the given |
|
|
517 | memory area, page-wise, that is, it reads (or reads and writes back) the |
|
|
518 | first octet of every page that spans the memory area. |
|
|
519 | |
|
|
520 | This can be used to page in some mmapped file, or dirty some pages. Note |
|
|
521 | that dirtying is an unlocked read-write access, so races can ensue when |
|
|
522 | the some other thread modifies the data stored in that memory area. |
|
|
523 | |
576 | =item eio_custom (void (*)(eio_req *) execute, int pri, eio_cb cb, void *data) |
524 | =item eio_custom (void (*)(eio_req *) execute, int pri, eio_cb cb, void *data) |
577 | |
525 | |
578 | Executes a custom request, i.e., a user-specified callback. |
526 | Executes a custom request, i.e., a user-specified callback. |
579 | |
527 | |
580 | The callback gets the C<eio_req *> as parameter and is expected to read |
528 | The callback gets the C<eio_req *> as parameter and is expected to read |
581 | and modify any request-specific members. Specifically, it should set C<< |
529 | and modify any request-specific members. Specifically, it should set C<< |
… | |
… | |
601 | req->result = open (req->data, O_RDONLY); |
549 | req->result = open (req->data, O_RDONLY); |
602 | } |
550 | } |
603 | |
551 | |
604 | eio_custom (my_open, 0, my_open_done, "/etc/passwd"); |
552 | eio_custom (my_open, 0, my_open_done, "/etc/passwd"); |
605 | |
553 | |
606 | =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) |
554 | =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) |
607 | |
555 | |
608 | This is a a request that takes C<delay> seconds to execute, but otherwise |
556 | This is a a request that takes C<delay> seconds to execute, but otherwise |
609 | does nothing - it simply puts one of the worker threads to sleep for this |
557 | does nothing - it simply puts one of the worker threads to sleep for this |
610 | long. |
558 | long. |
611 | |
559 | |
612 | This request can be used to artificially increase load, e.g. for debugging |
560 | This request can be used to artificially increase load, e.g. for debugging |
613 | or benchmarking reasons. |
561 | or benchmarking reasons. |
614 | |
562 | |
615 | =item eio_nop (int pri, eio_cb cb, void *data) |
563 | =item eio_nop (int pri, eio_cb cb, void *data) |
616 | |
564 | |
617 | This request does nothing, except go through the whole request cycle. This |
565 | This request does nothing, except go through the whole request cycle. This |
618 | can be used to measure latency or in some cases to simplify code, but is |
566 | can be used to measure latency or in some cases to simplify code, but is |
619 | not really of much use. |
567 | not really of much use. |
620 | |
568 | |
… | |
… | |
660 | |
608 | |
661 | zero |
609 | zero |
662 | |
610 | |
663 | #TODO |
611 | #TODO |
664 | |
612 | |
|
|
613 | =head2 CONFIGURATION |
|
|
614 | |
|
|
615 | The functions in this section can sometimes be useful, but the default |
|
|
616 | configuration will do in most case, so you should skip this section on |
|
|
617 | first reading. |
|
|
618 | |
|
|
619 | =over 4 |
|
|
620 | |
|
|
621 | =item eio_set_max_poll_time (eio_tstamp nseconds) |
|
|
622 | |
|
|
623 | This causes C<eio_poll ()> to return after it has detected that it was |
|
|
624 | running for C<nsecond> seconds or longer (this number can be fractional). |
|
|
625 | |
|
|
626 | This can be used to limit the amount of time spent handling eio requests, |
|
|
627 | for example, in interactive programs, you might want to limit this time to |
|
|
628 | C<0.01> seconds or so. |
|
|
629 | |
|
|
630 | Note that: |
|
|
631 | |
|
|
632 | a) libeio doesn't know how long your request callbacks take, so the time |
|
|
633 | spent in C<eio_poll> is up to one callback invocation longer then this |
|
|
634 | interval. |
|
|
635 | |
|
|
636 | b) this is implemented by calling C<gettimeofday> after each request, |
|
|
637 | which can be costly. |
|
|
638 | |
|
|
639 | c) at least one request will be handled. |
|
|
640 | |
|
|
641 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
|
|
642 | |
|
|
643 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
|
|
644 | C<nreqs> requests per invocation. This is a less costly way to limit the |
|
|
645 | amount of work done by C<eio_poll> then setting a time limit. |
|
|
646 | |
|
|
647 | If you know your callbacks are generally fast, you could use this to |
|
|
648 | encourage interactiveness in your programs by setting it to C<10>, C<100> |
|
|
649 | or even C<1000>. |
|
|
650 | |
|
|
651 | =item eio_set_min_parallel (unsigned int nthreads) |
|
|
652 | |
|
|
653 | Make sure libeio can handle at least this many requests in parallel. It |
|
|
654 | might be able handle more. |
|
|
655 | |
|
|
656 | =item eio_set_max_parallel (unsigned int nthreads) |
|
|
657 | |
|
|
658 | Set the maximum number of threads that libeio will spawn. |
|
|
659 | |
|
|
660 | =item eio_set_max_idle (unsigned int nthreads) |
|
|
661 | |
|
|
662 | Libeio uses threads internally to handle most requests, and will start and stop threads on demand. |
|
|
663 | |
|
|
664 | This call can be used to limit the number of idle threads (threads without |
|
|
665 | work to do): libeio will keep some threads idle in preparation for more |
|
|
666 | requests, but never longer than C<nthreads> threads. |
|
|
667 | |
|
|
668 | In addition to this, libeio will also stop threads when they are idle for |
|
|
669 | a few seconds, regardless of this setting. |
|
|
670 | |
|
|
671 | =item unsigned int eio_nthreads () |
|
|
672 | |
|
|
673 | Return the number of worker threads currently running. |
|
|
674 | |
|
|
675 | =item unsigned int eio_nreqs () |
|
|
676 | |
|
|
677 | Return the number of requests currently handled by libeio. This is the |
|
|
678 | total number of requests that have been submitted to libeio, but not yet |
|
|
679 | destroyed. |
|
|
680 | |
|
|
681 | =item unsigned int eio_nready () |
|
|
682 | |
|
|
683 | Returns the number of ready requests, i.e. requests that have been |
|
|
684 | submitted but have not yet entered the execution phase. |
|
|
685 | |
|
|
686 | =item unsigned int eio_npending () |
|
|
687 | |
|
|
688 | Returns the number of pending requests, i.e. requests that have been |
|
|
689 | executed and have results, but have not been finished yet by a call to |
|
|
690 | C<eio_poll>). |
|
|
691 | |
|
|
692 | =back |
|
|
693 | |
665 | =head1 EMBEDDING |
694 | =head1 EMBEDDING |
666 | |
695 | |
667 | Libeio can be embedded directly into programs. This functionality is not |
696 | Libeio can be embedded directly into programs. This functionality is not |
668 | documented and not (yet) officially supported. |
697 | documented and not (yet) officially supported. |
669 | |
698 | |