… | |
… | |
11 | The newest version of this document is also available as an html-formatted |
11 | The newest version of this document is also available as an html-formatted |
12 | web page you might find easier to navigate when reading it for the first |
12 | web page you might find easier to navigate when reading it for the first |
13 | time: L<http://pod.tst.eu/http://cvs.schmorp.de/libeio/eio.pod>. |
13 | time: L<http://pod.tst.eu/http://cvs.schmorp.de/libeio/eio.pod>. |
14 | |
14 | |
15 | Note that this library is a by-product of the C<IO::AIO> perl |
15 | Note that this library is a by-product of the C<IO::AIO> perl |
16 | module, and many of the subtler points regarding requets lifetime |
16 | module, and many of the subtler points regarding requests lifetime |
17 | and so on are only documented in its documentation at the |
17 | and so on are only documented in its documentation at the |
18 | moment: L<http://pod.tst.eu/http://cvs.schmorp.de/IO-AIO/AIO.pm>. |
18 | moment: L<http://pod.tst.eu/http://cvs.schmorp.de/IO-AIO/AIO.pm>. |
19 | |
19 | |
20 | =head2 FEATURES |
20 | =head2 FEATURES |
21 | |
21 | |
22 | This library provides fully asynchronous versions of most POSIX functions |
22 | This library provides fully asynchronous versions of most POSIX functions |
23 | dealign with I/O. Unlike most asynchronous libraries, this not only |
23 | dealing with I/O. Unlike most asynchronous libraries, this not only |
24 | includes C<read> and C<write>, but also C<open>, C<stat>, C<unlink> and |
24 | includes C<read> and C<write>, but also C<open>, C<stat>, C<unlink> and |
25 | similar functions, as well as less rarely ones such as C<mknod>, C<futime> |
25 | similar functions, as well as less rarely ones such as C<mknod>, C<futime> |
26 | or C<readlink>. |
26 | or C<readlink>. |
27 | |
27 | |
28 | It also offers wrappers around C<sendfile> (Solaris, Linux, HP-UX and |
28 | It also offers wrappers around C<sendfile> (Solaris, Linux, HP-UX and |
29 | FreeBSD, with emulation on other platforms) and C<readahead> (Linux, with |
29 | FreeBSD, with emulation on other platforms) and C<readahead> (Linux, with |
30 | emulation elsewhere>). |
30 | emulation elsewhere>). |
31 | |
31 | |
32 | The goal is to enbale you to write fully non-blocking programs. For |
32 | The goal is to enable you to write fully non-blocking programs. For |
33 | example, in a game server, you would not want to freeze for a few seconds |
33 | example, in a game server, you would not want to freeze for a few seconds |
34 | just because the server is running a backup and you happen to call |
34 | just because the server is running a backup and you happen to call |
35 | C<readdir>. |
35 | C<readdir>. |
36 | |
36 | |
37 | =head2 TIME REPRESENTATION |
37 | =head2 TIME REPRESENTATION |
38 | |
38 | |
39 | Libeio represents time as a single floating point number, representing the |
39 | Libeio represents time as a single floating point number, representing the |
40 | (fractional) number of seconds since the (POSIX) epoch (somewhere near |
40 | (fractional) number of seconds since the (POSIX) epoch (somewhere near |
41 | the beginning of 1970, details are complicated, don't ask). This type is |
41 | the beginning of 1970, details are complicated, don't ask). This type is |
42 | called C<eio_tstamp>, but it is guarenteed to be of type C<double> (or |
42 | called C<eio_tstamp>, but it is guaranteed to be of type C<double> (or |
43 | better), so you can freely use C<double> yourself. |
43 | better), so you can freely use C<double> yourself. |
44 | |
44 | |
45 | Unlike the name component C<stamp> might indicate, it is also used for |
45 | Unlike the name component C<stamp> might indicate, it is also used for |
46 | time differences throughout libeio. |
46 | time differences throughout libeio. |
47 | |
47 | |
48 | =head2 FORK SUPPORT |
48 | =head2 FORK SUPPORT |
49 | |
49 | |
50 | Calling C<fork ()> is fully supported by this module. It is implemented in these steps: |
50 | Calling C<fork ()> is fully supported by this module - but you must not |
|
|
51 | rely on this. It is currently implemented in these steps: |
51 | |
52 | |
52 | 1. wait till all requests in "execute" state have been handled |
53 | 1. wait till all requests in "execute" state have been handled |
53 | (basically requests that are already handed over to the kernel). |
54 | (basically requests that are already handed over to the kernel). |
54 | 2. fork |
55 | 2. fork |
55 | 3. in the parent, continue business as usual, done |
56 | 3. in the parent, continue business as usual, done |
56 | 4. in the child, destroy all ready and pending requests and free the |
57 | 4. in the child, destroy all ready and pending requests and free the |
57 | memory used by the worker threads. This gives you a fully empty |
58 | memory used by the worker threads. This gives you a fully empty |
58 | libeio queue. |
59 | libeio queue. |
|
|
60 | |
|
|
61 | Note, however, since libeio does use threads, the above guarantee doesn't |
|
|
62 | cover your libc, for example, malloc and other libc functions are not |
|
|
63 | fork-safe, so there is very little you can do after a fork, and in fact, |
|
|
64 | the above might crash, and thus change. |
59 | |
65 | |
60 | =head1 INITIALISATION/INTEGRATION |
66 | =head1 INITIALISATION/INTEGRATION |
61 | |
67 | |
62 | Before you can call any eio functions you first have to initialise the |
68 | Before you can call any eio functions you first have to initialise the |
63 | library. The library integrates into any event loop, but can also be used |
69 | library. The library integrates into any event loop, but can also be used |
… | |
… | |
97 | handled or C<done_poll> has been called, which signals the same. |
103 | handled or C<done_poll> has been called, which signals the same. |
98 | |
104 | |
99 | Note that C<eio_poll> might return after C<done_poll> and C<want_poll> |
105 | Note that C<eio_poll> might return after C<done_poll> and C<want_poll> |
100 | have been called again, so watch out for races in your code. |
106 | have been called again, so watch out for races in your code. |
101 | |
107 | |
102 | As with C<want_poll>, this callback is called while lcoks are being held, |
108 | As with C<want_poll>, this callback is called while locks are being held, |
103 | so you I<must not call any libeio functions form within this callback>. |
109 | so you I<must not call any libeio functions form within this callback>. |
104 | |
110 | |
105 | =item int eio_poll () |
111 | =item int eio_poll () |
106 | |
112 | |
107 | This function has to be called whenever there are pending requests that |
113 | This function has to be called whenever there are pending requests that |
… | |
… | |
119 | =back |
125 | =back |
120 | |
126 | |
121 | For libev, you would typically use an C<ev_async> watcher: the |
127 | For libev, you would typically use an C<ev_async> watcher: the |
122 | C<want_poll> callback would invoke C<ev_async_send> to wake up the event |
128 | C<want_poll> callback would invoke C<ev_async_send> to wake up the event |
123 | loop. Inside the callback set for the watcher, one would call C<eio_poll |
129 | loop. Inside the callback set for the watcher, one would call C<eio_poll |
124 | ()> (followed by C<ev_async_send> again if C<eio_poll> indicates that not |
130 | ()>. |
125 | all requests have been handled yet). The race is taken care of because |
131 | |
126 | libev resets/rearms the async watcher before calling your callback, |
132 | If C<eio_poll ()> is configured to not handle all results in one go |
127 | and therefore, before calling C<eio_poll>. This might result in (some) |
133 | (i.e. it returns C<-1>) then you should start an idle watcher that calls |
128 | spurious wake-ups, but is generally harmless. |
134 | C<eio_poll> until it returns something C<!= -1>. |
|
|
135 | |
|
|
136 | A full-featured connector between libeio and libev would look as follows |
|
|
137 | (if C<eio_poll> is handling all requests, it can of course be simplified a |
|
|
138 | lot by removing the idle watcher logic): |
|
|
139 | |
|
|
140 | static struct ev_loop *loop; |
|
|
141 | static ev_idle repeat_watcher; |
|
|
142 | static ev_async ready_watcher; |
|
|
143 | |
|
|
144 | /* idle watcher callback, only used when eio_poll */ |
|
|
145 | /* didn't handle all results in one call */ |
|
|
146 | static void |
|
|
147 | repeat (EV_P_ ev_idle *w, int revents) |
|
|
148 | { |
|
|
149 | if (eio_poll () != -1) |
|
|
150 | ev_idle_stop (EV_A_ w); |
|
|
151 | } |
|
|
152 | |
|
|
153 | /* eio has some results, process them */ |
|
|
154 | static void |
|
|
155 | ready (EV_P_ ev_async *w, int revents) |
|
|
156 | { |
|
|
157 | if (eio_poll () == -1) |
|
|
158 | ev_idle_start (EV_A_ &repeat_watcher); |
|
|
159 | } |
|
|
160 | |
|
|
161 | /* wake up the event loop */ |
|
|
162 | static void |
|
|
163 | want_poll (void) |
|
|
164 | { |
|
|
165 | ev_async_send (loop, &ready_watcher) |
|
|
166 | } |
|
|
167 | |
|
|
168 | void |
|
|
169 | my_init_eio () |
|
|
170 | { |
|
|
171 | loop = EV_DEFAULT; |
|
|
172 | |
|
|
173 | ev_idle_init (&repeat_watcher, repeat); |
|
|
174 | ev_async_init (&ready_watcher, ready); |
|
|
175 | ev_async_start (loop &watcher); |
|
|
176 | |
|
|
177 | eio_init (want_poll, 0); |
|
|
178 | } |
129 | |
179 | |
130 | For most other event loops, you would typically use a pipe - the event |
180 | For most other event loops, you would typically use a pipe - the event |
131 | loop should be told to wait for read readyness on the read end. In |
181 | loop should be told to wait for read readiness on the read end. In |
132 | C<want_poll> you would write a single byte, in C<done_poll> you would try |
182 | C<want_poll> you would write a single byte, in C<done_poll> you would try |
133 | to read that byte, and in the callback for the read end, you would call |
183 | to read that byte, and in the callback for the read end, you would call |
134 | C<eio_poll>. The race is avoided here because the event loop should invoke |
184 | C<eio_poll>. |
135 | your callback again and again until the byte has been read (as the pipe |
185 | |
136 | read callback does not read it, only C<done_poll>). |
186 | You don't have to take special care in the case C<eio_poll> doesn't handle |
|
|
187 | all requests, as the done callback will not be invoked, so the event loop |
|
|
188 | will still signal readiness for the pipe until I<all> results have been |
|
|
189 | processed. |
|
|
190 | |
|
|
191 | |
|
|
192 | =head1 HIGH LEVEL REQUEST API |
|
|
193 | |
|
|
194 | Libeio has both a high-level API, which consists of calling a request |
|
|
195 | function with a callback to be called on completion, and a low-level API |
|
|
196 | where you fill out request structures and submit them. |
|
|
197 | |
|
|
198 | This section describes the high-level API. |
|
|
199 | |
|
|
200 | =head2 REQUEST SUBMISSION AND RESULT PROCESSING |
|
|
201 | |
|
|
202 | You submit a request by calling the relevant C<eio_TYPE> function with the |
|
|
203 | required parameters, a callback of type C<int (*eio_cb)(eio_req *req)> |
|
|
204 | (called C<eio_cb> below) and a freely usable C<void *data> argument. |
|
|
205 | |
|
|
206 | The return value will either be 0, in case something went really wrong |
|
|
207 | (which can basically only happen on very fatal errors, such as C<malloc> |
|
|
208 | returning 0, which is rather unlikely), or a pointer to the newly-created |
|
|
209 | and submitted C<eio_req *>. |
|
|
210 | |
|
|
211 | The callback will be called with an C<eio_req *> which contains the |
|
|
212 | results of the request. The members you can access inside that structure |
|
|
213 | vary from request to request, except for: |
|
|
214 | |
|
|
215 | =over 4 |
|
|
216 | |
|
|
217 | =item C<ssize_t result> |
|
|
218 | |
|
|
219 | This contains the result value from the call (usually the same as the |
|
|
220 | syscall of the same name). |
|
|
221 | |
|
|
222 | =item C<int errorno> |
|
|
223 | |
|
|
224 | This contains the value of C<errno> after the call. |
|
|
225 | |
|
|
226 | =item C<void *data> |
|
|
227 | |
|
|
228 | The C<void *data> member simply stores the value of the C<data> argument. |
|
|
229 | |
|
|
230 | =back |
|
|
231 | |
|
|
232 | The return value of the callback is normally C<0>, which tells libeio to |
|
|
233 | continue normally. If a callback returns a nonzero value, libeio will |
|
|
234 | stop processing results (in C<eio_poll>) and will return the value to its |
|
|
235 | caller. |
|
|
236 | |
|
|
237 | Memory areas passed to libeio must stay valid as long as a request |
|
|
238 | executes, with the exception of paths, which are being copied |
|
|
239 | internally. Any memory libeio itself allocates will be freed after the |
|
|
240 | finish callback has been called. If you want to manage all memory passed |
|
|
241 | to libeio yourself you can use the low-level API. |
|
|
242 | |
|
|
243 | For example, to open a file, you could do this: |
|
|
244 | |
|
|
245 | static int |
|
|
246 | file_open_done (eio_req *req) |
|
|
247 | { |
|
|
248 | if (req->result < 0) |
|
|
249 | { |
|
|
250 | /* open() returned -1 */ |
|
|
251 | errno = req->errorno; |
|
|
252 | perror ("open"); |
|
|
253 | } |
|
|
254 | else |
|
|
255 | { |
|
|
256 | int fd = req->result; |
|
|
257 | /* now we have the new fd in fd */ |
|
|
258 | } |
|
|
259 | |
|
|
260 | return 0; |
|
|
261 | } |
|
|
262 | |
|
|
263 | /* the first three arguments are passed to open(2) */ |
|
|
264 | /* the remaining are priority, callback and data */ |
|
|
265 | if (!eio_open ("/etc/passwd", O_RDONLY, 0, 0, file_open_done, 0)) |
|
|
266 | abort (); /* something went wrong, we will all die!!! */ |
|
|
267 | |
|
|
268 | Note that you additionally need to call C<eio_poll> when the C<want_cb> |
|
|
269 | indicates that requests are ready to be processed. |
|
|
270 | |
|
|
271 | =head2 CANCELLING REQUESTS |
|
|
272 | |
|
|
273 | Sometimes the need for a request goes away before the request is |
|
|
274 | finished. In that case, one can cancel the request by a call to |
|
|
275 | C<eio_cancel>: |
|
|
276 | |
|
|
277 | =over 4 |
|
|
278 | |
|
|
279 | =item eio_cancel (eio_req *req) |
|
|
280 | |
|
|
281 | Cancel the request (and all its subrequests). If the request is currently |
|
|
282 | executing it might still continue to execute, and in other cases it might |
|
|
283 | still take a while till the request is cancelled. |
|
|
284 | |
|
|
285 | Even if cancelled, the finish callback will still be invoked - the |
|
|
286 | callbacks of all cancellable requests need to check whether the request |
|
|
287 | has been cancelled by calling C<EIO_CANCELLED (req)>: |
|
|
288 | |
|
|
289 | static int |
|
|
290 | my_eio_cb (eio_req *req) |
|
|
291 | { |
|
|
292 | if (EIO_CANCELLED (req)) |
|
|
293 | return 0; |
|
|
294 | } |
|
|
295 | |
|
|
296 | In addition, cancelled requests will I<either> have C<< req->result >> |
|
|
297 | set to C<-1> and C<errno> to C<ECANCELED>, or I<otherwise> they were |
|
|
298 | successfully executed, despite being cancelled (e.g. when they have |
|
|
299 | already been executed at the time they were cancelled). |
|
|
300 | |
|
|
301 | C<EIO_CANCELLED> is still true for requests that have successfully |
|
|
302 | executed, as long as C<eio_cancel> was called on them at some point. |
|
|
303 | |
|
|
304 | =back |
|
|
305 | |
|
|
306 | =head2 AVAILABLE REQUESTS |
|
|
307 | |
|
|
308 | The following request functions are available. I<All> of them return the |
|
|
309 | C<eio_req *> on success and C<0> on failure, and I<all> of them have the |
|
|
310 | same three trailing arguments: C<pri>, C<cb> and C<data>. The C<cb> is |
|
|
311 | mandatory, but in most cases, you pass in C<0> as C<pri> and C<0> or some |
|
|
312 | custom data value as C<data>. |
|
|
313 | |
|
|
314 | =head3 POSIX API WRAPPERS |
|
|
315 | |
|
|
316 | These requests simply wrap the POSIX call of the same name, with the same |
|
|
317 | arguments. If a function is not implemented by the OS and cannot be emulated |
|
|
318 | in some way, then all of these return C<-1> and set C<errorno> to C<ENOSYS>. |
|
|
319 | |
|
|
320 | =over 4 |
|
|
321 | |
|
|
322 | =item eio_open (const char *path, int flags, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
323 | |
|
|
324 | =item eio_truncate (const char *path, off_t offset, int pri, eio_cb cb, void *data) |
|
|
325 | |
|
|
326 | =item eio_chown (const char *path, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data) |
|
|
327 | |
|
|
328 | =item eio_chmod (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
329 | |
|
|
330 | =item eio_mkdir (const char *path, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
331 | |
|
|
332 | =item eio_rmdir (const char *path, int pri, eio_cb cb, void *data) |
|
|
333 | |
|
|
334 | =item eio_unlink (const char *path, int pri, eio_cb cb, void *data) |
|
|
335 | |
|
|
336 | =item eio_utime (const char *path, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data) |
|
|
337 | |
|
|
338 | =item eio_mknod (const char *path, mode_t mode, dev_t dev, int pri, eio_cb cb, void *data) |
|
|
339 | |
|
|
340 | =item eio_link (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
|
|
341 | |
|
|
342 | =item eio_symlink (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
|
|
343 | |
|
|
344 | =item eio_rename (const char *path, const char *new_path, int pri, eio_cb cb, void *data) |
|
|
345 | |
|
|
346 | =item eio_mlock (void *addr, size_t length, int pri, eio_cb cb, void *data) |
|
|
347 | |
|
|
348 | =item eio_close (int fd, int pri, eio_cb cb, void *data) |
|
|
349 | |
|
|
350 | =item eio_sync (int pri, eio_cb cb, void *data) |
|
|
351 | |
|
|
352 | =item eio_fsync (int fd, int pri, eio_cb cb, void *data) |
|
|
353 | |
|
|
354 | =item eio_fdatasync (int fd, int pri, eio_cb cb, void *data) |
|
|
355 | |
|
|
356 | =item eio_futime (int fd, eio_tstamp atime, eio_tstamp mtime, int pri, eio_cb cb, void *data) |
|
|
357 | |
|
|
358 | =item eio_ftruncate (int fd, off_t offset, int pri, eio_cb cb, void *data) |
|
|
359 | |
|
|
360 | =item eio_fchmod (int fd, mode_t mode, int pri, eio_cb cb, void *data) |
|
|
361 | |
|
|
362 | =item eio_fchown (int fd, uid_t uid, gid_t gid, int pri, eio_cb cb, void *data) |
|
|
363 | |
|
|
364 | =item eio_dup2 (int fd, int fd2, int pri, eio_cb cb, void *data) |
|
|
365 | |
|
|
366 | These have the same semantics as the syscall of the same name, their |
|
|
367 | return value is available as C<< req->result >> later. |
|
|
368 | |
|
|
369 | =item eio_read (int fd, void *buf, size_t length, off_t offset, int pri, eio_cb cb, void *data) |
|
|
370 | |
|
|
371 | =item eio_write (int fd, void *buf, size_t length, off_t offset, int pri, eio_cb cb, void *data) |
|
|
372 | |
|
|
373 | These two requests are called C<read> and C<write>, but actually wrap |
|
|
374 | C<pread> and C<pwrite>. On systems that lack these calls (such as cygwin), |
|
|
375 | libeio uses lseek/read_or_write/lseek and a mutex to serialise the |
|
|
376 | requests, so all these requests run serially and do not disturb each |
|
|
377 | other. However, they still disturb the file offset while they run, so it's |
|
|
378 | not safe to call these functions concurrently with non-libeio functions on |
|
|
379 | the same fd on these systems. |
|
|
380 | |
|
|
381 | Not surprisingly, pread and pwrite are not thread-safe on Darwin (OS/X), |
|
|
382 | so it is advised not to submit multiple requests on the same fd on this |
|
|
383 | horrible pile of garbage. |
|
|
384 | |
|
|
385 | =item eio_mlockall (int flags, int pri, eio_cb cb, void *data) |
|
|
386 | |
|
|
387 | Like C<mlockall>, but the flag value constants are called |
|
|
388 | C<EIO_MCL_CURRENT> and C<EIO_MCL_FUTURE>. |
|
|
389 | |
|
|
390 | =item eio_msync (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
|
|
391 | |
|
|
392 | Just like msync, except that the flag values are called C<EIO_MS_ASYNC>, |
|
|
393 | C<EIO_MS_INVALIDATE> and C<EIO_MS_SYNC>. |
|
|
394 | |
|
|
395 | =item eio_readlink (const char *path, int pri, eio_cb cb, void *data) |
|
|
396 | |
|
|
397 | If successful, the path read by C<readlink(2)> can be accessed via C<< |
|
|
398 | req->ptr2 >> and is I<NOT> null-terminated, with the length specified as |
|
|
399 | C<< req->result >>. |
|
|
400 | |
|
|
401 | if (req->result >= 0) |
|
|
402 | { |
|
|
403 | char *target = strndup ((char *)req->ptr2, req->result); |
|
|
404 | |
|
|
405 | free (target); |
|
|
406 | } |
|
|
407 | |
|
|
408 | =item eio_realpath (const char *path, int pri, eio_cb cb, void *data) |
|
|
409 | |
|
|
410 | Similar to the realpath libc function, but unlike that one, C<< |
|
|
411 | req->result >> is C<-1> on failure. On success, the result is the length |
|
|
412 | of the returned path in C<ptr2> (which is I<NOT> 0-terminated) - this is |
|
|
413 | similar to readlink. |
|
|
414 | |
|
|
415 | =item eio_stat (const char *path, int pri, eio_cb cb, void *data) |
|
|
416 | |
|
|
417 | =item eio_lstat (const char *path, int pri, eio_cb cb, void *data) |
|
|
418 | |
|
|
419 | =item eio_fstat (int fd, int pri, eio_cb cb, void *data) |
|
|
420 | |
|
|
421 | Stats a file - if C<< req->result >> indicates success, then you can |
|
|
422 | access the C<struct stat>-like structure via C<< req->ptr2 >>: |
|
|
423 | |
|
|
424 | EIO_STRUCT_STAT *statdata = (EIO_STRUCT_STAT *)req->ptr2; |
|
|
425 | |
|
|
426 | =item eio_statvfs (const char *path, int pri, eio_cb cb, void *data) |
|
|
427 | |
|
|
428 | =item eio_fstatvfs (int fd, int pri, eio_cb cb, void *data) |
|
|
429 | |
|
|
430 | Stats a filesystem - if C<< req->result >> indicates success, then you can |
|
|
431 | access the C<struct statvfs>-like structure via C<< req->ptr2 >>: |
|
|
432 | |
|
|
433 | EIO_STRUCT_STATVFS *statdata = (EIO_STRUCT_STATVFS *)req->ptr2; |
|
|
434 | |
|
|
435 | =back |
|
|
436 | |
|
|
437 | =head3 READING DIRECTORIES |
|
|
438 | |
|
|
439 | Reading directories sounds simple, but can be rather demanding, especially |
|
|
440 | if you want to do stuff such as traversing a directory hierarchy or |
|
|
441 | processing all files in a directory. Libeio can assist these complex tasks |
|
|
442 | with it's C<eio_readdir> call. |
|
|
443 | |
|
|
444 | =over 4 |
|
|
445 | |
|
|
446 | =item eio_readdir (const char *path, int flags, int pri, eio_cb cb, void *data) |
|
|
447 | |
|
|
448 | This is a very complex call. It basically reads through a whole directory |
|
|
449 | (via the C<opendir>, C<readdir> and C<closedir> calls) and returns either |
|
|
450 | the names or an array of C<struct eio_dirent>, depending on the C<flags> |
|
|
451 | argument. |
|
|
452 | |
|
|
453 | The C<< req->result >> indicates either the number of files found, or |
|
|
454 | C<-1> on error. On success, null-terminated names can be found as C<< req->ptr2 >>, |
|
|
455 | and C<struct eio_dirents>, if requested by C<flags>, can be found via C<< |
|
|
456 | req->ptr1 >>. |
|
|
457 | |
|
|
458 | Here is an example that prints all the names: |
|
|
459 | |
|
|
460 | int i; |
|
|
461 | char *names = (char *)req->ptr2; |
|
|
462 | |
|
|
463 | for (i = 0; i < req->result; ++i) |
|
|
464 | { |
|
|
465 | printf ("name #%d: %s\n", i, names); |
|
|
466 | |
|
|
467 | /* move to next name */ |
|
|
468 | names += strlen (names) + 1; |
|
|
469 | } |
|
|
470 | |
|
|
471 | Pseudo-entries such as F<.> and F<..> are never returned by C<eio_readdir>. |
|
|
472 | |
|
|
473 | C<flags> can be any combination of: |
|
|
474 | |
|
|
475 | =over 4 |
|
|
476 | |
|
|
477 | =item EIO_READDIR_DENTS |
|
|
478 | |
|
|
479 | If this flag is specified, then, in addition to the names in C<ptr2>, |
|
|
480 | also an array of C<struct eio_dirent> is returned, in C<ptr1>. A C<struct |
|
|
481 | eio_dirent> looks like this: |
|
|
482 | |
|
|
483 | struct eio_dirent |
|
|
484 | { |
|
|
485 | int nameofs; /* offset of null-terminated name string in (char *)req->ptr2 */ |
|
|
486 | unsigned short namelen; /* size of filename without trailing 0 */ |
|
|
487 | unsigned char type; /* one of EIO_DT_* */ |
|
|
488 | signed char score; /* internal use */ |
|
|
489 | ino_t inode; /* the inode number, if available, otherwise unspecified */ |
|
|
490 | }; |
|
|
491 | |
|
|
492 | The only members you normally would access are C<nameofs>, which is the |
|
|
493 | byte-offset from C<ptr2> to the start of the name, C<namelen> and C<type>. |
|
|
494 | |
|
|
495 | C<type> can be one of: |
|
|
496 | |
|
|
497 | C<EIO_DT_UNKNOWN> - if the type is not known (very common) and you have to C<stat> |
|
|
498 | the name yourself if you need to know, |
|
|
499 | one of the "standard" POSIX file types (C<EIO_DT_REG>, C<EIO_DT_DIR>, C<EIO_DT_LNK>, |
|
|
500 | C<EIO_DT_FIFO>, C<EIO_DT_SOCK>, C<EIO_DT_CHR>, C<EIO_DT_BLK>) |
|
|
501 | or some OS-specific type (currently |
|
|
502 | C<EIO_DT_MPC> - multiplexed char device (v7+coherent), |
|
|
503 | C<EIO_DT_NAM> - xenix special named file, |
|
|
504 | C<EIO_DT_MPB> - multiplexed block device (v7+coherent), |
|
|
505 | C<EIO_DT_NWK> - HP-UX network special, |
|
|
506 | C<EIO_DT_CMP> - VxFS compressed, |
|
|
507 | C<EIO_DT_DOOR> - solaris door, or |
|
|
508 | C<EIO_DT_WHT>). |
|
|
509 | |
|
|
510 | This example prints all names and their type: |
|
|
511 | |
|
|
512 | int i; |
|
|
513 | struct eio_dirent *ents = (struct eio_dirent *)req->ptr1; |
|
|
514 | char *names = (char *)req->ptr2; |
|
|
515 | |
|
|
516 | for (i = 0; i < req->result; ++i) |
|
|
517 | { |
|
|
518 | struct eio_dirent *ent = ents + i; |
|
|
519 | char *name = names + ent->nameofs; |
|
|
520 | |
|
|
521 | printf ("name #%d: %s (type %d)\n", i, name, ent->type); |
|
|
522 | } |
|
|
523 | |
|
|
524 | =item EIO_READDIR_DIRS_FIRST |
|
|
525 | |
|
|
526 | When this flag is specified, then the names will be returned in an order |
|
|
527 | where likely directories come first, in optimal C<stat> order. This is |
|
|
528 | useful when you need to quickly find directories, or you want to find all |
|
|
529 | directories while avoiding to stat() each entry. |
|
|
530 | |
|
|
531 | If the system returns type information in readdir, then this is used |
|
|
532 | to find directories directly. Otherwise, likely directories are names |
|
|
533 | beginning with ".", or otherwise names with no dots, of which names with |
|
|
534 | short names are tried first. |
|
|
535 | |
|
|
536 | =item EIO_READDIR_STAT_ORDER |
|
|
537 | |
|
|
538 | When this flag is specified, then the names will be returned in an order |
|
|
539 | suitable for stat()'ing each one. That is, when you plan to stat() |
|
|
540 | all files in the given directory, then the returned order will likely |
|
|
541 | be fastest. |
|
|
542 | |
|
|
543 | If both this flag and C<EIO_READDIR_DIRS_FIRST> are specified, then the |
|
|
544 | likely directories come first, resulting in a less optimal stat order. |
|
|
545 | |
|
|
546 | =item EIO_READDIR_FOUND_UNKNOWN |
|
|
547 | |
|
|
548 | This flag should not be specified when calling C<eio_readdir>. Instead, |
|
|
549 | it is being set by C<eio_readdir> (you can access the C<flags> via C<< |
|
|
550 | req->int1 >>, when any of the C<type>'s found were C<EIO_DT_UNKNOWN>. The |
|
|
551 | absence of this flag therefore indicates that all C<type>'s are known, |
|
|
552 | which can be used to speed up some algorithms. |
|
|
553 | |
|
|
554 | A typical use case would be to identify all subdirectories within a |
|
|
555 | directory - you would ask C<eio_readdir> for C<EIO_READDIR_DIRS_FIRST>. If |
|
|
556 | then this flag is I<NOT> set, then all the entries at the beginning of the |
|
|
557 | returned array of type C<EIO_DT_DIR> are the directories. Otherwise, you |
|
|
558 | should start C<stat()>'ing the entries starting at the beginning of the |
|
|
559 | array, stopping as soon as you found all directories (the count can be |
|
|
560 | deduced by the link count of the directory). |
|
|
561 | |
|
|
562 | =back |
|
|
563 | |
|
|
564 | =back |
|
|
565 | |
|
|
566 | =head3 OS-SPECIFIC CALL WRAPPERS |
|
|
567 | |
|
|
568 | These wrap OS-specific calls (usually Linux ones), and might or might not |
|
|
569 | be emulated on other operating systems. Calls that are not emulated will |
|
|
570 | return C<-1> and set C<errno> to C<ENOSYS>. |
|
|
571 | |
|
|
572 | =over 4 |
|
|
573 | |
|
|
574 | =item eio_sendfile (int out_fd, int in_fd, off_t in_offset, size_t length, int pri, eio_cb cb, void *data) |
|
|
575 | |
|
|
576 | Wraps the C<sendfile> syscall. The arguments follow the Linux version, but |
|
|
577 | libeio supports and will use similar calls on FreeBSD, HP/UX, Solaris and |
|
|
578 | Darwin. |
|
|
579 | |
|
|
580 | If the OS doesn't support some sendfile-like call, or the call fails, |
|
|
581 | indicating support for the given file descriptor type (for example, |
|
|
582 | Linux's sendfile might not support file to file copies), then libeio will |
|
|
583 | emulate the call in userspace, so there are almost no limitations on its |
|
|
584 | use. |
|
|
585 | |
|
|
586 | =item eio_readahead (int fd, off_t offset, size_t length, int pri, eio_cb cb, void *data) |
|
|
587 | |
|
|
588 | Calls C<readahead(2)>. If the syscall is missing, then the call is |
|
|
589 | emulated by simply reading the data (currently in 64kiB chunks). |
|
|
590 | |
|
|
591 | =item eio_sync_file_range (int fd, off_t offset, size_t nbytes, unsigned int flags, int pri, eio_cb cb, void *data) |
|
|
592 | |
|
|
593 | Calls C<sync_file_range>. If the syscall is missing, then this is the same |
|
|
594 | as calling C<fdatasync>. |
|
|
595 | |
|
|
596 | Flags can be any combination of C<EIO_SYNC_FILE_RANGE_WAIT_BEFORE>, |
|
|
597 | C<EIO_SYNC_FILE_RANGE_WRITE> and C<EIO_SYNC_FILE_RANGE_WAIT_AFTER>. |
|
|
598 | |
|
|
599 | =item eio_fallocate (int fd, int mode, off_t offset, off_t len, int pri, eio_cb cb, void *data) |
|
|
600 | |
|
|
601 | Calls C<fallocate> (note: I<NOT> C<posix_fallocate>!). If the syscall is |
|
|
602 | missing, then it returns failure and sets C<errno> to C<ENOSYS>. |
|
|
603 | |
|
|
604 | The C<mode> argument can be C<0> (for behaviour similar to |
|
|
605 | C<posix_fallocate>), or C<EIO_FALLOC_FL_KEEP_SIZE>, which keeps the size |
|
|
606 | of the file unchanged (but still preallocates space beyond end of file). |
|
|
607 | |
|
|
608 | =back |
|
|
609 | |
|
|
610 | =head3 LIBEIO-SPECIFIC REQUESTS |
|
|
611 | |
|
|
612 | These requests are specific to libeio and do not correspond to any OS call. |
|
|
613 | |
|
|
614 | =over 4 |
|
|
615 | |
|
|
616 | =item eio_mtouch (void *addr, size_t length, int flags, int pri, eio_cb cb, void *data) |
|
|
617 | |
|
|
618 | Reads (C<flags == 0>) or modifies (C<flags == EIO_MT_MODIFY) the given |
|
|
619 | memory area, page-wise, that is, it reads (or reads and writes back) the |
|
|
620 | first octet of every page that spans the memory area. |
|
|
621 | |
|
|
622 | This can be used to page in some mmapped file, or dirty some pages. Note |
|
|
623 | that dirtying is an unlocked read-write access, so races can ensue when |
|
|
624 | the some other thread modifies the data stored in that memory area. |
|
|
625 | |
|
|
626 | =item eio_custom (void (*)(eio_req *) execute, int pri, eio_cb cb, void *data) |
|
|
627 | |
|
|
628 | Executes a custom request, i.e., a user-specified callback. |
|
|
629 | |
|
|
630 | The callback gets the C<eio_req *> as parameter and is expected to read |
|
|
631 | and modify any request-specific members. Specifically, it should set C<< |
|
|
632 | req->result >> to the result value, just like other requests. |
|
|
633 | |
|
|
634 | Here is an example that simply calls C<open>, like C<eio_open>, but it |
|
|
635 | uses the C<data> member as filename and uses a hardcoded C<O_RDONLY>. If |
|
|
636 | you want to pass more/other parameters, you either need to pass some |
|
|
637 | struct or so via C<data> or provide your own wrapper using the low-level |
|
|
638 | API. |
|
|
639 | |
|
|
640 | static int |
|
|
641 | my_open_done (eio_req *req) |
|
|
642 | { |
|
|
643 | int fd = req->result; |
|
|
644 | |
|
|
645 | return 0; |
|
|
646 | } |
|
|
647 | |
|
|
648 | static void |
|
|
649 | my_open (eio_req *req) |
|
|
650 | { |
|
|
651 | req->result = open (req->data, O_RDONLY); |
|
|
652 | } |
|
|
653 | |
|
|
654 | eio_custom (my_open, 0, my_open_done, "/etc/passwd"); |
|
|
655 | |
|
|
656 | =item eio_busy (eio_tstamp delay, int pri, eio_cb cb, void *data) |
|
|
657 | |
|
|
658 | This is a request that takes C<delay> seconds to execute, but otherwise |
|
|
659 | does nothing - it simply puts one of the worker threads to sleep for this |
|
|
660 | long. |
|
|
661 | |
|
|
662 | This request can be used to artificially increase load, e.g. for debugging |
|
|
663 | or benchmarking reasons. |
|
|
664 | |
|
|
665 | =item eio_nop (int pri, eio_cb cb, void *data) |
|
|
666 | |
|
|
667 | This request does nothing, except go through the whole request cycle. This |
|
|
668 | can be used to measure latency or in some cases to simplify code, but is |
|
|
669 | not really of much use. |
|
|
670 | |
|
|
671 | =back |
|
|
672 | |
|
|
673 | =head3 GROUPING AND LIMITING REQUESTS |
|
|
674 | |
|
|
675 | There is one more rather special request, C<eio_grp>. It is a very special |
|
|
676 | aio request: Instead of doing something, it is a container for other eio |
|
|
677 | requests. |
|
|
678 | |
|
|
679 | There are two primary use cases for this: a) bundle many requests into a |
|
|
680 | single, composite, request with a definite callback and the ability to |
|
|
681 | cancel the whole request with its subrequests and b) limiting the number |
|
|
682 | of "active" requests. |
|
|
683 | |
|
|
684 | Further below you will find more discussion of these topics - first |
|
|
685 | follows the reference section detailing the request generator and other |
|
|
686 | methods. |
|
|
687 | |
|
|
688 | =over 4 |
|
|
689 | |
|
|
690 | =item eio_req *grp = eio_grp (eio_cb cb, void *data) |
|
|
691 | |
|
|
692 | Creates, submits and returns a group request. Note that it doesn't have a |
|
|
693 | priority, unlike all other requests. |
|
|
694 | |
|
|
695 | =item eio_grp_add (eio_req *grp, eio_req *req) |
|
|
696 | |
|
|
697 | Adds a request to the request group. |
|
|
698 | |
|
|
699 | =item eio_grp_cancel (eio_req *grp) |
|
|
700 | |
|
|
701 | Cancels all requests I<in> the group, but I<not> the group request |
|
|
702 | itself. You can cancel the group request I<and> all subrequests via a |
|
|
703 | normal C<eio_cancel> call. |
|
|
704 | |
|
|
705 | =back |
|
|
706 | |
|
|
707 | =head4 GROUP REQUEST LIFETIME |
|
|
708 | |
|
|
709 | Left alone, a group request will instantly move to the pending state and |
|
|
710 | will be finished at the next call of C<eio_poll>. |
|
|
711 | |
|
|
712 | The usefulness stems from the fact that, if a subrequest is added to a |
|
|
713 | group I<before> a call to C<eio_poll>, via C<eio_grp_add>, then the group |
|
|
714 | will not finish until all the subrequests have finished. |
|
|
715 | |
|
|
716 | So the usage cycle of a group request is like this: after it is created, |
|
|
717 | you normally instantly add a subrequest. If none is added, the group |
|
|
718 | request will finish on it's own. As long as subrequests are added before |
|
|
719 | the group request is finished it will be kept from finishing, that is the |
|
|
720 | callbacks of any subrequests can, in turn, add more requests to the group, |
|
|
721 | and as long as any requests are active, the group request itself will not |
|
|
722 | finish. |
|
|
723 | |
|
|
724 | =head4 CREATING COMPOSITE REQUESTS |
|
|
725 | |
|
|
726 | Imagine you wanted to create an C<eio_load> request that opens a file, |
|
|
727 | reads it and closes it. This means it has to execute at least three eio |
|
|
728 | requests, but for various reasons it might be nice if that request looked |
|
|
729 | like any other eio request. |
|
|
730 | |
|
|
731 | This can be done with groups: |
|
|
732 | |
|
|
733 | =over 4 |
|
|
734 | |
|
|
735 | =item 1) create the request object |
|
|
736 | |
|
|
737 | Create a group that contains all further requests. This is the request you |
|
|
738 | can return as "the load request". |
|
|
739 | |
|
|
740 | =item 2) open the file, maybe |
|
|
741 | |
|
|
742 | Next, open the file with C<eio_open> and add the request to the group |
|
|
743 | request and you are finished setting up the request. |
|
|
744 | |
|
|
745 | If, for some reason, you cannot C<eio_open> (path is a null ptr?) you |
|
|
746 | can set C<< grp->result >> to C<-1> to signal an error and let the group |
|
|
747 | request finish on its own. |
|
|
748 | |
|
|
749 | =item 3) open callback adds more requests |
|
|
750 | |
|
|
751 | In the open callback, if the open was not successful, copy C<< |
|
|
752 | req->errorno >> to C<< grp->errorno >> and set C<< grp->errorno >> to |
|
|
753 | C<-1> to signal an error. |
|
|
754 | |
|
|
755 | Otherwise, malloc some memory or so and issue a read request, adding the |
|
|
756 | read request to the group. |
|
|
757 | |
|
|
758 | =item 4) continue issuing requests till finished |
|
|
759 | |
|
|
760 | In the real callback, check for errors and possibly continue with |
|
|
761 | C<eio_close> or any other eio request in the same way. |
|
|
762 | |
|
|
763 | As soon as no new requests are added the group request will finish. Make |
|
|
764 | sure you I<always> set C<< grp->result >> to some sensible value. |
|
|
765 | |
|
|
766 | =back |
|
|
767 | |
|
|
768 | =head4 REQUEST LIMITING |
|
|
769 | |
|
|
770 | |
|
|
771 | #TODO |
|
|
772 | |
|
|
773 | void eio_grp_limit (eio_req *grp, int limit); |
|
|
774 | |
|
|
775 | |
|
|
776 | =back |
|
|
777 | |
|
|
778 | |
|
|
779 | =head1 LOW LEVEL REQUEST API |
|
|
780 | |
|
|
781 | #TODO |
|
|
782 | |
|
|
783 | |
|
|
784 | =head1 ANATOMY AND LIFETIME OF AN EIO REQUEST |
|
|
785 | |
|
|
786 | A request is represented by a structure of type C<eio_req>. To initialise |
|
|
787 | it, clear it to all zero bytes: |
|
|
788 | |
|
|
789 | eio_req req; |
|
|
790 | |
|
|
791 | memset (&req, 0, sizeof (req)); |
|
|
792 | |
|
|
793 | A more common way to initialise a new C<eio_req> is to use C<calloc>: |
|
|
794 | |
|
|
795 | eio_req *req = calloc (1, sizeof (*req)); |
|
|
796 | |
|
|
797 | In either case, libeio neither allocates, initialises or frees the |
|
|
798 | C<eio_req> structure for you - it merely uses it. |
|
|
799 | |
|
|
800 | zero |
|
|
801 | |
|
|
802 | #TODO |
137 | |
803 | |
138 | =head2 CONFIGURATION |
804 | =head2 CONFIGURATION |
139 | |
805 | |
140 | The functions in this section can sometimes be useful, but the default |
806 | The functions in this section can sometimes be useful, but the default |
141 | configuration will do in most case, so you should skip this section on |
807 | configuration will do in most case, so you should skip this section on |
… | |
… | |
152 | for example, in interactive programs, you might want to limit this time to |
818 | for example, in interactive programs, you might want to limit this time to |
153 | C<0.01> seconds or so. |
819 | C<0.01> seconds or so. |
154 | |
820 | |
155 | Note that: |
821 | Note that: |
156 | |
822 | |
|
|
823 | =over 4 |
|
|
824 | |
157 | a) libeio doesn't know how long your request callbacks take, so the time |
825 | =item a) libeio doesn't know how long your request callbacks take, so the |
158 | spent in C<eio_poll> is up to one callback invocation longer then this |
826 | time spent in C<eio_poll> is up to one callback invocation longer then |
159 | interval. |
827 | this interval. |
160 | |
828 | |
161 | b) this is implemented by calling C<gettimeofday> after each request, |
829 | =item b) this is implemented by calling C<gettimeofday> after each |
162 | which can be costly. |
830 | request, which can be costly. |
163 | |
831 | |
164 | c) at least one request will be handled. |
832 | =item c) at least one request will be handled. |
|
|
833 | |
|
|
834 | =back |
165 | |
835 | |
166 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
836 | =item eio_set_max_poll_reqs (unsigned int nreqs) |
167 | |
837 | |
168 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
838 | When C<nreqs> is non-zero, then C<eio_poll> will not handle more than |
169 | C<nreqs> requests per invocation. This is a less costly way to limit the |
839 | C<nreqs> requests per invocation. This is a less costly way to limit the |
… | |
… | |
185 | =item eio_set_max_idle (unsigned int nthreads) |
855 | =item eio_set_max_idle (unsigned int nthreads) |
186 | |
856 | |
187 | Libeio uses threads internally to handle most requests, and will start and stop threads on demand. |
857 | Libeio uses threads internally to handle most requests, and will start and stop threads on demand. |
188 | |
858 | |
189 | This call can be used to limit the number of idle threads (threads without |
859 | This call can be used to limit the number of idle threads (threads without |
190 | work to do): libeio will keep some threads idle in preperation for more |
860 | work to do): libeio will keep some threads idle in preparation for more |
191 | requests, but never longer than C<nthreads> threads. |
861 | requests, but never longer than C<nthreads> threads. |
192 | |
862 | |
193 | In addition to this, libeio will also stop threads when they are idle for |
863 | In addition to this, libeio will also stop threads when they are idle for |
194 | a few seconds, regardless of this setting. |
864 | a few seconds, regardless of this setting. |
195 | |
865 | |
… | |
… | |
214 | executed and have results, but have not been finished yet by a call to |
884 | executed and have results, but have not been finished yet by a call to |
215 | C<eio_poll>). |
885 | C<eio_poll>). |
216 | |
886 | |
217 | =back |
887 | =back |
218 | |
888 | |
219 | |
|
|
220 | =head1 ANATOMY OF AN EIO REQUEST |
|
|
221 | |
|
|
222 | #TODO |
|
|
223 | |
|
|
224 | |
|
|
225 | =head1 HIGH LEVEL REQUEST API |
|
|
226 | |
|
|
227 | #TODO |
|
|
228 | |
|
|
229 | =back |
|
|
230 | |
|
|
231 | |
|
|
232 | =head1 LOW LEVEL REQUEST API |
|
|
233 | |
|
|
234 | #TODO |
|
|
235 | |
|
|
236 | =head1 EMBEDDING |
889 | =head1 EMBEDDING |
237 | |
890 | |
238 | Libeio can be embedded directly into programs. This functionality is not |
891 | Libeio can be embedded directly into programs. This functionality is not |
239 | documented and not (yet) officially supported. |
892 | documented and not (yet) officially supported. |
240 | |
893 | |
241 | Note that, when including C<libeio.m4>, you are responsible for defining |
894 | Note that, when including C<libeio.m4>, you are responsible for defining |
242 | the compilation environment (C<_LARGEFILE_SOURCE>, C<_GNU_SOURCE> etc.). |
895 | the compilation environment (C<_LARGEFILE_SOURCE>, C<_GNU_SOURCE> etc.). |
243 | |
896 | |
244 | If you need to know how, check the C<IO::AIO> perl module, which does |
897 | If you need to know how, check the C<IO::AIO> perl module, which does |
245 | exactly that. |
898 | exactly that. |
|
|
899 | |
|
|
900 | |
|
|
901 | =head1 COMPILETIME CONFIGURATION |
|
|
902 | |
|
|
903 | These symbols, if used, must be defined when compiling F<eio.c>. |
|
|
904 | |
|
|
905 | =over 4 |
|
|
906 | |
|
|
907 | =item EIO_STACKSIZE |
|
|
908 | |
|
|
909 | This symbol governs the stack size for each eio thread. Libeio itself |
|
|
910 | was written to use very little stackspace, but when using C<EIO_CUSTOM> |
|
|
911 | requests, you might want to increase this. |
|
|
912 | |
|
|
913 | If this symbol is undefined (the default) then libeio will use its default |
|
|
914 | stack size (C<sizeof (void *) * 4096> currently). If it is defined, but |
|
|
915 | C<0>, then the default operating system stack size will be used. In all |
|
|
916 | other cases, the value must be an expression that evaluates to the desired |
|
|
917 | stack size. |
|
|
918 | |
|
|
919 | =back |
246 | |
920 | |
247 | |
921 | |
248 | =head1 PORTABILITY REQUIREMENTS |
922 | =head1 PORTABILITY REQUIREMENTS |
249 | |
923 | |
250 | In addition to a working ISO-C implementation, libeio relies on a few |
924 | In addition to a working ISO-C implementation, libeio relies on a few |