--- libev/ev.pod 2009/12/31 06:59:47 1.278 +++ libev/ev.pod 2010/10/14 04:29:34 1.303 @@ -77,7 +77,7 @@ on event-based programming, nor will it introduce event-based programming with libev. -Familarity with event based programming techniques in general is assumed +Familiarity with event based programming techniques in general is assumed throughout this document. =head1 ABOUT LIBEV @@ -126,13 +126,14 @@ =head2 TIME REPRESENTATION Libev represents time as a single floating point number, representing -the (fractional) number of seconds since the (POSIX) epoch (somewhere -near the beginning of 1970, details are complicated, don't ask). This -type is called C, which is what you should use too. It usually -aliases to the C type in C. When you need to do any calculations -on it, you should treat it as some floating point value. Unlike the name -component C might indicate, it is also used for time differences -throughout libev. +the (fractional) number of seconds since the (POSIX) epoch (in practise +somewhere near the beginning of 1970, details are complicated, don't +ask). This type is called C, which is what you should use +too. It usually aliases to the C type in C. When you need to do +any calculations on it, you should treat it as some floating point value. + +Unlike the name component C might indicate, it is also used for +time differences (e.g. delays) throughout libev. =head1 ERROR HANDLING @@ -193,7 +194,7 @@ not a problem. Example: Make sure we haven't accidentally been linked against the wrong -version. +version (note, however, that this will not detect ABI mismatches :). assert (("libev version mismatch", ev_version_major () == EV_VERSION_MAJOR @@ -347,9 +348,8 @@ =item C -Instead of calling C or C manually after -a fork, you can also make libev check for a fork in each iteration by -enabling this flag. +Instead of calling C manually after a fork, you can also +make libev check for a fork in each iteration by enabling this flag. This works by calling C on every iteration of the loop, and thus this might slow down your event loop if you do a lot of loop @@ -569,11 +569,9 @@ =item struct ev_loop *ev_loop_new (unsigned int flags) Similar to C, but always creates a new event loop that is -always distinct from the default loop. Unlike the default loop, it cannot -handle signal and child watchers, and attempts to do so will be greeted by -undefined behaviour (or a failed assertion if assertions are enabled). +always distinct from the default loop. -Note that this function I thread-safe, and the recommended way to use +Note that this function I thread-safe, and one common way to use libev with threads is indeed to create one loop per thread, and using the default loop in the "main" or "initial" thread. @@ -585,13 +583,12 @@ =item ev_default_destroy () -Destroys the default loop again (frees all memory and kernel state -etc.). None of the active event watchers will be stopped in the normal -sense, so e.g. C might still return true. It is your -responsibility to either stop all watchers cleanly yourself I -calling this function, or cope with the fact afterwards (which is usually -the easiest thing, you can just ignore the watchers and/or C them -for example). +Destroys the default loop (frees all memory and kernel state etc.). None +of the active event watchers will be stopped in the normal sense, so +e.g. C might still return true. It is your responsibility to +either stop all watchers cleanly yourself I calling this function, +or cope with the fact afterwards (which is usually the easiest thing, you +can just ignore the watchers and/or C them for example). Note that certain global state, such as signal state (and installed signal handlers), will not be freed by this function, and related watchers (such @@ -616,9 +613,15 @@ sense). You I call it in the child before using any of the libev functions, and it will only take effect at the next C iteration. +Again, you I to call it on I loop that you want to re-use after +a fork, I. This is +because some kernel interfaces *cough* I *cough* do funny things +during fork. + On the other hand, you only need to call this function in the child -process if and only if you want to use the event library in the child. If -you just fork+exec, you don't have to call it at all. +process if and only if you want to use the event loop in the child. If you +just fork+exec or create a new loop in the child, you don't have to call +it at all. The function itself is quite fast and it's usually not a problem to call it just in case after a fork. To make this easy, the function will fit in @@ -630,25 +633,26 @@ Like C, but acts on an event loop created by C. Yes, you have to call this on every allocated event loop -after fork that you want to re-use in the child, and how you do this is -entirely your own problem. +after fork that you want to re-use in the child, and how you keep track of +them is entirely your own problem. =item int ev_is_default_loop (loop) Returns true when the given loop is, in fact, the default loop, and false otherwise. -=item unsigned int ev_loop_count (loop) +=item unsigned int ev_iteration (loop) -Returns the count of loop iterations for the loop, which is identical to +Returns the current iteration count for the loop, which is identical to the number of times libev did poll for new events. It starts at C<0> and happily wraps around with enough iterations. This value can sometimes be useful as a generation counter of sorts (it "ticks" the number of loop iterations), as it roughly corresponds with -C and C calls. +C and C calls - and is incremented between the +prepare and check phases. -=item unsigned int ev_loop_depth (loop) +=item unsigned int ev_depth (loop) Returns the number of times C was entered minus the number of times C was exited, in other words, the recursion depth. @@ -658,7 +662,8 @@ in which case it is higher. Leaving C abnormally (setjmp/longjmp, cancelling the thread -etc.), doesn't count as exit. +etc.), doesn't count as "exit" - consider this as a hint to avoid such +ungentleman behaviour unless it's really convenient. =item unsigned int ev_backend (loop) @@ -702,7 +707,7 @@ Effectively, all C watchers will be delayed by the time spend between C and C, and all C watchers will be rescheduled (that is, they will lose any events that would have -occured while suspended). +occurred while suspended). After calling C you B call I function on the given loop other than C, and you B call C @@ -789,7 +794,7 @@ This "unloop state" will be cleared when entering C again. -It is safe to call C from otuside any C calls. +It is safe to call C from outside any C calls. =item ev_ref (loop) @@ -869,7 +874,7 @@ you do transactions with the outside world and you can't increase the parallelity, then this setting will limit your transaction rate (if you need to poll once per transaction and the I/O collect interval is 0.01, -then you can't do more than 100 transations per second). +then you can't do more than 100 transactions per second). Setting the I can improve the opportunity for saving power, as the program will "bundle" timer callback invocations that @@ -1034,7 +1039,7 @@ The file descriptor in the C watcher has become readable and/or writable. -=item C +=item C The C watcher has timed out. @@ -1377,7 +1382,7 @@ you can associate an C watcher to each such watcher, and in the normal watcher callback, you just start the idle watcher. The real processing is done in the idle watcher callback. This causes libev to -continously poll and process kernel event data for the watcher, but when +continuously poll and process kernel event data for the watcher, but when the lock-out case is known to be rare (which in turn is rare :), this is workable. @@ -1401,7 +1406,7 @@ // are not yet ready to handle it. ev_io_stop (EV_A_ w); - // start the idle watcher to ahndle the actual event. + // start the idle watcher to handle the actual event. // it will not be executed as long as other watchers // with the default priority are receiving events. ev_idle_start (EV_A_ &idle); @@ -1465,7 +1470,7 @@ known-to-be-good backend (at the time of this writing, this includes only C and C). The same applies to file descriptors for which non-blocking operation makes no sense (such as -files) - libev doesn't guarentee any specific behaviour in that case. +files) - libev doesn't guarantee any specific behaviour in that case. Another thing you have to watch out for is that it is quite easy to receive "spurious" readiness notifications, that is your callback might @@ -1540,6 +1545,44 @@ ignore SIGPIPE (and maybe make sure you log the exit status of your daemon somewhere, as that would have given you a big clue). +=head3 The special problem of accept()ing when you can't + +Many implementations of the POSIX C function (for example, +found in post-2004 Linux) have the peculiar behaviour of not removing a +connection from the pending queue in all error cases. + +For example, larger servers often run out of file descriptors (because +of resource limits), causing C to fail with C but not +rejecting the connection, leading to libev signalling readiness on +the next iteration again (the connection still exists after all), and +typically causing the program to loop at 100% CPU usage. + +Unfortunately, the set of errors that cause this issue differs between +operating systems, there is usually little the app can do to remedy the +situation, and no known thread-safe method of removing the connection to +cope with overload is known (to me). + +One of the easiest ways to handle this situation is to just ignore it +- when the program encounters an overload, it will just loop until the +situation is over. While this is a form of busy waiting, no OS offers an +event-based way to handle this situation, so it's the best one can do. + +A better way to handle the situation is to log any errors other than +C and C, making sure not to flood the log with such +messages, and continue as usual, which at least gives the user an idea of +what could be wrong ("raise the ulimit!"). For extra points one could stop +the C watcher on the listening fd "for a while", which reduces CPU +usage. + +If your program is single-threaded, then you could also keep a dummy file +descriptor for overload situations (e.g. by opening F), and +when you run into C or C, close it, run C, +close that fd, and create a new dummy fd. This will gracefully refuse +clients under typical overload conditions. + +The last way to handle it is to simply log the error and C, as +is often done with C failures, but this results in an easy +opportunity for a DoS attack. =head3 Watcher-Specific Functions @@ -1696,7 +1739,7 @@ // if last_activity + 60. is older than now, we did time out if (timeout < now) { - // timeout occured, take action + // timeout occurred, take action } else { @@ -1728,12 +1771,12 @@ ev_init (timer, callback); last_activity = ev_now (loop); - callback (loop, timer, EV_TIMEOUT); + callback (loop, timer, EV_TIMER); And when there is some activity, simply store the current time in C, no libev calls at all: - last_actiivty = ev_now (loop); + last_activity = ev_now (loop); This technique is slightly more complex, but in most cases where the time-out is unlikely to be triggered, much more efficient. @@ -1869,7 +1912,7 @@ the timeout value currently configured. That is, after an C, C returns -C<5>. When the timer is started and one second passes, C +C<5>. When the timer is started and one second passes, C will return C<4>. When the timer expires and is restarted, it will return roughly C<7> (likely slightly less as callback invocation takes some time, too), and so on. @@ -2082,7 +2125,7 @@ potentially a lot of jitter, but good long-term stability. static void - clock_cb (struct ev_loop *loop, ev_io *w, int revents) + clock_cb (struct ev_loop *loop, ev_periodic *w, int revents) { ... its now a full hour (UTC, or TAI or whatever your clock follows) } @@ -2924,7 +2967,7 @@ =head3 The special problem of life after fork - how is it possible? -Most uses of C consist of forking, then some simple calls to ste +Most uses of C consist of forking, then some simple calls to set up/change the process environment, followed by a call to C. This sequence should be handled by libev without any problems. @@ -2968,17 +3011,16 @@ =back -=head2 C - how to wake up another event loop +=head2 C - how to wake up an event loop In general, you cannot use an C from multiple threads or other asynchronous sources such as signal handlers (as opposed to multiple event loops - those are of course safe to use in different threads). -Sometimes, however, you need to wake up another event loop you do not -control, for example because it belongs to another thread. This is what -C watchers do: as long as the C watcher is active, you -can signal it by calling C, which is thread- and signal -safe. +Sometimes, however, you need to wake up an event loop you do not control, +for example because it belongs to another thread. This is what C +watchers do: as long as the C watcher is active, you can signal +it by calling C, which is thread- and signal safe. This functionality is very similar to C watchers, as signals, too, are asynchronous in nature, and signals, too, will be compressed @@ -3143,9 +3185,9 @@ started. Otherwise an C watcher with after = C (and repeat = 0) will be started. C<0> is a valid timeout. -The callback has the type C and gets +The callback has the type C and is passed an C set like normal event callbacks (a combination of -C, C, C or C) and the C +C, C, C or C) and the C value passed to C. Note that it is possible to receive I a timeout and an io event at the same time - you probably should give io events precedence. @@ -3156,7 +3198,7 @@ { if (revents & EV_READ) /* stdin might have data for us, joy! */; - else if (revents & EV_TIMEOUT) + else if (revents & EV_TIMER) /* doh, nothing entered */; } @@ -3301,8 +3343,6 @@ =item w->set (object *) -This is an B feature that might go away in a future version. - This is a variation of a method callback - leaving out the method to call will default the method to C, which makes it possible to use functor objects without having to manually specify the C all @@ -3453,8 +3493,8 @@ =item Lua -Brian Maher has written a partial interface to libev -for lua (only C and C), to be found at +Brian Maher has written a partial interface to libev for lua (at the +time of this writing, only C and C), to be found at L. =back @@ -3619,12 +3659,19 @@ =head2 PREPROCESSOR SYMBOLS/MACROS Libev can be configured via a variety of preprocessor symbols you have to -define before including any of its files. The default in the absence of -autoconf is documented for every option. +define before including (or compiling) any of its files. The default in +the absence of autoconf is documented for every option. + +Symbols marked with "(h)" do not change the ABI, and can have different +values when compiling libev vs. including F, so it is permissible +to redefine them before including F without breaking compatibility +to a compiled library. All other symbols change the ABI, which means all +users of libev and the libev code itself must be compiled with compatible +settings. =over 4 -=item EV_STANDALONE +=item EV_STANDALONE (h) Must always be C<1> if you do not use autoconf configuration, which keeps libev from including F, and it also defines dummy @@ -3784,24 +3831,24 @@ In the absence of this define, libev will use C (from F), which is usually good enough on most platforms. -=item EV_H +=item EV_H (h) The name of the F header file used to include it. The default if undefined is C<"ev.h"> in F, F and F. This can be used to virtually rename the F header file in case of conflicts. -=item EV_CONFIG_H +=item EV_CONFIG_H (h) If C isn't C<1>, this variable can be used to override F's idea of where to find the F file, similarly to C, above. -=item EV_EVENT_H +=item EV_EVENT_H (h) Similarly to C, this macro can be used to override F's idea of how the F header can be found, the default is C<"event.h">. -=item EV_PROTOTYPES +=item EV_PROTOTYPES (h) If defined to be C<0>, then F will not define any function prototypes, but still define all the structs and other symbols. This is @@ -3833,55 +3880,106 @@ If your embedding application does not need any priorities, defining these both to C<0> will save some memory and CPU. -=item EV_PERIODIC_ENABLE +=item EV_PERIODIC_ENABLE, EV_IDLE_ENABLE, EV_EMBED_ENABLE, EV_STAT_ENABLE, +EV_PREPARE_ENABLE, EV_CHECK_ENABLE, EV_FORK_ENABLE, EV_SIGNAL_ENABLE, +EV_ASYNC_ENABLE, EV_CHILD_ENABLE. + +If undefined or defined to be C<1> (and the platform supports it), then +the respective watcher type is supported. If defined to be C<0>, then it +is not. Disabling watcher types mainly saves code size. -If undefined or defined to be C<1>, then periodic timers are supported. If -defined to be C<0>, then they are not. Disabling them saves a few kB of -code. +=item EV_FEATURES -=item EV_IDLE_ENABLE +If you need to shave off some kilobytes of code at the expense of some +speed (but with the full API), you can define this symbol to request +certain subsets of functionality. The default is to enable all features +that can be enabled on the platform. + +A typical way to use this symbol is to define it to C<0> (or to a bitset +with some broad features you want) and then selectively re-enable +additional parts you want, for example if you want everything minimal, +but multiple event loop support, async and child watchers and the poll +backend, use this: + + #define EV_FEATURES 0 + #define EV_MULTIPLICITY 1 + #define EV_USE_POLL 1 + #define EV_CHILD_ENABLE 1 + #define EV_ASYNC_ENABLE 1 -If undefined or defined to be C<1>, then idle watchers are supported. If -defined to be C<0>, then they are not. Disabling them saves a few kB of -code. +The actual value is a bitset, it can be a combination of the following +values: -=item EV_EMBED_ENABLE +=over 4 -If undefined or defined to be C<1>, then embed watchers are supported. If -defined to be C<0>, then they are not. Embed watchers rely on most other -watcher types, which therefore must not be disabled. +=item C<1> - faster/larger code -=item EV_STAT_ENABLE +Use larger code to speed up some operations. -If undefined or defined to be C<1>, then stat watchers are supported. If -defined to be C<0>, then they are not. +Currently this is used to override some inlining decisions (enlarging the +code size by roughly 30% on amd64). -=item EV_FORK_ENABLE +When optimising for size, use of compiler flags such as C<-Os> with +gcc is recommended, as well as C<-DNDEBUG>, as libev contains a number of +assertions. -If undefined or defined to be C<1>, then fork watchers are supported. If -defined to be C<0>, then they are not. +=item C<2> - faster/larger data structures -=item EV_ASYNC_ENABLE +Replaces the small 2-heap for timer management by a faster 4-heap, larger +hash table sizes and so on. This will usually further increase code size +and can additionally have an effect on the size of data structures at +runtime. -If undefined or defined to be C<1>, then async watchers are supported. If -defined to be C<0>, then they are not. +=item C<4> - full API configuration -=item EV_MINIMAL +This enables priorities (sets C=2 and C=-2), and +enables multiplicity (C=1). -If you need to shave off some kilobytes of code at the expense of some -speed (but with the full API), define this symbol to C<1>. Currently this -is used to override some inlining decisions, saves roughly 30% code size -on amd64. It also selects a much smaller 2-heap for timer management over -the default 4-heap. - -You can save even more by disabling watcher types you do not need -and setting C == C. Also, disabling C -(C<-DNDEBUG>) will usually reduce code size a lot. - -Defining C to C<2> will additionally reduce the core API to -provide a bare-bones event library. See C for details on what parts -of the API are still available, and do not complain if this subset changes -over time. +=item C<8> - full API + +This enables a lot of the "lesser used" API functions. See C for +details on which parts of the API are still available without this +feature, and do not complain if this subset changes over time. + +=item C<16> - enable all optional watcher types + +Enables all optional watcher types. If you want to selectively enable +only some watcher types other than I/O and timers (e.g. prepare, +embed, async, child...) you can enable them manually by defining +C to C<1> instead. + +=item C<32> - enable all backends + +This enables all backends - without this feature, you need to enable at +least one backend manually (C is a good choice). + +=item C<64> - enable OS-specific "helper" APIs + +Enable inotify, eventfd, signalfd and similar OS-specific helper APIs by +default. + +=back + +Compiling with C +reduces the compiled size of libev from 24.7Kb code/2.8Kb data to 6.5Kb +code/0.3Kb data on my GNU/Linux amd64 system, while still giving you I/O +watchers, timers and monotonic clock support. + +With an intelligent-enough linker (gcc+binutils are intelligent enough +when you use C<-Wl,--gc-sections -ffunction-sections>) functions unused by +your program might be left out as well - a binary starting a timer and an +I/O watcher then might come out at only 5Kb. + +=item EV_AVOID_STDIO + +If this is set to C<1> at compiletime, then libev will avoid using stdio +functions (printf, scanf, perror etc.). This will increase the code size +somewhat, but if your program doesn't otherwise depend on stdio and your +libc allows it, this avoids linking in the stdio library which is quite +big. + +Note that error messages might become less precise when this option is +enabled. =item EV_NSIG @@ -3889,23 +3987,23 @@ signals): Normally, libev tries to deduce the maximum number of signals automatically, but sometimes this fails, in which case it can be specified. Also, using a lower number than detected (C<32> should be -good for about any system in existance) can save some memory, as libev +good for about any system in existence) can save some memory, as libev statically allocates some 12-24 bytes per signal number. =item EV_PID_HASHSIZE C watchers use a small hash table to distribute workload by -pid. The default size is C<16> (or C<1> with C), usually more -than enough. If you need to manage thousands of children you might want to -increase this value (I be a power of two). +pid. The default size is C<16> (or C<1> with C disabled), +usually more than enough. If you need to manage thousands of children you +might want to increase this value (I be a power of two). =item EV_INOTIFY_HASHSIZE C watchers use a small hash table to distribute workload by -inotify watch id. The default size is C<16> (or C<1> with C), -usually more than enough. If you need to manage thousands of C -watchers you might want to increase this value (I be a power of -two). +inotify watch id. The default size is C<16> (or C<1> with C +disabled), usually more than enough. If you need to manage thousands of +C watchers you might want to increase this value (I be a +power of two). =item EV_USE_4HEAP @@ -3914,8 +4012,8 @@ to C<1>. The 4-heap uses more complicated (longer) code but has noticeably faster performance with many (thousands) of watchers. -The default is C<1> unless C is set in which case it is C<0> -(disabled). +The default is C<1>, unless C overrides it, in which case it +will be C<0>. =item EV_HEAP_CACHE_AT @@ -3926,8 +4024,8 @@ but avoids random read accesses on heap changes. This improves performance noticeably with many (hundreds) of watchers. -The default is C<1> unless C is set in which case it is C<0> -(disabled). +The default is C<1>, unless C overrides it, in which case it +will be C<0>. =item EV_VERIFY @@ -3939,13 +4037,13 @@ verification code will be called very frequently, which will slow down libev considerably. -The default is C<1>, unless C is set, in which case it will be -C<0>. +The default is C<1>, unless C overrides it, in which case it +will be C<0>. =item EV_COMMON By default, all watchers have a C member. By redefining -this macro to a something else you can include more and other types of +this macro to something else you can include more and other types of members. You have to define it each time you include one of the files, though, and it must be identical each time. @@ -4008,15 +4106,14 @@ The usage in rxvt-unicode is simpler. It has a F header file that everybody includes and which overrides some configure choices: - #define EV_MINIMAL 1 - #define EV_USE_POLL 0 - #define EV_MULTIPLICITY 0 - #define EV_PERIODIC_ENABLE 0 - #define EV_STAT_ENABLE 0 - #define EV_FORK_ENABLE 0 + #define EV_FEATURES 8 + #define EV_USE_SELECT 1 + #define EV_PREPARE_ENABLE 1 + #define EV_IDLE_ENABLE 1 + #define EV_SIGNAL_ENABLE 1 + #define EV_CHILD_ENABLE 1 + #define EV_USE_STDEXCEPT 0 #define EV_CONFIG_H - #define EV_MINPRI 0 - #define EV_MAXPRI 0 #include "ev++.h" @@ -4255,7 +4352,7 @@ And of course, some compiler warnings are just plain stupid, or simply wrong (because they don't actually warn about the condition their message seems to warn about). For example, certain older gcc versions had some -warnings that resulted an extreme number of false positives. These have +warnings that resulted in an extreme number of false positives. These have been fixed, but some people still insist on making code warn-free with such buggy versions. @@ -4301,19 +4398,101 @@ =head1 PORTABILITY NOTES +=head2 GNU/LINUX 32 BIT LIMITATIONS + +GNU/Linux is the only common platform that supports 64 bit file/large file +interfaces but I them by default. + +That means that libev compiled in the default environment doesn't support +files larger than 2GiB or so, which mainly affects C watchers. + +Unfortunately, many programs try to work around this GNU/Linux issue +by enabling the large file API, which makes them incompatible with the +standard libev compiled for their system. + +Likewise, libev cannot enable the large file API itself as this would +suddenly make it incompatible to the default compile time environment, +i.e. all programs not using special compile switches. + +=head2 OS/X AND DARWIN BUGS + +The whole thing is a bug if you ask me - basically any system interface +you touch is broken, whether it is locales, poll, kqueue or even the +OpenGL drivers. + +=head3 C is buggy + +The kqueue syscall is broken in all known versions - most versions support +only sockets, many support pipes. + +=head3 C is buggy + +Instead of fixing C, Apple replaced their (working) C +implementation by something calling C internally around the 10.5.6 +release, so now C I C are broken. + +Libev tries to work around this by neither using C nor C by +default on this rotten platform, but of course you cna still ask for them +when creating a loop. + +=head3 C, and of course Apple found a way to fuck this +one up as well: On OS/X, C backends. + +=head2 AIX POLL BUG + +AIX unfortunately has a broken C header. Libev works around +this by trying to avoid the poll backend altogether (i.e. it's not even +compiled in), which normally isn't a big problem as C function The winsocket C