… | |
… | |
864 | |
864 | |
865 | |
865 | |
866 | =head1 BENCHMARK |
866 | =head1 BENCHMARK |
867 | |
867 | |
868 | To give you an idea of the performance and overheads that AnyEvent adds |
868 | To give you an idea of the performance and overheads that AnyEvent adds |
869 | over the backends directly, here is a benchmark of various supported event |
869 | over the event loops themselves (and to give you an impression of the |
|
|
870 | speed of various event loops), here is a benchmark of various supported |
870 | models natively and with anyevent. The benchmark creates a lot of timers |
871 | event models natively and with anyevent. The benchmark creates a lot of |
871 | (with a zero timeout) and io watchers (watching STDOUT, a pty, to become |
872 | timers (with a zero timeout) and io watchers (watching STDOUT, a pty, to |
872 | writable, which it is), lets them fire exactly once and destroys them |
873 | become writable, which it is), lets them fire exactly once and destroys |
873 | again. |
874 | them again. |
874 | |
875 | |
875 | Explanation of the fields: |
876 | =head2 Explanation of the columns |
876 | |
877 | |
877 | I<watcher> is the number of event watchers created/destroyed. Sicne |
878 | I<watcher> is the number of event watchers created/destroyed. Since |
878 | different event models have vastly different performance each backend was |
879 | different event models feature vastly different performances, each event |
879 | handed a number of watchers so that overall runtime is acceptable and |
880 | loop was given a number of watchers so that overall runtime is acceptable |
880 | similar to all backends (and keep them from crashing). |
881 | and similar between tested event loop (and keep them from crashing): Glib |
|
|
882 | would probably take thousands of years if asked to process the same number |
|
|
883 | of watchers as EV in this benchmark. |
881 | |
884 | |
882 | I<bytes> is the number of bytes (as measured by resident set size) used by |
885 | I<bytes> is the number of bytes (as measured by the resident set size, |
883 | each watcher. |
886 | RSS) consumed by each watcher. This method of measuring captures both C |
|
|
887 | and Perl-based overheads. |
884 | |
888 | |
885 | I<create> is the time, in microseconds, to create a single watcher. |
889 | I<create> is the time, in microseconds (millionths of seconds), that it |
|
|
890 | takes to create a single watcher. The callback is a closure shared between |
|
|
891 | all watchers, to avoid adding memory overhead. That means closure creation |
|
|
892 | and memory usage is not included in the figures. |
886 | |
893 | |
887 | I<invoke> is the time, in microseconds, used to invoke a simple callback |
894 | I<invoke> is the time, in microseconds, used to invoke a simple |
888 | that simply counts down. |
895 | callback. The callback simply counts down a Perl variable and after it was |
|
|
896 | invoked "watcher" times, it would C<< ->broadcast >> a condvar once to |
|
|
897 | signal the end of this phase. |
889 | |
898 | |
890 | I<destroy> is the time, in microseconds, to destroy a single watcher. |
899 | I<destroy> is the time, in microseconds, that it takes to destroy a single |
|
|
900 | watcher. |
|
|
901 | |
|
|
902 | =head2 Results |
891 | |
903 | |
892 | name watcher bytes create invoke destroy comment |
904 | name watcher bytes create invoke destroy comment |
893 | EV/EV 400000 244 0.56 0.46 0.31 EV native interface |
905 | EV/EV 400000 244 0.56 0.46 0.31 EV native interface |
894 | EV/Any 100000 610 3.52 0.91 0.75 |
906 | EV/Any 100000 610 3.52 0.91 0.75 EV + AnyEvent watchers |
895 | CoroEV/Any 100000 610 3.49 0.92 0.75 coroutines + Coro::Signal |
907 | CoroEV/Any 100000 610 3.49 0.92 0.75 coroutines + Coro::Signal |
896 | Perl/Any 10000 654 4.64 1.22 0.77 pure perl implementation |
908 | Perl/Any 16000 654 4.64 1.22 0.77 pure perl implementation |
897 | Event/Event 10000 523 28.05 21.38 5.22 Event native interface |
909 | Event/Event 16000 523 28.05 21.38 0.86 Event native interface |
898 | Event/Any 10000 943 34.43 20.48 1.39 |
910 | Event/Any 16000 943 34.43 20.48 1.39 Event + AnyEvent watchers |
899 | Glib/Any 16000 1357 96.99 12.55 55.51 quadratic behaviour |
911 | Glib/Any 16000 1357 96.99 12.55 55.51 quadratic behaviour |
900 | Tk/Any 2000 1855 27.01 66.61 14.03 SEGV with >> 2000 watchers |
912 | Tk/Any 2000 1855 27.01 66.61 14.03 SEGV with >> 2000 watchers |
|
|
913 | POE/Event 2000 6644 108.15 768.19 14.33 via POE::Loop::Event |
901 | POE/Select 2000 6343 94.69 807.65 562.69 POE::Loop::Select |
914 | POE/Select 2000 6343 94.69 807.65 562.69 via POE::Loop::Select |
902 | POE/Event 2000 6644 108.15 768.19 14.33 POE::Loop::Event |
|
|
903 | |
915 | |
904 | Discussion: The benchmark does I<not> bench scalability of the |
916 | =head2 Discussion |
|
|
917 | |
|
|
918 | The benchmark does I<not> measure scalability of the event loop very |
905 | backend. For example a select-based backend (such as the pureperl one) can |
919 | well. For example, a select-based event loop (such as the pure perl one) |
906 | never compete with a backend using epoll. In this benchmark, only a single |
920 | can never compete with an event loop that uses epoll when the number of |
907 | filehandle is used. |
921 | file descriptors grows high. In this benchmark, only a single filehandle |
|
|
922 | is used (although some of the AnyEvent adaptors dup() its file descriptor |
|
|
923 | to worka round bugs). |
908 | |
924 | |
909 | EV is the sole leader regarding speed and memory use, which are both |
925 | C<EV> is the sole leader regarding speed and memory use, which are both |
910 | maximal/minimal. Even when going through AnyEvent, there is only one event |
926 | maximal/minimal, respectively. Even when going through AnyEvent, there is |
911 | loop that uses less memory (the Event module natively), and no faster |
927 | only one event loop that uses less memory (the C<Event> module natively), and |
912 | event model. |
928 | no faster event model, not event C<Event> natively. |
913 | |
929 | |
914 | The pure perl implementation is hit in a few sweet spots (both the |
930 | The pure perl implementation is hit in a few sweet spots (both the |
915 | zero timeout and the use of a single fd hit optimisations in the perl |
931 | zero timeout and the use of a single fd hit optimisations in the perl |
916 | interpreter and the backend itself), but it shows that it adds very little |
932 | interpreter and the backend itself). Nevertheless tis shows that it |
917 | overhead in itself. Like any select-based backend it's performance becomes |
933 | adds very little overhead in itself. Like any select-based backend its |
918 | really bad with lots of file descriptors. |
934 | performance becomes really bad with lots of file descriptors, of course, |
|
|
935 | but this was not subjetc of this benchmark. |
919 | |
936 | |
920 | The Event module has a relatively high setup and callback invocation cost, |
937 | The C<Event> module has a relatively high setup and callback invocation cost, |
921 | but overall scores on the third place. |
938 | but overall scores on the third place. |
922 | |
939 | |
923 | Glib has a little higher memory cost, a bit fster callback invocation and |
940 | C<Glib>'s memory usage is quite a bit bit higher, but it features a |
924 | has a similar speed as Event. |
941 | faster callback invocation and overall ends up in the same class as |
|
|
942 | C<Event>. However, Glib scales extremely badly, doubling the number of |
|
|
943 | watchers increases the processing time by more than a factor of four, |
|
|
944 | making it completely unusable when using larger numbers of watchers |
|
|
945 | (note that only a single file descriptor was used in the benchmark, so |
|
|
946 | inefficiencies of C<poll> do not account for this). |
925 | |
947 | |
926 | The Tk backend works relatively well, the fact that it crashes with |
948 | The C<Tk> adaptor works relatively well. The fact that it crashes with |
927 | more than 2000 watchers is a big setback, however, as correctness takes |
949 | more than 2000 watchers is a big setback, however, as correctness takes |
928 | precedence over speed. |
950 | precedence over speed. Nevertheless, its performance is surprising, as the |
|
|
951 | file descriptor is dup()ed for each watcher. This shows that the dup() |
|
|
952 | employed by some adaptors is not a big performance issue (it does incur a |
|
|
953 | hidden memory cost inside the kernel, though). |
929 | |
954 | |
930 | POE, regardless of backend (wether it's pure perl select backend or the |
955 | C<POE>, regardless of underlying event loop (wether using its pure perl |
931 | Event backend) shows abysmal performance and memory usage: Watchers use |
956 | select-based backend or the Event module) shows abysmal performance and |
932 | almost 30 times as much memory as EV watchers, and 10 times as much memory |
957 | memory usage: Watchers use almost 30 times as much memory as EV watchers, |
933 | as both Event or EV via AnyEvent. |
958 | and 10 times as much memory as both Event or EV via AnyEvent. Watcher |
|
|
959 | invocation is almost 700 times slower than with AnyEvent's pure perl |
|
|
960 | implementation. The design of the POE adaptor class in AnyEvent can not |
|
|
961 | really account for this, as session creation overhead is small compared |
|
|
962 | to execution of the state machine, which is coded pretty optimally within |
|
|
963 | L<AnyEvent::Impl::POE>. POE simply seems to be abysmally slow. |
934 | |
964 | |
|
|
965 | =head2 Summary |
|
|
966 | |
935 | Summary: using EV through AnyEvent is faster than any other event |
967 | Using EV through AnyEvent is faster than any other event loop, but most |
936 | loop. The overhead AnyEvent adds can be very small, and you should avoid |
968 | event loops have acceptable performance with or without AnyEvent. |
937 | POE like the plague if you want performance or reasonable memory usage. |
969 | |
|
|
970 | The overhead AnyEvent adds is usually much smaller than the overhead of |
|
|
971 | the actual event loop, only with extremely fast event loops such as the EV |
|
|
972 | adds AnyEvent significant overhead. |
|
|
973 | |
|
|
974 | And you should simply avoid POE like the plague if you want performance or |
|
|
975 | reasonable memory usage. |
938 | |
976 | |
939 | |
977 | |
940 | =head1 FORK |
978 | =head1 FORK |
941 | |
979 | |
942 | Most event libraries are not fork-safe. The ones who are usually are |
980 | Most event libraries are not fork-safe. The ones who are usually are |