… | |
… | |
863 | $quit->wait; |
863 | $quit->wait; |
864 | |
864 | |
865 | |
865 | |
866 | =head1 BENCHMARK |
866 | =head1 BENCHMARK |
867 | |
867 | |
868 | To give you an idea of the performance an doverheads that AnyEvent adds |
868 | To give you an idea of the performance and overheads that AnyEvent adds |
869 | over the backends, here is a benchmark of various supported backends. The |
869 | over the event loops directly, here is a benchmark of various supported |
870 | benchmark creates a lot of timers (with zero timeout) and io events |
870 | event models natively and with anyevent. The benchmark creates a lot of |
871 | (watching STDOUT, a pty, to become writable). |
871 | timers (with a zero timeout) and io watchers (watching STDOUT, a pty, to |
|
|
872 | become writable, which it is), lets them fire exactly once and destroys |
|
|
873 | them again. |
872 | |
874 | |
873 | Explanation of the fields: |
875 | =head2 Explanation of the columns |
874 | |
876 | |
875 | I<watcher> is the number of event watchers created/destroyed. Sicne |
877 | I<watcher> is the number of event watchers created/destroyed. Since |
876 | different event models have vastly different performance each backend was |
878 | different event models feature vastly different performances, each event |
877 | handed a number of watchers so that overall runtime is acceptable and |
879 | loop was given a number of watchers so that overall runtime is acceptable |
878 | similar to all backends (and keep them from crashing). |
880 | and similar between tested event loop (and keep them from crashing): Glib |
|
|
881 | would probably take thousands of years if asked to process the same number |
|
|
882 | of watchers as EV in this benchmark. |
879 | |
883 | |
880 | I<bytes> is the number of bytes (as measured by resident set size) used by |
884 | I<bytes> is the number of bytes (as measured by the resident set size, |
881 | each watcher. |
885 | RSS) consumed by each watcher. This method of measuring captures both C |
|
|
886 | and Perl-based overheads. |
882 | |
887 | |
883 | I<create> is the time, in microseconds, to create a single watcher. |
888 | I<create> is the time, in microseconds (millionths of seconds), that it |
|
|
889 | takes to create a single watcher. The callback is a closure shared between |
|
|
890 | all watchers, to avoid adding memory overhead. That means closure creation |
|
|
891 | and memory usage is not included in the figures. |
884 | |
892 | |
885 | I<invoke> is the time, in microseconds, used to invoke a simple callback |
893 | I<invoke> is the time, in microseconds, used to invoke a simple |
886 | that simply counts down. |
894 | callback. The callback simply counts down a Perl variable and after it was |
|
|
895 | invoked "watcher" times, it would C<< ->broadcast >> a condvar once to |
|
|
896 | signal the end of this phase. |
887 | |
897 | |
888 | I<destroy> is the time, in microseconds, to destroy a single watcher. |
898 | I<destroy> is the time, in microseconds, that it takes destroy a single |
|
|
899 | watcher. |
|
|
900 | |
|
|
901 | =head2 Results |
889 | |
902 | |
890 | name watcher bytes create invoke destroy comment |
903 | name watcher bytes create invoke destroy comment |
891 | EV/EV 400000 244 0.56 0.46 0.31 EV native interface |
904 | EV/EV 400000 244 0.56 0.46 0.31 EV native interface |
892 | EV/Any 100000 610 3.52 0.91 0.75 |
905 | EV/Any 100000 610 3.52 0.91 0.75 EV + AnyEvent watchers |
893 | CoroEV/Any 100000 610 3.49 0.92 0.75 coroutines + Coro::Signal |
906 | CoroEV/Any 100000 610 3.49 0.92 0.75 coroutines + Coro::Signal |
894 | Perl/Any 10000 654 4.64 1.22 0.77 pure perl implementation |
907 | Perl/Any 16000 654 4.64 1.22 0.77 pure perl implementation |
895 | Event/Event 10000 523 28.05 21.38 5.22 Event native interface |
908 | Event/Event 16000 523 28.05 21.38 0.86 Event native interface |
896 | Event/Any 10000 943 34.43 20.48 1.39 |
909 | Event/Any 16000 943 34.43 20.48 1.39 Event + AnyEvent watchers |
897 | Glib/Any 16000 1357 96.99 12.55 55.51 quadratic behaviour |
910 | Glib/Any 16000 1357 96.99 12.55 55.51 quadratic behaviour |
898 | Tk/Any 2000 1855 27.01 66.61 14.03 SEGV with >> 2000 watchers |
911 | Tk/Any 2000 1855 27.01 66.61 14.03 SEGV with >> 2000 watchers |
|
|
912 | POE/Event 2000 6644 108.15 768.19 14.33 via POE::Loop::Event |
899 | POE/Select 2000 6343 94.69 807.65 562.69 POE::Loop::Select |
913 | POE/Select 2000 6343 94.69 807.65 562.69 via POE::Loop::Select |
900 | POE/Event 2000 6644 108.15 768.19 14.33 POE::Loop::Event |
|
|
901 | |
914 | |
902 | Discussion: The benchmark does I<not> bench scalability of the |
915 | =head2 Discussion |
|
|
916 | |
|
|
917 | The benchmark does I<not> measure scalability of the event loop very |
903 | backend. For example a select-based backend (such as the pureperl one) can |
918 | well. For example, a select-based event loop (such as the pure perl one) |
904 | never compete with a backend using epoll. In this benchmark, only a single |
919 | can never compete with an event loop that uses epoll when the number of |
905 | filehandle is used. |
920 | file descriptors grows high. In this benchmark, only a single filehandle |
|
|
921 | is used (although some of the AnyEvent adaptors dup() its file descriptor |
|
|
922 | to worka round bugs). |
906 | |
923 | |
907 | EV is the sole leader regarding speed and memory use, which are both |
924 | C<EV> is the sole leader regarding speed and memory use, which are both |
908 | maximal/minimal. Even when going through AnyEvent, there is only one event |
925 | maximal/minimal, respectively. Even when going through AnyEvent, there is |
909 | loop that uses less memory (the Event module natively), and no faster |
926 | only one event loop that uses less memory (the C<Event> module natively), and |
910 | event model. |
927 | no faster event model, not event C<Event> natively. |
911 | |
928 | |
912 | The pure perl implementation is hit in a few sweet spots (both the |
929 | The pure perl implementation is hit in a few sweet spots (both the |
913 | zero timeout and the use of a single fd hit optimisations in the perl |
930 | zero timeout and the use of a single fd hit optimisations in the perl |
914 | interpreter and the backend itself), but it shows that it adds very little |
931 | interpreter and the backend itself). Nevertheless tis shows that it |
915 | overhead in itself. Like any select-based backend it's performance becomes |
932 | adds very little overhead in itself. Like any select-based backend its |
916 | really bad with lots of file descriptors. |
933 | performance becomes really bad with lots of file descriptors, of course, |
|
|
934 | but this was not subjetc of this benchmark. |
917 | |
935 | |
918 | The Event module has a relatively high setup and callback invocation cost, |
936 | The C<Event> module has a relatively high setup and callback invocation cost, |
919 | but overall scores on the third place. |
937 | but overall scores on the third place. |
920 | |
938 | |
921 | Glib has a little higher memory cost, a bit fster callback invocation and |
939 | C<Glib>'s memory usage is quite a bit bit higher, features a faster |
922 | has a similar speed as Event. |
940 | callback invocation and overall lands in the same class as C<Event>. |
923 | |
941 | |
924 | The Tk backend works relatively well, the fact that it crashes with |
942 | The C<Tk> adaptor works relatively well, the fact that it crashes with |
925 | more than 2000 watchers is a big setback, however, as correctness takes |
943 | more than 2000 watchers is a big setback, however, as correctness takes |
926 | precedence over speed. |
944 | precedence over speed. Nevertheless, its performance is surprising, as the |
|
|
945 | file descriptor is dup()ed for each watcher. This shows that the dup() |
|
|
946 | employed by some adaptors is not a big performance issue (it does incur a |
|
|
947 | hidden memory cost inside the kernel, though). |
927 | |
948 | |
928 | POE, regardless of backend (wether it's pure perl select backend or the |
949 | C<POE>, regardless of backend (wether using its pure perl select-based |
929 | Event backend) shows abysmal performance and memory usage: Watchers use |
950 | backend or the Event backend) shows abysmal performance and memory |
930 | almost 30 times as much memory as EV watchers, and 10 times as much memory |
951 | usage: Watchers use almost 30 times as much memory as EV watchers, and 10 |
931 | as both Event or EV via AnyEvent. |
952 | times as much memory as both Event or EV via AnyEvent. Watcher invocation |
|
|
953 | is almost 700 times slower as with AnyEvent's pure perl implementation. |
932 | |
954 | |
933 | Summary: using EV through AnyEvent is faster than any other event |
955 | Summary: using EV through AnyEvent is faster than any other event |
934 | loop. The overhead AnyEvent adds can be very small, and you should avoid |
956 | loop. The overhead AnyEvent adds can be very small, and you should avoid |
935 | POE like the plague if you want performance or reasonable memory usage. |
957 | POE like the plague if you want performance or reasonable memory usage. |
936 | |
958 | |