| 1 |
=head1 NAME |
| 2 |
|
| 3 |
Coro::Multicore - make coro threads on multiple cores with specially supported modules |
| 4 |
|
| 5 |
=head1 SYNOPSIS |
| 6 |
|
| 7 |
# when you DO control the main event loop, e.g. in the main program |
| 8 |
|
| 9 |
use Coro::Multicore; # enable by default |
| 10 |
|
| 11 |
Coro::Multicore::scoped_disable; |
| 12 |
AE::cv->recv; # or EV::run, AnyEvent::Loop::run, Event::loop, ... |
| 13 |
|
| 14 |
# when you DO NOT control the event loop, e.g. in a module on CPAN |
| 15 |
# do nothing (see HOW TO USE IT) or something like this: |
| 16 |
|
| 17 |
use Coro::Multicore (); # disable by default |
| 18 |
|
| 19 |
async { |
| 20 |
Coro::Multicore::scoped_enable; |
| 21 |
|
| 22 |
# blocking is safe in your own threads |
| 23 |
... |
| 24 |
}; |
| 25 |
|
| 26 |
=head1 DESCRIPTION |
| 27 |
|
| 28 |
While L<Coro> threads (unlike ithreads) provide real threads similar to |
| 29 |
pthreads, python threads and so on, they do not run in parallel to each |
| 30 |
other even on machines with multiple CPUs or multiple CPU cores. |
| 31 |
|
| 32 |
This module lifts this restriction under two very specific but useful |
| 33 |
conditions: firstly, the coro thread executes in XS code and does not |
| 34 |
touch any perl data structures, and secondly, the XS code is specially |
| 35 |
prepared to allow this. |
| 36 |
|
| 37 |
This means that, when you call an XS function of a module prepared for it, |
| 38 |
this XS function can execute in parallel to any other Coro threads. This |
| 39 |
is useful for both CPU bound tasks (such as cryptography) as well as I/O |
| 40 |
bound tasks (such as loading an image from disk). It can also be used |
| 41 |
to do stuff in parallel via APIs that were not meant for this, such as |
| 42 |
database accesses via DBI. |
| 43 |
|
| 44 |
The mechanism to support this is easily added to existing modules |
| 45 |
and is independent of L<Coro> or L<Coro::Multicore>, and therefore |
| 46 |
could be used, without changes, with other, similar, modules, or even |
| 47 |
the perl core, should it gain real thread support anytime soon. See |
| 48 |
L<http://perlmulticore.schmorp.de/> for more info on how to prepare a |
| 49 |
module to allow parallel execution. Preparing an existing module is easy, |
| 50 |
doesn't add much overhead and no dependencies. |
| 51 |
|
| 52 |
This module is an L<AnyEvent> user (and also, if not obvious, uses |
| 53 |
L<Coro>). |
| 54 |
|
| 55 |
=head1 HOW TO USE IT |
| 56 |
|
| 57 |
Quick explanation: decide whether you control the main program/the event |
| 58 |
loop and choose one of the two styles from the SYNOPSIS. |
| 59 |
|
| 60 |
Longer explanation: There are two major modes this module can used in - |
| 61 |
supported operations run asynchronously either by default, or only when |
| 62 |
requested. The reason you might not want to enable this module for all |
| 63 |
operations by default is compatibility with existing code: |
| 64 |
|
| 65 |
Since this module integrates into an event loop and you must not normally |
| 66 |
block and wait for something in an event loop callbacks. Now imagine |
| 67 |
somebody patches your favourite module (e.g. Digest::MD5) to take |
| 68 |
advantage of of the Perl Multicore API. |
| 69 |
|
| 70 |
Then code that runs in an event loop callback and executes |
| 71 |
Digest::MD5::md5 would work fine without C<Coro::Multicore> - it would |
| 72 |
simply calculate the MD5 digest and block execution of anything else. But |
| 73 |
with C<Coro::Multicore> enabled, the same operation would try to run other |
| 74 |
threads. And when those wait for events, there is no event loop anymore, |
| 75 |
as the event loop thread is busy doing the MD5 calculation, leading to a |
| 76 |
deadlock. |
| 77 |
|
| 78 |
=head2 USE IT IN THE MAIN PROGRAM |
| 79 |
|
| 80 |
One way to avoid this is to not run perlmulticore enabled functions |
| 81 |
in any callbacks. A simpler way to ensure it works is to disable |
| 82 |
C<Coro::Multicore> thread switching in event loop callbacks, and enable it |
| 83 |
everywhere else. |
| 84 |
|
| 85 |
Therefore, if you control the event loop, as is usually the case when |
| 86 |
you write I<program> and not a I<module>, then you can enable C<Coro::Multicore> |
| 87 |
by default, and disable it in your event loop thread: |
| 88 |
|
| 89 |
# example 1, separate thread for event loop |
| 90 |
|
| 91 |
use EV; |
| 92 |
use Coro; |
| 93 |
use Coro::Multicore; |
| 94 |
|
| 95 |
async { |
| 96 |
Coro::Multicore::scoped_disable; |
| 97 |
EV::run; |
| 98 |
}; |
| 99 |
|
| 100 |
# do something else |
| 101 |
|
| 102 |
# example 2, run event loop as main program |
| 103 |
|
| 104 |
use EV; |
| 105 |
use Coro; |
| 106 |
use Coro::Multicore; |
| 107 |
|
| 108 |
Coro::Multicore::scoped_disable; |
| 109 |
|
| 110 |
... initialisation |
| 111 |
|
| 112 |
EV::run; |
| 113 |
|
| 114 |
The latter form is usually better and more idiomatic - the main thread is |
| 115 |
the best place to run the event loop. |
| 116 |
|
| 117 |
Often you want to do some initialisation before running the event |
| 118 |
loop. The most efficient way to do that is to put your intialisation code |
| 119 |
(and main program) into its own thread and run the event loop in your main |
| 120 |
program: |
| 121 |
|
| 122 |
use AnyEvent::Loop; |
| 123 |
use Coro::Multicore; # enable by default |
| 124 |
|
| 125 |
async { |
| 126 |
load_data; |
| 127 |
do_other_init; |
| 128 |
bind_socket; |
| 129 |
... |
| 130 |
}; |
| 131 |
|
| 132 |
Coro::Multicore::scoped_disable; |
| 133 |
AnyEvent::Loop::run; |
| 134 |
|
| 135 |
This has the effect of running the event loop first, so the initialisation |
| 136 |
code can block if it wants to. |
| 137 |
|
| 138 |
If this is too cumbersome but you still want to make sure you can |
| 139 |
call blocking functions before entering the event loop, you can keep |
| 140 |
C<Coro::Multicore> disabled till you cna run the event loop: |
| 141 |
|
| 142 |
use AnyEvent::Loop; |
| 143 |
use Coro::Multicore (); # disable by default |
| 144 |
|
| 145 |
load_data; |
| 146 |
do_other_init; |
| 147 |
bind_socket; |
| 148 |
... |
| 149 |
|
| 150 |
Coro::Multicore::scoped_disable; # disable for event loop |
| 151 |
Coro::Multicore::enable 1; # enable for the rest of the program |
| 152 |
AnyEvent::Loop::run; |
| 153 |
|
| 154 |
=head2 USE IT IN A MODULE |
| 155 |
|
| 156 |
When you I<do not> control the event loop, for example, because you want |
| 157 |
to use this from a module you published on CPAN, then the previous method |
| 158 |
doesn't work. |
| 159 |
|
| 160 |
However, this is not normally a problem in practise - most modules only |
| 161 |
do work at request of the caller. In that case, you might not care |
| 162 |
whether it does block other threads or not, as this would be the callers |
| 163 |
responsibility (or decision), and by extension, a decision for the main |
| 164 |
program. |
| 165 |
|
| 166 |
So unless you use XS and want your XS functions to run asynchronously, |
| 167 |
you don't have to worry about C<Coro::Multicore> at all - if you |
| 168 |
happen to call XS functions that are multicore-enabled and your |
| 169 |
caller has configured things correctly, they will automatically run |
| 170 |
asynchronously. Or in other words: nothing needs to be done at all, which |
| 171 |
also means that this method works fine for existing pure-perl modules, |
| 172 |
without having to change them at all. |
| 173 |
|
| 174 |
Only if your module runs it's own L<Coro> threads could it be an |
| 175 |
issue - maybe your module implements some kind of job pool and relies |
| 176 |
on certain operations to run asynchronously. Then you can still use |
| 177 |
C<Coro::Multicore> by not enabling it be default and only enabling it in |
| 178 |
your own threads: |
| 179 |
|
| 180 |
use Coro; |
| 181 |
use Coro::Multicore (); # note the () to disable by default |
| 182 |
|
| 183 |
async { |
| 184 |
Coro::Multicore::scoped_enable; |
| 185 |
|
| 186 |
# do things asynchronously by calling perlmulticore-enabled functions |
| 187 |
}; |
| 188 |
|
| 189 |
=head2 EXPORTS |
| 190 |
|
| 191 |
This module does not (at the moment) export any symbols. It does, however, |
| 192 |
export "behaviour" - if you use the default import, then Coro::Multicore |
| 193 |
will be enabled for all threads and all callers in the whole program: |
| 194 |
|
| 195 |
use Coro::Multicore; |
| 196 |
|
| 197 |
In a module where you don't control what else might be loaded and run, you |
| 198 |
might want to be more conservative, and not import anything. This has the |
| 199 |
effect of not enabling the functionality by default, so you have to enable |
| 200 |
it per scope: |
| 201 |
|
| 202 |
use Coro::Multicore (); |
| 203 |
|
| 204 |
sub myfunc { |
| 205 |
Coro::Multicore::scoped_enable; |
| 206 |
|
| 207 |
# from here to the end of this function, and in any functions |
| 208 |
# called from this function, tasks will be executed asynchronously. |
| 209 |
} |
| 210 |
|
| 211 |
=head1 API FUNCTIONS |
| 212 |
|
| 213 |
=over 4 |
| 214 |
|
| 215 |
=item $previous = Coro::Multicore::enable [$enable] |
| 216 |
|
| 217 |
This function enables (if C<$enable> is true) or disables (if C<$enable> |
| 218 |
is false) the multicore functionality globally. By default, it is enabled. |
| 219 |
|
| 220 |
This can be used to effectively disable this module's functionality by |
| 221 |
default, and enable it only for selected threads or scopes, by calling |
| 222 |
C<Coro::Multicore::scoped_enable>. |
| 223 |
|
| 224 |
Note that this setting nonly affects the I<global default> - it will not |
| 225 |
reflect whether multicore functionality is enabled for the current thread. |
| 226 |
|
| 227 |
The function returns the previous value of the enable flag. |
| 228 |
|
| 229 |
=item Coro::Multicore::scoped_enable |
| 230 |
|
| 231 |
This function instructs Coro::Multicore to handle all requests executed |
| 232 |
in the current coro thread, from the call to the end of the current scope. |
| 233 |
|
| 234 |
Calls to C<scoped_enable> and C<scoped_disable> don't nest very well at |
| 235 |
the moment, so don't nest them. |
| 236 |
|
| 237 |
=item Coro::Multicore::scoped_disable |
| 238 |
|
| 239 |
The opposite of C<Coro::Multicore::scope_disable>: instructs Coro::Multicore to |
| 240 |
I<not> handle the next multicore-enabled request. |
| 241 |
|
| 242 |
=back |
| 243 |
|
| 244 |
=cut |
| 245 |
|
| 246 |
package Coro::Multicore; |
| 247 |
|
| 248 |
use Coro (); |
| 249 |
|
| 250 |
BEGIN { |
| 251 |
our $VERSION = '1.07'; |
| 252 |
|
| 253 |
use XSLoader; |
| 254 |
XSLoader::load __PACKAGE__, $VERSION; |
| 255 |
} |
| 256 |
|
| 257 |
|
| 258 |
sub import { |
| 259 |
if (@_ > 1) { |
| 260 |
require Carp; |
| 261 |
Carp::croak ("Coro::Multicore does not export any symbols"); |
| 262 |
} |
| 263 |
|
| 264 |
enable 1; |
| 265 |
} |
| 266 |
|
| 267 |
our $WATCHER; |
| 268 |
|
| 269 |
# called when first thread is started, on first release. can |
| 270 |
# be called manually, but is not currently a public interface. |
| 271 |
sub init { |
| 272 |
require AnyEvent; # maybe load it unconditionally? |
| 273 |
$WATCHER ||= AE::io (fd, 0, \&poll); |
| 274 |
} |
| 275 |
|
| 276 |
=head1 THREAD SAFETY OF SUPPORTING XS MODULES |
| 277 |
|
| 278 |
Just because an XS module supports perlmulticore might not immediately |
| 279 |
make it reentrant. For example, while you can (try to) call C<execute> |
| 280 |
on the same database handle for the patched C<DBD::mysql> (see the |
| 281 |
L<registry|http://perlmulticore.schmorp.de/registry>), this will almost |
| 282 |
certainly not work, despite C<DBD::mysql> and C<libmysqlclient> being |
| 283 |
thread safe and reentrant - just not on the same database handle. |
| 284 |
|
| 285 |
Many modules have limitations such as these - some can only be called |
| 286 |
concurrently from a single thread as they use global variables, some |
| 287 |
can only be called concurrently on different I<handles> (e.g. database |
| 288 |
connections for DBD modules, or digest objects for Digest modules), |
| 289 |
and some can be called at any time (such as the C<md5> function in |
| 290 |
C<Digest::MD5>). |
| 291 |
|
| 292 |
Generally, you only have to be careful with the very few modules that use |
| 293 |
global variables or rely on C libraries that aren't thread-safe, which |
| 294 |
should be documented clearly in the module documentation. |
| 295 |
|
| 296 |
Most modules are either perfectly reentrant, or at least reentrant as long |
| 297 |
as you give every thread it's own I<handle> object. |
| 298 |
|
| 299 |
=head1 EXCEPTIONS AND THREAD CANCELLATION |
| 300 |
|
| 301 |
L<Coro> allows you to cancel threads even when they execute within an XS |
| 302 |
function (C<cancel> vs. C<cancel> methods). Similarly, L<Coro> allows you |
| 303 |
to send exceptions (e.g. via the C<throw> method) to threads executing |
| 304 |
inside an XS function. |
| 305 |
|
| 306 |
While doing this is questionable and dangerous with normal Coro threads |
| 307 |
already, they are both supported in this module, although with potentially |
| 308 |
unwanted effects. The following describes the current implementation and |
| 309 |
is subject to change. It is described primarily so you can understand what |
| 310 |
went wrong, if things go wrong. |
| 311 |
|
| 312 |
=over 4 |
| 313 |
|
| 314 |
=item EXCEPTIONS |
| 315 |
|
| 316 |
When a thread that has currently released the perl interpreter (e.g. |
| 317 |
because it is executing a perlmulticore enabled XS function) receives an exception, it will |
| 318 |
at first continue normally. |
| 319 |
|
| 320 |
After acquiring the perl interpreter again, it will throw the |
| 321 |
exception it previously received. More specifically, when a thread |
| 322 |
calls C<perlinterp_acquire ()> and has received an exception, then |
| 323 |
C<perlinterp_acquire ()> will not return but instead C<die>. |
| 324 |
|
| 325 |
Most code that has been updated for perlmulticore support will not expect |
| 326 |
this, and might leave internal state corrupted to some extent. |
| 327 |
|
| 328 |
=item CANCELLATION |
| 329 |
|
| 330 |
Unsafe cancellation on a thread that has released the perl interpreter |
| 331 |
frees its resources, but let's the XS code continue at first. This should |
| 332 |
not lead to corruption on the perl level, as the code isn't allowed to |
| 333 |
touch perl data structures until it reacquires the interpreter. |
| 334 |
|
| 335 |
The call to C<perlinterp_acquire ()> will then block indefinitely, leaking |
| 336 |
the (OS level) thread. |
| 337 |
|
| 338 |
Safe cancellation will simply fail in this case, so is still "safe" to |
| 339 |
call. |
| 340 |
|
| 341 |
=back |
| 342 |
|
| 343 |
=head1 INTERACTION WITH OTHER SOFTWARE |
| 344 |
|
| 345 |
This module is very similar to other environments where perl interpreters |
| 346 |
are moved between threads, such as mod_perl2, and the same caveats apply. |
| 347 |
|
| 348 |
I want to spell out the most important ones: |
| 349 |
|
| 350 |
=over 4 |
| 351 |
|
| 352 |
=item pthreads usage |
| 353 |
|
| 354 |
Any creation of pthreads make it impossible to fork portably from a |
| 355 |
perl program, as forking from within a threaded program will leave the |
| 356 |
program in a state similar to a signal handler. While it might work on |
| 357 |
some platforms (as an extension), this might also result in silent data |
| 358 |
corruption. It also seems to work most of the time, so it's hard to test |
| 359 |
for this. |
| 360 |
|
| 361 |
I recommend using something like L<AnyEvent::Fork>, which can create |
| 362 |
subprocesses safely (via L<Proc::FastSpawn>). |
| 363 |
|
| 364 |
Similar issues exist for signal handlers, although this module works hard |
| 365 |
to keep safe perl signals safe. |
| 366 |
|
| 367 |
=item module support |
| 368 |
|
| 369 |
This module moves the same perl interpreter between different |
| 370 |
threads. Some modules might get confused by that (although this can |
| 371 |
usually be considered a bug). This is a rare case though. |
| 372 |
|
| 373 |
=item event loop reliance |
| 374 |
|
| 375 |
To be able to wake up programs waiting for results, this module relies on |
| 376 |
an active event loop (via L<AnyEvent>). This is used to notify the perl |
| 377 |
interpreter when the asynchronous task is done. |
| 378 |
|
| 379 |
Since event loops typically fail to work properly after a fork, this means |
| 380 |
that some operations that were formerly working will now hang after fork. |
| 381 |
|
| 382 |
A workaround is to call C<Coro::Multicore::enable 0> after a fork to |
| 383 |
disable the module. |
| 384 |
|
| 385 |
Future versions of this module might do this automatically. |
| 386 |
|
| 387 |
=back |
| 388 |
|
| 389 |
=head1 BUGS & LIMITATIONS |
| 390 |
|
| 391 |
=over 4 |
| 392 |
|
| 393 |
=item (OS-) threads are never released |
| 394 |
|
| 395 |
At the moment, threads that were created once will never be freed. They |
| 396 |
will be reused for asynchronous requests, though, so as long as you limit |
| 397 |
the maximum number of concurrent asynchronous tasks, this will also limit |
| 398 |
the maximum number of threads created. |
| 399 |
|
| 400 |
The idle threads are not necessarily using a lot of resources: on |
| 401 |
GNU/Linux + glibc, each thread takes about 8KiB of userspace memory + |
| 402 |
whatever the kernel needs (probably less than 8KiB). |
| 403 |
|
| 404 |
Future versions will likely lift this limitation. |
| 405 |
|
| 406 |
=item The enable_times feature of Coro is messed up |
| 407 |
|
| 408 |
The enable_times feature uses the per-thread timer to measure per-thread |
| 409 |
execution time, but since Coro::Multicore runs threads on different |
| 410 |
pthreads it will get the wrong times. Real times are not affected. |
| 411 |
|
| 412 |
=item Fork support |
| 413 |
|
| 414 |
Due to the nature of threads, you are not allowed to use this module in a |
| 415 |
forked child normally, with one exception: If you don't create any threads |
| 416 |
in the parent, then it is safe to start using it in a forked child. |
| 417 |
|
| 418 |
=back |
| 419 |
|
| 420 |
=head1 AUTHOR |
| 421 |
|
| 422 |
Marc Lehmann <schmorp@schmorp.de> |
| 423 |
http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html |
| 424 |
|
| 425 |
Additional thanks to Zsbán Ambrus, who gave considerable desing input for |
| 426 |
this module and the perl multicore specification. |
| 427 |
|
| 428 |
=cut |
| 429 |
|
| 430 |
1 |
| 431 |
|