ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Coro/README
(Generate patch)

Comparing Coro/README (file contents):
Revision 1.17 by root, Wed Nov 5 15:38:10 2008 UTC vs.
Revision 1.20 by root, Tue Nov 25 20:48:41 2008 UTC

1NAME 1NAME
2 Coro - coroutine process abstraction 2 Coro - the only real threads in perl
3 3
4SYNOPSIS 4SYNOPSIS
5 use Coro; 5 use Coro;
6 6
7 async { 7 async {
… …
23 $lock->down; 23 $lock->down;
24 $locked = 1; 24 $locked = 1;
25 $lock->up; 25 $lock->up;
26 26
27DESCRIPTION 27DESCRIPTION
28 This module collection manages coroutines. Coroutines are similar to 28 For a tutorial-style introduction, please read the Coro::Intro manpage.
29 threads but don't (in general) run in parallel at the same time even on 29 This manpage mainly contains reference information.
30 SMP machines. The specific flavor of coroutine used in this module also 30
31 guarantees you that it will not switch between coroutines unless 31 This module collection manages continuations in general, most often in
32 the form of cooperative threads (also called coroutines in the
33 documentation). They are similar to kernel threads but don't (in
34 general) run in parallel at the same time even on SMP machines. The
35 specific flavor of thread offered by this module also guarantees you
36 that it will not switch between threads unless necessary, at
32 necessary, at easily-identified points in your program, so locking and 37 easily-identified points in your program, so locking and parallel access
33 parallel access are rarely an issue, making coroutine programming much 38 are rarely an issue, making thread programming much safer and easier
34 safer and easier than threads programming. 39 than using other thread models.
35 40
36 Unlike a normal perl program, however, coroutines allow you to have 41 Unlike the so-called "Perl threads" (which are not actually real threads
37 multiple running interpreters that share data, which is especially 42 but only the windows process emulation ported to unix), Coro provides a
38 useful to code pseudo-parallel processes and for event-based 43 full shared address space, which makes communication between threads
39 programming, such as multiple HTTP-GET requests running concurrently. 44 very easy. And threads are fast, too: disabling the Windows process
40 See Coro::AnyEvent to learn more. 45 emulation code in your perl and using Coro can easily result in a two to
46 four times speed increase for your programs.
41 47
42 Coroutines are also useful because Perl has no support for threads (the 48 Coro achieves that by supporting multiple running interpreters that
43 so called "threads" that perl offers are nothing more than the (bad) 49 share data, which is especially useful to code pseudo-parallel processes
44 process emulation coming from the Windows platform: On standard 50 and for event-based programming, such as multiple HTTP-GET requests
45 operating systems they serve no purpose whatsoever, except by making 51 running concurrently. See Coro::AnyEvent to learn more on how to
46 your programs slow and making them use a lot of memory. Best disable 52 integrate Coro into an event-based environment.
47 them when building perl, or aks your software vendor/distributor to do
48 it for you).
49 53
50 In this module, coroutines are defined as "callchain + lexical variables 54 In this module, a thread is defined as "callchain + lexical variables +
51 + @_ + $_ + $@ + $/ + C stack), that is, a coroutine has its own 55 @_ + $_ + $@ + $/ + C stack), that is, a thread has its own callchain,
52 callchain, its own set of lexicals and its own set of perls most 56 its own set of lexicals and its own set of perls most important global
53 important global variables (see Coro::State for more configuration). 57 variables (see Coro::State for more configuration and background info).
54 58
59 See also the "SEE ALSO" section at the end of this document - the Coro
60 module family is quite large.
61
62GLOBAL VARIABLES
55 $Coro::main 63 $Coro::main
56 This variable stores the coroutine object that represents the main 64 This variable stores the coroutine object that represents the main
57 program. While you cna "ready" it and do most other things you can 65 program. While you cna "ready" it and do most other things you can
58 do to coroutines, it is mainly useful to compare again 66 do to coroutines, it is mainly useful to compare again
59 $Coro::current, to see whether you are running in the main program 67 $Coro::current, to see whether you are running in the main program
60 or not. 68 or not.
61 69
62 $Coro::current 70 $Coro::current
63 The coroutine object representing the current coroutine (the last 71 The coroutine object representing the current coroutine (the last
64 coroutine that the Coro scheduler switched to). The initial value is 72 coroutine that the Coro scheduler switched to). The initial value is
65 $main (of course). 73 $Coro::main (of course).
66 74
67 This variable is strictly *read-only*. You can take copies of the 75 This variable is strictly *read-only*. You can take copies of the
68 value stored in it and use it as any other coroutine object, but you 76 value stored in it and use it as any other coroutine object, but you
69 must not otherwise modify the variable itself. 77 must not otherwise modify the variable itself.
70 78
71 $Coro::idle 79 $Coro::idle
72 This variable is mainly useful to integrate Coro into event loops. 80 This variable is mainly useful to integrate Coro into event loops.
73 It is usually better to rely on Coro::AnyEvent or L"Coro::EV", as 81 It is usually better to rely on Coro::AnyEvent or Coro::EV, as this
74 this is pretty low-level functionality. 82 is pretty low-level functionality.
75 83
76 This variable stores a callback that is called whenever the 84 This variable stores either a coroutine or a callback.
77 scheduler finds no ready coroutines to run. The default 85
78 implementation prints "FATAL: deadlock detected" and exits, because 86 If it is a callback, the it is called whenever the scheduler finds
87 no ready coroutines to run. The default implementation prints
88 "FATAL: deadlock detected" and exits, because the program has no
79 the program has no other way to continue. 89 other way to continue.
80 90
91 If it is a coroutine object, then this object will be readied
92 (without invoking any ready hooks, however) when the scheduler finds
93 no other ready coroutines to run.
94
81 This hook is overwritten by modules such as "Coro::Timer" and 95 This hook is overwritten by modules such as "Coro::EV" and
82 "Coro::AnyEvent" to wait on an external event that hopefully wake up 96 "Coro::AnyEvent" to wait on an external event that hopefully wake up
83 a coroutine so the scheduler can run it. 97 a coroutine so the scheduler can run it.
84 98
85 Note that the callback *must not*, under any circumstances, block 99 Note that the callback *must not*, under any circumstances, block
86 the current coroutine. Normally, this is achieved by having an "idle 100 the current coroutine. Normally, this is achieved by having an "idle
87 coroutine" that calls the event loop and then blocks again, and then 101 coroutine" that calls the event loop and then blocks again, and then
88 readying that coroutine in the idle handler. 102 readying that coroutine in the idle handler, or by simply placing
103 the idle coroutine in this variable.
89 104
90 See Coro::Event or Coro::AnyEvent for examples of using this 105 See Coro::Event or Coro::AnyEvent for examples of using this
91 technique. 106 technique.
92 107
93 Please note that if your callback recursively invokes perl (e.g. for 108 Please note that if your callback recursively invokes perl (e.g. for
94 event handlers), then it must be prepared to be called recursively 109 event handlers), then it must be prepared to be called recursively
95 itself. 110 itself.
96 111
97 SIMPLE COROUTINE CREATION 112SIMPLE COROUTINE CREATION
98 async { ... } [@args...] 113 async { ... } [@args...]
99 Create a new coroutine and return it's coroutine object (usually 114 Create a new coroutine and return its coroutine object (usually
100 unused). The coroutine will be put into the ready queue, so it will 115 unused). The coroutine will be put into the ready queue, so it will
101 start running automatically on the next scheduler run. 116 start running automatically on the next scheduler run.
102 117
103 The first argument is a codeblock/closure that should be executed in 118 The first argument is a codeblock/closure that should be executed in
104 the coroutine. When it returns argument returns the coroutine is 119 the coroutine. When it returns argument returns the coroutine is
… …
126 Similar to "async", but uses a coroutine pool, so you should not 141 Similar to "async", but uses a coroutine pool, so you should not
127 call terminate or join on it (although you are allowed to), and you 142 call terminate or join on it (although you are allowed to), and you
128 get a coroutine that might have executed other code already (which 143 get a coroutine that might have executed other code already (which
129 can be good or bad :). 144 can be good or bad :).
130 145
131 On the plus side, this function is faster than creating (and 146 On the plus side, this function is about twice as fast as creating
132 destroying) a completly new coroutine, so if you need a lot of 147 (and destroying) a completely new coroutine, so if you need a lot of
133 generic coroutines in quick successsion, use "async_pool", not 148 generic coroutines in quick successsion, use "async_pool", not
134 "async". 149 "async".
135 150
136 The code block is executed in an "eval" context and a warning will 151 The code block is executed in an "eval" context and a warning will
137 be issued in case of an exception instead of terminating the 152 be issued in case of an exception instead of terminating the
… …
153 168
154 If you are concerned about pooled coroutines growing a lot because a 169 If you are concerned about pooled coroutines growing a lot because a
155 single "async_pool" used a lot of stackspace you can e.g. 170 single "async_pool" used a lot of stackspace you can e.g.
156 "async_pool { terminate }" once per second or so to slowly replenish 171 "async_pool { terminate }" once per second or so to slowly replenish
157 the pool. In addition to that, when the stacks used by a handler 172 the pool. In addition to that, when the stacks used by a handler
158 grows larger than 16kb (adjustable via $Coro::POOL_RSS) it will also 173 grows larger than 32kb (adjustable via $Coro::POOL_RSS) it will also
159 be destroyed. 174 be destroyed.
160 175
161 STATIC METHODS 176STATIC METHODS
162 Static methods are actually functions that operate on the current 177 Static methods are actually functions that implicitly operate on the
163 coroutine. 178 current coroutine.
164 179
165 schedule 180 schedule
166 Calls the scheduler. The scheduler will find the next coroutine that 181 Calls the scheduler. The scheduler will find the next coroutine that
167 is to be run from the ready queue and switches to it. The next 182 is to be run from the ready queue and switches to it. The next
168 coroutine to be run is simply the one with the highest priority that 183 coroutine to be run is simply the one with the highest priority that
… …
180 yours to call "->ready" on that once some event happens, and last 195 yours to call "->ready" on that once some event happens, and last
181 you call "schedule" to put yourself to sleep. Note that a lot of 196 you call "schedule" to put yourself to sleep. Note that a lot of
182 things can wake your coroutine up, so you need to check whether the 197 things can wake your coroutine up, so you need to check whether the
183 event indeed happened, e.g. by storing the status in a variable. 198 event indeed happened, e.g. by storing the status in a variable.
184 199
185 The canonical way to wait on external events is this: 200 See HOW TO WAIT FOR A CALLBACK, below, for some ways to wait for
186 201 callbacks.
187 {
188 # remember current coroutine
189 my $current = $Coro::current;
190
191 # register a hypothetical event handler
192 on_event_invoke sub {
193 # wake up sleeping coroutine
194 $current->ready;
195 undef $current;
196 };
197
198 # call schedule until event occurred.
199 # in case we are woken up for other reasons
200 # (current still defined), loop.
201 Coro::schedule while $current;
202 }
203 202
204 cede 203 cede
205 "Cede" to other coroutines. This function puts the current coroutine 204 "Cede" to other coroutines. This function puts the current coroutine
206 into the ready queue and calls "schedule", which has the effect of 205 into the ready queue and calls "schedule", which has the effect of
207 giving up the current "timeslice" to other coroutines of the same or 206 giving up the current "timeslice" to other coroutines of the same or
… …
227 Note that while this will try to free some of the main programs 226 Note that while this will try to free some of the main programs
228 resources, you cannot free all of them, so if a coroutine that is 227 resources, you cannot free all of them, so if a coroutine that is
229 not the main program calls this function, there will be some 228 not the main program calls this function, there will be some
230 one-time resource leak. 229 one-time resource leak.
231 230
232 COROUTINE METHODS 231COROUTINE OBJECT METHODS
233 These are the methods you can call on coroutine objects (or to create 232 These are the methods you can call on coroutine objects (or to create
234 them). 233 them).
235 234
236 new Coro \&sub [, @args...] 235 new Coro \&sub [, @args...]
237 Create a new coroutine and return it. When the sub returns, the 236 Create a new coroutine and return it. When the sub returns, the
… …
258 $coroutine->cancel (arg...) 257 $coroutine->cancel (arg...)
259 Terminates the given coroutine and makes it return the given 258 Terminates the given coroutine and makes it return the given
260 arguments as status (default: the empty list). Never returns if the 259 arguments as status (default: the empty list). Never returns if the
261 coroutine is the current coroutine. 260 coroutine is the current coroutine.
262 261
262 $coroutine->schedule_to
263 Puts the current coroutine to sleep (like "Coro::schedule"), but
264 instead of continuing with the next coro from the ready queue,
265 always switch to the given coroutine object (regardless of priority
266 etc.). The readyness state of that coroutine isn't changed.
267
268 This is an advanced method for special cases - I'd love to hear
269 about any uses for this one.
270
271 $coroutine->cede_to
272 Like "schedule_to", but puts the current coroutine into the ready
273 queue. This has the effect of temporarily switching to the given
274 coroutine, and continuing some time later.
275
276 This is an advanced method for special cases - I'd love to hear
277 about any uses for this one.
278
263 $coroutine->throw ([$scalar]) 279 $coroutine->throw ([$scalar])
264 If $throw is specified and defined, it will be thrown as an 280 If $throw is specified and defined, it will be thrown as an
265 exception inside the coroutine at the next convenient point in time 281 exception inside the coroutine at the next convenient point in time.
266 (usually after it gains control at the next schedule/transfer/cede).
267 Otherwise clears the exception object. 282 Otherwise clears the exception object.
283
284 Coro will check for the exception each time a schedule-like-function
285 returns, i.e. after each "schedule", "cede",
286 "Coro::Semaphore->down", "Coro::Handle->readable" and so on. Most of
287 these functions detect this case and return early in case an
288 exception is pending.
268 289
269 The exception object will be thrown "as is" with the specified 290 The exception object will be thrown "as is" with the specified
270 scalar in $@, i.e. if it is a string, no line number or newline will 291 scalar in $@, i.e. if it is a string, no line number or newline will
271 be appended (unlike with "die"). 292 be appended (unlike with "die").
272 293
… …
322 with a coroutine. 343 with a coroutine.
323 344
324 This method simply sets the "$coroutine->{desc}" member to the given 345 This method simply sets the "$coroutine->{desc}" member to the given
325 string. You can modify this member directly if you wish. 346 string. You can modify this member directly if you wish.
326 347
327 GLOBAL FUNCTIONS 348GLOBAL FUNCTIONS
328 Coro::nready 349 Coro::nready
329 Returns the number of coroutines that are currently in the ready 350 Returns the number of coroutines that are currently in the ready
330 state, i.e. that can be switched to by calling "schedule" directory 351 state, i.e. that can be switched to by calling "schedule" directory
331 or indirectly. The value 0 means that the only runnable coroutine is 352 or indirectly. The value 0 means that the only runnable coroutine is
332 the currently running one, so "cede" would have no effect, and 353 the currently running one, so "cede" would have no effect, and
… …
359 while the original code ref will be called (with parameters) from 380 while the original code ref will be called (with parameters) from
360 within another coroutine. 381 within another coroutine.
361 382
362 The reason this function exists is that many event libraries (such 383 The reason this function exists is that many event libraries (such
363 as the venerable Event module) are not coroutine-safe (a weaker form 384 as the venerable Event module) are not coroutine-safe (a weaker form
364 of thread-safety). This means you must not block within event 385 of reentrancy). This means you must not block within event
365 callbacks, otherwise you might suffer from crashes or worse. The 386 callbacks, otherwise you might suffer from crashes or worse. The
366 only event library currently known that is safe to use without 387 only event library currently known that is safe to use without
367 "unblock_sub" is EV. 388 "unblock_sub" is EV.
368 389
369 This function allows your callbacks to block by executing them in 390 This function allows your callbacks to block by executing them in
… …
383 when you use a module that uses AnyEvent (and you use 404 when you use a module that uses AnyEvent (and you use
384 Coro::AnyEvent) and it provides callbacks that are the result of 405 Coro::AnyEvent) and it provides callbacks that are the result of
385 some event callback, then you must not block either, or use 406 some event callback, then you must not block either, or use
386 "unblock_sub". 407 "unblock_sub".
387 408
409 $cb = Coro::rouse_cb
410 Create and return a "rouse callback". That's a code reference that,
411 when called, will remember a copy of its arguments and notify the
412 owner coroutine of the callback.
413
414 See the next function.
415
416 @args = Coro::rouse_wait [$cb]
417 Wait for the specified rouse callback (or the last one that was
418 created in this coroutine).
419
420 As soon as the callback is invoked (or when the callback was invoked
421 before "rouse_wait"), it will return the arguments originally passed
422 to the rouse callback.
423
424 See the section HOW TO WAIT FOR A CALLBACK for an actual usage
425 example.
426
427HOW TO WAIT FOR A CALLBACK
428 It is very common for a coroutine to wait for some callback to be
429 called. This occurs naturally when you use coroutines in an otherwise
430 event-based program, or when you use event-based libraries.
431
432 These typically register a callback for some event, and call that
433 callback when the event occured. In a coroutine, however, you typically
434 want to just wait for the event, simplyifying things.
435
436 For example "AnyEvent->child" registers a callback to be called when a
437 specific child has exited:
438
439 my $child_watcher = AnyEvent->child (pid => $pid, cb => sub { ... });
440
441 But from withina coroutine, you often just want to write this:
442
443 my $status = wait_for_child $pid;
444
445 Coro offers two functions specifically designed to make this easy,
446 "Coro::rouse_cb" and "Coro::rouse_wait".
447
448 The first function, "rouse_cb", generates and returns a callback that,
449 when invoked, will save its arguments and notify the coroutine that
450 created the callback.
451
452 The second function, "rouse_wait", waits for the callback to be called
453 (by calling "schedule" to go to sleep) and returns the arguments
454 originally passed to the callback.
455
456 Using these functions, it becomes easy to write the "wait_for_child"
457 function mentioned above:
458
459 sub wait_for_child($) {
460 my ($pid) = @_;
461
462 my $watcher = AnyEvent->child (pid => $pid, cb => Coro::rouse_cb);
463
464 my ($rpid, $rstatus) = Coro::rouse_wait;
465 $rstatus
466 }
467
468 In the case where "rouse_cb" and "rouse_wait" are not flexible enough,
469 you can roll your own, using "schedule":
470
471 sub wait_for_child($) {
472 my ($pid) = @_;
473
474 # store the current coroutine in $current,
475 # and provide result variables for the closure passed to ->child
476 my $current = $Coro::current;
477 my ($done, $rstatus);
478
479 # pass a closure to ->child
480 my $watcher = AnyEvent->child (pid => $pid, cb => sub {
481 $rstatus = $_[1]; # remember rstatus
482 $done = 1; # mark $rstatus as valud
483 });
484
485 # wait until the closure has been called
486 schedule while !$done;
487
488 $rstatus
489 }
490
388BUGS/LIMITATIONS 491BUGS/LIMITATIONS
492 fork with pthread backend
493 When Coro is compiled using the pthread backend (which isn't
494 recommended but required on many BSDs as their libcs are completely
495 broken), then coroutines will not survive a fork. There is no known
496 workaround except to fix your libc and use a saner backend.
497
498 perl process emulation ("threads")
389 This module is not perl-pseudo-thread-safe. You should only ever use 499 This module is not perl-pseudo-thread-safe. You should only ever use
390 this module from the same thread (this requirement might be removed in 500 this module from the first thread (this requirement might be removed
391 the future to allow per-thread schedulers, but Coro::State does not yet 501 in the future to allow per-thread schedulers, but Coro::State does
392 allow this). I recommend disabling thread support and using processes, 502 not yet allow this). I recommend disabling thread support and using
393 as this is much faster and uses less memory. 503 processes, as having the windows process emulation enabled under
504 unix roughly halves perl performance, even when not used.
505
506 coroutine switching not signal safe
507 You must not switch to another coroutine from within a signal
508 handler (only relevant with %SIG - most event libraries provide safe
509 signals).
510
511 That means you *MUST NOT* call any function that might "block" the
512 current coroutine - "cede", "schedule" "Coro::Semaphore->down" or
513 anything that calls those. Everything else, including calling
514 "ready", works.
394 515
395SEE ALSO 516SEE ALSO
396 Event-Loop integration: Coro::AnyEvent, Coro::EV, Coro::Event. 517 Event-Loop integration: Coro::AnyEvent, Coro::EV, Coro::Event.
397 518
398 Debugging: Coro::Debug. 519 Debugging: Coro::Debug.
399 520
400 Support/Utility: Coro::Specific, Coro::Util. 521 Support/Utility: Coro::Specific, Coro::Util.
401 522
402 Locking/IPC: Coro::Signal, Coro::Channel, Coro::Semaphore, 523 Locking and IPC: Coro::Signal, Coro::Channel, Coro::Semaphore,
403 Coro::SemaphoreSet, Coro::RWLock. 524 Coro::SemaphoreSet, Coro::RWLock.
404 525
405 IO/Timers: Coro::Timer, Coro::Handle, Coro::Socket, Coro::AIO. 526 I/O and Timers: Coro::Timer, Coro::Handle, Coro::Socket, Coro::AIO.
406 527
407 Compatibility: Coro::LWP, Coro::BDB, Coro::Storable, Coro::Select. 528 Compatibility with other modules: Coro::LWP (but see also AnyEvent::HTTP
529 for a better-working alternative), Coro::BDB, Coro::Storable,
530 Coro::Select.
408 531
409 XS API: Coro::MakeMaker. 532 XS API: Coro::MakeMaker.
410 533
411 Low level Configuration, Coroutine Environment: Coro::State. 534 Low level Configuration, Thread Environment, Continuations: Coro::State.
412 535
413AUTHOR 536AUTHOR
414 Marc Lehmann <schmorp@schmorp.de> 537 Marc Lehmann <schmorp@schmorp.de>
415 http://home.schmorp.de/ 538 http://home.schmorp.de/
416 539

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines