1 |
=head1 NAME |
2 |
|
3 |
Coro::Multicore - make coro threads on multiple cores with specially supported modules |
4 |
|
5 |
=head1 SYNOPSIS |
6 |
|
7 |
use Coro::Multicore; |
8 |
|
9 |
# or, if you want it disabled by default (e.g. to use it from a module) |
10 |
use Coro::Multicore (); |
11 |
|
12 |
=head1 DESCRIPTION |
13 |
|
14 |
EXPERIMENTAL WARNING: This module is in its early stages of |
15 |
development. It's fine to try out, but it didn't receive the normal amount |
16 |
of testing and real-world usage that my other modules have gone through. |
17 |
|
18 |
While L<Coro> threads (unlike ithreads) provide real threads similar to |
19 |
pthreads, python threads and so on, they do not run in parallel to each |
20 |
other even on machines with multiple CPUs or multiple CPU cores. |
21 |
|
22 |
This module lifts this restriction under two very specific but useful |
23 |
conditions: firstly, the coro thread executes in XS code and does not |
24 |
touch any perl data structures, and secondly, the XS code is specially |
25 |
prepared to allow this. |
26 |
|
27 |
This means that, when you call an XS function of a module prepared for it, |
28 |
this XS function can execute in parallel to any other Coro threads. |
29 |
|
30 |
The mechanism to support this is easily added to existing modules |
31 |
and is independent of L<Coro> or L<Coro::Multicore>, and therefore |
32 |
could be used, without changes, with other, similar, modules, or even |
33 |
the perl core, should it gain real thread support anytime soon. See |
34 |
L<http://perlmulticore.schmorp.de/> for more info on how to prepare a |
35 |
module to allow parallel execution. Preparing an existing module is easy, |
36 |
doesn't add much overhead and no dependencies. |
37 |
|
38 |
This module is an L<AnyEvent> user (and also, if not obvious, uses |
39 |
L<Coro>). |
40 |
|
41 |
=head1 HOW TO USE IT |
42 |
|
43 |
It could hardly be simpler - if you use coro threads, and before you call |
44 |
a supported lengthy operation implemented in XS, use this module and other |
45 |
coro threads can run in parallel: |
46 |
|
47 |
use Coro::Multicore; |
48 |
|
49 |
This module has no important API functions to learn or remember. All you |
50 |
need to do is I<load> it before you can take advantage of it. |
51 |
|
52 |
=head2 EXPORTS |
53 |
|
54 |
This module does not (at the moment) export any symbols. It does, however, |
55 |
export "behaviour" - if you use the default import, then Coro::Multicore |
56 |
will be enabled for all threads and all callers in the whole program: |
57 |
|
58 |
use Coro::Multicore; |
59 |
|
60 |
In a module where you don't control what else might be loaded and run, you |
61 |
might want to be more conservative, and not import anything. This has the |
62 |
effect of not enabling the functionality by default, so you have to enable |
63 |
it per scope: |
64 |
|
65 |
use Coro::Multicore (); |
66 |
|
67 |
sub myfunc { |
68 |
Coro::Multicore::scoped_enable; |
69 |
|
70 |
# from here to the end of this function, and in any functions |
71 |
# called from this function, tasks will be executed asynchronously. |
72 |
} |
73 |
|
74 |
=head1 API FUNCTIONS |
75 |
|
76 |
=over 4 |
77 |
|
78 |
=item $previous = Coro::Multicore::enable [$enable] |
79 |
|
80 |
This function enables (if C<$enable> is true) or disables (if C<$enable> |
81 |
is false) the multicore functionality globally. By default, it is enabled. |
82 |
|
83 |
This can be used to effectively disable this module's functionality by |
84 |
default, and enable it only for selected threads or scopes, by calling |
85 |
C<Coro::Multicore::scope_enable>. |
86 |
|
87 |
The function returns the previous value of the enable flag. |
88 |
|
89 |
=item Coro::Multicore::scoped_enable |
90 |
|
91 |
This function instructs Coro::Multicore to handle all requests executed |
92 |
in the current coro thread, from the call to the end of the current scope. |
93 |
|
94 |
Calls to C<scoped_enable> and C<scoped_disable> don't nest very well at |
95 |
the moment, so don't nest them. |
96 |
|
97 |
=item Coro::Multicore::scoped_disable |
98 |
|
99 |
The opposite of C<Coro::Multicore::scope_disable>: instructs Coro::Multicore to |
100 |
I<not> handle the next multicore-enabled request. |
101 |
|
102 |
=back |
103 |
|
104 |
=cut |
105 |
|
106 |
package Coro::Multicore; |
107 |
|
108 |
use Coro (); |
109 |
use AnyEvent (); |
110 |
|
111 |
BEGIN { |
112 |
our $VERSION = 0.02; |
113 |
|
114 |
use XSLoader; |
115 |
XSLoader::load __PACKAGE__, $VERSION; |
116 |
} |
117 |
|
118 |
|
119 |
sub import { |
120 |
if (@_ > 1) { |
121 |
require Carp; |
122 |
Carp::croak ("Coro::Multicore does not export any symbols"); |
123 |
} |
124 |
|
125 |
enable 1; |
126 |
} |
127 |
|
128 |
our $WATCHER = AE::io fd, 0, \&poll; |
129 |
|
130 |
=head1 INTERACTION WITH OTHER SOFTWARE |
131 |
|
132 |
This module is very similar to other environments where perl interpreters |
133 |
are moved between threads, such as mod_perl2, and the same caveats apply. |
134 |
|
135 |
I want to spell out the most important ones: |
136 |
|
137 |
=over 4 |
138 |
|
139 |
=item pthreads usage |
140 |
|
141 |
Any creation of pthreads make it impossible to fork portably from a |
142 |
perl program, as forking from within a threaded program will leave the |
143 |
program in a state similar to a signal handler. While it might work on |
144 |
some platforms (as an extension), this might also result in silent data |
145 |
corruption. It also seems to work most of the time, so it's hard to test |
146 |
for this. |
147 |
|
148 |
I recommend using something like L<AnyEvent::Fork>, which can create |
149 |
subprocesses safely (via L<Proc::FastSpawn>). |
150 |
|
151 |
Similar issues exist for signal handlers, although this module works hard |
152 |
to keep safe perl signals safe. |
153 |
|
154 |
=item module support |
155 |
|
156 |
This module moves the same perl interpreter between different |
157 |
threads. Some modules might get confused by that (although this can |
158 |
usually be considered a bug). This is a rare case though. |
159 |
|
160 |
=item event loop reliance |
161 |
|
162 |
To be able to wake up programs waiting for results, this module relies on |
163 |
an active event loop (via L<AnyEvent>). This is used to notify the perl |
164 |
interpreter when the asynchronous task is done. |
165 |
|
166 |
Since event loops typically fail to work properly after a fork, this means |
167 |
that some operations that were formerly working will now hang after fork. |
168 |
|
169 |
A workaround is to call C<Coro::Multicore::enable 0> after a fork to |
170 |
disable the module. |
171 |
|
172 |
Future versions of this module might do this automatically. |
173 |
|
174 |
=back |
175 |
|
176 |
=head1 BUGS |
177 |
|
178 |
=over 4 |
179 |
|
180 |
=item (OS-) threads are never released |
181 |
|
182 |
At the moment, threads that were created once will never be freed. They |
183 |
will be reused for asynchronous requests, though, so a slong as you limit |
184 |
the maximum number of concurrent asynchronous tasks, this will also limit |
185 |
the maximum number of threads created. |
186 |
|
187 |
Future versions will likely lift this limitation. |
188 |
|
189 |
=item AnyEvent is initalised at module load time |
190 |
|
191 |
AnyEvent is initialised on module load, as opposed to at a later time. |
192 |
|
193 |
Future versions will likely change this. |
194 |
|
195 |
=back |
196 |
|
197 |
=head1 AUTHOR |
198 |
|
199 |
Marc Lehmann <schmorp@schmorp.de> |
200 |
http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html |
201 |
|
202 |
Additional thanks to Zsbán Ambrus, who gave considerable desing input for |
203 |
this module and the perl multicore specification. |
204 |
|
205 |
=cut |
206 |
|
207 |
1 |
208 |
|