cvsroot/Coro-Multicore/Multicore.pm

=head1 NAME

Coro::Multicore - make coro threads on multiple cores with specially supported modules

=head1 SYNOPSIS

 # when you DO control the main event loop, e.g. in the main program

 use Coro::Multicore; # enable by default

 Coro::Multicore::scoped_disable;
 AE::cv->recv; # or EV::loop, AnyEvent::Loop::run, Event::loop, ...

 # when you DO NOT control the event loop, e.g. in a module on CPAN
 # do nothing (see HOW TO USE IT) or something like this:

 use Coro::Multicore (); # disable by default

 async {
    Coro::Multicore::scoped_enable;

    # blocking is safe in your own threads
    ...
 };

=head1 DESCRIPTION

EXPERIMENTAL WARNING: This module is in its early stages of
development. It's fine to try out, but it didn't receive the normal amount
of testing and real-world usage that my other modules have gone through.

While L<Coro> threads (unlike ithreads) provide real threads similar to
pthreads, python threads and so on, they do not run in parallel to each
other even on machines with multiple CPUs or multiple CPU cores.

This module lifts this restriction under two very specific but useful
conditions: firstly, the coro thread executes in XS code and does not
touch any perl data structures, and secondly, the XS code is specially
prepared to allow this.

This means that, when you call an XS function of a module prepared for it,
this XS function can execute in parallel to any other Coro threads. This
is useful for both CPU bound tasks (such as cryptography) as well as I/O
bound tasks (such as loading an image from disk). It can also be used
to do stuff in parallel via APIs that were not meant for this, such as
database accesses via DBI.

The mechanism to support this is easily added to existing modules
and is independent of L<Coro> or L<Coro::Multicore>, and therefore
could be used, without changes, with other, similar, modules, or even
the perl core, should it gain real thread support anytime soon. See
L<http://perlmulticore.schmorp.de/> for more info on how to prepare a
module to allow parallel execution. Preparing an existing module is easy,
doesn't add much overhead and no dependencies.

This module is an L<AnyEvent> user (and also, if not obvious, uses
L<Coro>).

=head1 HOW TO USE IT

Quick explanation: decide whether you control the main program/the event
loop and choose one of the two styles from the SYNOPSIS.

Longer explanation: There are two major modes this module can used in -
supported operations run asynchronously either by default, or only when
requested. The reason you might not want to enable this module for all
operations by default is compatibility with existing code:

Since this module integrates into an event loop and you must not normally
block and wait for something in an event loop callbacks. Now imagine
somebody patches your favourite module (e.g. Digest::MD5) to take
advantage of of the Perl Multicore API.

Then code that runs in an event loop callback and executes
Digest::MD5::md5 would work fine without C<Coro::Multicore> - it would
simply calculate the MD5 digest and block execution of anything else. But
with C<Coro::Multicore> enabled, the same operation would try to run other
threads. And when those wait for events, there is no event loop anymore,
as the event loop thread is busy doing the MD5 calculation, leading to a
deadlock.

=head2 USE IT IN THE MAIN PROGRAM

One way to avoid this is to not run perlmulticore enabled functions
in any callbacks. A simpler way to snure it works is to disable
C<Coro::Multicore> thread switching in event loop callbacks, and enable it
everywhere else.

Therefore, if you control the event loop, as is usually the case when
you write I<program> and not a I<module>, then you can enable C<Coro::Multicore>
by default, and disable it in your event loop thread:

   # example 1, separate thread for event loop

   use EV;
   use Coro;
   use Coro::Multicore;

   async {
      Coro::Multicore::scoped_disable;
      EV::loop;
   };

   # do something else

   # example 2, run event loop as main program

   use EV;
   use Coro;
   use Coro::Multicore;

   Coro::Multicore::scoped_disable;

   ... initialisation

   EV::loop;

The latter form is usually better and more idiomatic - the main thread is
the best place to run the event loop.

=head2 USE IT IN A MODULE

When you I<do not> control the event loop, for example, because you want
to use this from a module you published on CPAN, then the previous method
doesn't work.

However, this is not normally a problem in practise - most modules only
do work at request of the caller. In that case, you might not care
whether it does block other threads or not, as this would be the callers
responsibility (or decision), and by extension, a decision for the main
program.

So unless you use XS and want your XS functions to run asynchronously,
you don't have to worry about C<Coro::Multicore> at all - if you
happen to call XS functions that are multicore-enabled and your
caller has configured things correctly, they will automatically run
asynchronously. Or in other words: nothing needs to be done at all, which
also means that this method works fine for existing pure-perl modules,
without having to change them at all.

Only if your module runs it's own L<Coro> threads could it be an
issue - maybe your module implements some kind of job pool and relies
on certain operations to run asynchronously. Then you can still use
C<Coro::Multicore> by not enabling it be default and only enabling it in
your own threads:

   use Coro;
   use Coro::Multicore (); # note the () to disable by default

   async {
      Coro::Multicore::scoped_enable;

      # do things asynchronously by calling perlmulticore-enabled functions
   };

=head2 EXPORTS

This module does not (at the moment) export any symbols. It does, however,
export "behaviour" - if you use the default import, then Coro::Multicore
will be enabled for all threads and all callers in the whole program:

   use Coro::Multicore;

In a module where you don't control what else might be loaded and run, you
might want to be more conservative, and not import anything. This has the
effect of not enabling the functionality by default, so you have to enable
it per scope:

   use Coro::Multicore ();

   sub myfunc {
      Coro::Multicore::scoped_enable;

      # from here to the end of this function, and in any functions
      # called from this function, tasks will be executed asynchronously.
   }

=head1 API FUNCTIONS

=over 4

=item $previous = Coro::Multicore::enable [$enable]

This function enables (if C<$enable> is true) or disables (if C<$enable>
is false) the multicore functionality globally. By default, it is enabled.

This can be used to effectively disable this module's functionality by
default, and enable it only for selected threads or scopes, by calling
C<Coro::Multicore::scoped_enable>.

The function returns the previous value of the enable flag.

=item Coro::Multicore::scoped_enable

This function instructs Coro::Multicore to handle all requests executed
in the current coro thread, from the call to the end of the current scope.

Calls to C<scoped_enable> and C<scoped_disable> don't nest very well at
the moment, so don't nest them.

=item Coro::Multicore::scoped_disable

The opposite of C<Coro::Multicore::scope_disable>: instructs Coro::Multicore to
I<not> handle the next multicore-enabled request.

=back

=cut

package Coro::Multicore;

use Coro ();
use AnyEvent ();

BEGIN {
   our $VERSION = 0.03;

   use XSLoader;
   XSLoader::load __PACKAGE__, $VERSION;
}


sub import {
   if (@_ > 1) {
      require Carp;
      Carp::croak ("Coro::Multicore does not export any symbols");
   }

   enable 1;
}

our $WATCHER = AE::io fd, 0, \&poll;

=head1 THREAD SAFETY OF SUPPORTING XS MODULES

Just because an XS module supports perlmulticore might not immediately
make it reentrant. For example, while you can (try to) call C<execute>
on the same database handle for the patched C<DBD::mysql> (see the
L<registry|http://perlmulticore.schmorp.de/registry>), this will almost
certainly not work, despite C<DBD::mysql> and C<libmysqlclient> being
thread safe and reentrant - just not on the same database handle.

Many modules have limitations such as these - some can only be called
concurrently from a single thread as they use global variables, some
can only be called concurrently on different I<handles> (e.g. database
connections for DBD modules, or digest objects for Digest modules),
and some can be called at any time (such as the C<md5> function in
C<Digest::MD5>).

Generally, you only have to be careful with the very few modules that use
global variables or rely on C libraries that aren't thread-safe, which
should be documented clearly in the module documentation.

Most modules are either perfectly reentrant, or at least reentrant as long
as you give every thread it's own I<handle> object.

=head1 EXCEPTIONS AND THREAD CANCELLATION

L<Coro> allows you to cancel threads even when they execute within an XS
function (C<cancel> vs. C<cancel> methods). Similarly, L<Coro> allows you
to send exceptions (e.g. via the C<throw> method) to threads executing
inside an XS function.

While doing this is questionable and dangerous with normal Coro threads
already, they are both supported in this module, although with potentially
unwanted effects. The following describes the current implementation and
is subject to change. It is described primarily so you can understand what
went wrong, if things go wrong.

=over 4

=item EXCEPTIONS

When a thread that has currently released the perl interpreter (e.g.
because it is executing a perlmulticore enabled XS function) receives an exception, it will
at first continue normally.

After acquiring the perl interpreter again, it will throw the
exception it previously received. More specifically, when a thread
calls C<perlinterp_acquire ()> and has received an exception, then
C<perlinterp_acquire ()> will not return but instead C<die>.

Most code that has been updated for perlmulticore support will not expect
this, and might leave internal state corrupted to some extent.

=item CANCELLATION

Unsafe cancellation on a thread that has released the perl interpreter
frees its resources, but let's the XS code continue at first. This should
not lead to corruption on the perl level, as the code isn't allowed to
touch perl data structures until it reacquires the interpreter.

The call to C<perlinterp_acquire ()> will then block indefinitely, leaking
the (OS level) thread.

Safe cancellation will simply fail in this case, so is still "safe" to
call.

=back

=head1 INTERACTION WITH OTHER SOFTWARE

This module is very similar to other environments where perl interpreters
are moved between threads, such as mod_perl2, and the same caveats apply.

I want to spell out the most important ones:

=over 4

=item pthreads usage

Any creation of pthreads make it impossible to fork portably from a
perl program, as forking from within a threaded program will leave the
program in a state similar to a signal handler. While it might work on
some platforms (as an extension), this might also result in silent data
corruption. It also seems to work most of the time, so it's hard to test
for this.

I recommend using something like L<AnyEvent::Fork>, which can create
subprocesses safely (via L<Proc::FastSpawn>).

Similar issues exist for signal handlers, although this module works hard
to keep safe perl signals safe.

=item module support

This module moves the same perl interpreter between different
threads. Some modules might get confused by that (although this can
usually be considered a bug). This is a rare case though.

=item event loop reliance

To be able to wake up programs waiting for results, this module relies on
an active event loop (via L<AnyEvent>). This is used to notify the perl
interpreter when the asynchronous task is done.

Since event loops typically fail to work properly after a fork, this means
that some operations that were formerly working will now hang after fork.

A workaround is to call C<Coro::Multicore::enable 0> after a fork to
disable the module.

Future versions of this module might do this automatically.

=back

=head1 BUGS

=over 4

=item (OS-) threads are never released

At the moment, threads that were created once will never be freed. They
will be reused for asynchronous requests, though, so as long as you limit
the maximum number of concurrent asynchronous tasks, this will also limit
the maximum number of threads created.

The idle threads are not necessarily using a lot of resources: on
GNU/Linux + glibc, each thread takes about 8KiB of userspace memory +
whatever the kernel needs (probably less than 8KiB).

Future versions will likely lift this limitation.

=item AnyEvent is initalised at module load time

AnyEvent is initialised on module load, as opposed to at a later time.

Future versions will likely change this.

=back

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html

Additional thanks to Zsbán Ambrus, who gave considerable desing input for
this module and the perl multicore specification.

=cut

1

Revision:	1.10
Committed:	Mon Jul 27 14:32:33 2015 UTC (8 years, 11 months ago) by root
Branch:	MAIN
Changes since 1.9:	+167 -9 lines
Log Message:	* empty log message *
#	Content
1	=head1 NAME
2
3	Coro::Multicore - make coro threads on multiple cores with specially supported modules
4
5	=head1 SYNOPSIS
6
7	# when you DO control the main event loop, e.g. in the main program
8
9	use Coro::Multicore; # enable by default
10
11	Coro::Multicore::scoped_disable;
12	AE::cv->recv; # or EV::loop, AnyEvent::Loop::run, Event::loop, ...
13
14	# when you DO NOT control the event loop, e.g. in a module on CPAN
15	# do nothing (see HOW TO USE IT) or something like this:
16
17	use Coro::Multicore (); # disable by default
18
19	async {
20	Coro::Multicore::scoped_enable;
21
22	# blocking is safe in your own threads
23	...
24	};
25
26	=head1 DESCRIPTION
27
28	EXPERIMENTAL WARNING: This module is in its early stages of
29	development. It's fine to try out, but it didn't receive the normal amount
30	of testing and real-world usage that my other modules have gone through.
31
32	While L<Coro> threads (unlike ithreads) provide real threads similar to
33	pthreads, python threads and so on, they do not run in parallel to each
34	other even on machines with multiple CPUs or multiple CPU cores.
35
36	This module lifts this restriction under two very specific but useful
37	conditions: firstly, the coro thread executes in XS code and does not
38	touch any perl data structures, and secondly, the XS code is specially
39	prepared to allow this.
40
41	This means that, when you call an XS function of a module prepared for it,
42	this XS function can execute in parallel to any other Coro threads. This
43	is useful for both CPU bound tasks (such as cryptography) as well as I/O
44	bound tasks (such as loading an image from disk). It can also be used
45	to do stuff in parallel via APIs that were not meant for this, such as
46	database accesses via DBI.
47
48	The mechanism to support this is easily added to existing modules
49	and is independent of L<Coro> or L<Coro::Multicore>, and therefore
50	could be used, without changes, with other, similar, modules, or even
51	the perl core, should it gain real thread support anytime soon. See
52	L<http://perlmulticore.schmorp.de/> for more info on how to prepare a
53	module to allow parallel execution. Preparing an existing module is easy,
54	doesn't add much overhead and no dependencies.
55
56	This module is an L<AnyEvent> user (and also, if not obvious, uses
57	L<Coro>).
58
59	=head1 HOW TO USE IT
60
61	Quick explanation: decide whether you control the main program/the event
62	loop and choose one of the two styles from the SYNOPSIS.
63
64	Longer explanation: There are two major modes this module can used in -
65	supported operations run asynchronously either by default, or only when
66	requested. The reason you might not want to enable this module for all
67	operations by default is compatibility with existing code:
68
69	Since this module integrates into an event loop and you must not normally
70	block and wait for something in an event loop callbacks. Now imagine
71	somebody patches your favourite module (e.g. Digest::MD5) to take
72	advantage of of the Perl Multicore API.
73
74	Then code that runs in an event loop callback and executes
75	Digest::MD5::md5 would work fine without C<Coro::Multicore> - it would
76	simply calculate the MD5 digest and block execution of anything else. But
77	with C<Coro::Multicore> enabled, the same operation would try to run other
78	threads. And when those wait for events, there is no event loop anymore,
79	as the event loop thread is busy doing the MD5 calculation, leading to a
80	deadlock.
81
82	=head2 USE IT IN THE MAIN PROGRAM
83
84	One way to avoid this is to not run perlmulticore enabled functions
85	in any callbacks. A simpler way to snure it works is to disable
86	C<Coro::Multicore> thread switching in event loop callbacks, and enable it
87	everywhere else.
88
89	Therefore, if you control the event loop, as is usually the case when
90	you write I<program> and not a I<module>, then you can enable C<Coro::Multicore>
91	by default, and disable it in your event loop thread:
92
93	# example 1, separate thread for event loop
94
95	use EV;
96	use Coro;
97	use Coro::Multicore;
98
99	async {
100	Coro::Multicore::scoped_disable;
101	EV::loop;
102	};
103
104	# do something else
105
106	# example 2, run event loop as main program
107
108	use EV;
109	use Coro;
110	use Coro::Multicore;
111
112	Coro::Multicore::scoped_disable;
113
114	... initialisation
115
116	EV::loop;
117
118	The latter form is usually better and more idiomatic - the main thread is
119	the best place to run the event loop.
120
121	=head2 USE IT IN A MODULE
122
123	When you I<do not> control the event loop, for example, because you want
124	to use this from a module you published on CPAN, then the previous method
125	doesn't work.
126
127	However, this is not normally a problem in practise - most modules only
128	do work at request of the caller. In that case, you might not care
129	whether it does block other threads or not, as this would be the callers
130	responsibility (or decision), and by extension, a decision for the main
131	program.
132
133	So unless you use XS and want your XS functions to run asynchronously,
134	you don't have to worry about C<Coro::Multicore> at all - if you
135	happen to call XS functions that are multicore-enabled and your
136	caller has configured things correctly, they will automatically run
137	asynchronously. Or in other words: nothing needs to be done at all, which
138	also means that this method works fine for existing pure-perl modules,
139	without having to change them at all.
140
141	Only if your module runs it's own L<Coro> threads could it be an
142	issue - maybe your module implements some kind of job pool and relies
143	on certain operations to run asynchronously. Then you can still use
144	C<Coro::Multicore> by not enabling it be default and only enabling it in
145	your own threads:
146
147	use Coro;
148	use Coro::Multicore (); # note the () to disable by default
149
150	async {
151	Coro::Multicore::scoped_enable;
152
153	# do things asynchronously by calling perlmulticore-enabled functions
154	};
155
156	=head2 EXPORTS
157
158	This module does not (at the moment) export any symbols. It does, however,
159	export "behaviour" - if you use the default import, then Coro::Multicore
160	will be enabled for all threads and all callers in the whole program:
161
162	use Coro::Multicore;
163
164	In a module where you don't control what else might be loaded and run, you
165	might want to be more conservative, and not import anything. This has the
166	effect of not enabling the functionality by default, so you have to enable
167	it per scope:
168
169	use Coro::Multicore ();
170
171	sub myfunc {
172	Coro::Multicore::scoped_enable;
173
174	# from here to the end of this function, and in any functions
175	# called from this function, tasks will be executed asynchronously.
176	}
177
178	=head1 API FUNCTIONS
179
180	=over 4
181
182	=item $previous = Coro::Multicore::enable [$enable]
183
184	This function enables (if C<$enable> is true) or disables (if C<$enable>
185	is false) the multicore functionality globally. By default, it is enabled.
186
187	This can be used to effectively disable this module's functionality by
188	default, and enable it only for selected threads or scopes, by calling
189	C<Coro::Multicore::scoped_enable>.
190
191	The function returns the previous value of the enable flag.
192
193	=item Coro::Multicore::scoped_enable
194
195	This function instructs Coro::Multicore to handle all requests executed
196	in the current coro thread, from the call to the end of the current scope.
197
198	Calls to C<scoped_enable> and C<scoped_disable> don't nest very well at
199	the moment, so don't nest them.
200
201	=item Coro::Multicore::scoped_disable
202
203	The opposite of C<Coro::Multicore::scope_disable>: instructs Coro::Multicore to
204	I<not> handle the next multicore-enabled request.
205
206	=back
207
208	=cut
209
210	package Coro::Multicore;
211
212	use Coro ();
213	use AnyEvent ();
214
215	BEGIN {
216	our $VERSION = 0.03;
217
218	use XSLoader;
219	XSLoader::load __PACKAGE__, $VERSION;
220	}
221
222
223	sub import {
224	if (@_ > 1) {
225	require Carp;
226	Carp::croak ("Coro::Multicore does not export any symbols");
227	}
228
229	enable 1;
230	}
231
232	our $WATCHER = AE::io fd, 0, \&poll;
233
234	=head1 THREAD SAFETY OF SUPPORTING XS MODULES
235
236	Just because an XS module supports perlmulticore might not immediately
237	make it reentrant. For example, while you can (try to) call C<execute>
238	on the same database handle for the patched C<DBD::mysql> (see the
239	L<registry\|http://perlmulticore.schmorp.de/registry>), this will almost
240	certainly not work, despite C<DBD::mysql> and C<libmysqlclient> being
241	thread safe and reentrant - just not on the same database handle.
242
243	Many modules have limitations such as these - some can only be called
244	concurrently from a single thread as they use global variables, some
245	can only be called concurrently on different I<handles> (e.g. database
246	connections for DBD modules, or digest objects for Digest modules),
247	and some can be called at any time (such as the C<md5> function in
248	C<Digest::MD5>).
249
250	Generally, you only have to be careful with the very few modules that use
251	global variables or rely on C libraries that aren't thread-safe, which
252	should be documented clearly in the module documentation.
253
254	Most modules are either perfectly reentrant, or at least reentrant as long
255	as you give every thread it's own I<handle> object.
256
257	=head1 EXCEPTIONS AND THREAD CANCELLATION
258
259	L<Coro> allows you to cancel threads even when they execute within an XS
260	function (C<cancel> vs. C<cancel> methods). Similarly, L<Coro> allows you
261	to send exceptions (e.g. via the C<throw> method) to threads executing
262	inside an XS function.
263
264	While doing this is questionable and dangerous with normal Coro threads
265	already, they are both supported in this module, although with potentially
266	unwanted effects. The following describes the current implementation and
267	is subject to change. It is described primarily so you can understand what
268	went wrong, if things go wrong.
269
270	=over 4
271
272	=item EXCEPTIONS
273
274	When a thread that has currently released the perl interpreter (e.g.
275	because it is executing a perlmulticore enabled XS function) receives an exception, it will
276	at first continue normally.
277
278	After acquiring the perl interpreter again, it will throw the
279	exception it previously received. More specifically, when a thread
280	calls C<perlinterp_acquire ()> and has received an exception, then
281	C<perlinterp_acquire ()> will not return but instead C<die>.
282
283	Most code that has been updated for perlmulticore support will not expect
284	this, and might leave internal state corrupted to some extent.
285
286	=item CANCELLATION
287
288	Unsafe cancellation on a thread that has released the perl interpreter
289	frees its resources, but let's the XS code continue at first. This should
290	not lead to corruption on the perl level, as the code isn't allowed to
291	touch perl data structures until it reacquires the interpreter.
292
293	The call to C<perlinterp_acquire ()> will then block indefinitely, leaking
294	the (OS level) thread.
295
296	Safe cancellation will simply fail in this case, so is still "safe" to
297	call.
298
299	=back
300
301	=head1 INTERACTION WITH OTHER SOFTWARE
302
303	This module is very similar to other environments where perl interpreters
304	are moved between threads, such as mod_perl2, and the same caveats apply.
305
306	I want to spell out the most important ones:
307
308	=over 4
309
310	=item pthreads usage
311
312	Any creation of pthreads make it impossible to fork portably from a
313	perl program, as forking from within a threaded program will leave the
314	program in a state similar to a signal handler. While it might work on
315	some platforms (as an extension), this might also result in silent data
316	corruption. It also seems to work most of the time, so it's hard to test
317	for this.
318
319	I recommend using something like L<AnyEvent::Fork>, which can create
320	subprocesses safely (via L<Proc::FastSpawn>).
321
322	Similar issues exist for signal handlers, although this module works hard
323	to keep safe perl signals safe.
324
325	=item module support
326
327	This module moves the same perl interpreter between different
328	threads. Some modules might get confused by that (although this can
329	usually be considered a bug). This is a rare case though.
330
331	=item event loop reliance
332
333	To be able to wake up programs waiting for results, this module relies on
334	an active event loop (via L<AnyEvent>). This is used to notify the perl
335	interpreter when the asynchronous task is done.
336
337	Since event loops typically fail to work properly after a fork, this means
338	that some operations that were formerly working will now hang after fork.
339
340	A workaround is to call C<Coro::Multicore::enable 0> after a fork to
341	disable the module.
342
343	Future versions of this module might do this automatically.
344
345	=back
346
347	=head1 BUGS
348
349	=over 4
350
351	=item (OS-) threads are never released
352
353	At the moment, threads that were created once will never be freed. They
354	will be reused for asynchronous requests, though, so as long as you limit
355	the maximum number of concurrent asynchronous tasks, this will also limit
356	the maximum number of threads created.
357
358	The idle threads are not necessarily using a lot of resources: on
359	GNU/Linux + glibc, each thread takes about 8KiB of userspace memory +
360	whatever the kernel needs (probably less than 8KiB).
361
362	Future versions will likely lift this limitation.
363
364	=item AnyEvent is initalised at module load time
365
366	AnyEvent is initialised on module load, as opposed to at a later time.
367
368	Future versions will likely change this.
369
370	=back
371
372	=head1 AUTHOR
373
374	Marc Lehmann <schmorp@schmorp.de>
375	http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html
376
377	Additional thanks to Zsbán Ambrus, who gave considerable desing input for
378	this module and the perl multicore specification.
379
380	=cut
381
382	1
383