Coro/Coro/State.pm

=head1 NAME

Coro::State - create and manage simple coroutines

=head1 SYNOPSIS

 use Coro::State;

 $new = new Coro::State sub {
    print "in coroutine (called with @_), switching back\n";
    $new->transfer ($main);
    print "in coroutine again, switching back\n";
    $new->transfer ($main);
 }, 5;

 $main = new Coro::State;

 print "in main, switching to coroutine\n";
 $main->transfer ($new);
 print "back in main, switch to coroutine again\n";
 $main->transfer ($new);
 print "back in main\n";

=head1 DESCRIPTION

This module implements coroutines. Coroutines, similar to continuations,
allow you to run more than one "thread of execution" in parallel. Unlike
threads, there is no parallelism and only voluntary switching is used so
locking problems are greatly reduced.

This can be used to implement non-local jumps, exception handling,
(non-clonable) continuations and more.

This module provides only low-level functionality. See L<Coro> and related
modules for a higher level process abstraction including scheduling.

=head2 MODEL

Coro::State implements two different coroutine models: Perl and C. The
C coroutines (called cctx's) are basically simplified perl interpreters
running/interpreting the Perl coroutines. A single interpreter can run any
number of Perl coroutines, so usually there are very few C coroutines.

When Perl code calls a C function (e.g. in an extension module) and that
C function then calls back into Perl or does a coroutine switch the C
coroutine can no longer execute other Perl coroutines, so it stays tied to
the specific coroutine until it returns to the original Perl caller, after
which it is again avaikable to run other Perl coroutines.

The main program always has its own "C coroutine" (which really is
*the* Perl interpreter running the whole program), so there will always
be at least one additional C coroutine. You can use the debugger (see
L<Coro::Debug>) to find out which coroutines are tied to their cctx and
which aren't.

=head2 MEMORY CONSUMPTION

A newly created coroutine that has not been used only allocates a
relatively small (a hundred bytes) structure. Only on the first
C<transfer> will perl allocate stacks (a few kb, 64 bit architetcures
use twice as much, i.e. a few kb :) and optionally a C stack/coroutine
(cctx) for coroutines that recurse through C functions. All this is very
system-dependent. On my x86-pc-linux-gnu system this amounts to about 2k
per (non-trivial but simple) coroutine.

You can view the actual memory consumption using Coro::Debug. Keep in mind
that a for loop or other block constructs can easily consume 100-200 bytes
per nesting level.

=cut

package Coro::State;

use strict;
no warnings "uninitialized";

use Carp;

use Time::HiRes (); # currently only used for PerlIO::cede

our $DIEHOOK;
our $WARNHOOK;

BEGIN {
   $DIEHOOK  = sub { };
   $WARNHOOK = sub { warn $_[0] };
}

sub diehook  { &$DIEHOOK  }
sub warnhook { &$WARNHOOK }

use XSLoader;

BEGIN {
   our $VERSION = 4.91;

   # must be done here because the xs part expects it to exist
   # it might exist already because Coro::Specific created it.
   $Coro::current ||= { };

   {
      # save/restore the handlers before/after overwriting %SIG magic
      local $SIG{__DIE__};
      local $SIG{__WARN__};

      XSLoader::load __PACKAGE__, $VERSION;
   }

   # need to do it after overwriting the %SIG magic
   $SIG{__DIE__}  ||= \&diehook;
   $SIG{__WARN__} ||= \&warnhook;
}

use Exporter;
use base Exporter::;

=head2 GLOBAL VARIABLES

=over 4

=item $Coro::State::DIEHOOK

This works similarly to C<$SIG{__DIE__}> and is used as the default die
hook for newly created coroutines. This is useful if you want some generic
logging function that works for all coroutines that don't set their own
hook.

When Coro::State is first loaded it will install these handlers for the
main program, too, unless they have been overwritten already.

The default handlers provided will behave like the built-in ones (as if
they weren't there).

Note 1: You I<must> store a valid code reference in these variables,
C<undef> will I<not> do.

Note 2: The value of this variable will be shared among all coroutines, so
changing its value will change it in all coroutines that don't have their
own die handler.

=item $Coro::State::WARNHOOK

Similar to above die hook, but augments C<$SIG{__WARN__}>.

=back

=head2 FUNCTIONS

=over 4

=item $coro = new Coro::State [$coderef[, @args...]]

Create a new coroutine and return it. The first C<transfer> call to this
coroutine will start execution at the given coderef.

If the subroutine returns the program will be terminated as if execution
of the main program ended.

If it throws an exception the program will terminate unless the exception
is caught, exactly like in the main program.

Calling C<exit> in a coroutine does the same as calling it in the main
program.

If the coderef is omitted this function will create a new "empty"
coroutine, i.e. a coroutine that cannot be transfered to but can be used
to save the current coroutine state in (note that this is dangerous, as no
reference is taken to ensure that the "current coroutine state" survives,
the caller is responsible to ensure that the cloned state does not go
away).

The returned object is an empty hash which can be used for any purpose
whatsoever, for example when subclassing Coro::State.

Certain variables are "localised" to each coroutine, that is, certain
"global" variables are actually per coroutine. Not everything that would
sensibly be localised currently is, and not everything that is localised
makes sense for every application, and the future might bring changes.

The following global variables can have different values per coroutine,
and have the stated initial values:

   Variable       Initial Value
   @_             whatever arguments were passed to the Coro
   $_             undef
   $@             undef
   $/             "\n"
   $SIG{__DIE__}  aliased to $Coro::State::DIEHOOK(*)
   $SIG{__WARN__} aliased to $Coro::State::WARNHOOK(*)
   (default fh)   *STDOUT
   $1, $2...      all regex results are initially undefined

   (*) reading the value from %SIG is not supported, but local'ising is.

If you feel that something important is missing then tell me. Also
remember that every function call that might call C<transfer> (such
as C<Coro::Channel::put>) might clobber any global and/or special
variables. Yes, this is by design ;) You can always create your own
process abstraction model that saves these variables.

The easiest way to do this is to create your own scheduling primitive like
in the code below, and use it in your coroutines:

  sub my_cede {
     local ($;, ...);
     Coro::cede;
  }

=cut

# this is called for each newly created C coroutine,
# and is being artificially injected into the opcode flow.
# its sole purpose is to call transfer() once so it knows
# the top level stack frame for stack sharing.
sub _cctx_init {
   &_set_stacklevel;
}

=item $state->throw ([$scalar])

See L<< Coro->throw >>.

=item $state->call ($coderef)

Try to call the given C<$coderef> in the context of the given state. This
works even when the state is currently within an XS function, and can
be very dangerous. You can use it to acquire stack traces etc. (see the
Coro::Debug module for more details). The coderef MUST NOT EVER transfer
to another state.

=item $state->eval ($string)

Like C<call>, but eval's the string. Dangerous.

=item $state->swap_defsv

=item $state->swap_defav

Swap the current C<$_> (swap_defsv) or C<@_> (swap_defav) with the
equivalent in the saved state of C<$state>. This can be used to give the
coroutine a defined content for C<@_> and C<$_> before transfer'ing to it.

=item $state->trace ($flags)

Internal function to control tracing. I just mention this so you can stay
away from abusing it.

=item $prev->transfer ($next)

Save the state of the current subroutine in C<$prev> and switch to the
coroutine saved in C<$next>.

The "state" of a subroutine includes the scope, i.e. lexical variables and
the current execution state (subroutine, stack).

=item $bytes = $state->rss

Returns the memory allocated by the coroutine (which includes static
structures, various perl stacks but NOT local variables, arguments or any
C context data). This is a rough indication of how much memory it might
use.

=item $state->has_cctx

Returns whether the state currently uses a cctx/C context. An active
state always has a cctx, as well as the main program. Other states only
use a cctxts when needed.

=item Coro::State::force_cctx

Forces the allocation of a C context for the currently running coroutine
(if not already done). Apart from benchmarking there is little point
in doing so, however.

=item $ncctx = Coro::State::cctx_count

Returns the number of C-level coroutines allocated. If this number is
very high (more than a dozen) it might help to identify points of C-level
recursion in your code and moving this into a separate coroutine.

=item $nidle = Coro::State::cctx_idle

Returns the number of allocated but idle (free for reuse) C level
coroutines. Currently, Coro will limit the number of idle/unused cctxs to
8.

=item $old = Coro::State::cctx_stacksize [$new_stacksize]

Returns the current C stack size and optionally sets the new I<minimum>
stack size to C<$new_stacksize> I<long>s. Existing stacks will not
be changed, but Coro will try to replace smaller stacks as soon as
possible. Any Coro::State that starts to use a stack after this call is
guaranteed this minimum stack size.

Please note that Coroutines will only need to use a C-level stack if the
interpreter recurses or calls a function in a module that calls back into
the interpreter, so use of this feature is usually never needed.

=item $old = Coro::State::cctx_max_idle [$new_count]

Coro caches C contexts that are not in use currently, as creating them
from scratch has some overhead.

This function returns the current maximum number of idle C contexts and
optionally sets the new amount. The count must be at least C<1>, with the
default being C<4>.

=item @states = Coro::State::list

Returns a list of all states currently allocated.

=cut

# used by Coro::Debug only atm.
sub debug_desc {
   $_[0]{desc}
}

1;

=back

=head1 BUGS

This module is not thread-safe. You must only ever use this module from
the same thread (this requirement might be removed in the future).

=head1 SEE ALSO

L<Coro>.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

Revision:	1.119
Committed:	Mon Nov 10 04:37:23 2008 UTC (15 years, 7 months ago) by root
Branch:	MAIN
CVS Tags:	rel-4_91, rel-4_901, rel-4_911
Changes since 1.118:	+7 -3 lines
Log Message:	4.91
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Coro::State - create and manage simple coroutines
4
5			=head1 SYNOPSIS
6
7			use Coro::State;
8
9			$new = new Coro::State sub {
10	root	1.3	print "in coroutine (called with @_), switching back\n";
11	root	1.43	$new->transfer ($main);
12	root	1.1	print "in coroutine again, switching back\n";
13	root	1.43	$new->transfer ($main);
14	root	1.3	}, 5;
15	root	1.1
16			$main = new Coro::State;
17
18			print "in main, switching to coroutine\n";
19	root	1.43	$main->transfer ($new);
20	root	1.1	print "back in main, switch to coroutine again\n";
21	root	1.43	$main->transfer ($new);
22	root	1.1	print "back in main\n";
23
24			=head1 DESCRIPTION
25
26			This module implements coroutines. Coroutines, similar to continuations,
27			allow you to run more than one "thread of execution" in parallel. Unlike
28	root	1.42	threads, there is no parallelism and only voluntary switching is used so
29			locking problems are greatly reduced.
30
31			This can be used to implement non-local jumps, exception handling,
32	root	1.94	(non-clonable) continuations and more.
33	root	1.1
34			This module provides only low-level functionality. See L<Coro> and related
35	root	1.42	modules for a higher level process abstraction including scheduling.
36	root	1.1
37	root	1.86	=head2 MODEL
38
39			Coro::State implements two different coroutine models: Perl and C. The
40			C coroutines (called cctx's) are basically simplified perl interpreters
41			running/interpreting the Perl coroutines. A single interpreter can run any
42			number of Perl coroutines, so usually there are very few C coroutines.
43
44			When Perl code calls a C function (e.g. in an extension module) and that
45			C function then calls back into Perl or does a coroutine switch the C
46			coroutine can no longer execute other Perl coroutines, so it stays tied to
47			the specific coroutine until it returns to the original Perl caller, after
48			which it is again avaikable to run other Perl coroutines.
49
50			The main program always has its own "C coroutine" (which really is
51			the Perl interpreter running the whole program), so there will always
52	root	1.94	be at least one additional C coroutine. You can use the debugger (see
53	root	1.86	L<Coro::Debug>) to find out which coroutines are tied to their cctx and
54			which aren't.
55
56	root	1.9	=head2 MEMORY CONSUMPTION
57
58			A newly created coroutine that has not been used only allocates a
59	root	1.84	relatively small (a hundred bytes) structure. Only on the first
60	root	1.94	C<transfer> will perl allocate stacks (a few kb, 64 bit architetcures
61			use twice as much, i.e. a few kb :) and optionally a C stack/coroutine
62			(cctx) for coroutines that recurse through C functions. All this is very
63			system-dependent. On my x86-pc-linux-gnu system this amounts to about 2k
64			per (non-trivial but simple) coroutine.
65
66			You can view the actual memory consumption using Coro::Debug. Keep in mind
67			that a for loop or other block constructs can easily consume 100-200 bytes
68			per nesting level.
69	root	1.1
70			=cut
71
72			package Coro::State;
73
74	root	1.47	use strict;
75			no warnings "uninitialized";
76
77	root	1.84	use Carp;
78	root	1.87
79	root	1.109	use Time::HiRes (); # currently only used for PerlIO::cede
80
81	root	1.87	our $DIEHOOK;
82			our $WARNHOOK;
83
84			BEGIN {
85			$DIEHOOK = sub { };
86			$WARNHOOK = sub { warn $_[0] };
87			}
88
89			sub diehook { &$DIEHOOK }
90			sub warnhook { &$WARNHOOK }
91	root	1.84
92	root	1.47	use XSLoader;
93	root	1.18
94	root	1.1	BEGIN {
95	root	1.119	our $VERSION = 4.91;
96	root	1.1
97	root	1.67	# must be done here because the xs part expects it to exist
98			# it might exist already because Coro::Specific created it.
99			$Coro::current \|\|= { };
100
101	root	1.101	{
102			# save/restore the handlers before/after overwriting %SIG magic
103			local $SIG{__DIE__};
104			local $SIG{__WARN__};
105
106			XSLoader::load __PACKAGE__, $VERSION;
107			}
108
109			# need to do it after overwriting the %SIG magic
110			$SIG{__DIE__} \|\|= \&diehook;
111			$SIG{__WARN__} \|\|= \&warnhook;
112	root	1.1	}
113
114	root	1.51	use Exporter;
115	root	1.47	use base Exporter::;
116	root	1.5
117	root	1.84	=head2 GLOBAL VARIABLES
118
119			=over 4
120
121			=item $Coro::State::DIEHOOK
122
123			This works similarly to C<$SIG{__DIE__}> and is used as the default die
124			hook for newly created coroutines. This is useful if you want some generic
125			logging function that works for all coroutines that don't set their own
126			hook.
127
128			When Coro::State is first loaded it will install these handlers for the
129	root	1.101	main program, too, unless they have been overwritten already.
130	root	1.84
131	root	1.100	The default handlers provided will behave like the built-in ones (as if
132	root	1.84	they weren't there).
133
134	root	1.94	Note 1: You I<must> store a valid code reference in these variables,
135			C<undef> will I<not> do.
136	root	1.84
137	root	1.89	Note 2: The value of this variable will be shared among all coroutines, so
138	root	1.100	changing its value will change it in all coroutines that don't have their
139			own die handler.
140	root	1.84
141			=item $Coro::State::WARNHOOK
142
143			Similar to above die hook, but augments C<$SIG{__WARN__}>.
144
145			=back
146
147			=head2 FUNCTIONS
148
149			=over 4
150
151	root	1.65	=item $coro = new Coro::State [$coderef[, @args...]]
152	root	1.1
153			Create a new coroutine and return it. The first C<transfer> call to this
154	root	1.94	coroutine will start execution at the given coderef.
155
156			If the subroutine returns the program will be terminated as if execution
157			of the main program ended.
158
159			If it throws an exception the program will terminate unless the exception
160			is caught, exactly like in the main program.
161	root	1.74
162			Calling C<exit> in a coroutine does the same as calling it in the main
163			program.
164	root	1.1
165			If the coderef is omitted this function will create a new "empty"
166			coroutine, i.e. a coroutine that cannot be transfered to but can be used
167	root	1.104	to save the current coroutine state in (note that this is dangerous, as no
168			reference is taken to ensure that the "current coroutine state" survives,
169			the caller is responsible to ensure that the cloned state does not go
170			away).
171	root	1.1
172	root	1.55	The returned object is an empty hash which can be used for any purpose
173			whatsoever, for example when subclassing Coro::State.
174
175	root	1.79	Certain variables are "localised" to each coroutine, that is, certain
176			"global" variables are actually per coroutine. Not everything that would
177			sensibly be localised currently is, and not everything that is localised
178			makes sense for every application, and the future might bring changes.
179	root	1.1
180	root	1.84	The following global variables can have different values per coroutine,
181			and have the stated initial values:
182	root	1.5
183	root	1.83	Variable Initial Value
184			@_ whatever arguments were passed to the Coro
185			$_ undef
186			$@ undef
187			$/ "\n"
188	root	1.88	$SIG{__DIE__} aliased to $Coro::State::DIEHOOK(*)
189			$SIG{__WARN__} aliased to $Coro::State::WARNHOOK(*)
190	root	1.83	(default fh) *STDOUT
191	root	1.84	$1, $2... all regex results are initially undefined
192	root	1.2
193	root	1.88	(*) reading the value from %SIG is not supported, but local'ising is.
194
195	root	1.70	If you feel that something important is missing then tell me. Also
196	root	1.2	remember that every function call that might call C<transfer> (such
197			as C<Coro::Channel::put>) might clobber any global and/or special
198			variables. Yes, this is by design ;) You can always create your own
199			process abstraction model that saves these variables.
200	root	1.1
201	root	1.9	The easiest way to do this is to create your own scheduling primitive like
202	root	1.94	in the code below, and use it in your coroutines:
203	root	1.1
204	root	1.84	sub my_cede {
205	root	1.80	local ($;, ...);
206	root	1.84	Coro::cede;
207	root	1.1	}
208
209	root	1.79	=cut
210
211			# this is called for each newly created C coroutine,
212			# and is being artificially injected into the opcode flow.
213			# its sole purpose is to call transfer() once so it knows
214	root	1.119	# the top level stack frame for stack sharing.
215	root	1.79	sub _cctx_init {
216	root	1.119	&_set_stacklevel;
217	root	1.79	}
218
219	root	1.119	=item $state->throw ([$scalar])
220
221			See L<< Coro->throw >>.
222
223	root	1.76	=item $state->call ($coderef)
224
225	root	1.94	Try to call the given C<$coderef> in the context of the given state. This
226	root	1.76	works even when the state is currently within an XS function, and can
227			be very dangerous. You can use it to acquire stack traces etc. (see the
228			Coro::Debug module for more details). The coderef MUST NOT EVER transfer
229			to another state.
230
231			=item $state->eval ($string)
232
233	root	1.94	Like C<call>, but eval's the string. Dangerous.
234	root	1.76
235	root	1.90	=item $state->swap_defsv
236
237			=item $state->swap_defav
238
239			Swap the current C<$_> (swap_defsv) or C<@_> (swap_defav) with the
240			equivalent in the saved state of C<$state>. This can be used to give the
241			coroutine a defined content for C<@_> and C<$_> before transfer'ing to it.
242
243	root	1.77	=item $state->trace ($flags)
244
245			Internal function to control tracing. I just mention this so you can stay
246	root	1.90	away from abusing it.
247	root	1.77
248	root	1.70	=item $prev->transfer ($next)
249
250			Save the state of the current subroutine in C<$prev> and switch to the
251			coroutine saved in C<$next>.
252
253			The "state" of a subroutine includes the scope, i.e. lexical variables and
254			the current execution state (subroutine, stack).
255
256	root	1.117	=item $bytes = $state->rss
257
258			Returns the memory allocated by the coroutine (which includes static
259			structures, various perl stacks but NOT local variables, arguments or any
260			C context data). This is a rough indication of how much memory it might
261			use.
262
263	root	1.94	=item $state->has_cctx
264
265	root	1.117	Returns whether the state currently uses a cctx/C context. An active
266	root	1.94	state always has a cctx, as well as the main program. Other states only
267			use a cctxts when needed.
268
269	root	1.117	=item Coro::State::force_cctx
270	root	1.94
271	root	1.117	Forces the allocation of a C context for the currently running coroutine
272			(if not already done). Apart from benchmarking there is little point
273			in doing so, however.
274	root	1.94
275	root	1.117	=item $ncctx = Coro::State::cctx_count
276	root	1.64
277			Returns the number of C-level coroutines allocated. If this number is
278			very high (more than a dozen) it might help to identify points of C-level
279			recursion in your code and moving this into a separate coroutine.
280
281	root	1.117	=item $nidle = Coro::State::cctx_idle
282	root	1.64
283			Returns the number of allocated but idle (free for reuse) C level
284	root	1.76	coroutines. Currently, Coro will limit the number of idle/unused cctxs to
285			8.
286	root	1.64
287	root	1.117	=item $old = Coro::State::cctx_stacksize [$new_stacksize]
288	root	1.72
289			Returns the current C stack size and optionally sets the new I<minimum>
290			stack size to C<$new_stacksize> I<long>s. Existing stacks will not
291			be changed, but Coro will try to replace smaller stacks as soon as
292	root	1.91	possible. Any Coro::State that starts to use a stack after this call is
293	root	1.94	guaranteed this minimum stack size.
294
295			Please note that Coroutines will only need to use a C-level stack if the
296			interpreter recurses or calls a function in a module that calls back into
297			the interpreter, so use of this feature is usually never needed.
298	root	1.72
299	root	1.117	=item $old = Coro::State::cctx_max_idle [$new_count]
300
301			Coro caches C contexts that are not in use currently, as creating them
302			from scratch has some overhead.
303	root	1.92
304	root	1.117	This function returns the current maximum number of idle C contexts and
305			optionally sets the new amount. The count must be at least C<1>, with the
306			default being C<4>.
307	root	1.92
308	root	1.76	=item @states = Coro::State::list
309
310			Returns a list of all states currently allocated.
311
312	root	1.1	=cut
313
314	root	1.115	# used by Coro::Debug only atm.
315	root	1.75	sub debug_desc {
316			$_[0]{desc}
317			}
318
319	root	1.1	1;
320
321			=back
322
323			=head1 BUGS
324
325	root	1.5	This module is not thread-safe. You must only ever use this module from
326	root	1.94	the same thread (this requirement might be removed in the future).
327	root	1.1
328			=head1 SEE ALSO
329
330			L<Coro>.
331
332			=head1 AUTHOR
333
334	root	1.41	Marc Lehmann <schmorp@schmorp.de>
335	root	1.39	http://home.schmorp.de/
336	root	1.1
337			=cut
338