[ViewVC] Annotation of: cvs/Coro-Multicore/README

NAME
    Coro::Multicore - make coro threads on multiple cores with specially
    supported modules

SYNOPSIS
     use Coro::Multicore;

     # or, if you want it disabled by default (e.g. to use it from a module)
     use Coro::Multicore ();

DESCRIPTION
    EXPERIMENTAL WARNING: This module is in its early stages of development.
    It's fine to try out, but it didn't receive the normal amount of testing
    and real-world usage that my other modules have gone through.

    While Coro threads (unlike ithreads) provide real threads similar to
    pthreads, python threads and so on, they do not run in parallel to each
    other even on machines with multiple CPUs or multiple CPU cores.

    This module lifts this restriction under two very specific but useful
    conditions: firstly, the coro thread executes in XS code and does not
    touch any perl data structures, and secondly, the XS code is specially
    prepared to allow this.

    This means that, when you call an XS function of a module prepared for
    it, this XS function can execute in parallel to any other Coro threads.

    The mechanism to support this is easily added to existing modules and is
    independent of Coro or Coro::Multicore, and therefore could be used,
    without changes, with other, similar, modules, or even the perl core,
    should it gain real thread support anytime soon. See
    <http://perlmulticore.schmorp.de/> for more info on how to prepare a
    module to allow parallel execution. Preparing an existing module is
    easy, doesn't add much overhead and no dependencies.

    This module is an AnyEvent user (and also, if not obvious, uses Coro).

HOW TO USE IT
    It could hardly be simpler - if you use coro threads, and before you
    call a supported lengthy operation implemented in XS, use this module
    and other coro threads can run in parallel:

       use Coro::Multicore;

    This module has no important API functions to learn or remember. All you
    need to do is *load* it before you can take advantage of it.

  EXPORTS
    This module does not (at the moment) export any symbols. It does,
    however, export "behaviour" - if you use the default import, then
    Coro::Multicore will be enabled for all threads and all callers in the
    whole program:

       use Coro::Multicore;

    In a module where you don't control what else might be loaded and run,
    you might want to be more conservative, and not import anything. This
    has the effect of not enabling the functionality by default, so you have
    to enable it per scope:

       use Coro::Multicore ();

       sub myfunc {
          Coro::Multicore::scoped_enable;

          # from here to the end of this function, and in any functions
          # called from this function, tasks will be executed asynchronously.
       }

API FUNCTIONS
    $previous = Coro::Multicore::enable [$enable]
        This function enables (if $enable is true) or disables (if $enable
        is false) the multicore functionality globally. By default, it is
        enabled.

        This can be used to effectively disable this module's functionality
        by default, and enable it only for selected threads or scopes, by
        calling "Coro::Multicore::scope_enable".

        The function returns the previous value of the enable flag.

    Coro::Multicore::scoped_enable
        This function instructs Coro::Multicore to handle all requests
        executed in the current coro thread, from the call to the end of the
        current scope.

        Calls to "scoped_enable" and "scoped_disable" don't nest very well
        at the moment, so don't nest them.

    Coro::Multicore::scoped_disable
        The opposite of "Coro::Multicore::scope_disable": instructs
        Coro::Multicore to *not* handle the next multicore-enabled request.

INTERACTION WITH OTHER SOFTWARE
    This module is very similar to other environments where perl
    interpreters are moved between threads, such as mod_perl2, and the same
    caveats apply.

    I want to spell out the most important ones:

    pthreads usage
        Any creation of pthreads make it impossible to fork portably from a
        perl program, as forking from within a threaded program will leave
        the program in a state similar to a signal handler. While it might
        work on some platforms (as an extension), this might also result in
        silent data corruption. It also seems to work most of the time, so
        it's hard to test for this.

        I recommend using something like AnyEvent::Fork, which can create
        subprocesses safely (via Proc::FastSpawn).

        Similar issues exist for signal handlers, although this module works
        hard to keep safe perl signals safe.

    module support
        This module moves the same perl interpreter between different
        threads. Some modules might get confused by that (although this can
        usually be considered a bug). This is a rare case though.

    event loop reliance
        To be able to wake up programs waiting for results, this module
        relies on an active event loop (via AnyEvent). This is used to
        notify the perl interpreter when the asynchronous task is done.

        Since event loops typically fail to work properly after a fork, this
        means that some operations that were formerly working will now hang
        after fork.

        A workaround is to call "Coro::Multicore::enable 0" after a fork to
        disable the module.

        Future versions of this module might do this automatically.

BUGS
    (OS-) threads are never released
        At the moment, threads that were created once will never be freed.
        They will be reused for asynchronous requests, though, so a slong as
        you limit the maximum number of concurrent asynchronous tasks, this
        will also limit the maximum number of threads created.

        Future versions will likely lift this limitation.

    AnyEvent is initalised at module load time
        AnyEvent is initialised on module load, as opposed to at a later
        time.

        Future versions will likely change this.

AUTHOR
     Marc Lehmann <schmorp@schmorp.de>
     http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html

    Additional thanks to Zsb�n Ambrus, who gave considerable desing input
    for this module and the perl multicore specification.

Revision:	1.3
Committed:	Wed Jul 1 22:53:37 2015 UTC (8 years, 10 months ago) by root
Branch:	MAIN
CVS Tags:	rel-0_02
Changes since 1.2:	+43 -2 lines
Log Message:	0.02
#	User	Rev	Content
1	root	1.2	NAME
2			Coro::Multicore - make coro threads on multiple cores with specially
3			supported modules
4
5			SYNOPSIS
6			use Coro::Multicore;
7
8			# or, if you want it disabled by default (e.g. to use it from a module)
9			use Coro::Multicore ();
10
11			DESCRIPTION
12	root	1.3	EXPERIMENTAL WARNING: This module is in its early stages of development.
13			It's fine to try out, but it didn't receive the normal amount of testing
14			and real-world usage that my other modules have gone through.
15
16	root	1.2	While Coro threads (unlike ithreads) provide real threads similar to
17			pthreads, python threads and so on, they do not run in parallel to each
18			other even on machines with multiple CPUs or multiple CPU cores.
19
20			This module lifts this restriction under two very specific but useful
21			conditions: firstly, the coro thread executes in XS code and does not
22			touch any perl data structures, and secondly, the XS code is specially
23			prepared to allow this.
24
25			This means that, when you call an XS function of a module prepared for
26			it, this XS function can execute in parallel to any other Coro threads.
27
28			The mechanism to support this is easily added to existing modules and is
29			independent of Coro or Coro::Multicore, and therefore could be used,
30			without changes, with other, similar, modules, or even the perl core,
31			should it gain real thread support anytime soon. See
32			<http://perlmulticore.schmorp.de/> for more info on how to prepare a
33			module to allow parallel execution. Preparing an existing module is
34			easy, doesn't add much overhead and no dependencies.
35
36			This module is an AnyEvent user (and also, if not obvious, uses Coro).
37
38			HOW TO USE IT
39			It could hardly be simpler - if you use coro threads, and before you
40			call a supported lengthy operation implemented in XS, use this module
41			and other coro threads can run in parallel:
42
43			use Coro::Multicore;
44
45			This module has no important API functions to learn or remember. All you
46			need to do is load it before you can take advantage of it.
47
48			EXPORTS
49			This module does not (at the moment) export any symbols. It does,
50			however, export "behaviour" - if you use the default import, then
51			Coro::Multicore will be enabled for all threads and all callers in the
52			whole program:
53
54			use Coro::Multicore;
55
56			In a module where you don't control what else might be loaded and run,
57			you might want to be more conservative, and not import anything. This
58			has the effect of not enabling the functionality by default, so you have
59			to enable it per scope:
60
61			use Coro::Multicore ();
62
63			sub myfunc {
64			Coro::Multicore::scoped_enable;
65
66			# from here to the end of this function, and in any functions
67			# called from this function, tasks will be executed asynchronously.
68			}
69
70			API FUNCTIONS
71			$previous = Coro::Multicore::enable [$enable]
72			This function enables (if $enable is true) or disables (if $enable
73			is false) the multicore functionality globally. By default, it is
74			enabled.
75
76			This can be used to effectively disable this module's functionality
77			by default, and enable it only for selected threads or scopes, by
78			calling "Coro::Multicore::scope_enable".
79
80			The function returns the previous value of the enable flag.
81
82			Coro::Multicore::scoped_enable
83			This function instructs Coro::Multicore to handle all requests
84			executed in the current coro thread, from the call to the end of the
85			current scope.
86
87			Calls to "scoped_enable" and "scoped_disable" don't nest very well
88			at the moment, so don't nest them.
89
90			Coro::Multicore::scoped_disable
91			The opposite of "Coro::Multicore::scope_disable": instructs
92			Coro::Multicore to not handle the next multicore-enabled request.
93
94			INTERACTION WITH OTHER SOFTWARE
95	root	1.3	This module is very similar to other environments where perl
96			interpreters are moved between threads, such as mod_perl2, and the same
97			caveats apply.
98
99			I want to spell out the most important ones:
100
101			pthreads usage
102			Any creation of pthreads make it impossible to fork portably from a
103			perl program, as forking from within a threaded program will leave
104			the program in a state similar to a signal handler. While it might
105			work on some platforms (as an extension), this might also result in
106			silent data corruption. It also seems to work most of the time, so
107			it's hard to test for this.
108
109			I recommend using something like AnyEvent::Fork, which can create
110			subprocesses safely (via Proc::FastSpawn).
111
112			Similar issues exist for signal handlers, although this module works
113			hard to keep safe perl signals safe.
114
115			module support
116			This module moves the same perl interpreter between different
117			threads. Some modules might get confused by that (although this can
118			usually be considered a bug). This is a rare case though.
119
120			event loop reliance
121			To be able to wake up programs waiting for results, this module
122			relies on an active event loop (via AnyEvent). This is used to
123			notify the perl interpreter when the asynchronous task is done.
124
125			Since event loops typically fail to work properly after a fork, this
126			means that some operations that were formerly working will now hang
127			after fork.
128
129			A workaround is to call "Coro::Multicore::enable 0" after a fork to
130			disable the module.
131
132			Future versions of this module might do this automatically.
133	root	1.2
134			BUGS
135			(OS-) threads are never released
136			At the moment, threads that were created once will never be freed.
137			They will be reused for asynchronous requests, though, so a slong as
138			you limit the maximum number of concurrent asynchronous tasks, this
139			will also limit the maximum number of threads created.
140
141			Future versions will likely lift this limitation.
142
143	root	1.3	AnyEvent is initalised at module load time
144	root	1.2	AnyEvent is initialised on module load, as opposed to at a later
145			time.
146
147			Future versions will likely change this.
148
149			AUTHOR
150			Marc Lehmann <schmorp@schmorp.de>
151			http://software.schmorp.de/pkg/AnyEvent-XSThreadPool.html
152
153			Additional thanks to Zsb�n Ambrus, who gave considerable desing input
154			for this module and the perl multicore specification.
155