ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.62 by root, Mon Nov 25 20:21:09 2013 UTC vs.
Revision 1.63 by root, Wed Nov 26 13:36:18 2014 UTC

… …
56or L<AnyEvent::Subprocess>. There are modules that implement their own 56or L<AnyEvent::Subprocess>. There are modules that implement their own
57process management, such as L<AnyEvent::DBI>. 57process management, such as L<AnyEvent::DBI>.
58 58
59The problems that all these modules try to solve are real, however, none 59The problems that all these modules try to solve are real, however, none
60of them (from what I have seen) tackle the very real problems of unwanted 60of them (from what I have seen) tackle the very real problems of unwanted
61memory sharing, efficiency, not being able to use event processing or 61memory sharing, efficiency or not being able to use event processing, GUI
62similar modules in the processes they create. 62toolkits or similar modules in the processes they create.
63 63
64This module doesn't try to replace any of them - instead it tries to solve 64This module doesn't try to replace any of them - instead it tries to solve
65the problem of creating processes with a minimum of fuss and overhead (and 65the problem of creating processes with a minimum of fuss and overhead (and
66also luxury). Ideally, most of these would use AnyEvent::Fork internally, 66also luxury). Ideally, most of these would use AnyEvent::Fork internally,
67except they were written before AnyEvent:Fork was available, so obviously 67except they were written before AnyEvent:Fork was available, so obviously
… …
89 89
90=item Forking usually creates a copy-on-write copy of the parent 90=item Forking usually creates a copy-on-write copy of the parent
91process. 91process.
92 92
93For example, modules or data files that are loaded will not use additional 93For example, modules or data files that are loaded will not use additional
94memory after a fork. When exec'ing a new process, modules and data files 94memory after a fork. Exec'ing a new process, in contrast, means modules
95might need to be loaded again, at extra CPU and memory cost. But when 95and data files might need to be loaded again, at extra CPU and memory
96forking, literally all data structures are copied - if the program frees 96cost.
97
98But when forking, you still create a copy of your data structures - if
97them and replaces them by new data, the child processes will retain the 99the program frees them and replaces them by new data, the child processes
98old version even if it isn't used, which can suddenly and unexpectedly 100will retain the old version even if it isn't used, which can suddenly and
99increase memory usage when freeing memory. 101unexpectedly increase memory usage when freeing memory.
100 102
103For example, L<Gtk2::CV> is an image viewer optimised for large
104directories (millions of pictures). It also forks subprocesses for
105thumbnail generation, which inherit the data structure that stores all
106file information. If the user changes the directory, it gets freed in
107the main process, leaving a copy in the thumbnailer processes. This can
108lead to many times the memory usage that would actually be required. The
109solution is to fork early (and being unable to dynamically generate more
110subprocesses or do this from a module)... or to use L<AnyEvent:Fork>.
111
101The trade-off is between more sharing with fork (which can be good or 112There is a trade-off between more sharing with fork (which can be good or
102bad), and no sharing with exec. 113bad), and no sharing with exec.
103 114
104This module allows the main program to do a controlled fork, and allows 115This module allows the main program to do a controlled fork, and allows
105modules to exec processes safely at any time. When creating a custom 116modules to exec processes safely at any time. When creating a custom
106process pool you can take advantage of data sharing via fork without 117process pool you can take advantage of data sharing via fork without
… …
111shared and what isn't, at all times. 122shared and what isn't, at all times.
112 123
113=item Exec'ing a new perl process might be difficult. 124=item Exec'ing a new perl process might be difficult.
114 125
115For example, it is not easy to find the correct path to the perl 126For example, it is not easy to find the correct path to the perl
116interpreter - C<$^X> might not be a perl interpreter at all. 127interpreter - C<$^X> might not be a perl interpreter at all. Worse, there
128might not even be a perl binary installed on the system.
117 129
118This module tries hard to identify the correct path to the perl 130This module tries hard to identify the correct path to the perl
119interpreter. With a cooperative main program, exec'ing the interpreter 131interpreter. With a cooperative main program, exec'ing the interpreter
120might not even be necessary, but even without help from the main program, 132might not even be necessary, but even without help from the main program,
121it will still work when used from a module. 133it will still work when used from a module.
… …
127and modules are no longer loadable because they refer to a different 139and modules are no longer loadable because they refer to a different
128perl version, or parts of a distribution are newer than the ones already 140perl version, or parts of a distribution are newer than the ones already
129loaded. 141loaded.
130 142
131This module supports creating pre-initialised perl processes to be used as 143This module supports creating pre-initialised perl processes to be used as
132a template for new processes. 144a template for new processes at a later time, e.g. for use in a process
145pool.
133 146
134=item Forking might be impossible when a program is running. 147=item Forking might be impossible when a program is running.
135 148
136For example, POSIX makes it almost impossible to fork from a 149For example, POSIX makes it almost impossible to fork from a
137multi-threaded program while doing anything useful in the child - in 150multi-threaded program while doing anything useful in the child - in
138fact, if your perl program uses POSIX threads (even indirectly via 151fact, if your perl program uses POSIX threads (even indirectly via
139e.g. L<IO::AIO> or L<threads>), you cannot call fork on the perl level 152e.g. L<IO::AIO> or L<threads>), you cannot call fork on the perl level
140anymore without risking corruption issues on a number of operating 153anymore without risking memory corruption or worse on a number of
141systems. 154operating systems.
142 155
143This module can safely fork helper processes at any time, by calling 156This module can safely fork helper processes at any time, by calling
144fork+exec in C, in a POSIX-compatible way (via L<Proc::FastSpawn>). 157fork+exec in C, in a POSIX-compatible way (via L<Proc::FastSpawn>).
145 158
146=item Parallel processing with fork might be inconvenient or difficult 159=item Parallel processing with fork might be inconvenient or difficult
… …
1080Cygwin perl is not supported at the moment due to some hilarious 1093Cygwin perl is not supported at the moment due to some hilarious
1081shortcomings of its API - see L<IO::FDPoll> for more details. If you never 1094shortcomings of its API - see L<IO::FDPoll> for more details. If you never
1082use C<send_fh> and always use C<new_exec> to create processes, it should 1095use C<send_fh> and always use C<new_exec> to create processes, it should
1083work though. 1096work though.
1084 1097
1098=head1 USING AnyEvent::Fork IN SUBPROCESSES
1099
1100AnyEvent::Fork itself cannot generally be used in subprocesses. As long as
1101only one process ever forks new processes, sharing the template processes
1102is possible (you could use a pipe as a lock by writing a byte into it to
1103unlock, and reading the byte to lock for example)
1104
1105To make concurrent calls possible after fork, you should get rid of the
1106template and early fork processes. AnyEvent::Fork will create a new
1107template process as needed.
1108
1109 undef $AnyEvent::Fork::EARLY;
1110 undef $AnyEvent::Fork::TEMPLATE;
1111
1112It doesn't matter whether you get rid of them in the parent or child after
1113a fork.
1114
1085=head1 SEE ALSO 1115=head1 SEE ALSO
1086 1116
1087L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all 1117L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
1088(part of this distribution). 1118(part of this distribution).
1089 1119

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines