--- AnyEvent-Fork/README 2013/09/25 11:05:30 1.9 +++ AnyEvent-Fork/README 2016/05/12 16:54:01 1.10 @@ -53,8 +53,9 @@ The problems that all these modules try to solve are real, however, none of them (from what I have seen) tackle the very real problems of - unwanted memory sharing, efficiency, not being able to use event - processing or similar modules in the processes they create. + unwanted memory sharing, efficiency or not being able to use event + processing, GUI toolkits or similar modules in the processes they + create. This module doesn't try to replace any of them - instead it tries to solve the problem of creating processes with a minimum of fuss and @@ -81,16 +82,28 @@ Forking usually creates a copy-on-write copy of the parent process. For example, modules or data files that are loaded will not use - additional memory after a fork. When exec'ing a new process, modules - and data files might need to be loaded again, at extra CPU and - memory cost. But when forking, literally all data structures are - copied - if the program frees them and replaces them by new data, - the child processes will retain the old version even if it isn't - used, which can suddenly and unexpectedly increase memory usage when - freeing memory. + additional memory after a fork. Exec'ing a new process, in contrast, + means modules and data files might need to be loaded again, at extra + CPU and memory cost. + + But when forking, you still create a copy of your data structures - + if the program frees them and replaces them by new data, the child + processes will retain the old version even if it isn't used, which + can suddenly and unexpectedly increase memory usage when freeing + memory. + + For example, Gtk2::CV is an image viewer optimised for large + directories (millions of pictures). It also forks subprocesses for + thumbnail generation, which inherit the data structure that stores + all file information. If the user changes the directory, it gets + freed in the main process, leaving a copy in the thumbnailer + processes. This can lead to many times the memory usage that would + actually be required. The solution is to fork early (and being + unable to dynamically generate more subprocesses or do this from a + module)... or to use . - The trade-off is between more sharing with fork (which can be good - or bad), and no sharing with exec. + There is a trade-off between more sharing with fork (which can be + good or bad), and no sharing with exec. This module allows the main program to do a controlled fork, and allows modules to exec processes safely at any time. When creating a @@ -103,7 +116,8 @@ Exec'ing a new perl process might be difficult. For example, it is not easy to find the correct path to the perl - interpreter - $^X might not be a perl interpreter at all. + interpreter - $^X might not be a perl interpreter at all. Worse, + there might not even be a perl binary installed on the system. This module tries hard to identify the correct path to the perl interpreter. With a cooperative main program, exec'ing the @@ -118,15 +132,16 @@ already loaded. This module supports creating pre-initialised perl processes to be - used as a template for new processes. + used as a template for new processes at a later time, e.g. for use + in a process pool. Forking might be impossible when a program is running. For example, POSIX makes it almost impossible to fork from a multi-threaded program while doing anything useful in the child - in fact, if your perl program uses POSIX threads (even indirectly via e.g. IO::AIO or threads), you cannot call fork on the perl level - anymore without risking corruption issues on a number of operating - systems. + anymore without risking memory corruption or worse on a number of + operating systems. This module can safely fork helper processes at any time, by calling fork+exec in C, in a POSIX-compatible way (via Proc::FastSpawn). @@ -152,6 +167,8 @@ loops or window interfaces safely. EXAMPLES + This is where the wall of text ends and code speaks. + Create a single new process, tell it to run your worker function. AnyEvent::Fork ->new @@ -171,7 +188,7 @@ my ($slave_filehandle) = @_; # now $slave_filehandle is connected to the $master_filehandle - # in the original prorcess. have fun! + # in the original process. have fun! } Create a pool of server processes all accepting on the same socket. @@ -240,7 +257,7 @@ my $stderr = $cv->recv; For stingy users: put the worker code into a "DATA" section. - When you want to be stingy with files, you cna put your code into the + When you want to be stingy with files, you can put your code into the "DATA" section of your module (or program): use AnyEvent::Fork; @@ -259,7 +276,7 @@ For stingy standalone programs: do not rely on external files at all. For single-file scripts it can be inconvenient to rely on external files - - even when using < "DATA" section, you still need to "exec" an external + - even when using a "DATA" section, you still need to "exec" an external perl interpreter, which might not be available when using App::Staticperl, Urlader or PAR::Packer for example. @@ -287,7 +304,7 @@ } # now preserve everything so far as AnyEvent::Fork object - # in §TEMPLATE. + # in $TEMPLATE. use AnyEvent::Fork::Template; # do not put code outside of BEGIN blocks until here @@ -484,6 +501,13 @@ Returns the process object for easy chaining of method calls. + It's common to want to call an iniitalisation function with some + arguments. Make sure you actually pass @_ to that function (for + example by using &name syntax), and do not just specify a function + name: + + $proc->eval ('&MyModule::init', $string1, $string2); + $proc = $proc->require ($module, ...) Tries to load the given module(s) into the process @@ -767,6 +791,22 @@ use "send_fh" and always use "new_exec" to create processes, it should work though. +USING AnyEvent::Fork IN SUBPROCESSES + AnyEvent::Fork itself cannot generally be used in subprocesses. As long + as only one process ever forks new processes, sharing the template + processes is possible (you could use a pipe as a lock by writing a byte + into it to unlock, and reading the byte to lock for example) + + To make concurrent calls possible after fork, you should get rid of the + template and early fork processes. AnyEvent::Fork will create a new + template process as needed. + + undef $AnyEvent::Fork::EARLY; + undef $AnyEvent::Fork::TEMPLATE; + + It doesn't matter whether you get rid of them in the parent or child + after a fork. + SEE ALSO AnyEvent::Fork::Early, to avoid executing a perl interpreter at all (part of this distribution).