Proc-FastSpawn/FastSpawn.pm

=head1 NAME

Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible

=head1 SYNOPSIS

   use Proc::FastSpawn;

   # simple use
   my $pid = spawn "/bin/echo", ["echo", "hello, world"];
   ...
   waitpid $pid, 0;

   # with environment
   my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"];

   # inheriting file descriptors
   pipe R, W or die;
   fd_inherit fileno W;
   my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W];
   close W;
   print <R>;

=head1 DESCRIPTION

The purpose of this small (in scope and footprint) module is simple:
spawn a subprocess asynchronously as efficiently and/or fast as
possible. Basically the same as calling fork+exec (on POSIX), but
hopefully faster than those two syscalls.

Apart from fork overhead, this module also allows you to fork+exec
programs when otherwise you couldn't - for example, when you use POSIX
threads in your perl process then it generally isn't safe to call
fork from perl, but it is safe to use this module to execute external
processes.

If neither of these are problems for you, you can safely ignore this
module.

So when is fork+exec not fast enough, how can you do it faster, and why
would it matter?

Forking a process requires making a complete copy of a process. Even
thougth almost every implementation only copies page tables and not the
memory istelf, this is still not free. For example, on my 3.6GHz amd64
box, I can fork a 5GB process only twenty times a second. For a realtime
process that must meet stricter deadlines, this is too slow. For a busy
and big webserver, starting CGI scripts might mean unacceptable overhead.

A workaround is to use C<vfork> - this function isn't very portable, but
it avoids the memory copy that C<fork> has to do. Some systems have an
optimised implementation of C<spawn>, and some systems have nothing.

This module tries to abstract these differences away.

As for what improvements to expect - on the 3.6GHz amd64 box that this
module was originally developed on, a 3MB perl process (basically just
perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using
fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the
difference is already 5.0s vs 2.6s, and so on.

=head1 FUNCTIONS

All the following functions are currently exported by default.

=over 4

=cut

package Proc::FastSpawn;

# only used on WIN32 - maddeningly complex and doesn't even work
sub _quote {
   $_[0] = [@{ $_[0] }]; # make copy

   for (@{ $_[0] }) {
      if (/[\x01-\x20"]/) { # some sources say only space, "\t\n\v need to be escaped, microsoft says space and tab
         s/(\\*)"/$1$1\\"/g; # double + extra escape before "
         s/(\\+)$/$1$1/;     # just double at end
         $_ = '"' . $_ . '"';
      }
   }
}

BEGIN {
   $VERSION = '0.1';

   our @ISA = qw(Exporter);
   our @EXPORT = qw(spawn fd_inherit);
   require Exporter;

   require XSLoader;
   XSLoader::load (__PACKAGE__, $VERSION);
}

=item $pid = spawn $path, \@argv[, \@envp]

Creates a new process and tries to make it execute C<$path>, with the given
arguments and optionally the given environment variables, similar to
calling fork + execv, or execve.

Returns the PID of the new process if successful. On any error, C<undef>
is currently returned. Failure to execution might or might not be reported
as C<undef>, or via a subprocess exit status of C<127>.

=item fd_inherit $fileno[, $on]

File descriptors can be inherited by the spawned proceses or not. This is
decided on a per file descriptor basis. This module does nothing to any
preexisting handles, but with this call, you can change the state of a
single file descriptor to either be inherited (C<$on> is true or missing)
or not C<$on> is false).

=back

=head1 PORTABILITY NOTES

On POSIX systems, this module currently calls vfork+exec, spawn, or
fork+exec, depending on the platform. If your platform has a good vfork or
spawn but is misdetected and falls back to slow fork+exec, drop me a note.

On win32, the C<_spawn> family of functions is used, and the module tries
hard to patch the new process into perl's internal pid table, so the pid
returned should work with other perl functions such as waitpid. Also,
win32 doesn't have a meaningful way to quote arguments containing
"special" characters, so this module tries it's best to quote those
strings itself. Other typical platform limitations (such as being able to
only have 64 or so subprocesses) are not worked around.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

1;

Revision:	1.2
Committed:	Sat Mar 30 12:54:13 2013 UTC (11 years, 1 month ago) by root
Branch:	MAIN
Changes since 1.1:	+18 -3 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Proc::FastSpawn - fork+exec, or spawn, a subprocess as quickly as possible
4
5			=head1 SYNOPSIS
6
7	root	1.2	use Proc::FastSpawn;
8
9			# simple use
10			my $pid = spawn "/bin/echo", ["echo", "hello, world"];
11			...
12			waitpid $pid, 0;
13
14			# with environment
15			my $pid = spawn "/bin/echo", ["echo", "hello, world"], ["PATH=/bin", "HOME=/tmp"];
16
17			# inheriting file descriptors
18			pipe R, W or die;
19			fd_inherit fileno W;
20			my $pid = spawn "/bin/sh", ["sh", "-c", "echo a pipe >&" . fileno W];
21			close W;
22			print <R>;
23	root	1.1
24			=head1 DESCRIPTION
25
26			The purpose of this small (in scope and footprint) module is simple:
27			spawn a subprocess asynchronously as efficiently and/or fast as
28			possible. Basically the same as calling fork+exec (on POSIX), but
29			hopefully faster than those two syscalls.
30
31			Apart from fork overhead, this module also allows you to fork+exec
32			programs when otherwise you couldn't - for example, when you use POSIX
33			threads in your perl process then it generally isn't safe to call
34			fork from perl, but it is safe to use this module to execute external
35			processes.
36
37			If neither of these are problems for you, you can safely ignore this
38			module.
39
40			So when is fork+exec not fast enough, how can you do it faster, and why
41			would it matter?
42
43			Forking a process requires making a complete copy of a process. Even
44			thougth almost every implementation only copies page tables and not the
45			memory istelf, this is still not free. For example, on my 3.6GHz amd64
46			box, I can fork a 5GB process only twenty times a second. For a realtime
47			process that must meet stricter deadlines, this is too slow. For a busy
48			and big webserver, starting CGI scripts might mean unacceptable overhead.
49
50			A workaround is to use C<vfork> - this function isn't very portable, but
51			it avoids the memory copy that C<fork> has to do. Some systems have an
52			optimised implementation of C<spawn>, and some systems have nothing.
53
54			This module tries to abstract these differences away.
55
56			As for what improvements to expect - on the 3.6GHz amd64 box that this
57			module was originally developed on, a 3MB perl process (basically just
58			perl + Proc::FastSpawn) takes 3.6s to run /bin/true 10000 times using
59			fork+exec, and only 2.6s when using vfork+exec. In a 22MB process, the
60			difference is already 5.0s vs 2.6s, and so on.
61
62			=head1 FUNCTIONS
63
64			All the following functions are currently exported by default.
65
66			=over 4
67
68			=cut
69
70			package Proc::FastSpawn;
71
72			# only used on WIN32 - maddeningly complex and doesn't even work
73			sub _quote {
74			$_[0] = [@{ $_[0] }]; # make copy
75
76			for (@{ $_[0] }) {
77			if (/[\x01-\x20"]/) { # some sources say only space, "\t\n\v need to be escaped, microsoft says space and tab
78			s/(\\*)"/$1$1\\"/g; # double + extra escape before "
79			s/(\\+)$/$1$1/; # just double at end
80			$_ = '"' . $_ . '"';
81			}
82			}
83			}
84
85			BEGIN {
86			$VERSION = '0.1';
87
88			our @ISA = qw(Exporter);
89			our @EXPORT = qw(spawn fd_inherit);
90			require Exporter;
91
92			require XSLoader;
93			XSLoader::load (__PACKAGE__, $VERSION);
94			}
95
96			=item $pid = spawn $path, \@argv[, \@envp]
97
98			Creates a new process and tries to make it execute C<$path>, with the given
99			arguments and optionally the given environment variables, similar to
100			calling fork + execv, or execve.
101
102			Returns the PID of the new process if successful. On any error, C<undef>
103			is currently returned. Failure to execution might or might not be reported
104			as C<undef>, or via a subprocess exit status of C<127>.
105
106			=item fd_inherit $fileno[, $on]
107
108			File descriptors can be inherited by the spawned proceses or not. This is
109			decided on a per file descriptor basis. This module does nothing to any
110			preexisting handles, but with this call, you can change the state of a
111			single file descriptor to either be inherited (C<$on> is true or missing)
112			or not C<$on> is false).
113
114			=back
115
116			=head1 PORTABILITY NOTES
117
118			On POSIX systems, this module currently calls vfork+exec, spawn, or
119			fork+exec, depending on the platform. If your platform has a good vfork or
120	root	1.2	spawn but is misdetected and falls back to slow fork+exec, drop me a note.
121	root	1.1
122			On win32, the C<_spawn> family of functions is used, and the module tries
123			hard to patch the new process into perl's internal pid table, so the pid
124			returned should work with other perl functions such as waitpid. Also,
125			win32 doesn't have a meaningful way to quote arguments containing
126			"special" characters, so this module tries it's best to quote those
127			strings itself. Other typical platform limitations (such as being able to
128	root	1.2	only have 64 or so subprocesses) are not worked around.
129	root	1.1
130			=head1 AUTHOR
131
132			Marc Lehmann <schmorp@schmorp.de>
133			http://home.schmorp.de/
134
135			=cut
136
137			1;
138