AnyEvent-Watchdog/Watchdog/Util.pm

=head1 NAME

AnyEvent::Watchdog::Util - watchdog control and process management

=head1 SYNOPSIS

   use AnyEvent::Watchdog::Util;

=head1 DESCRIPTION

This module can control the watchdog started by using
L<AnyEvent::Watchdog> in your main program, but it has useful
functionality even when not running under the watchdog at all, such as
program exit hooks.

=head1 VARIABLES/FUNCTIONS

The module supports the following variables and functions:

=over 4

=cut

package AnyEvent::Watchdog::Util;

# load modules we will use later anyways
use common::sense;
use AnyEvent ();
use Carp ();

our $VERSION = '1.0';

our $C;
BEGIN {
   *C = \$AnyEvent::Watchdog::C;
}

our $AUTORESTART;
our $HEARTBEAT_W;

=item AnyEvent::Watchdog::Util::enabled

Return true when the program is running under the regime of
AnyEvent::Watchdog, false otherwise.

   AnyEvent::Watchdog::Util::enabled
      or die "watchdog not enabled...";
   AnyEvent::Watchdog::Util::restart;

Note that if it returns defined, but false, then AnyEvent::Watchdog is
running, but you are in the watchdog process - you probably did something
very wrong in this case.

=cut

sub enabled() {
   $AnyEvent::Watchdog::ENABLED
}

=item AnyEvent::Watchdog::restart_in [$timeout]

Tells the supervisor to restart the process when it exits (enable
autorestart), or forcefully after C<$timeout> seconds (minimum 1, maximum
255, default 60).

This function disables the heartbeat, if it was enabled. Also, after
calling this function the watchdog will ignore any further requests until
the program has restarted.

Good to call before you intend to exit, in case your clean-up handling
gets stuck.

=cut

sub restart_in(;$) {
   my ($timeout) = @_;

   return unless $C;

   undef $HEARTBEAT_W;

   $timeout =  60 unless defined $timeout;
   $timeout =   1 if $timeout <   1;
   $timeout = 255 if $timeout > 255;

   syswrite $C, "\x01\x02" . chr $timeout;

   # now make sure we dont' send it any further requests
   our $OLD_C = $C; undef $C;
}

=item AnyEvent::Watchdog::restart [$timeout]

Just like C<restart_in>, but also calls C<exit 0>. This means that this is
the ideal method to force a restart.

=cut

sub restart(;$) {
   &restart_in;
   exit 0;
}

=item AnyEvent::Watchdog::Util::autorestart [$boolean]

=item use AnyEvent::Watchdog autorestart => $boolean

Enables or disables autorestart (initially disabled, default for
C<$boolean> is to enable): By default, the supervisor will exit if the
program exits or dies in any way. When enabling autorestart behaviour,
then the supervisor will try to restart the program after it dies.

Note that the supervisor will never autorestart when the child died with
SIGINT or SIGTERM.

=cut

sub autorestart(;$) {
   my $AUTORESTART = !@_ || $_[0];

   return unless $C;

   unless (enabled) {
      warn "AnyEvent::Watchdog: watchdog not running, cannot enable autorestart, ignoring.\n"
         if $AUTORESTART;

      $AUTORESTART = 0;

      return;
   }

   syswrite $C, $AUTORESTART ? "\x01" : "\x00";
}

=item AnyEvent::Watchdog::heartbeat [$interval]

=item use AnyEvent::Watchdog heartbeat => $interval

Tells the supervisor to automatically kill the program if it doesn't
react for C<$interval> seconds (minium 1, maximum 255, default 60) , then
installs an AnyEvent timer the sends a regular heartbeat to the supervisor
twice as often.

Exit behaviour isn't changed, so if you want a restart instead of an exit,
you have to call C<autorestart>.

The heartbeat frequency can be changed as often as you want, an interval
of C<0> disables the heartbeat check again.

=cut

sub heartbeat(;$) {
   my ($interval) = @_;

   unless (enabled) {
      warn "AnyEvent::Watchdog: watchdog not running, cannot enable heartbeat, ignoring.\n";
      return;
   }

   $interval =  60 unless defined $interval;
   $interval =   0 if $interval <   0;
   $interval = 255 if $interval > 255;

   syswrite $C, "\x03" . chr int $interval
      if $C;

   $HEARTBEAT_W = $interval && AE::timer (0, $interval * 0.5, sub {
      syswrite $C, "\x04"
         if $C;
   });
}

=item AnyEvent::Watchdog::on_exit { BLOCK; shift->() }

Installs an exit hook that is executed when the program is about to exit,
while event processing is still active to some extent.

The hook should do whatever it needs to do (close active connections,
disable listeners, write state, free resources etc.). When it is done, it
should call the code reference that has been passed to it.

This means you can install event handlers and return from the block, and
the program will not exit until the callback is invoked.

Exiting "the right way" is surprisingly difficult. This is what C<on_exit>
does:

It installs watchers for C<SIGTERM>, C<SIGINT>, C<SIGXCPU> and C<SIGXFSZ>,
and well as an C<END> block (the END block is actually registered
in L<AnyEvent::Watchdog>, if possible, so it executes as late as
possible). The signal handlers remember the signal and then call C<exit>,
invoking the C<END> callback.

The END block then checks for an exit code of C<255>, in which case
nothing happens (C<255> is the exit code that results from a program
error), otherwise it runs all C<on_exit> hooks and waits for their
completion using the event loop.

After all C<on_exit> hooks have finished, the program will either be
C<exit>ed with the relevant status code (if C<exit> was the cause for the
program exit), or it will reset the signal handler, unblock the signal and
kill itself with the signal, to ensure that the exit status is correct.

If the program is running under the watchdog, and autorestart is enabled,
then the heartbeat is disabled and the watchdog is told that the program
wishes to exit within C<60> seconds, after which it will be forcefully
killed.

All of this should ensure that C<on_exit> hooks are only executed when the
program is in a sane state and data structures are still intact. This only
works when the program does not install it's own TERM (etc.) watchers, of
course, as there is no control over them.

There is currently no way to unregister C<on_exit> hooks.

=cut

our @ON_EXIT;
our %SIG_W;
our $EXIT_STATUS; # >= 0 exit status; arrayref => signal, undef if exit was just called

# in case AnyEvent::Watchdog is not loaded, use our own END block
END { $AnyEvent::Watchdog::end && &$AnyEvent::Watchdog::end }

sub _exit {
   $EXIT_STATUS = $? unless defined $EXIT_STATUS;

   # we might have two END blocks trying to call us.
   undef $AnyEvent::Watchdog::end;

   if (enabled) {
      undef $HEARTBEAT_W;
      restart_in 60;
   }

   my $cv = AE::cv;
   my $cb = sub { $cv->end };

   $cv->begin;
   while (@ON_EXIT) {
      $cv->begin;
      (pop @ON_EXIT)->($cb);
   }
   $cv->end;
   $cv->recv;

   if (ref $EXIT_STATUS) {
      # signal
      # reset to default, hopefully this overrides any C-level handlers
      $SIG{$EXIT_STATUS->[0]} = 'DEFAULT';

      eval {
         # try to unblock
         require POSIX;

         my $set = POSIX::SigSet->new;
         $set->addset ($EXIT_STATUS->[1]);
         POSIX::sigprocmask (POSIX::SIG_UNBLOCK (), $set);
      };

      # now raise the signal
      kill $EXIT_STATUS->[1], $$;

      # well, if we can't force it even now, try exit 255
      $? = 255;
   } else {
      # exit status
      $? = $EXIT_STATUS;
   }

}

sub on_exit(&) {
   unless ($AnyEvent::Watchdog::end) {
      $AnyEvent::Watchdog::end = \&_exit;

      push @ON_EXIT, $_[0];

      for my $signal (qw(TERM INT XFSZ XCPU)) {
         my $signum = AnyEvent::Base::sig2num $signal
            or next;
         $SIG_W{$signum} = AE::signal $signal => sub {
            $EXIT_STATUS = [$signal => $signum];
            exit 124;
         };
      }
   }
}

=back

=head1 SEE ALSO

L<AnyEvent>.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

1

Revision:	1.3
Committed:	Tue Sep 1 16:30:47 2009 UTC (15 years, 1 month ago) by root
Branch:	MAIN
Changes since 1.2:	+45 -9 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			AnyEvent::Watchdog::Util - watchdog control and process management
4
5			=head1 SYNOPSIS
6
7			use AnyEvent::Watchdog::Util;
8
9			=head1 DESCRIPTION
10
11			This module can control the watchdog started by using
12			L<AnyEvent::Watchdog> in your main program, but it has useful
13			functionality even when not running under the watchdog at all, such as
14			program exit hooks.
15
16			=head1 VARIABLES/FUNCTIONS
17
18			The module supports the following variables and functions:
19
20			=over 4
21
22			=cut
23
24			package AnyEvent::Watchdog::Util;
25
26			# load modules we will use later anyways
27			use common::sense;
28			use AnyEvent ();
29			use Carp ();
30
31			our $VERSION = '1.0';
32
33			our $C;
34			BEGIN {
35			*C = \$AnyEvent::Watchdog::C;
36			}
37
38			our $AUTORESTART;
39			our $HEARTBEAT_W;
40
41			=item AnyEvent::Watchdog::Util::enabled
42
43			Return true when the program is running under the regime of
44			AnyEvent::Watchdog, false otherwise.
45
46			AnyEvent::Watchdog::Util::enabled
47			or die "watchdog not enabled...";
48			AnyEvent::Watchdog::Util::restart;
49
50			Note that if it returns defined, but false, then AnyEvent::Watchdog is
51			running, but you are in the watchdog process - you probably did something
52			very wrong in this case.
53
54			=cut
55
56			sub enabled() {
57			$AnyEvent::Watchdog::ENABLED
58			}
59
60	root	1.3	=item AnyEvent::Watchdog::restart_in [$timeout]
61	root	1.1
62	root	1.3	Tells the supervisor to restart the process when it exits (enable
63			autorestart), or forcefully after C<$timeout> seconds (minimum 1, maximum
64			255, default 60).
65
66			This function disables the heartbeat, if it was enabled. Also, after
67			calling this function the watchdog will ignore any further requests until
68			the program has restarted.
69	root	1.1
70	root	1.3	Good to call before you intend to exit, in case your clean-up handling
71			gets stuck.
72	root	1.1
73			=cut
74
75	root	1.3	sub restart_in(;$) {
76	root	1.1	my ($timeout) = @_;
77
78	root	1.3	return unless $C;
79
80			undef $HEARTBEAT_W;
81
82	root	1.1	$timeout = 60 unless defined $timeout;
83			$timeout = 1 if $timeout < 1;
84			$timeout = 255 if $timeout > 255;
85
86			syswrite $C, "\x01\x02" . chr $timeout;
87	root	1.3
88			# now make sure we dont' send it any further requests
89			our $OLD_C = $C; undef $C;
90			}
91
92			=item AnyEvent::Watchdog::restart [$timeout]
93
94			Just like C<restart_in>, but also calls C<exit 0>. This means that this is
95			the ideal method to force a restart.
96
97			=cut
98
99			sub restart(;$) {
100			&restart_in;
101	root	1.1	exit 0;
102			}
103
104			=item AnyEvent::Watchdog::Util::autorestart [$boolean]
105
106			=item use AnyEvent::Watchdog autorestart => $boolean
107
108			Enables or disables autorestart (initially disabled, default for
109			C<$boolean> is to enable): By default, the supervisor will exit if the
110			program exits or dies in any way. When enabling autorestart behaviour,
111			then the supervisor will try to restart the program after it dies.
112
113			Note that the supervisor will never autorestart when the child died with
114			SIGINT or SIGTERM.
115
116			=cut
117
118			sub autorestart(;$) {
119			my $AUTORESTART = !@_ \|\| $_[0];
120
121	root	1.3	return unless $C;
122
123	root	1.1	unless (enabled) {
124			warn "AnyEvent::Watchdog: watchdog not running, cannot enable autorestart, ignoring.\n"
125			if $AUTORESTART;
126
127			$AUTORESTART = 0;
128
129			return;
130			}
131
132			syswrite $C, $AUTORESTART ? "\x01" : "\x00";
133			}
134
135			=item AnyEvent::Watchdog::heartbeat [$interval]
136
137			=item use AnyEvent::Watchdog heartbeat => $interval
138
139			Tells the supervisor to automatically kill the program if it doesn't
140			react for C<$interval> seconds (minium 1, maximum 255, default 60) , then
141			installs an AnyEvent timer the sends a regular heartbeat to the supervisor
142			twice as often.
143
144			Exit behaviour isn't changed, so if you want a restart instead of an exit,
145			you have to call C<autorestart>.
146
147			The heartbeat frequency can be changed as often as you want, an interval
148			of C<0> disables the heartbeat check again.
149
150			=cut
151
152			sub heartbeat(;$) {
153			my ($interval) = @_;
154
155			unless (enabled) {
156			warn "AnyEvent::Watchdog: watchdog not running, cannot enable heartbeat, ignoring.\n";
157			return;
158			}
159
160			$interval = 60 unless defined $interval;
161	root	1.3	$interval = 0 if $interval < 0;
162	root	1.1	$interval = 255 if $interval > 255;
163
164	root	1.3	syswrite $C, "\x03" . chr int $interval
165			if $C;
166	root	1.1
167	root	1.3	$HEARTBEAT_W = $interval && AE::timer (0, $interval * 0.5, sub {
168			syswrite $C, "\x04"
169			if $C;
170	root	1.1	});
171			}
172
173			=item AnyEvent::Watchdog::on_exit { BLOCK; shift->() }
174
175			Installs an exit hook that is executed when the program is about to exit,
176			while event processing is still active to some extent.
177
178			The hook should do whatever it needs to do (close active connections,
179			disable listeners, write state, free resources etc.). When it is done, it
180			should call the code reference that has been passed to it.
181
182			This means you can install event handlers and return from the block, and
183			the program will not exit until the callback is invoked.
184
185			Exiting "the right way" is surprisingly difficult. This is what C<on_exit>
186			does:
187
188			It installs watchers for C<SIGTERM>, C<SIGINT>, C<SIGXCPU> and C<SIGXFSZ>,
189			and well as an C<END> block (the END block is actually registered
190			in L<AnyEvent::Watchdog>, if possible, so it executes as late as
191			possible). The signal handlers remember the signal and then call C<exit>,
192			invoking the C<END> callback.
193
194			The END block then checks for an exit code of C<255>, in which case
195			nothing happens (C<255> is the exit code that results from a program
196			error), otherwise it runs all C<on_exit> hooks and waits for their
197			completion using the event loop.
198
199			After all C<on_exit> hooks have finished, the program will either be
200			C<exit>ed with the relevant status code (if C<exit> was the cause for the
201			program exit), or it will reset the signal handler, unblock the signal and
202			kill itself with the signal, to ensure that the exit status is correct.
203
204			If the program is running under the watchdog, and autorestart is enabled,
205			then the heartbeat is disabled and the watchdog is told that the program
206			wishes to exit within C<60> seconds, after which it will be forcefully
207			killed.
208
209			All of this should ensure that C<on_exit> hooks are only executed when the
210			program is in a sane state and data structures are still intact. This only
211			works when the program does not install it's own TERM (etc.) watchers, of
212			course, as there is no control over them.
213
214			There is currently no way to unregister C<on_exit> hooks.
215
216			=cut
217
218			our @ON_EXIT;
219			our %SIG_W;
220			our $EXIT_STATUS; # >= 0 exit status; arrayref => signal, undef if exit was just called
221
222			# in case AnyEvent::Watchdog is not loaded, use our own END block
223			END { $AnyEvent::Watchdog::end && &$AnyEvent::Watchdog::end }
224
225			sub _exit {
226			$EXIT_STATUS = $? unless defined $EXIT_STATUS;
227
228	root	1.3	# we might have two END blocks trying to call us.
229	root	1.1	undef $AnyEvent::Watchdog::end;
230
231	root	1.3	if (enabled) {
232			undef $HEARTBEAT_W;
233			restart_in 60;
234			}
235
236	root	1.1	my $cv = AE::cv;
237			my $cb = sub { $cv->end };
238
239			$cv->begin;
240			while (@ON_EXIT) {
241			$cv->begin;
242			(pop @ON_EXIT)->($cb);
243			}
244			$cv->end;
245			$cv->recv;
246
247			if (ref $EXIT_STATUS) {
248			# signal
249			# reset to default, hopefully this overrides any C-level handlers
250			$SIG{$EXIT_STATUS->[0]} = 'DEFAULT';
251	root	1.3
252	root	1.1	eval {
253			# try to unblock
254			require POSIX;
255
256			my $set = POSIX::SigSet->new;
257			$set->addset ($EXIT_STATUS->[1]);
258			POSIX::sigprocmask (POSIX::SIG_UNBLOCK (), $set);
259			};
260	root	1.3
261	root	1.1	# now raise the signal
262			kill $EXIT_STATUS->[1], $$;
263
264			# well, if we can't force it even now, try exit 255
265			$? = 255;
266			} else {
267			# exit status
268			$? = $EXIT_STATUS;
269			}
270
271			}
272
273			sub on_exit(&) {
274			unless ($AnyEvent::Watchdog::end) {
275			$AnyEvent::Watchdog::end = \&_exit;
276
277			push @ON_EXIT, $_[0];
278
279			for my $signal (qw(TERM INT XFSZ XCPU)) {
280			my $signum = AnyEvent::Base::sig2num $signal
281			or next;
282			$SIG_W{$signum} = AE::signal $signal => sub {
283			$EXIT_STATUS = [$signal => $signum];
284			exit 124;
285			};
286			}
287			}
288			}
289
290			=back
291
292			=head1 SEE ALSO
293
294			L<AnyEvent>.
295
296			=head1 AUTHOR
297
298			Marc Lehmann <schmorp@schmorp.de>
299			http://home.schmorp.de/
300
301			=cut
302
303			1
304