Ruby Graceful Application Shutdown with SignalException and SIGTERM
In Container-based DevOps solutions like Kubernetes, ECS, Heroku an application must be able to stop accepting new client requests before termination, and, most importantly, it must successfully complete already running requests and processes.
By default in most systems Graceful Shutdown implements by sending SIGTERM
signal and 30 seconds delay before terminating the instance.
How to handle SIGTERM in Ruby?
There are two similar ways:
- Using Signal.trap
- Rescuing SignalException
Both constructions allow handling Linux signals, below I will give some examples with handling SIGTERM
and describe the differences. Also, you can use the full list of POSIX signals.
Catch Signal with Trap
I will write an app that publishes messages to the queue and pop messages from the queue. To illustrate graceful shutdown messages will be published faster than the app will consume them and after receiving SIGTERM
process the rest of the messages.
touch signal_trap.rb
Run that app ruby signal_trap.rb
and copy PID to send a signal.
App PID: 32103Pop:
Queue: [9]
Queue: [9, 5]
Queue: [9, 5, 0]
Pop: 0
Queue: [9, 5, 3]
Pop: 3
Queue: [9, 5, 5]
Queue: [9, 5, 5, 4]
Send SIGTERM
from the terminal.
kill -s 32103 TERM
The application will process the rest of the messages before terminating.
Received Signal
Process the remaining messages:
9
5
5
4
In the trap
method block argument needs to use exit
to close the application, otherwise, it will continue working.
Rescuing SignalException
Now let’s write an app with the same shutdown behavior by using SignalException
touch signal_exception.rb
Run that app ruby signal_exception.rb
and copy PID to send a signal.
App PID: 82730Pop:
Queue: [7]
Queue: [7, 8]
Queue: [7, 8, 7]
Pop: 7
Queue: [7, 8, 2]
Pop: 2
Queue: [7, 8, 8]
Queue: [7, 8, 8, 4]
Send SIGTERM
from the terminal.
kill -s 82730 TERM
The application will process the rest of the messages before terminating through logic in rescue
block.
Received Signal SIGTERM
Process the remaining messages:
7
8
8
4
Behavior will be almost the same as with Signal.trap
, but in this case, the application handles any signal.
SignalException vs Signal.trap
Signal.trap
overrides other signal handlers, so this can lead to conflicts with handlers in used libraries. For most applications the best choice would be using SignalException
, especially if external process control gems are used.