pg_crash: Crashing PostgreSQL automatically

PostgreSQL is a rock-solid database that is widely used for highly critical applications. Bugs are rarely seen. However, everything fails once in a while. The entire stack (hardware, operating system, etc) are subjected to occasional failure. Thus one has to prepare for that.

The main problem is: How can one simulate failure? In many cases manual tests are made. People test their cluster (Patroni, repmgr, etc …) a couple of times and this is it. The downside of this approach is that many corner cases are not properly discovered and are only found later in production. A better approach is needed. Doing a handful of failovers is not enough to ensure 24x7 operations in a reliable and trustworthy manner.

pg_crash: Professional database torture

One way to approach the problem is to use pg_crash. What does pg_crash do? It simply crashes your database automatically by using signals of your choice.
Here is how the configuration works:

The first thing to do is to load pg_crash into your server. The way to do that is to use the shared_preload_libraries setting. Once the package has been installed all you need to do is to add pg_crash to this variable and restart the server (shared libraries are loaded on startup). Then we can tell pg_crash how often to send which kind of signal.

This gives you some flexibility.

Running pg_crash over a long period of time helps to detect hidden flaws in your setup or even in your database architecture. We recommend using pg_crash for QA to uncover potential issues before they kill your production system.

Finally…

If you want to learn more about PostgreSQL right now we want to recommend one of our recent works: How can one use pg_stat_statements for troubleshooting? It will give you some vital insights into one of the most powerful and useful extensions currently available for PostgreSQL.
Also, if you want to build PostgreSQL applications more rapidly than ever before we suggest to check out CYPEX which is an excellent tool to build PostgreSQL applications.