To reach the same level of simplicity for NodeJS developers I late last year created an NPM package called failure-lambda (https://github.com/gunnargrosch/failure-lambda). The goal with failure-lambda is in short to have an easy way to do failure injection in AWS Lambda using several different failure modes. To make it even easier I decided to use a single wrapper and instead have the failure mode selectable. That way you don’t have to make code changes if you want to switch between for example latency or exception injection, you just change a setting.
As we know serverless isn’t just about AWS and chaos engineering for serverless isn’t only about AWS Lambda. For that reason, there are now also the same failure injection options for NodeJS developers building serverless using Azure Functions and Cloud Functions. This with the NPM packages failure-azurefunctions (https://github.com/gunnargrosch/failure-azurefunctions) and failure-cloudfunctions (https://github.com/gunnargrosch/failure-cloudfunctions).
Failure modes and rate of failure
Even though this all started with latency injection as in Yan Cui’s articles, latency is far from the only possible failure we can have in our serverless applications. In failure-lambda, failure-azurefunctions and failure-cloudfunctions there are now five different failure modes to choose from:
Latency – Injects latency to the executed function, controlled using a minimum and maximum span of milliseconds. This can for example be used to simulate service latency or to test and help set your timeout values.
Exception – Throws an exception in the function. Helps you test how your application and code handles exceptions.
Status code – Your function will return a status code of choice, for instance 502 or 404 instead of the normal 200. This gives you the possibility to test what happens when there are errors.
Disk space – Will fill your temporary disk with files to create a failure. If you’re using disk to store temporary files you can test how your application behaves if that disk gets full or you are unable to store to it.
Blacklist (courtesy of Jason Barto) – Blocks connections to specified hosts. Use to simulate services or third parties being unavailable.
All these failure modes can be used together with a rate of failure that you set. The default is to inject failure on every invocation but in reality, it is likely that for example a third party is unavailable on 50% of the calls made to that host or that an exception is thrown on a quarter of the invocations. Setting rate will allow you to achieve this.
