March 7, 2025|12:22 PM
Whether it’s IT operations, cloud migration, or AI-driven innovation – let’s explore how we can support your success.
Creating modern applications using serverless technology and managed services means that anyone can build distributed and highly available systems without worrying about the underlying infrastructure. But it’s not all fun and games. This talk I did at AWS re:Invent 2019 with the title “Performing chaos engineering in a serverless world” explains some of the challenges with serverless, common weaknesses in serverless applications as well as challenges with using chaos engineering in serverless. Please refer the video.
Even though this all started with latency injection as in Yan Cui’s articles, latency is far from the only possible failure we can have in our serverless applications. In failure-lambda, failure-azurefunctions and failure-cloudfunctions there are now five different failure modes to choose from:
Injects latency to the executed function, controlled using a minimum and maximum span of milliseconds. This can for example be used to simulate service latency or to test and help set your timeout values.
Throws an exception in the function. Helps you test how your application and code handles exceptions.
Your function will return a status code of choice, for instance 502 or 404 instead of the normal 200. This gives you the possibility to test what happens when there are errors.
Will fill your temporary disk with files to create a failure. If you’re using disk to store temporary files you can test how your application behaves if that disk gets full or you are unable to store to it.
Blocks connections to specified hosts. Use to simulate services or third parties being unavailable.
All these failure modes can be used together with a rate of failure that you set. The default is to inject failure on every invocation but in reality, it is likely that for example a third party is unavailable on 50% of the calls made to that host or that an exception is thrown on a quarter of the invocations. Setting rate will allow you to achieve this.
