Get Slack notifications when Serilog records errors

Introduction

Here at Swoop we work really hard to prevent errors creeping into our software but inevitably problems will occasionally occur. When they do it is critical that we know about them as soon as possible. Observability is a key requirement of any logging system, because of course if we don’t know about errors, we are not going to be able to fix them. Therefore being proactively notified about errors is a must-have requirement. Doing this gives us the best chance of fixing problems before they have a chance to cause too much damage. It also enables us to respond quickly to any users who encounter the issue.

All of the APIs we develop are .NET or .NET Framework based and our logging infrastructure needs to work consistently across both of these platforms. To enable this, we use the Serilog framework as it’s a mature package that has a myriad of plug-ins available that allow logs to be written to different data stores. These range from text file to email to pretty much any serious database you can think of. Our main log storage platform is Elasticsearch and it provides us with a great way of storing all of our log data and the capability to search. However we decided to look for other options when it came to sending error notifications.

We use Slack company-wide for instant messaging and of course it comes with built-in notification capabilities across a range of operating systems and devices. Therefore using it for error notifications was a no-brainer and as it happens there is an existing open source Serilog plug-in that enables writing of log events to Slack and it requires just a few simple steps to enable. Before we get into the code though, here are some hints and tips.

• In theory every log event could be written to Slack but we are careful that only the really important ones are sent as too many notifications will quickly lead people to start ignoring all of them.
• We have created a separate Slack channel for each environment, so that development and test errors aren’t being mistaken for errors in production and vice versa.

Now, let’s dive into the steps needed to configure this setup. We’re assuming that you already have Slack setup and that you’ve configured Serilog in your application.

Configuring Slack

• Create a new Slack channel that will be used when writing the log events.

• Create a new Slack application or use an already existing one.
• Within that Slack application, activate the Incoming Webhooks feature.

• Add a new webhook to the workspace and assign it to the newly created Slack channel.
• Store the Webhook URL that is generated securely in a password manager or secret vault. 

Configuring the application

• Install the Serilog Slack sink NuGet package.

Install-Package Serilog.Sinks.Slack

• Update the Serilog configuration with the address of the Slack app and tell it to only write events that have a level of Error or above. 

Log.Logger = new LoggerConfiguration()
…… Existing configuration settings ……
    .WriteTo.Slack(new SlackSinkOptions()
     {
         WebHookUrl = “https://hooks.slack.com/services/???",
          RestrictedToMinimumLevel = "Error"
     })
     .CreateLogger();

There are quite a few other options that can be specified to further control which events are written and the properties that are sent to Slack so take the time to investigate the NuGet package’s documentation. 

Conclusion

From now on whenever an error occurs in the application a new message that looks like this will be immediately posted to the selected Slack channel.