Don't lose sight of your bugs: How to improve your defect capture by 20%

Here we assume you correctly set up Sentry. Both your front-end and your back-end are able to send errors to Sentry and you set up Sentry releases (https://docs.sentry.io/workflow/releases/?platform=javascript).

Checkpoints

This is what we’ll want to achieve

ALL unexpected behaviours (bugs) are captured in Sentry: no false negative
No expected behaviours are captured in Sentry: no false positive
Sentry issues contain all data needed to debug the issue

The first point is about identifying all bugs, the second about avoiding noise. This is all about making the workplace clean, uncluttered, safe, and well organized to help reduce waste and optimize productivity: see 5s.

Capture ALL exceptions

Why aren’t all exceptions caught

Out of the box, Sentry works by listening to uncaught exceptions in your application. In addition, there are some integrations to listen to some technology-specific errors: eg redux-sentry-middleware.

Something to be aware of is that Sentry does not automatically capture caught exceptions: if you wrote a try/catch without re-throwing an exception, this exception will never appear in Sentry. Example:

async function getTodoList() {
  try {
    return await axios.get('/todos')
  } catch (error) {
    toaster.error('Error while retrieving todos')
    // `error` will never appear in Sentry! The only way for you to
    // know that this bug happens is waiting for users to report it,
    // and even then you'll have no data to debug it
    return []
  }
}

This is called “silencing” or “swallowing” exceptions.

Manually capture exceptions

The user experience of the code snippet above is fine: we show a toaster and do not crash the entire application. The only problem is with reporting: we need to know that this bug happened.

Sentry gives you an easy tool for this, that allows you to manually capture exceptions:

async function getTodoList() {
  try {
    return await axios.get('/todos')
  } catch (error) {
    Sentry.captureException(error) // this exception will now show up on Sentry
    toaster.error('Error while retrieving todos')
    return []
  }
}

Summary: exception handling rule

When you catch an exception, you have 2 exclusive choices:

re-throw it (throw error in Javascript)
Sentry.captureException

If this exception is a “normal” behaviour, make it clear with a comment. This should be very rare, you shouldn’t use exceptions as “normal” control flow! But Python/Django sometimes do not leave you much choice, sadly.

Capture exceptions at the right place

This mostly applies to backend.

When an exception happens, it’s usually quite deep down the call stack: for example accessing the database in a Django model’s method called by a service called by a Django view. You want all exceptions to be captured, so you have 4 options here:

Let the exception propagate until it crashes your request (HTTP 500 internal error) or Celery task. That would be automatically caught by Sentry
Capture the exception as close to its source as possible: in the model’s method
Capture the exception where there’s most business logic context: in the service
Capture the exception as close to the app’s edges as possible: in the Django view

Remember the exception handling rule: any exception caught should be either re-thrown, or manually captured. This means that you don’t really have any choice:

An exception usually is due to an internal error: it should absolutely crash your request or your Celery task. It is important so that your logs and metrics are accurate, don’t hide problems to yourself.
If for some reason you do not want the exception crashing your request/celery task (maybe it’s something not-so-important that is acceptable to fail), you want to handle it as close to the edge of your application as possible: in the Django view for example. This allows all intermediate functions to have a chance to enrich the exception (raise SomeException from err in Python, or Sentry breadcrumbs) and is a good practice to avoid your core services making too much assumptions.

Remember that the exception handling rule is an either/or: you should not both capture the exception and re-raise it. The following is wrong:

def getById(id):
  try:
    User.objects.get(id=id)
  except Exception as e:
    Sentry.capture_exception(e) # This is wrong!
    raise UserNotFound from e # This is the exception that should be caught, higher in the call stack

It is wrong because it will lead to your exceptions being captured by Sentry multiple times, which means multiple Sentry issues for the same underlying problem, which means noise. Noise is bad, see 5S principles.

Capture non-exception bugs

Most bugs should trigger an exception (that you raise yourself if needed): invalid input, invalid state… But in some (hopefully rare) cases you might need to capture a bug in Sentry but still continue your work. For this, you can use Sentry.captureEvent() to capture fully-custom events.

This should be very rare: if you’re in an invalid situation (bad input for example), probably you should raise an exception and let the caller handle it!

How to add extra data to Sentry events

Capturing the information is nice, but sometimes you have some data available in the code that you’d like to make available to Sentry for later debugging. Make extensive use of it: this makes the difference between an instant fix and a half-day debugging session because you can’t reproduce a bug!

Breadcrumbs

Breadcrumbs are basically info logs. They are not errors themselves: they are only sent to Sentry when an error is actually captured. It allows you to get more info on what happened in the lifecycle of a request (backend) or in the user journey (frontend). You can probably assume that every log should also be a breadcrumb.

They are very easy to use, see the Sentry docs

function onLoginSubmit(event) {
  // ...
  Sentry.addBreadcrumb({
    category: 'auth',
    message: 'User logged in ' + user.email,
    level: Sentry.Severity.Info
  })
}

Context

You can set some “context” (set of key-value pairs, called “tags” or “extra data”) to send to Sentry if and when an error is captured. For example, you might want to send the current user ID, the ID of the model targeted in a Django view, the referrer header, a tracing ID… For this you can use configureScope (https://docs.sentry.io/enriching-error-data/context/?platform=javascript#extra-context).

By default, the scope of the context is the entire request (backend) or lifetime of the app (frontend). You can set a scope local to a block of code (https://docs.sentry.io/enriching-error-data/scopes/?platform=javascript#local-scopes)

Extra data

Arbitrary key-value pair, that you can see on Sentry issues. You cannot filter issues based on these.

Example: value of local variable, message, timestamp…

Summary

Never swallow exceptions
Never captureException AND raise
In the backend, captureException as close to the edge as possible (Django views, Celery tasks)
Use configureScope extensively to add data to errors captured by Sentry