Error Handling

How an application handles errors ultimately also decides how the application fails. As we strive to produce very reliable software, it is important we get our error handling right. So this document includes some best practices for error handling that this project should follow.

In general this follows a priority list, although exceptions might exist:

1. Errors that can be avoided should be avoided. E.g. use strongly typed function parameters instead of parsing inputs repeatedly.
2. If errors can be handled gracefully they should be handled. Errors should be handled as closely to the source as possible.
3. Errors that can’t be handled should be clear, descriptive, but also not leak sensitive information across external API boundaries.
4. If nobody can handle an error, the application should fail early and predictably.

Panicking

In the ideal version of our code, a panic never happens, no matter how badly a user may mess up. Any code that the compiler thinks may panic, should have an annotation, describing why this can never happen:

  let a = vec![0];
  // SAFETY: We just created a vector with one element, so it will have a first
  // element
  #[allow(clippy::unwrap_used)]
  let first = a.get(0).unwrap();

Instead, we generally use the thiserror crate to make it easier to define errors.

In services, we try to use thiserror for custom error types when it makes sense to do so. color_eyre may also be used, but this should be done sparingly and only when a custom error type is too much boilerplate than it’s worth. When handling an error with color_eyre, the actual error must be passed, any text representation of the error should be attached with .context().

For invalid configuration, we also don’t use panics, for aesthetic reasons. The code

panic!("Missing environment variable");

returns

thread 'main' panicked at /app/example.rs:2:5:
Missing environment variable
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

which is just down-right ugly. Instead, with an error type, we can use the typing system for our errors, define them centrally, and completely control how they’re displayed.

Parse Don’t Validate

To avoid errors as much as possible, we should use strongly typed parameters. This means that the function:

pub fn add_sender_to_event(event: &mut Event, sender: String) -> Result<(), SenderUserIdIsInvalid>

is bad. Instead, we should prefer

pub fn parse_user_id(user_id: String) -> Result<UserId, UserIdIsInvalid>

pub fn add_sender_to_event(event: &mut Event, sender: UserId)

This has the advantage that all error handling, in regards to the format of the user ID that we are working with, is done only once, instead of every function having to handle the error.

Low-Level Library Functions.

For low-level library functions, that may be used in multiple places by different components in Arminius, it’s often a good idea to define one error type per function. In some cases similar functions might share an error type, if they fail in the same ways. Such as in crates/libs/arminius-lib-roomservice-client/src/subject_builder.rs::

/// Errors that occur when unescaping a NATS token
#[allow(missing_docs)]
#[derive(Debug, thiserror::Error, Serialize, Deserialize, PartialEq)]
pub enum TokenEscapeError {
	#[error("Unknown escape code in token {0}")]
	UnknownEscapeCode(char),
	#[error("Last character of escaped token is backslash")]
	LastCharIsBackslash,
}

/// Unescape a string that has been replaced with [escape_token]
fn unescape_token(escaped_token: &str) -> Result<String, TokenEscapeError>

This code is nice, because it’s modular. Everything you need to know about the unescape_token function, and how it can go wrong is contained in the same file as the function, which makes it easy to find, and easy to modify without it impacting the rest of our code.

Returning Errors to Users

Note

For more information on #[derive(MatrixApiError)], see the relevant rustdocs

Error types that return to users should implement the MatrixApiError trait (when returned over an API specified by the Matrix specification). A good example is our implementation of the Matrix endpoint GET /_matrix/client/*/user/{userId}/account_data/{type}:

/// Error for when unset account data is queried.
#[derive(thiserror::Error, Debug, MatrixApiError)]
#[error("Account data not found")]
#[matrix_api_error(status_code = StatusCode::NOT_FOUND, error_kind = ErrorKind::NotFound)]
struct AccountDataNotFound;

#[allow(missing_docs, clippy::missing_docs_in_private_items)]
#[derive(thiserror::Error, Debug, MatrixApiError, FromDbError)]
enum GetAccountDataError {
	/// Unrecoverable error.
	/// See <https://bundesmessenger.gitlab.io/backend/neo/arminius/arminius/03-development/errors.html#unrecoverable-errors>
	#[error(transparent)]
	InternalServerError(#[from] InternalServerError),
	#[error(transparent)]
	AccountActionNotAuthorized(#[from] AccountActionNotAuthorized),
	#[error(transparent)]
	AccountDataNotFound(#[from] AccountDataNotFound),
}

/// Endpoint for getting global account data.
async fn global_account_data_get(
	database: State<Database>,
	AuthenticatedClaims(claims): AuthenticatedClaims,
	RumaRequest(request): RumaRequest<get_global_account_data::v3::Request>,
) -> Result<ArminiusResponse<get_global_account_data::v3::Response>, GetAccountDataError> {
	auth::check_user_matches(claims.user_id.as_ref(), request.user_id.as_ref())?;

  let time_tomorrow = chrono::Utc::now()
    .checked_add_days(1)
    .unrecoverable_error("Timestamp incorrect, near overflow")?;

	Ok(ArminiusResponse(get_global_account_data::v3::Response::new(Raw::from_json(
		db::account_data::fetch_global_account_data_by_user_and_type(
			database.get_pool(),
			request.user_id.as_str(),
			&request.event_type.to_string(),
		)
		.await?
		.ok_or(AccountDataNotFound)?,
	))))
}

To define an individual error type, we define the struct AccountDataNotFound. We use the thiserror::Error derive macro to make a rust error type, which uses the error attribute to set an error message. We then use the MatrixApiError trait to extend this error type to a MatrixAPIError, which using the attribute matrix_api_error will have the correct HTTP status code, and matrix error kind when turned into a response. The error message is taken from the Rust Error.

For the function global_account_data_get, we then define an enum of the errors this function can return (these enums will often end up shared between functions that return similar errors). Using the MatrixApiError macro, we implement MatrixApiError for the enum. When GetAccountDataError is turned into a response, it will use the MatrixApiError trait of the corresponding value, to create the HTTP response.

In the function, we return errors at three places:

• On the first line, we call check_user_matches, which may return AccountActionNotAuthorized, handling it simply with ?
• Further down, we get the time tomorrow. If adding a single day to the current time overflow, we cannot recover from this, and use .unrecoverable_error (See the next section)
• Further down, we use the From<DatabaseError> implementation provided by the #[derive(FromDbError)] attribute on GetAccountDataError, as a substitute for .unrecoverable_error
• Finally, if the database returns a None, we return AccountDataNotFound

Tip

You don’t have to implement one error type for each handler. In client-component/src/account_data.rs, the setters and getters for global and room-local account data ended up with the same error type.

Unrecoverable Errors

Note

For more information about InternalServerError, see the relevant rustdoc

In some cases the type system does not allow us to express, that certain errors should never happen. An example of this might be that a string in the database does not follow the usual schema (e.g. a wrongly formatted user ID, such as ":jim@homeserver.gov"). If an administrator manually modified those strings to not follow our schema, we have no way to recover from this.

For these kind of errors, we create an InternalServerError. For any Result type, we have an extension trait which provides the function unrecoverable_error(self, context: str), which maps any error into an InternalServerError. If there is an error, the context string and callsite will be logged, and the user will get

  {
		"errcode": "M_UNKNOWN",
		"error": "Internal server error"
	}

Errors in Tests

To make tests easier to develop, we have all our tests return color_eyre::eyre::Result<()>. Instead of having to write .expect("some message") after every theoretically fallible function which should never actually fail, we can just use the try-operator ?. And we get prettier output, that’s easier to parse at a glance. There is an ast-grep rule that checks whether all test functions actually return color_eyre::eyre::Result<()>.