Some Dagster Plus Serverless Runs Failing to Start

Incident Report for Dagster Cloud

Resolved

The fix for this problem is now live.

Posted May 14, 2024 - 20:16 UTC

Identified

Our cloud provider has identified the root cause of the problem and is in the process of rolling back the change that caused it. We'll continue to provide updates as we work to resolve the issue.

Posted May 14, 2024 - 19:57 UTC

Update

We are continuing to investigate the root cause of this issue with our cloud provider. We are rolling out a change to add additional automatic retries to runs that are affected by this issue.

Posted May 14, 2024 - 19:34 UTC

Investigating

Some Dagster Plus Serverless runs are failing to start with an "Agent detected that run worker failed." error. We are working with our cloud provider to investigate the root cause of the problem as well as adding automatic retries for the underlying error that is causing the runs to fail to start.

Posted May 14, 2024 - 18:53 UTC

This incident affected: API.