Affected services:

  • Crane

Crane: /work filesystem unplanned downtime

Opened on Tuesday 14th July 2020, last updated

Resolved

The /work filesystem on Crane is back in service.

We do not believe there was data loss or corruption. Jobs which ended between 4:15pm and 7:00pm may have incomplete output and we encourage you to check them for consistency.

Posted by John Thiltges

Investigating

One of the OSS storage servers backing Crane's /work filesystem unexpectedly rebooted at 4:17pm today. We are currently investigating the issue and checking the filesystem.

Expect degraded performance and possible issues accessing certain files. Pending jobs will be held until the filesystem is back in service. We will work to resolve this as soon as possible and provide updates as necessary.

Posted by John Thiltges