NRAO Status

At approximately 2PM Thursday, June 20, ‘cvfiler’ experienced a disk failure which impact the performance of one the volumes. This impact was severe enough to bring Unix and Exchange email services to a halt. Email services were down until around 5PM when the volume stabilized (after a power-cycle) and stale lock issues were resolved.

We are currently working with NetApp to verify the cause of the failure and determine if further maintenance is needed to reduce the possibility of a recurring failure.

Update: There was another brief outage on Friday at 3PM but services were restored shortly thereafter. At this time we believe the filer is stable but we are still investigating the root cause of the issue with the system vendor.

An apparent disk issue with cvfiler has forced email services offline. We are currently working to power-cycle the filer in hopes that this resolves the issues.

Update: 6PM – The filer and all mail services is back online after an apparent hardware glitch. The vendor is being brought in to determine what went wrong so we are leaving this case open. All services are available at this time.

Power has been restored to CV Edgemont Road. Services are being restored as quickly as possible. Email should be functioning normally at this time, although a moderate amount of spam mail has slipped past a malfunctioning filter during startup.

Update: 9:20PM – Web services, including www.nrao.edu, science.nrao.edu and helpdesks have been restored.