[Development] Our CI is suffering from malfunctions currently

[Tony Sarajärvi]

We don't have tier1 or tier2 images available for the hosts, so they can't do much currently. It was a close call that would have lost them permanently as well, but luckily it seems that btrfs restore can still read them.

We really need to have tier1 images stored somewhere else as well. The NFS on the compellent running on btrfs is not safe enough. We need more copies distributed and that's something we'll P0 prioritize after we get things running.

-T

-----Original Message-----
From: Volker Hilsheimer <volker.hilsheimer at qt.io> 
Sent: Wednesday, June 24, 2020 10:57 AM
To: Tony Sarajärvi <tony.sarajarvi at qt.io>
Cc: development at qt-project.org
Subject: Re: Our CI is suffering from malfunctions currently

Hi Tony,

Thanks for the update, and good luck with the fixing!

What are the implications? Should we not stage anything until you give the all-clear?

Cheers,
Volker


> On 24 Jun 2020, at 09:30, Tony Sarajärvi <tony.sarajarvi at qt.io> wrote:
> 
> Seems the BTRFS file system is permanently busted. We have to recreate the thing and I estimate it taking the rest of the day. Sorry!
>  
> -T
>  
> From: Tony Sarajärvi 
> Sent: Wednesday, June 24, 2020 8:13 AM
> To: Qt development mailing list <development at qt-project.org>
> Subject: Our CI is suffering from malfunctions currently
>  
> Hi
>  
> Yesterday our NFS server’s root partition got filled up by logs apparently. As we got everything fixed, it did not take long for it to start filling up again. Within ours it got filled up again. So, the CI is currently down until we fix things…
>  
> Bear with us
> -Tony