unscheduled hypervisor reboot

classic Classic list List threaded Threaded
11 messages Options
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

unscheduled hypervisor reboot

Hello,

earlier today, one of our hypervisors rebooted and brought down a couple
of services.

Infra is currently working on bringing the services back online again.

Amongst services currently AFFECTED are:

- BugZilla: available, but throws error messages for certain write
operations

- the TDF and LibreOffice website

- the help system

- the pad

- the download archive

NOT affected are e.g. Gerrit, Wiki, AskBot, mailing lists, Jitsi, SSO,
Redmine and the main download website.

The infra team is available at the #tdf-infra IRC channel on Freenode
for questions. The status website at
https://status.documentfoundation.org will be updated as soon as there
are news.

We apologize for the inconveniences.

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Hello,

a quick update on the situation. Still AFFECTED are:

- the main websites documentfoundation.org and libreoffice.org
- the pad
- OpenGrok
- the downloadarchive
- some Bugzilla tables are affected too, we're trying to reindex these

Infra is working on bringing the services up bit by bit, with the
priority on the website, and then will analyse the situation to find out
why the hypervisor rebooted itself.

The infra team is available at the #tdf-infra IRC channel on Freenode
for questions. The status website at
https://status.documentfoundation.org will be updated as soon as there
are news.

We apologize for the inconveniences.

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Hello,

another update:

- the main websites documentfoundation.org and libreoffice.org are back
ONLINE

- OpenGrok is back ONLINE

- the download archive is back ONLINE

- the pad is still OFFLINE

- Bugzilla is ONLINE; if you spot errors, please let us know

The infra team is available at the #tdf-infra IRC channel on Freenode
for questions.

Thanks to Cloph for looking into the issues immediately after they were
spottted!

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
m.a.riosv m.a.riosv
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Hi Florian,
seems Ask is partially broken, not all but in notifications and activity I
receive the message:

/Internal server error

system error log is recorded, error will be fixed as soon as possible
please report the error to the site administrators
 /

Miguel Ángel




--
Sent from: http://document-foundation-mail-archive.969070.n3.nabble.com/Website-f1644957.html

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Hello,

m.a.riosv wrote:
> seems Ask is partially broken, not all but in notifications and activity I
> receive the message:

thanks a lot for the report - we will have a look and I'll send an
update to this list.

In the meantime, all the other services are up and running again, since
yesterday evening (Berlin time).

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

In reply to this post by m.a.riosv
Hello,

m.a.riosv wrote:
> seems Ask is partially broken, not all but in notifications and activity I
> receive the message:
>
> /Internal server error
>
> system error log is recorded, error will be fixed as soon as possible
> please report the error to the site administrators
>   /

to add another finding we received from a user: When editing questions
or comments, the error

        "invalid page in block 20670 of relation base/16386/14472111"

seems to be thrown.

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
m.a.riosv m.a.riosv
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Guilhem Moulin Guilhem Moulin
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

On Wed, 31 Jul 2019 at 13:10:06 -0700, m.a.riosv wrote:
> the issues in Ask are still there.

Working on it.  See https://status.documentfoundation.org and the
message I sent yesterday to that very thread.

--
Guilhem.

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
m.a.riosv m.a.riosv
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

Thanks Guilhem,
the message was thinking to avoid the matter from being forgotten, not to
press. Although It's bit annoying when you are used to, at least myself can
live without it.
Regards.
Miguel Ángel.



--
Sent from: http://document-foundation-mail-archive.969070.n3.nabble.com/Website-f1644957.html

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
Florian Effenberger Florian Effenberger
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

In reply to this post by Florian Effenberger
Hi,

Guilhem Moulin wrote:
> AFAICT the corruption is now fully resolved, and no data was lost.

that's good news, thanks a lot for your work!

Florian

--
To unsubscribe e-mail to: [hidden email]
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy
m.a.riosv m.a.riosv
Reply | Threaded
Open this post in threaded view
|

Re: unscheduled hypervisor reboot

In reply to this post by m.a.riosv