Service Unavailable
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Additionally, a 503 Service Unavailable error was encountered while trying to use an ErrorDocument to handle the request.

While accessing one of the website saw the error mentioned above

Normally this 503 error are related to module PHP-fpm.

You just need to restart it from WHM to fix the issue

This issue is due to some core files in wordpress being corrupt.

The solution to this issue is to download new version of wordpress and upload new files and folders mentioned below

wp-includes , wp-admin and below files

wp-activate.php
wp-blog-header.php
wp-comments-post.php
wp-config.php
wp-config-sample.php
wp-cron.php
wp-links-opml.php
wp-load.php
wp-login.php
wp-mail.php
wp-settings.php
wp-signup.php
wp-trackback.php
xmlrpc.php

This resolved the issue and wordpres site was working fine

We had a Inoddb database crash in server , Where Mysqld was not starting.

So first we checked on what databases tables are crashed or corrupted using

command mysqlcheck -A –all-databases and repaired it using

mysqlcheck --auto-repair –all-databases

You can check in below URL for additional functions using mysqlcheck

https://mariadb.com/kb/en/mysqlcheck/

First thing we did was got mysql in recovery mode using innodb_force

As with 0,1,2 mysql was not coming up we used value 3 to get innodb online in read only mode.

innodb_force_recovery = 3

now as mysql was online we made a full dump of mysql which include all databases

mysqldump –all-databases > all_databases.sql

Once complete dump was ready

We stopped the Mysql services using below command

whmapi1 configureservice service=mysql enabled=1 monitored=0

,We moved the the

mv /var/lib/mysql /var/lib/bkp_mysql

/scripts/restartsrv_mysql –stop

We now reinitialized the mysql using below command

mysqld –initialize –user=mysql

Now we restored the dump which we had created using command

Mysql Mysql < all_databases.sql

After the dump was restored , The mysql was restarted

/scripts/restartsrv_mysql –start

whmapi1 configureservice service=mysql enabled=1 monitored=1

Mysql root password was reset using below command

/scripts/mysqlconnectioncheck

I was getting error “Row size too large (> 8126). Changing some columns to TEXT or BLOB… | Mysql / MariaDB” while dumping database

The fix to this issue is to add below parameter in /etc/my.cnf

innodb_strict_mode = 0


and restart mysql

While Checking Mysql error logs could see below error

2021-03-31 0:04:00 1222 [Warning] InnoDB: Table mysql/innodb_index_stats has length mismatch in the column name table_name. Please run mysql_upgrade
2021-03-31 0:04:00 1222 [Warning] InnoDB: Table mysql/innodb_table_stats has length mismatch in the column name table_name. Please run mysql_upgrade
2021-03-31 0:04:00 1222 [Warning] InnoDB: Table mysql/innodb_index_stats has length mismatch in the column name table_name. Please run mysql_upgrade

To fix this issue you need to use command

mysql_upgrade

Reference is from https://forums.cpanel.net/threads/cpanel-22320-table-mysql-innodb_index_stats-has-length-mismatch.641493/

Had an opportunity an Operation Notification open source application callled Cachet

Below are the steps I used to successfully complete it and get it working

SSH to Ubuntu server

Set up right time on server using

root@status:~# sudo dpkg-reconfigure tzdata

Current default time zone: ‘America/Chicago’
Local time is now:      Mon Feb 22 10:48:44 CST 2021.
Universal Time is now:  Mon Feb 22 16:48:44 UTC 2021.

Next update the existing system using

root@status:~# sudo apt update && sudo apt upgrade -y

Next install the PHP7 and mariadb using below command

root@status:~# sudo apt install -y php php-cli php-fpm php-common php-xml php-gd php-zip php-mbstring php-mysql php-pgsql php-sqlite3 php-apcu

Next start mysql in secure mode

root@status:~# sudo mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we’ll need the current
password for the root user.  If you’ve just installed MariaDB, and
you haven’t set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on…

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

You already have a root password set, so you can safely answer ‘n’.

Change the root password? [Y/n] n
 … skipping.

By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
 … Success!

Normally, root should only be allowed to connect from ‘localhost’.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
 … Success!

By default, MariaDB comes with a database named ‘test’ that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] y
 – Dropping test database…
 … Success!
 – Removing privileges on test database…
 … Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y
 … Success!

Cleaning up…

All done!  If you’ve completed all of the above steps, your MariaDB
installation should now be secure.

Next Create database and user and assign database to user

root@status:~# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 47
Server version: 10.1.47-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MariaDB [(none)]> CREATE DATABASE nocdb;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> GRANT ALL ON nocdb.* to nocusr@localhost identified by “your-password”;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> quit
Bye

Now install Nginx on server using below command

root@status:~# sudo apt install -y nginx

Now install git and curl on server using

root@status:~# sudo apt install curl git

Now install composer using

root@status:~# curl -sS https://getcomposer.org/installer | sudo php — –install-dir=/usr/local/bin –filename=composer

Now create folder cachet using

root@status:~# sudo mkdir -p /var/www/cachet

root@status:~# sudo chown -R user.user /var/www/cachet
root@status:~# cd /var/www/cachet

root@status:/var/www/cachet# git clone -b 2.4 –single-branch https://github.com/cachethq/Cachet.git .

root@status:/var/www/cachet# mv .env.example .env

root@status:/var/www/cachet# composer install –no-dev -o

root@status:/var/www/cachet# php artisan cachet:install

 Do you want to configure Cachet before installing? (yes/no) [no]:
 > yes

Environment file already exists. Moving on.
**************************************
*     Application In Production!     *
**************************************

 Do you really wish to run this command? (yes/no) [no]:
 > yes

Application key set successfully.

 Which database driver do you want to use?:
  [mysql ] MySQL
  [pgsql ] PostgreSQL
  [sqlite] SQLite
 > mysql

 What is the host of your mysql database?:
 > localhost

Using ‘localhost’ will result in the usage of a local unix socket. Use 127.0.0.1 if you want to connect over TCP

 What is the name of the database that Cachet should use?:
 > nocdb

 What username should we connect with?:
 > nocusr

 What password should we connect with?:
 >

 Is your database listening on a non-standard port number? (yes/no) [no]:
 > no

 Do you want to use a prefix on the table names? (yes/no) [no]:
 > no

+————-+—————-+
| Setting     | Value          |
+————-+—————-+
| DB_DRIVER   | mysql          |
| DB_HOST     | localhost      |
| DB_DATABASE | nocdb       |
| DB_USERNAME | nocusr      |
| DB_PASSWORD | xx345444$ |
| DB_PORT     | 3306           |
| DB_PREFIX   |                |
+————-+—————-+

 Are these settings correct? (yes/no) [no]:
 > yes

+—————-+——-+
| Setting        | Value |
+—————-+——-+
| CACHE_DRIVER   |       |
| SESSION_DRIVER |       |
| QUEUE_DRIVER   |       |
+—————-+——-+

 Which cache driver do you want to use?:
  [apc      ] APC(u)
  [array    ] Array
  [database ] Database
  [file     ] File
  [memcached] Memcached
  [redis    ] Redis
 > apc

 Which session driver do you want to use?:
  [apc      ] APC(u)
  [array    ] Array
  [database ] Database
  [file     ] File
  [memcached] Memcached
  [redis    ] Redis
 > apc

 Which queue driver do you want to use?:
  [null      ] None
  [sync      ] Synchronous
  [database  ] Database
  [beanstalkd] Beanstalk
  [sqs       ] Amazon SQS
  [redis     ] Redis
 > database

+—————-+———-+
| Setting        | Value    |
+—————-+———-+
| CACHE_DRIVER   | apc      |
| SESSION_DRIVER | apc      |
| QUEUE_DRIVER   | database |
+—————-+———-+

 Are these settings correct? (yes/no) [no]:
 > yes

 Do you want Cachet to send mail notifications? (yes/no) [no]:
 > no

 Do you wish to use Cachet Beacon? (yes/no) [no]:
 > no

 Do you wish to use Emoji? This requires a GitHub oAuth Token! (yes/no) [no]:
 > no

 Do you want to create an admin user? (yes/no) [no]:
 > yes

 Please enter your username:
 > your name

 Please enter your email:
 > your email address

Application key set successfully.
Configuration cache cleared!
Configuration cached successfully!
Route cache cleared!
Routes cached successfully!
Copied Directory [/vendor/laravel/framework/src/Illuminate/Foundation/Exceptions/views] To [/resources/views/errors]
Copied Directory [/vendor/laravel/framework/src/Illuminate/Notifications/resources/views] To [/resources/views/vendor/notifications]
Copied Directory [/vendor/laravel/framework/src/Illuminate/Pagination/resources/views] To [/resources/views/vendor/pagination]
Copied Directory [/vendor/laravel/framework/src/Illuminate/Mail/resources/views] To [/resources/views/vendor/mail]
Publishing complete.
Migration table created successfully.
Migrating: 2015_01_05_201324_CreateComponentGroupsTable
Migrated:  2015_01_05_201324_CreateComponentGroupsTable
Migrating: 2015_01_05_201444_CreateComponentsTable
Migrated:  2015_01_05_201444_CreateComponentsTable
Migrating: 2015_01_05_202446_CreateIncidentTemplatesTable
Migrated:  2015_01_05_202446_CreateIncidentTemplatesTable
Migrating: 2015_01_05_202609_CreateIncidentsTable
Migrated:  2015_01_05_202609_CreateIncidentsTable
Migrating: 2015_01_05_202730_CreateMetricPointsTable
Migrated:  2015_01_05_202730_CreateMetricPointsTable
Migrating: 2015_01_05_202826_CreateMetricsTable
Migrated:  2015_01_05_202826_CreateMetricsTable
Migrating: 2015_01_05_203014_CreateSettingsTable
Migrated:  2015_01_05_203014_CreateSettingsTable
Migrating: 2015_01_05_203235_CreateSubscribersTable
Migrated:  2015_01_05_203235_CreateSubscribersTable
Migrating: 2015_01_05_203341_CreateUsersTable
Migrated:  2015_01_05_203341_CreateUsersTable
Migrating: 2015_01_09_083419_AlterTableUsersAdd2FA
Migrated:  2015_01_09_083419_AlterTableUsersAdd2FA
Migrating: 2015_01_16_083825_CreateTagsTable
Migrated:  2015_01_16_083825_CreateTagsTable
Migrating: 2015_01_16_084030_CreateComponentTagTable
Migrated:  2015_01_16_084030_CreateComponentTagTable
Migrating: 2015_02_28_214642_UpdateIncidentsAddScheduledAt
Migrated:  2015_02_28_214642_UpdateIncidentsAddScheduledAt
Migrating: 2015_05_19_214534_AlterTableComponentGroupsAddOrder
Migrated:  2015_05_19_214534_AlterTableComponentGroupsAddOrder
Migrating: 2015_05_20_073041_AlterTableIncidentsAddVisibileColumn
Migrated:  2015_05_20_073041_AlterTableIncidentsAddVisibileColumn
Migrating: 2015_05_24_210939_create_jobs_table
Migrated:  2015_05_24_210939_create_jobs_table
Migrating: 2015_05_24_210948_create_failed_jobs_table
Migrated:  2015_05_24_210948_create_failed_jobs_table
Migrating: 2015_06_10_122216_AlterTableComponentsDropUserIdColumn
Migrated:  2015_06_10_122216_AlterTableComponentsDropUserIdColumn
Migrating: 2015_06_10_122229_AlterTableIncidentsDropUserIdColumn
Migrated:  2015_06_10_122229_AlterTableIncidentsDropUserIdColumn
Migrating: 2015_08_02_120436_AlterTableSubscribersRemoveDeletedAt
Migrated:  2015_08_02_120436_AlterTableSubscribersRemoveDeletedAt
Migrating: 2015_08_13_214123_AlterTableMetricsAddDecimalPlacesColumn
Migrated:  2015_08_13_214123_AlterTableMetricsAddDecimalPlacesColumn
Migrating: 2015_10_31_211944_CreateInvitesTable
Migrated:  2015_10_31_211944_CreateInvitesTable
Migrating: 2015_11_03_211049_AlterTableComponentsAddEnabledColumn
Migrated:  2015_11_03_211049_AlterTableComponentsAddEnabledColumn
Migrating: 2015_12_26_162258_AlterTableMetricsAddDefaultViewColumn
Migrated:  2015_12_26_162258_AlterTableMetricsAddDefaultViewColumn
Migrating: 2016_01_09_141852_CreateSubscriptionsTable
Migrated:  2016_01_09_141852_CreateSubscriptionsTable
Migrating: 2016_01_29_154937_AlterTableComponentGroupsAddCollapsedColumn
Migrated:  2016_01_29_154937_AlterTableComponentGroupsAddCollapsedColumn
Migrating: 2016_02_18_085210_AlterTableMetricPointsChangeValueColumn
Migrated:  2016_02_18_085210_AlterTableMetricPointsChangeValueColumn
Migrating: 2016_03_01_174858_AlterTableMetricPointsAddCounterColumn
Migrated:  2016_03_01_174858_AlterTableMetricPointsAddCounterColumn
Migrating: 2016_03_08_125729_CreateIncidentUpdatesTable
Migrated:  2016_03_08_125729_CreateIncidentUpdatesTable
Migrating: 2016_03_10_144613_AlterTableComponentGroupsMakeColumnInteger
Migrated:  2016_03_10_144613_AlterTableComponentGroupsMakeColumnInteger
Migrating: 2016_04_05_142933_create_sessions_table
Migrated:  2016_04_05_142933_create_sessions_table
Migrating: 2016_04_29_061916_AlterTableSubscribersAddGlobalColumn
Migrated:  2016_04_29_061916_AlterTableSubscribersAddGlobalColumn
Migrating: 2016_06_02_075012_AlterTableMetricsAddOrderColumn
Migrated:  2016_06_02_075012_AlterTableMetricsAddOrderColumn
Migrating: 2016_06_05_091615_create_cache_table
Migrated:  2016_06_05_091615_create_cache_table
Migrating: 2016_07_25_052444_AlterTableComponentGroupsAddVisibleColumn
Migrated:  2016_07_25_052444_AlterTableComponentGroupsAddVisibleColumn
Migrating: 2016_08_23_114610_AlterTableUsersAddWelcomedColumn
Migrated:  2016_08_23_114610_AlterTableUsersAddWelcomedColumn
Migrating: 2016_09_04_100000_AlterTableIncidentsAddStickiedColumn
Migrated:  2016_09_04_100000_AlterTableIncidentsAddStickiedColumn
Migrating: 2016_10_24_183415_AlterTableIncidentsAddOccurredAtColumn
Migrated:  2016_10_24_183415_AlterTableIncidentsAddOccurredAtColumn
Migrating: 2016_10_30_174400_CreateSchedulesTable
Migrated:  2016_10_30_174400_CreateSchedulesTable
Migrating: 2016_10_30_174410_CreateScheduleComponentsTable
Migrated:  2016_10_30_174410_CreateScheduleComponentsTable
Migrating: 2016_10_30_182324_AlterTableIncidentsRemoveScheduledColumns
Migrated:  2016_10_30_182324_AlterTableIncidentsRemoveScheduledColumns
Migrating: 2016_12_04_163502_AlterTableMetricsAddVisibleColumn
Migrated:  2016_12_04_163502_AlterTableMetricsAddVisibleColumn
Migrating: 2016_12_05_185045_AlterTableComponentsAddMetaColumn
Migrated:  2016_12_05_185045_AlterTableComponentsAddMetaColumn
Migrating: 2016_12_29_124643_AlterTableSubscribersAddPhoneNumberSlackColumns
Migrated:  2016_12_29_124643_AlterTableSubscribersAddPhoneNumberSlackColumns
Migrating: 2016_12_29_155956_AlterTableComponentsMakeLinkNullable
Migrated:  2016_12_29_155956_AlterTableComponentsMakeLinkNullable
Migrating: 2017_01_03_143916_create_notifications_table
Migrated:  2017_01_03_143916_create_notifications_table
Migrating: 2017_02_03_222218_CreateActionsTable
Migrated:  2017_02_03_222218_CreateActionsTable
Migrating: 2017_06_13_181049_CreateMetaTable
Migrated:  2017_06_13_181049_CreateMetaTable
Migrating: 2017_07_18_214718_CreateIncidentComponents
Migrated:  2017_07_18_214718_CreateIncidentComponents
Migrating: 2017_09_14_180434_AlterIncidentsAddUserId
Migrated:  2017_09_14_180434_AlterIncidentsAddUserId
Migrating: 2018_04_02_163328_CreateTaggablesTable
Migrated:  2018_04_02_163328_CreateTaggablesTable
Migrating: 2018_04_02_163658_MigrateComponentTagTable
Migrated:  2018_04_02_163658_MigrateComponentTagTable
Migrating: 2018_06_14_201440_AlterSchedulesSoftDeletes
Migrated:  2018_06_14_201440_AlterSchedulesSoftDeletes
Migrating: 2018_06_17_182507_AlterIncidentsAddNotifications
Migrated:  2018_06_17_182507_AlterIncidentsAddNotifications
Migrating: 2019_12_12_131400_AlterJobsDropReserved
Migrated:  2019_12_12_131400_AlterJobsDropReserved
Database seeding completed successfully.
Clearing cache…
Application cache cleared!
Cache cleared!
The [public/storage] directory has been linked.
Cachet is installed ⚡

We were facing this issue on newly installed Centos 7 with Virtualizor on KVM

Where The viifbr0 is not started. Please run service virtnetwork start

error was showing up.

We went ahead and changed interface settings to enp5s0f0 based on our interface settings

However the issue still was persisting.

[root@PS1122 ~]# service virtnetwork start
SIOCADDRT: Network is unreachable
Bringing up viifbr0 – OK
Waiting for 10 seconds…1…2…3…4…5…6…7…8…9…10
Checking network connectivity through ping to 4.2.2.1
connect: Network is unreachable
Internet connectivity check failed … Restarting normal network
Stopping viifbr0 … Restarting normal network
Restarting network (via systemctl):                        [  OK  ]

So what we did was checked our file vi /etc/sysconfig/network-scripts/ifcfg-enp5s0f0

The netmask had value of /21

We changed it to 255.255.248.0 and restarted network

This resolved our issue and virtualizor issue was resolved

[root@PS1122 ~]# service virtnetwork start
Bringing up viifbr0 – OK
Waiting for 10 seconds…1…2…3…4…5…6…7…8…9…10
Internet connectivity check successful viifbr0 – OK
[root@OK1122 ~]#

While trying to add new user and ssh using the user login details after adding user ssh keys

was getting below error

Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

The issue was due to permissions

Changing to below permissions resolved the issue

chmod 0700 /home/surmeet/.ssh/

chmod 0600 /home/surmeet/.ssh/authorized_keys

Attempting to start the MySQL service results in the following error:

[ERROR] Unix socket lock file is empty /var/lib/mysql/mysql.sock.lock. 
[ERROR] Unable to setup unix socket lock file.

Below were the errors in mysql error log

2021-01-06T01:29:39.630848Z 0 [Note] – ‘127.0.0.1’ resolves to ‘127.0.0.1’;
2021-01-06T01:29:39.630871Z 0 [Note] Server socket created on IP: ‘127.0.0.1’.
2021-01-06T01:29:39.630932Z 0 [ERROR] Unix socket lock file is empty /var/lib/mysql/mysql.sock.lock.
2021-01-06T01:29:39.630943Z 0 [ERROR] Unable to setup unix socket lock file.
2021-01-06T01:29:39.630952Z 0 [ERROR] Aborting

Description 

A stale lock file is preventing the MySQL service from starting up. The ‘stat’ command can be used to compare the age of the lock file to the current time, in order to confirm that the lock file is stale:

# stat /var/lib/mysql/mysql.sock.lock

The above command should confirm the presence of an old lock file that is empty.

Workaround

It can be difficult to determine why a stale lock file was left behind, however, moving the stale lock file aside should allow MySQL to start up normally. The following commands will need to be executed as the root user via WHM’s Terminal application, or via shell:

# mkdir -v /root/stale_mysql_lock_files
# mv -v /var/lib/mysql/mysql.sock.lock /root/stale_mysql_lock_files
# /usr/local/cpanel/scripts/restartsrv_mysql

Currently aware of an issue when updating to MariaDB 10.3.26 or MariaDB 10.2.35. Which causes MySQL Databases interface within cPanel to display invalid errors such as, but not limited to: “The MySQL server is currently offline”.

While the error in cPanel may indicate that the MySQL service is offline, it is an issue with grants and MySQL is just inaccessible for the cPanel user. We do recommend checking the service directly to confirm that it is online – in the cases, we have seen MySQL is still online.

You may temporarily resolve the issue by downgrading to the previous minor version of MariaDB using the steps outlined

Follow below steps

yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel

If your server has not yet been affected by the issue, we recommend yum-locking the MariaDB packages in place to avoid the package from being updated. This can be completed by executing the following:

yum install -y yum-plugin-versionlock && yum versionlock MariaDB-* 

And restart mysql to resolve the issue