https://quicktasks.ismael.casimpan.com/tags/mysql/Recent content in mysql on IT QuicktasksHugo -- gohugo.ioCopyright © 2018–2022, Ismael Casimpan Jr.; All Rights ReservedSun, 27 Mar 2022 00:20:25 +0800https://quicktasks.ismael.casimpan.com/post/docker-mysql-backup-to-local/Sun, 27 Mar 2022 00:20:25 +0800https://quicktasks.ismael.casimpan.com/post/docker-mysql-backup-to-local/ Doing a backup of your mysql container's DB can be automated just like any other task. Script below is one way to do it. You may or may not provide a parameter to the script and you'll always get the gzipped version of the backup + its sha256 checksum. Sample Use Case: use it without parameters 1~$ ./backup_db.sh 2[INFO] DB backup for 'cooltool_db' copied to /home/icasimpan/work/backups... 3[INFO] Checking for backup integrity: 4cooltool_db-backup-2022.https://quicktasks.ismael.casimpan.com/post/check-if-databases-are-empty/Wed, 02 Sep 2020 23:11:25 +0800https://quicktasks.ismael.casimpan.com/post/check-if-databases-are-empty/ To check if MariaDB/MySQL databases are empty in your DB Server, run the following query: 1MariaDB [(none)]> SELECT table_schema, # "DB Name", 2-> Round(Sum(data_length + index_length) / 1024 / 1024, 1) # "DB Size in MB" 3-> FROM information_schema.tables 4-> GROUP BY table_schema; 5+--------------------+---------------------------------------------------------+ 6| table_schema | Round(Sum(data_length + index_length) / 1024 / 1024, 1) | 7+--------------------+---------------------------------------------------------+ 8| facultyblog | 970.5 | 9| information_schema | 0.2 | 10| laxnews | 65.https://quicktasks.ismael.casimpan.com/post/mysqldump-except-locked-tables/Wed, 17 Jun 2020 23:11:25 +0800https://quicktasks.ismael.casimpan.com/post/mysqldump-except-locked-tables/ MySQL will go down if locked tables are touched during dump. Use the following mysqldump syntax instead: 1mysqldump --single-transaction --skip-lock-tableshttps://quicktasks.ismael.casimpan.com/post/truncate-mysql-table/Wed, 10 Jun 2020 23:17:11 +0800https://quicktasks.ismael.casimpan.com/post/truncate-mysql-table/ To delete the entries BUT not the table, use: 1TRUNCATE TABLE <table_name>; See details in https://www.mysqltutorial.org/mysql-truncate-table/https://quicktasks.ismael.casimpan.com/post/mysql-with-credentials-file/Tue, 12 May 2020 23:11:25 +0800https://quicktasks.ismael.casimpan.com/post/mysql-with-credentials-file/ Using a configuration file for mysql is a good practice as you don't immediately expose credentials. Below, you are going to mysql cli and immediately using the db named dbName. 1mysql --defaults-file=/path/to/credentials.ini dbName where credentials.ini has this format: 1[client] 2host=<replace_with_servername_or_ip> 3user=<replace_with_db_user> 4password=<replace_with_db_password> Of course, there are other security precautions you could take, not just the above but it's a good start. See further info on securing mysql in https://dev.mysql.com/doc/refman/8.0/en/password-security-user.htmlhttps://quicktasks.ismael.casimpan.com/post/cannot-create-pid-file/Wed, 19 Feb 2020 23:11:25 +0800https://quicktasks.ismael.casimpan.com/post/cannot-create-pid-file/ MariaDB seems to have this issue and can't start as shown in /var/log/mariadb/mariadb.log 12020-02-19 15:18:49 0 [ERROR] mysqld: Can't create/write to file '/var/run/mariadb/mariadb.pid' (Errcode: 2 "No such file or directory") I was able to fix one instance of this issue by doing: 1sudo mkdir /var/run/mariadb 2sudo chown mysql.mysql /var/run/mariadb Then afterwards, starting the mariadb service went fine. Some possible solutions in case my solution doesn't work https://stackoverflow.com/questions/15408643/cant-connect-to-mysql-server-cant-create-write-the-pid-filehttps://quicktasks.ismael.casimpan.com/post/table-sizes/Tue, 21 Jan 2020 23:17:11 +0800https://quicktasks.ismael.casimpan.com/post/table-sizes/ Use the query below and replace "database_name" with the correct db name. 1SELECT table_name AS "Table", 2ROUND(((data_length + index_length) / 1024 / 1024), 2) AS "Size (MB)" 3FROM information_schema.TABLES 4WHERE table_schema = "database_name" 5ORDER BY (data_length + index_length) DESC;https://quicktasks.ismael.casimpan.com/post/shrink-huge-ibdata1/Sun, 24 Nov 2019 23:15:37 +0800https://quicktasks.ismael.casimpan.com/post/shrink-huge-ibdata1/ Best way is to make one file per table and not use default which baloons to a huge file and never shrinks when some tables are removed and unused. See https://www.thegeekstuff.com/2016/02/mysql-innodb-file-per-table Some other reference: https://www.percona.com/blog/2013/08/20/why-is-the-ibdata1-file-continuously-growing-in-mysql/ https://www.percona.com/blog/2013/09/25/how-to-reclaim-space-in-innodb-when-innodb_file_per_table-is-on/ https://www.percona.com/blog/2014/08/21/the-mysql-ibdata1-disk-space-issue-and-big-tables-part-1/https://quicktasks.ismael.casimpan.com/post/dump-database-skip-some-table/Sun, 24 Nov 2019 12:38:49 +0800https://quicktasks.ismael.casimpan.com/post/dump-database-skip-some-table/ 1mysqldump -u<username> -p<password> --no-create-db myDB --ignore-table=myDB.table1 --ignore-table=myDB.table3-h localhost > mysql.backup See https://tecadmin.net/tutorial/mysql/skip-tables-in-mysqldump/https://quicktasks.ismael.casimpan.com/post/slow-query-logs/Tue, 12 Feb 2019 22:47:36 +0800https://quicktasks.ismael.casimpan.com/post/slow-query-logs/ 1slow_query_log = 1 2long_query_time = 1 3slow_query_log_file = /var/log/mariadb/slow-query.log 4log_queries_not_using_indexes See details in https://www.cyberciti.biz/faq/how-to-set-and-enable-mariadb-slow-query-log-linux-unix/https://quicktasks.ismael.casimpan.com/post/innodb-fatal-error-cannot-allocate-memory-buffer-pool/Sat, 29 Dec 2018 09:47:00 +0800https://quicktasks.ismael.casimpan.com/post/innodb-fatal-error-cannot-allocate-memory-buffer-pool/ From /var/log/mariadb/mariadb.log, something like 1150515 12:10:45 InnoDB: Completed initialization of buffer pool 2150515 12:10:45 InnoDB: Fatal error: cannot allocate memory for the buffer pool 3150515 12:10:45 [ERROR] Plugin 'InnoDB' init function returned error. 4150515 12:10:45 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 5150515 12:10:45 [ERROR] Unknown/unsupported storage engine: InnoDB 6150515 12:10:45 [ERROR] Aborting Ensure to allocate 50% of max machine RAM. Assuming you have 2GB in the machine, you'll needhttps://quicktasks.ismael.casimpan.com/post/mariadb-tablejoin-limit/Tue, 23 Oct 2018 22:30:18 +0800https://quicktasks.ismael.casimpan.com/post/mariadb-tablejoin-limit/ Maximum number of table joins is 61 as per https://stackoverflow.com/questions/23389820/what-is-the-maximum-number-of-table-joins-in-mariadb Tested out as follows: 1CREATE TABLE t (i INT(10) NOT NULL); 2select * 3from t a01 join t a02 join t a03 join t a04 join t a05 join t a06 join t a07 join t a08 join t a09 join t a10 4join t a11 join t a12 join t a13 join t a14 join t a15 join t a16 join t a17 join t a18 join t a19 join t a20 5join t a21 join t a22 join t a23 join t a24 join t a25 join t a26 join t a27 join t a28 join t a29 join t a30 6join t a31 join t a32 join t a33 join t a34 join t a35 join t a36 join t a37 join t a38 join t a39 join t a40 7join t a41 join t a42 join t a43 join t a44 join t a45 join t a46 join t a47 join t a48 join t a49 join t a50 8join t a51 join t a52 join t a53 join t a54 join t a55 join t a56 join t a57 join t a58 join t a59 join t a60 9join t a61 join t a62; Results in error for more than 61 table used (tested on MariaDB 10.https://quicktasks.ismael.casimpan.com/post/get-number-of-seconds-slave-is-behind-master/Mon, 15 Oct 2018 12:45:19 +0800https://quicktasks.ismael.casimpan.com/post/get-number-of-seconds-slave-is-behind-master/ Get replication status, and look for "Seconds_Behind_Master"https://quicktasks.ismael.casimpan.com/post/get-replication-status-in-slave/Mon, 15 Oct 2018 12:44:45 +0800https://quicktasks.ismael.casimpan.com/post/get-replication-status-in-slave/ 1mysql> show slave status\G;https://quicktasks.ismael.casimpan.com/post/count-number-of-tables-in-a-mysql-database/Mon, 15 Oct 2018 12:43:31 +0800https://quicktasks.ismael.casimpan.com/post/count-number-of-tables-in-a-mysql-database/ 1mysql> select distinct(TABLE_SCHEMA) from information_schema.tables; 2+--------------------+ 3| TABLE_SCHEMA | 4+--------------------+ 5| information_schema | 6| drcongo | 7| mysql | 8| performance_schema | 9| pseademo | 10| pseairaq | 11| sys | 12+--------------------+ 1314mysql> select count(*) from information_schema.tables where TABLE_SCHEMA='drcongo'; 15+----------+ 16| count(*) | 17+----------+ 18| 333 | 19+----------+ 201 row in set (0.00 sec)https://quicktasks.ismael.casimpan.com/post/list-table-names-only-for-shell-scripting-purposes/Mon, 15 Oct 2018 12:42:34 +0800https://quicktasks.ismael.casimpan.com/post/list-table-names-only-for-shell-scripting-purposes/ 1mysql -s --skip-column-names -ne "use DB_to_Use; show tables;"https://quicktasks.ismael.casimpan.com/post/some-mysql-tweaking-tips/Mon, 15 Oct 2018 12:39:54 +0800https://quicktasks.ismael.casimpan.com/post/some-mysql-tweaking-tips/ See https://serverfault.com/questions/42789/how-to-increase-memory-usage-in-mysql-server-to-improve-speedhttps://quicktasks.ismael.casimpan.com/post/dump-mysql-database/Mon, 15 Oct 2018 12:38:49 +0800https://quicktasks.ismael.casimpan.com/post/dump-mysql-database/ 1mysqldump -u<username> -p<password> --no-create-db myDB -h localhost > mysql.backup That way, it would strip the original database name and NOT become a problem when importing to another database name. See https://stackoverflow.com/questions/30835874/how-to-avoid-use-database-statement-on-mysqldump-backupshttps://quicktasks.ismael.casimpan.com/post/automysqlbackup-looking-for-.my.cnf-on-your-home-instead-of-in-root/Mon, 15 Oct 2018 12:32:03 +0800https://quicktasks.ismael.casimpan.com/post/automysqlbackup-looking-for-.my.cnf-on-your-home-instead-of-in-root/ 1root@rh:~# /usr/local/bin/automysqlbackup /etc/automysqlbackup/rh.conf 2Parsed config file "/etc/automysqlbackup/automysqlbackup.conf" 34grep: /home/icasimpan/.my.cnf: No such file or directory 5grep: /home/icasimpan/.my.cnf: No such file or directory 6# Checking for permissions to write to folders: 7base folder /var/lib ... exists ... ok. 8backup folder /var/lib/automysqlbackup ... exists ... writable? yes. Proceeding. 9checking directory "/var/lib/automysqlbackup/daily" ... exists. 10checking directory "/var/lib/automysqlbackup/weekly" ... exists. 11checking directory "/var/lib/automysqlbackup/monthly" ... exists. 12checking directory "/var/lib/automysqlbackup/latest" ... exists. 13checking directory "/var/lib/automysqlbackup/tmp" ... exists. 14checking directory "/var/lib/automysqlbackup/fullschema" .https://quicktasks.ismael.casimpan.com/post/automysqlbackup-fails-due-to-issues-in-information_schema/Mon, 15 Oct 2018 12:11:14 +0800https://quicktasks.ismael.casimpan.com/post/automysqlbackup-fails-due-to-issues-in-information_schema/ In a machine 'rh', tested automysqlbackup cron job if it would work and got the issue below: 1[root@rh icasimpan]# /usr/local/bin/automysqlbackup /etc/automysqlbackup/rh.conf 2Parsed config file "/etc/automysqlbackup/automysqlbackup.conf" 34# Checking for permissions to write to folders: 5base folder /var/lib ... exists ... ok. 6backup folder /var/lib/automysqlbackup ... exists ... writable? yes. Proceeding. 7checking directory "/var/lib/automysqlbackup/daily" ... exists. 8checking directory "/var/lib/automysqlbackup/weekly" ... exists. 9checking directory "/var/lib/automysqlbackup/monthly" ... exists. 10checking directory "/var/lib/automysqlbackup/latest" ... exists. 11checking directory "/var/lib/automysqlbackup/tmp" .https://quicktasks.ismael.casimpan.com/post/automysqlbackup-cannot-use-root-credentials-from-mysql-cli/Mon, 15 Oct 2018 12:00:13 +0800https://quicktasks.ismael.casimpan.com/post/automysqlbackup-cannot-use-root-credentials-from-mysql-cli/ 1[root@rh icasimpan]# /usr/local/bin/automysqlbackup /etc/automysqlbackup/rh.conf 2Parsed config file "/etc/automysqlbackup/automysqlbackup.conf" 34# Checking for permissions to write to folders: 5base folder /var/lib ... exists ... ok. 6backup folder /var/lib/automysqlbackup ... exists ... writable? yes. Proceeding. 7checking directory "/var/lib/automysqlbackup/daily" ... exists. 8checking directory "/var/lib/automysqlbackup/weekly" ... exists. 9checking directory "/var/lib/automysqlbackup/monthly" ... exists. 10checking directory "/var/lib/automysqlbackup/latest" ... exists. 11checking directory "/var/lib/automysqlbackup/tmp" ... exists. 12checking directory "/var/lib/automysqlbackup/fullschema" ... exists. 13checking directory "/var/lib/automysqlbackup/status" ... exists. 1415# Testing for installed programs 16mysql .https://quicktasks.ismael.casimpan.com/post/check-current-character-set-settings-in-mysql/Mon, 15 Oct 2018 07:46:36 +0800https://quicktasks.ismael.casimpan.com/post/check-current-character-set-settings-in-mysql/ 1mysql> SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%'; Sample output as follows: Image not found a.warning-link { color: inherit !important; font-weight: inherit !important; text-decoration: underline !important; border-bottom: none !important; } Web path: https://quicktasks.ismael.casimpan.com/img/check-mysql-charset-settings.png Disk path: static/img/check-mysql-charset-settings.png Using Page Bundles: falsehttps://quicktasks.ismael.casimpan.com/post/checking-supported-character-set/Mon, 15 Oct 2018 07:41:11 +0800https://quicktasks.ismael.casimpan.com/post/checking-supported-character-set/ 1mysql> SHOW CHARACTER SET;https://quicktasks.ismael.casimpan.com/post/undefined-class-constant-mysql_attr_ssl_ca/Mon, 15 Oct 2018 07:27:21 +0800https://quicktasks.ismael.casimpan.com/post/undefined-class-constant-mysql_attr_ssl_ca/ 1drupal@ubuntu:/var/www/sites/openedu.localhost/docroot$ drush si site-name='OpenEDU Testing' --db-url='mysql://openedu_DBA:openedu_PASS@127.0.0.1/openedu_DB' openedu 2PHP Fatal error: Uncaught Error: Undefined class constant 'MYSQL_ATTR_SSL_CA' in /usr/local/share/drush/lib/Drush/Sql/Sqlmysql.php:56 3Stack trace: 4#0 /usr/local/share/drush/lib/Drush/Sql/SqlBase.php(164): Drush\Sql\Sqlmysql->creds() 5#1 /usr/local/share/drush/lib/Drush/Sql/Sqlmysql.php(107): Drush\Sql\SqlBase->query('SELECT 1;', '/tmp/drush_uX7T...', '/dev/null') 6#2 /usr/local/share/drush/commands/core/site_install.drush.inc(146): Drush\Sql\Sqlmysql->db_exists() 7#3 /usr/local/share/drush/includes/command.inc(422): drush_core_pre_site_install('site-name=OpenE...') 8#4 /usr/local/share/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array) 9#5 /usr/local/share/drush/includes/command.inc(199): drush_command('site-name=OpenE...') 10#6 /usr/local/share/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array) 11#7 /usr/local/share/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch() 12#8 /usr/local/share/drush/drush.php(12): drush_main() 13#9 {main} 14thrown in /usr/local/share/drush/lib/Drush/Sql/Sqlmysql.php on line 56 15Drush command terminated abnormally due to an unrecoverable error. [error] 16Error: Uncaught Error: Undefined class constant 'MYSQL_ATTR_SSL_CA' in /usr/local/share/drush/lib/Drush/Sql/Sqlmysql.https://quicktasks.ismael.casimpan.com/post/drush-drop-mysql-tables-without-deleting-database/Mon, 09 Apr 2018 00:21:46 +0800https://quicktasks.ismael.casimpan.com/post/drush-drop-mysql-tables-without-deleting-database/ In drupal root: 1drush sql-drophttps://quicktasks.ismael.casimpan.com/post/hardening-mysql/Sun, 08 Apr 2018 23:31:14 +0800https://quicktasks.ismael.casimpan.com/post/hardening-mysql/ 1icasimpan@am2018:~$ sudo mysql_secure_installation 23Securing the MySQL server deployment. 45Connecting to MySQL using a blank password. 67VALIDATE PASSWORD PLUGIN can be used to test passwords 8and improve security. It checks the strength of password 9and allows the users to set only those passwords which are 10secure enough. Would you like to setup VALIDATE PASSWORD plugin? 1112Press y|Y for Yes, any other key for No: Y 1314There are three levels of password validation policy: 1516LOW Length >= 8 17MEDIUM Length >= 8, numeric, mixed case, and special characters 18STRONG Length >= 8, numeric, mixed case, special characters and dictionary file 1920Please enter 0 = LOW, 1 = MEDIUM and 2 = STRONG: 8 2122Invalid option provided.https://quicktasks.ismael.casimpan.com/post/error-1273-hy000-unknown-collation-utf8mb4_unicode_520_ci/Sun, 08 Apr 2018 23:28:33 +0800https://quicktasks.ismael.casimpan.com/post/error-1273-hy000-unknown-collation-utf8mb4_unicode_520_ci/ Caused by difference in database versions where the latest version you're running does not support 'utf8mb4unicode520_ci'. Search and replace it from within the dump file with 'utf8mb4_unicode_ci'. See details in https://www.digitalocean.com/community/questions/error-1273-hy000-at-line-25-unknown-collation-utf8mb4_unicode_520_cihttps://quicktasks.ismael.casimpan.com/post/mysql-not-restarting-nothing-in-logs/Sun, 08 Apr 2018 23:24:04 +0800https://quicktasks.ismael.casimpan.com/post/mysql-not-restarting-nothing-in-logs/ Check /etc/mysql/my.cnf and examine "datadir" value. If it isn't valid or you know the disk is bad, change it. Restart service and all will be well.https://quicktasks.ismael.casimpan.com/post/restore-one-table/Sun, 08 Apr 2018 23:22:49 +0800https://quicktasks.ismael.casimpan.com/post/restore-one-table/ 1mysql -u root -p your_DB < ./your_system.sql More details in https://stackoverflow.com/questions/9696249/restoring-a-mysql-table-back-to-the-databasehttps://quicktasks.ismael.casimpan.com/post/dump-one-or-more-table/Sun, 08 Apr 2018 23:21:30 +0800https://quicktasks.ismael.casimpan.com/post/dump-one-or-more-table/ 1mysqldump -u root -p yourDB system > yourDB_system.sql More details in https://dba.stackexchange.com/questions/9306/how-do-you-mysqldump-specific-tableshttps://quicktasks.ismael.casimpan.com/post/change-mysql-user-credentials/Sun, 08 Apr 2018 23:20:10 +0800https://quicktasks.ismael.casimpan.com/post/change-mysql-user-credentials/ For as long as you have root access to MySQL, this would work: 1UPDATE mysql.user SET Password=PASSWORD('new-password-here') WHERE USER='user-name-here' AND Host='host-name-here'; more details from https://www.cyberciti.biz/faq/mysql-change-user-password/https://quicktasks.ismael.casimpan.com/post/delete-user/Sun, 08 Apr 2018 23:17:11 +0800https://quicktasks.ismael.casimpan.com/post/delete-user/ 1DROP USER 'yourDBA'@'localhost'https://quicktasks.ismael.casimpan.com/post/create-db-with-acl/Sun, 08 Apr 2018 23:15:37 +0800https://quicktasks.ismael.casimpan.com/post/create-db-with-acl/ 1create database yourDB; 2CREATE USER 'yourDBA'@'localhost' IDENTIFIED BY 'yourPASS'; 3GRANT ALL PRIVILEGES ON yourDB.* TO 'yourDBA'@'localhost'; 4FLUSH PRIVILEGES;https://quicktasks.ismael.casimpan.com/post/allow-remote-connections-to-mysql/Sun, 08 Apr 2018 23:11:25 +0800https://quicktasks.ismael.casimpan.com/post/allow-remote-connections-to-mysql/ Use format below: 1GRANT ALL ON yourDB.* TO yourDBA@'23.216.39.44' IDENTIFIED BY 'yourPASS'; and make sure that 'bind-address' in /etc/my.cnf is either 0.0.0.0 or commented out for this to take effect. Turn-off as well any firewall, SELinux, etc. Further details in https://www.digitalocean.com/community/questions/how-to-allow-remote-mysql-database-connectionhttps://quicktasks.ismael.casimpan.com/post/mysql-server-has-gone-away-fix/Sun, 08 Apr 2018 22:33:18 +0800https://quicktasks.ismael.casimpan.com/post/mysql-server-has-gone-away-fix/ If you see an error like the following: 1ERROR 2006 (HY000): MySQL server has gone away 2No connection. Trying to reconnect... 3Connection id: 990 4Current database: yourDB Add the the following to /etc/my.cnf, increasing the value if error persists. 1'max_allowed_packet=64M' See https://stackoverflow.com/questions/7942154/mysql-error-2006-mysql-server-has-gone-away for reference.