#maria Log v0.1

logs Catalogue

This page loads from the db, refresh as and when. It displays 24 hrs
For older data http://marialog.archivist.info/previous.php, logs go back to 2009-01-18
Some user stats
Table logs
date_time
user
message
2017-03-25 08:51:14
hojuruku
bugger I keep getting my mariadb go down because of checksum failure.
2017-03-25 08:51:29
hojuruku
usually takes a bout a hour
2017-03-25 08:53:12
hojuruku
2017-03-25 8:47:37 140351763355392 [ERROR] InnoDB: Corruption: Block in space_id 0 in file /var/lib/mysql/ibdata1 corrupted.
2017-03-25 08:54:18
hojuruku
2017-03-25 08:47:38 7fa630ff9700 InnoDB: uncompressed page, stored checksum in field1 0, calculated checksums for field1: crc32 536728786, innodb 1575996416, none 3735928559, stored checksum in field2 0, calculated checksums for field2: crc32 536728786, innodb 1371122432, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
2017-03-25 08:54:18
hojuruku
InnoDB: page type 0 meaning ALLOCATED
2017-03-25 08:54:18
hojuruku
InnoDB: Page may be a freshly allocated page
2017-03-25 08:54:20
hojuruku
any ideas?
2017-03-25 10:42:45
AspiringMind
Can anyone Voluntarily help me who has significant knowlegde of MariaDB?
2017-03-25 10:49:13
AspiringMind
In which table authentication related data is stored?
2017-03-25 12:22:10
jimmy__
hi there, anyone around with some experience using mariadb columnstore?
2017-03-25 12:22:33
LinuxJedi
jimmy__: I'm one of the senior engineers on ColumnStore
2017-03-25 12:22:50
jimmy__
that should be enough :)
2017-03-25 12:23:15
jimmy__
I'm trying to understand the partitioning applied to my table
2017-03-25 12:23:41
jimmy__
ive just loaded a test table with 100m records of which one field is an int date YYYYMMDD
2017-03-25 12:23:54
jimmy__
when looking at the calShowPartitions for this column
2017-03-25 12:24:23
jimmy__
I see 8 entries all having the same min and max date (20100622 20170323)
2017-03-25 12:25:19
jimmy__
I would have expected the partitions to have different ranges?
2017-03-25 12:25:49
LinuxJedi
so, partitioning currently happens at import time without any awareness of what is going to be loaded in. This means if the data is loaded in with the date in a random order the extents will have a wide min/max range.
2017-03-25 12:26:13
LinuxJedi
so partition elimination on that column would be difficult
2017-03-25 12:26:49
jimmy__
right, so Ive batch imported the same data set 10 times
2017-03-25 12:27:12
jimmy__
with the entire range
2017-03-25 12:28:37
LinuxJedi
at the moment you would need the data loaded in with as close to date order as you can to have a good min/max range on each extent
2017-03-25 12:29:03
jimmy__
let's say i import data every day with a narrow range but sometimes I get only a few old dates in as well would that throw of the whole partitioning as well?
2017-03-25 12:29:50
LinuxJedi
yes
2017-03-25 12:30:01
jimmy__
right, that explains it all.
2017-03-25 12:30:10
jimmy__
it sounds like this might change in the future though?
2017-03-25 12:30:25
LinuxJedi
yes, not for 1.1 but I have ideas for later releases
2017-03-25 12:31:10
LinuxJedi
at the moment that behaviour dates back to how InfiniDB worked. Which isn't always ideal.
2017-03-25 12:31:57
jimmy__
right, thanks for the explanation. Let me ask one more question then
2017-03-25 12:32:03
LinuxJedi
sure thing
2017-03-25 12:33:09
jimmy__
so in case I batch load my data to given date ranges and import them separately would that keep the partitions clean. e.g. loading batch1/range1, then batch2/range2 and then batch3/range1
2017-03-25 12:34:16
LinuxJedi
as long as the batches are > 8m rows. Otherwise they would be put in the same extent as the previous one.
2017-03-25 12:34:59
jimmy__
got it!
2017-03-25 12:35:27
jimmy__
thanks again and I'm looking forward to the next releases then :)
2017-03-25 12:36:03
LinuxJedi
no problem. If you have any more questions please feel free to use the MariaDB ColumnStore Google Group too, the rest of my team also monitor that
2017-03-25 12:36:23
jimmy__
will do!
2017-03-25 14:35:28
hojuruku
YAY :) The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
2017-03-25 14:35:28
hojuruku
information that should help you find out what is causing the crash. - from mysqld.err :P
2017-03-25 14:35:53
hojuruku
does mariadb have a page on crashing? cause it's directing me to mysql dot com ;)
2017-03-25 14:38:05
hojuruku
"About 95% of all bugs are related to a particular query. Normally, this is one of the last queries in the log file just before the server restarts." - WOOT I'm the lucky one. The server crashes at idle after exactly 1 hour. I'm thinking it's the defragmentation cycle.
2017-03-25 14:49:09
hojuruku
i'm turning my stack trace into something meaningful
2017-03-25 15:13:14
PGNd
Sanity check for myself -- anyone running a current 10.2 build on linux? If so, are you able to shut down cleanly/safely? Specifically, without crash?
2017-03-25 15:20:05
hojuruku
looks like i can't get symbols / readable stack trace after following those steps.
2017-03-25 15:20:23
hojuruku
https://dev.mysql.com/doc/refman/5.7/en/using-stack-trace.html
2017-03-25 15:20:34
hojuruku
the symbols were just _end
2017-03-25 15:22:35
PGNd
hojuruku: You *both* built with debug on, and launched with debug on?
2017-03-25 15:23:12
hojuruku
at least it's replicatable - after 1 hour exactly (the time defragmenting kicks in) she dies. Yes I am going to debug this i'm building now with debug and i'll launch with debug on
2017-03-25 15:23:47
hojuruku
i thought it was changing the page size to 32k and the log block size to 4096... but no it wasn't. i ran hammerdb on it for an hour before it crashed no problems.
2017-03-25 15:25:23
hojuruku
PGNd, would you have the honor of reviewing my my.cnf (don't worry the password for encryption isn't in there - i spent all yesterday making a systemd script using systemd-ask-password and fixing a bug with the cirrus drm driver working with plymouth so the encryption keychain password is only in the my.cnf for .5 seconds in a tmp filesystem when it starts)
2017-03-25 15:25:36
hojuruku
i'll pastbin it.
2017-03-25 15:26:41
PGNd
hojuruku: Not sure how useful I'll be, but can take a look ...
2017-03-25 15:27:47
hojuruku
does anyone want v1 of that systemd script. it needs to be done properly with password checking using openssl, when I'm finished I might block how to do it. kind of like dm-crypt on boot with plymouth only for mariadb ;)
2017-03-25 15:28:06
hojuruku
blog how to do it i mean.. ok let me see if this box can cmdline up to a pastebin.
2017-03-25 15:30:11
hojuruku
pastebunz -f /etc/mysql/my.cnf (403 forbidden) well time to remove that bloat that came from sabayon.org since most of what I care about i compile myself and block from sabyon's entropy package management system
2017-03-25 15:31:15
AspiringMindpro
Hi everyone, I am working on Authentication plugins
2017-03-25 15:31:38
AspiringMindpro
Can anyone please tell where can we store new authentication rules
2017-03-25 15:33:13
AspiringMindpro
Shall I make new table for this or is there any existing table in which I can add rules?
2017-03-25 15:35:18
hojuruku
PGNd wgetpaste -s bpaste /etc/mysql/my.cnf
2017-03-25 15:35:18
hojuruku
Your paste can be seen here: http://bpaste.net/show/1855b9c887c3 --- did I screw up with my my.cnf. I did at least try to RTFM. I started out with what percona gave me and tuned away.
2017-03-25 15:36:16
PGNd
hojuruku: is there something specific that's not working?
2017-03-25 15:36:41
hojuruku
no.. just after a hour with no activity - absolutely nothing from doing a msyql_install_db - boom off she goes.
2017-03-25 15:37:00
hojuruku
you can be hammering it in a load test, or leaving it idle, after one hour when the defragmenter kicks in it's going to get a checksum fail on an innodb block.
2017-03-25 15:37:16
hojuruku
i'm running on btrfs subvolume with lsattr confirming copy on write is disabled.
2017-03-25 15:37:51
hojuruku
percona tested mysql on btrfs it can't be that bad... https://www.percona.com/live/mysql-conference-2013/sites/default/files/slides/Percona%20Live%20-%20Linux%20Filesystems%20and%20MySQL.pdf
2017-03-25 15:40:49
PGNd
hojuruku: Have you tested the exact same setup on !btrfs? ext4 of xfs, e.g.?
2017-03-25 15:41:04
hojuruku
strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version --- no gentoo why did you do this.. if i want meaningful stack traces I can't strip the binaries right? i compiled with debug on
2017-03-25 15:41:20
hojuruku
no because i don't have the free space to do it on this box it's a VPS
2017-03-25 15:41:43
hojuruku
i need unstripped binaries of course to get decent stack traces right on a bug?
2017-03-25 15:41:50
hojuruku
let me tweak the ebuild to fix this.
2017-03-25 15:42:02
PGNd
hojuruku: Personally, my 1st suspicion is *always* btrfs ... fun, but not 4 production. 4 me, anyway.
2017-03-25 15:42:07
PGNd
correct -- no stripping
2017-03-25 15:42:24
hojuruku
yeah we'll it's faster for mostly read only databases, up their with XFS
2017-03-25 15:43:59
hojuruku
but i have this old CPANEL installation sitting there ready to be nuked, some paki contractor insisted on it and i'm doing some volunteer work setting up a webserver for lukesarmy.com facebook group exposing the government and fostercare.lwb.org.au (who only advertises offers for free childeren in gay mags). Anyway once the site is up I've got 10 gb's for my var filesystem (xfs) and qmail queue (ext2) / qmail xfs put on LVM2.
2017-03-25 15:45:11
hojuruku
i'll come back in a hour once i've tweaked the ebuild / enabled debug mode and replicated the defect again. i'll pastebin the mysql error log in the meaintime
2017-03-25 15:46:12
PGNd
Nothing jumps out at me re the my.cnf. Then again, it seldom does. Debugging usually finds the problem if its a config issue. Of course, it's not for me CURRENTLY on my stuff :-/
2017-03-25 15:47:12
hojuruku
http://bpaste.net/show/ce2f5ed387da
2017-03-25 15:47:29
hojuruku
that's the log it's been happening with and without the "experimental options" enabled.
2017-03-25 15:48:04
hojuruku
with encyrption on, if the slave has the encryption key you can still use binlogs for replication instead of WRSEP right?
2017-03-25 15:49:04
PGNd
hojuruku: Well, this stands out initially:
2017-03-25 15:49:05
PGNd
[ERROR] InnoDB: Corruption: Block in space_id 0 in file /var/lib/mysql/ibdata1 corrupted.
2017-03-25 15:49:21
PGNd
fs? i.e., btrfs?
2017-03-25 15:49:29
hojuruku
yeah but it only hits it after a hour when the defrag starts - yes it's btrfs.
2017-03-25 15:49:41
hojuruku
btrfs 32k node size - copy on write off.
2017-03-25 15:55:14
hojuruku
ok i'll backup the old cpanel crap offsite and reclaim the disk space now to test with XFS
2017-03-25 15:56:45
PGNd
You don't have to look far to see issues re: btrfs+mysql+defrag. E.g., http://www.spinics.net/lists/linux-btrfs/msg44651.html . Not to say it isn't fixed/or fixable, just keeps me suspicious about it.
2017-03-25 16:01:23
hojuruku
talking to my mate - the guy i do volunteer work for from lukesarmy.com he's going to filezilla down the cpanel crap we don't need and i'll move it to xfs. glad you didn't find anything wrong with my my.cnf :P though i wish that was the case not btrfs... i like btrfs
2017-03-25 16:01:44
hojuruku
when xfs was new in the 90s it had the same bad rap.
2017-03-25 16:03:01
PGNd
hojuruku: " didn't find anything wrong " .... don't bank on MY review! I didn't see anything -- does NOT mean that it's 100%. *my* build's not working atm -- COULD BE my cnf .... CaveatEmptor.
2017-03-25 16:03:16
hojuruku
PGNd, he's complaing about mysql and COW - you can only turn cow off for NEW extents that are written into a file, and yes I chattr +C /var/lib/mysql - and better than that i set up a subvolume in btrfs with /mnt/btrfs/mysql also chattr +C
2017-03-25 16:03:30
hojuruku
i'm 110% sure mysql is NOT dealing with btrfs copy on write here.
2017-03-25 16:04:00
musclewhite
why not use ZFS and the deadline scheduler?
2017-03-25 16:04:07
musclewhite
oops XFS
2017-03-25 16:04:15
hojuruku
PGNd, I think it's something to do with my config personally. something that causes the writes and reads to go out of sync. I thought it was DIRECT_IO and btrfs, so I tried a different flush method, no joy
2017-03-25 16:04:48
PGNd
All my data's on ext4. XFS is maybe a good arguments .... ext4 is a reliable old horse 4 me. I have too many other places to fubar stuff to mess with my fs.
2017-03-25 16:04:57
hojuruku
musclewhite, I'm using the NOOP sheduler because I'm on a VPS with hardware raid by the looks of it - I get reads 130-160 IOPS write and ~400 read IOPS
2017-03-25 16:04:58
PGNd
btrfs -- different strokes, for different folks!
2017-03-25 16:05:27
musclewhite
raid 10 on 24 sas disk is very nice with xfs and deadline
2017-03-25 16:05:31
musclewhite
cloud who knows
2017-03-25 16:05:35
hojuruku
if you run mysql on btrfs it needs it's own subvolume because if you take a snapshot it turns on COW.. that's an important tip i learnt today from #btrfs
2017-03-25 16:06:33
hojuruku
yeah cloud who knows :P that's me - the hosting provider abuses customers if you logged ticket. The first ticket they fixed they were not using -cpu host on qemu with KVM - so I didn't have many flags in /proc/cpuinfo - which was an epic fail - then they delete the tickets to cover their tracks.
2017-03-25 16:07:09
hojuruku
they disabled vmvga driver that doesn't work, and they won't confirm or deny if they are using Transparent HugePages with their host kvm setup so I don't know how to best optomize sql.
2017-03-25 16:09:01
musclewhite
better to spend less time messing with btfs and more buying many slot server and hardware raid 10 then zfs then deadline
2017-03-25 16:09:04
musclewhite
:)
2017-03-25 16:09:17
musclewhite
ah the joys of write speeed
2017-03-25 16:09:26
musclewhite
economics
2017-03-25 16:09:28
musclewhite
capitalism
2017-03-25 16:09:48
PGNd
hojuruku: If you're in a KVM guest, you should be able to simply turn off THP in your container.
2017-03-25 16:10:31
hojuruku
PGNd, I did in troubleshooting!!! I wanted to use it for tmpfs so I set it to madvise before.. but now it's off copmletely
2017-03-25 16:11:32
hojuruku
also i'm seeing reserved hugepages in /proc/meminfo when mysql starts but i'm not seeing ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB - is that normal?
2017-03-25 16:12:09
PGNd
Too much voudou with it. Latency probs on redis drove me nuts b4 I discovered the prob. Now I just set GRUB_CMDLINE_LINUX_DEFAULT=" ... transparent_hugepage=never ...", and am happier.
2017-03-25 16:12:16
hojuruku
yeah i hate vps - especially if it's not my vps. thinking of getting together with a few mates and doing a cloudmin setup together on a deicated box.
2017-03-25 16:12:29
hojuruku
PGNd, I already did it - here's my kernel args.
2017-03-25 16:13:13
hojuruku
ro video=VGA-1:800x600@60 quiet splash dobtrfs rootfstype=btrfs spinlock=unfair elevator=noop zswap.enabled=1 zswap.max_pool_percent=20 zswap.compressor=lz4 zswap.zpool=z3fold transparent_hugepage=never cirrus.modeset=1 init=/usr/lib/systemd/systemd root=/dev/vda3
2017-03-25 16:13:14
musclewhite
a database in a container is a fail
2017-03-25 16:13:19
musclewhite
same for vm
2017-03-25 16:13:33
musclewhite
by far your slowest thing is disk write
2017-03-25 16:13:36
musclewhite
fix that
2017-03-25 16:13:37
musclewhite
first
2017-03-25 16:14:13
hojuruku
i know! but there are costs limits. this is not a profitable enterprise fighting the government's crony capitalists. And I'm no socialist. I'm a political refugee from the west though - bit.ly/aurefugee
2017-03-25 16:14:42
hojuruku
musclewhite, that's why i tuned mysql to only aim for a max of 140 iops - the default is 200 anyway
2017-03-25 16:14:46
hojuruku
except when flusing
2017-03-25 16:15:11
PGNd
Ime, DB apps in particular are tweaky. Too tweaky to NOT have a full-control'd OS env. Lots of options, of course. I'm happy with Linodes ...
2017-03-25 16:25:28
hojuruku
PGNd, I hate linode. I caught them doing a warrantless search of my xen box. I have the whole dossier on them. Do you know their customer service manager is a gay scoutmaster and they didn't like the victims of gay bullying blog being run on their system. PM me and I can send you the dossier on them caught freezing the memory for an unauthorized search - and the guy who sued my father for vilifying gay marriage activists who had sex with russian
2017-03-25 16:25:28
hojuruku
babies bragging about the warrantless search. My dad's interview is here about being accused of the crime of hating sex with babies. You must never vilify anyone who does a homosexual act by law in Austrlaia - without limitation of age of their sex partners - in this case being 2 months old: http://www.truthnews.com.au/web/radio/story/cultural_marxism_101 I'm not making this up. My dad was just found guilty of not respecting those who have sex with
2017-03-25 16:25:29
hojuruku
babies by an Australian court that orders the whole world (in this case my dad was in vietnam when he made the online comment and linked to a video on mark newton and peter truong) is a boylover.net member respect zone. Put "mckee burns" into austlii.edu.au to find the case law.
2017-03-25 16:25:49
musclewhite
lol
2017-03-25 16:27:14
hojuruku
yeah i know it's a crazy world we live in. Now my friend is being sued for the crime of being against the religion of the beast 666 alister crowley otoaustralia.org.au. By law you must hail satan too - see the government letterhead saying you must respect OTO and their child sex magick [tm] https://johnsunol.blogspot.com/2016/07/if-nsw-anti-discrimination-act-1977-was.html (my dads / friends case law) - here's the new satanic pedophile cult lawsuit
2017-03-25 16:27:14
hojuruku
we just copped.
2017-03-25 16:27:39
PGNd
uhhh ... wow.
2017-03-25 16:29:22
hojuruku
https://garryburnsantidiscriminationactivist.com/ the boylover child sex normalization activist and first gay marriage activist on national tv in Australia after keven rudd announced gay marriage just put my father's photo on the front page of his blog. Here's the comment my dad made....
2017-03-25 16:31:03
hojuruku
https://web.archive.org/web/20140722080525/https://www.causes.com/posts/891061 you will notice that link is in the case law. The court ruled youtube video links and causes.com/posts/ links don't autoplay when you go to them so the video of mark newton and peter truong baby rapists saying it's homophobic to oppose us baby rapists getting kid's isn't relavent to my dad's comment. The internet archive says youtube and causes video links autoplay. The
2017-03-25 16:31:03
hojuruku
gay court of australia says no - we beleive the boylover not the internet archive.org. So the judgement stands youtube video links don't autoplay.
2017-03-25 16:35:07
hojuruku
last off topic links. My friend sued for the writing of a political dissident on his blog using my own google account to publish on a server in America. The Australian courts says my friend in Australia is liable for what I write against the religion of Satanic Child sex magick started by Alister Crowley the worlds most infamous satanist. Because he started a religion - this man into sex magickal orgies and child sacrafice by law must be respected.
2017-03-25 16:35:08
hojuruku
Se the government letterhead saying HAIL SATAN here. I'm agnostic btw - the government shouldnt' be involved in defending satanic cults. http://johnsunol.blogspot.com/2017/03/satanic-oto-child-rapists-threaten.html (selection of youtube videos exposing otoaustralia.org.au's child sex magick practises - very interesting stuff)
2017-03-25 16:39:43
hojuruku
the rant's over btw... someone mentioned linode. I had to spew hate. musclewhite is right - if you want a fast system don't use a VPS.
2017-03-25 16:40:19
hojuruku
and i was a linode customer for over 10 years since they were using user mode linux.... damn them.
2017-03-25 20:28:21
hojuruku
EXT4-fs: Warning: mounting with data=journal disables delayed allocation and O_DIRECT support! - kernel message from 3.9 - intersting
2017-03-25 20:28:43
hojuruku
i'm going to disable doubelwrite and let ext4 do it - so i'll have to use fsync() with innodb
2017-03-25 20:29:25
hojuruku
with O_DIRECT / O_DSYNC (the same now on +3.6) checksums fail with innodb on my vps setup - i've disabled the readahead too
2017-03-25 21:28:55
hojuruku
it could be crc32 checksums - i changed the checksumming to innodb because i was getting checksum failures in the log and it seems to be ok
2017-03-25 21:39:24
hojuruku
yep i enabled crc32 for binary log checksums and she died instantly
2017-03-25 21:39:46
hojuruku
crc32 is unstable on my vps - maybe SSE_4.2 isn't working well.
2017-03-25 22:08:29
hojuruku
if any global variable is set to crc32 my server goes down. i'm running sql-bench and hammerdb without it fine.