#maria Log v0.1

logs Catalogue

This page loads from the db, refresh as and when. It displays 24 hrs
For older data http://marialog.archivist.info/previous.php, logs go back to 2009-01-18
Some user stats
Table logs
date_time
user
message
2014-08-26 23:59:14
erkules
AL13N: haven't you been talking about socket activation?
2014-08-26 23:59:46
AL13N
>_>
2014-08-27 00:00:06
AL13N
what's this completely out of the blue?
2014-08-27 00:00:10
AL13N
erkules: ?
2014-08-27 00:00:16
erkules
I wonder if mysql/mariadb supports it anyway
2014-08-27 00:00:25
erkules
AL13N: systemd socket activation
2014-08-27 00:02:29
mgriffin_
as a lowly end user, i dunno that i want to modify systemd service just to change socket path
2014-08-27 00:02:40
mgriffin_
(if that would be required)
2014-08-27 00:03:04
kolbe
wow, what?
2014-08-27 00:05:37
erkules
mgriffin_: ?
2014-08-27 00:07:36
mgriffin_
I assume we are talking about http://www.freedesktop.org/software/systemd/man/systemd.socket.html in which case, i probably have to modify /etc/systemd/system/mariadb.socket if i change the path to unix socket
2014-08-27 00:08:13
mgriffin_
i would like to not have to do that, but that is just like my opinion
2014-08-27 00:08:42
mgriffin_
on the other hand, i could be completely wrong about these assertions
2014-08-27 00:09:11
thumbs
I would also like to see systemd die in a fire, but that's my opinion :)
2014-08-27 00:09:21
mgriffin_
systemd is pretty sweet so far
2014-08-27 00:09:28
kolbe
thumbs: too late for all that, i think
2014-08-27 00:09:34
thumbs
kolbe: :(
2014-08-27 00:09:35
mgriffin_
(played with fedora20 a bit lately)
2014-08-27 00:10:12
erkules
mgriffin_: I still don't get you :(
2014-08-27 00:11:21
mgriffin_
erkules: Ich möchte nicht eine Sache, an zwei Stellen ändern
2014-08-27 00:11:29
mgriffin_
sorry, that was just a troll
2014-08-27 00:11:41
erkules
mgriffin_: hehe great
2014-08-27 00:11:43
mgriffin_
(care of google translate, which means it was likely nonesense)
2014-08-27 00:11:58
mgriffin_
"i dont want to change one thing in two places"
2014-08-27 00:12:14
erkules
mgriffin_: Im talking about socket activation not working with mysql/mariab
2014-08-27 00:13:20
mgriffin_
erkules: feel free to ignore me going forward, i really dont know what s a does.
2014-08-27 00:14:50
erkules
mgriffin_: with s a systemd listen to a socket and starts the daemon when a connection arrives
2014-08-27 00:14:50
grknight
i think that socket activation of a DB isn't a great idea because of how heavyweight of a process it is. also more data can be cached the longer it runs
2014-08-27 00:15:04
erkules
grknight: agree
2014-08-27 00:15:39
erkules
grknight: but think about running 1000 mysqld on on host
2014-08-27 00:16:09
erkules
grknight: all having small databases and not often using it
2014-08-27 00:16:46
grknight
why would you do something like that? even shared hosts don't do that
2014-08-27 00:17:12
erkules
grknight: to make money
2014-08-27 00:17:19
erkules
\o/
2014-08-27 00:17:24
grknight
makes no sense
2014-08-27 00:17:26
Jb-boin
then consider migrating to sqlite....</troll>
2014-08-27 00:17:48
Jb-boin
that sound strange indeed
2014-08-27 00:18:15
erkules
with socket activation it would work nice
2014-08-27 00:18:42
grknight
until a good portion try to start with ENOMEMORY
2014-08-27 00:19:32
AL13N
grknight: well, i don't mean to shut it back down again automatically
2014-08-27 00:19:33
erkules
anyway socket activation is not important for my daily business
2014-08-27 00:19:44
erkules
grknight: welcome to the cloud business
2014-08-27 00:19:50
AL13N
but socket activation might be nice if you have an emergency restart of server?
2014-08-27 00:20:02
AL13N
or not
2014-08-27 00:20:34
AL13N
anyway, it's a good option for stuff that uses mariadb in a desktop
2014-08-27 00:21:01
AL13N
also, users won't be changing systemd socket files, that would be the sysadmin task, not the developer
2014-08-27 00:21:07
kolbe
is there stuff that uses MariaDB to support a desktop application? i'm surprised people still do that
2014-08-27 00:21:19
AL13N
akonadi in KDE for one
2014-08-27 00:21:26
grknight
kolbe: amarok still uses it embedded
2014-08-27 00:21:35
grknight
but can be external too
2014-08-27 00:21:56
kolbe
do you mean libmysqld ?
2014-08-27 00:22:11
grknight
yes, libmysqld or it can connect to an external mysqld
2014-08-27 00:22:22
AL13N
for such people it might make sense to have socket activation, cause they don't always play music or something
2014-08-27 00:22:44
AL13N
i'm not saying it's the most important feature there is though
2014-08-27 00:23:11
kolbe
Amarok =~ iTunes?
2014-08-27 00:23:33
AL13N
don't exaggerate
2014-08-27 00:23:48
kolbe
i have never heard of amarok, i have no idea what it is
2014-08-27 00:23:49
kolbe
i'm asking
2014-08-27 00:23:55
AL13N
you can still actually play files on your disk
2014-08-27 00:23:59
kolbe
i guess it is basically an equivalent thing to itunes
2014-08-27 00:24:00
kolbe
huh?
2014-08-27 00:24:10
kolbe
oh, you're going to go and complain about iTunes now? ok
2014-08-27 00:24:13
kolbe
kolbe will wait
2014-08-27 00:24:23
AL13N
no, it's ok, i've already stopped
2014-08-27 00:24:47
AL13N
but amarok is a app to play music and stuff
2014-08-27 00:24:53
kolbe
uh huh, thanks
2014-08-27 00:25:06
AL13N
and it has a catalog which it stores in mysql database
2014-08-27 00:25:47
AL13N
anyway, the point is, even if you have the option to do it, the socket activation doesn't need to be enabled
2014-08-27 00:26:15
AL13N
allthough the main point is really to not use mysqld anymore
2014-08-27 00:26:23
AL13N
mysqld_safe i mean
2014-08-27 00:26:32
AL13N
i'm getting sleepy /o\
2014-08-27 00:26:51
erkules
Anyway building a service unit for the masses is complicated (imho)
2014-08-27 00:26:59
AL13N
true
2014-08-27 00:27:00
erkules
There are a lot of thing to rething
2014-08-27 00:27:06
grknight
Gentoo hasn't used mysqld_safe since 4.1 days :)
2014-08-27 00:27:08
AL13N
keep in mind this:
2014-08-27 00:27:22
erkules
trying to reuse old code is broken
2014-08-27 00:27:25
AL13N
any large organisation will inevitably change it anyway
2014-08-27 00:27:41
AL13N
so the default doesn't need to be overthinked
2014-08-27 00:27:49
AL13N
something for a small-middle company is good
2014-08-27 00:28:02
erkules
grknight: galera without mysqld_safe is going to suffer on gentoo then :)
2014-08-27 00:28:18
AL13N
erkules: why is that?
2014-08-27 00:28:26
AL13N
erkules: it can't play with systemd?
2014-08-27 00:28:52
erkules
AL13N: to much logic in mysqld_safe
2014-08-27 00:28:53
grknight
erkules: Gentoo is using it just fine in the main infrastructure now
2014-08-27 00:29:15
grknight
running via mysqld
2014-08-27 00:29:21
erkules
AL13N: So dropping myslqd_safe means to put some effort into the service unit
2014-08-27 00:29:34
erkules
grknight: I doubt that
2014-08-27 00:29:36
Jb-boin
well, i could still run a simpler script for the very specific functions that couldnt be implemented on mysqld_safe
2014-08-27 00:29:46
grknight
and we don't use systemd by default
2014-08-27 00:29:54
Jb-boin
s/mysqld_safe/systemd/
2014-08-27 00:30:03
erkules
Jb-boin: yes
2014-08-27 00:30:18
erkules
Jb-boin: Best is to throw away all code and start new
2014-08-27 00:30:33
Jb-boin
yes, we agree
2014-08-27 00:32:43
erkules
# Copyright Abandoned 1996 TCX DataKonsult AB & Monty Program KB & Detron HB
2014-08-27 00:37:25
kolbe
rewrite MySQL in go!
2014-08-27 00:37:52
erkules
kolbe: don't be silly
2014-08-27 00:37:56
erkules
thats my job!
2014-08-27 00:40:50
erkules
mysqld_safe is a relict from a time where you need to handle the monitoring on your own. No need for that with Upstart, systemd, openrc or even pacemaker
2014-08-27 00:41:10
mgriffin_
NEEDS MOAR DJB
2014-08-27 00:41:18
erkules
erkules had seen people using mysqld_safe for there resource agent
2014-08-27 00:41:22
kolbe
:(
2014-08-27 00:41:31
kolbe
yeah mysqld_safe is particularly bizarre and unnecessary
2014-08-27 00:41:38
erkules
erkules never liked djb
2014-08-27 00:45:48
Jb-boin
i must confess that i use mysqld_safe to make coffee and clean the dishes
2014-08-27 00:47:36
mgriffin_
418 I'm a teapot
2014-08-27 00:48:10
kolbe
i finally polished up my really horrific bash scripts to manage my local mysql test instances
2014-08-27 00:49:59
kolbe
i had been relying on all the weird tricks done by mysqld_safe to set different environment variables and add various command-line options
2014-08-27 00:50:22
thumbs
I have to admit that I've hated the way mysqld_safe behave in many ways
2014-08-27 00:50:27
thumbs
*behaves
2014-08-27 00:52:12
erkules
kolbe: So you run your local instances in the same namespace?
2014-08-27 00:52:42
grknight
with openrc, to create mysql instances with different options, you just symlink the original service and cp the conf.d file it sources and change the file created
2014-08-27 00:52:43
kolbe
erkules: i use VMs for some stuff, i use local instances for other things
2014-08-27 00:52:48
erkules
erkules was shocked reading http://www.mysqlperformanceblog.com/2014/08/26/mysqld_multi-how-to-run-multiple-instances-of-mysql/
2014-08-27 00:52:53
kolbe
http://sprunge.us/jVAZ
2014-08-27 00:53:33
erkules
kolbe: ah ok osc
2014-08-27 00:53:35
erkules
osx
2014-08-27 00:53:41
erkules
so you are doomed anyway
2014-08-27 00:53:45
kolbe
sure, but this would all work the same on linux *shrug*
2014-08-27 00:54:01
erkules
grknight: same with systemd templates :)
2014-08-27 00:54:11
kolbe
i wanted to be able to see and interact with them using bash job control, and the extremely unfortunate answer to that was to do a lot of *really* outrageous eval stuff in my bash function ;)
2014-08-27 00:54:28
erkules
kolbe: but on linux you don't need it to run in the same namespace :)
2014-08-27 00:54:36
kolbe
erkules: i don't even know what you mean by "namespace"
2014-08-27 00:54:48
erkules
kolbe: you need a linux update
2014-08-27 00:54:51
kolbe
but for what i am doing i sure do not want anything other than what i am doing here
2014-08-27 00:55:58
kolbe
erkules: no i don't!
2014-08-27 00:55:58
kolbe
[root@db1 ~]# cat /etc/redhat-release
2014-08-27 00:55:58
kolbe
CentOS release 6.5 (Final)
2014-08-27 00:55:59
kolbe
[root@db1 ~]# yum update
2014-08-27 00:56:02
kolbe
No Packages marked for Update
2014-08-27 00:56:04
kolbe
;)
2014-08-27 00:56:16
erkules
hehe
2014-08-27 00:56:40
erkules
no realy not knowing what namespaces are is so last millenium
2014-08-27 00:56:53
erkules
kolbe: feel free to visit my training next week :D
2014-08-27 00:57:20
kolbe
i don't know anything about any of the new virtualization/compartmentalization/whatever stuff available in linux... no clue :-/
2014-08-27 00:57:54
erkules
do you have a linux?
2014-08-27 00:58:00
erkules
oh yes you do
2014-08-27 00:58:12
Jb-boin
well, to put it simply its like chroot but way more advanced
2014-08-27 00:58:18
erkules
readlink /proc/self/ns/*
2014-08-27 00:58:29
erkules
every process is in namespaces all the time
2014-08-27 00:58:36
erkules
Jb-boin: no
2014-08-27 00:58:49
erkules
namespace are there anyway
2014-08-27 00:59:35
Jb-boin
for sure, but to explain what the use of namespaces/containers could do, the comparison with chroot is a good start
2014-08-27 00:59:39
erkules
Jb-boin: But you are right. chroot/pivot_root+namespace+cgroups+capabilites => LXC aka LXC/docker/systemd/lxc@libvirt
2014-08-27 01:01:54
kolbe
erkules: ok, i see /proc/self/ns/* ... now what :)
2014-08-27 01:02:32
erkules
There you see all the namespaces your and every process is into
2014-08-27 01:02:56
erkules
knowing that you can run a lot of mysqld in separate namespaces
2014-08-27 01:03:16
kolbe
googling around for this hasn't been very useful so far
2014-08-27 01:04:03
erkules
may be not that important for you doing tests. But knowing that technique using myslqd_mulid or my_env is wrotten
2014-08-27 01:05:14
erkules
The big benefit is every mysqld instance feels like the same. I.e. having the same port
2014-08-27 01:05:34
thumbs
umm
2014-08-27 01:06:05
erkules
you can easy assign an ip to every running instance
2014-08-27 01:06:24
kolbe
erkules: do you know of a good tutorial or guide for this?
2014-08-27 01:06:27
erkules
You can even run bash having his own IP :)
2014-08-27 01:06:42
erkules
kolbe: have a drink with me ;)
2014-08-27 01:06:57
kolbe
erkules: i'll buy the drinks, just come find me! i'm in Seattle for a while :)
2014-08-27 01:07:16
erkules
ihh
2014-08-27 01:07:48
grknight
i don't get the difference between that and binding to an IP in the server
2014-08-27 01:09:08
erkules
grknight: would look the same in a way. But
2014-08-27 01:09:19
erkules
1. we are talking about the net namespace right now
2014-08-27 01:09:41
erkules
2. without using different net namespace you got to change the configuration for every instance
2014-08-27 01:10:27
erkules
The nice thing about namespaces->container is you run the same my.cnf on all instances without doing configuration fiddeling
2014-08-27 01:10:48
Jb-boin
its also more secure as the process cant bind/have access to another ip
2014-08-27 01:10:59
erkules
Jb-boin++
2014-08-27 01:11:14
grknight
color me old fashioned then
2014-08-27 01:11:16
Jb-boin
and it works the same for the pid namespace for example, even if ran as root it will only see the processes on its own namespace
2014-08-27 01:11:32
Jb-boin
and then cant interfere with them
2014-08-27 01:11:33
erkules
Jb-boin: no
2014-08-27 01:11:43
erkules
Jb-boin: thats what all the people do wrong
2014-08-27 01:12:16
erkules
Jb-boin: The pid namespace is about sending signals
2014-08-27 01:12:22
Jb-boin
ah
2014-08-27 01:12:43
erkules
Jb-boin: You can still see other processes as you see them using /proc -> mount namespace
2014-08-27 01:13:08
Jb-boin
its the cgroups, sorry
2014-08-27 01:13:09
erkules
Jb-boin: do an unshare -pf
2014-08-27 01:13:40
erkules
Jb-boin: doing a ps ax you see all processes on the host but you will not be able to kill them, even as root
2014-08-27 01:13:56
erkules
cgroups is orthognal
2014-08-27 01:14:49
kolbe
kolbe scratches his head
2014-08-27 01:15:13
Jb-boin
i must confess that i only use lxc with all the separations at once, never fiddled with them one by one
2014-08-27 01:15:31
erkules
lxc is a good choice
2014-08-27 01:16:10
Jb-boin
and i mostly use vserver and vz but it also relies more and more on namespaces
2014-08-27 01:16:58
erkules
vanilla kernel will prevail
2014-08-27 01:17:31
erkules
kolbe: that all is quite important
2014-08-27 01:17:47
erkules
only when you know namespace, cgroups and capabilities you see the benefit of systemd
2014-08-27 01:18:07
erkules
as systemd is much more than a daemon monitor
2014-08-27 01:19:03
erkules
you can drop capas, define resources and even use namespaces i.e. for a private /tmp
2014-08-27 01:19:53
kolbe
erkules: is this something that is likely to come into play in a dedicated server environment where a host is running only mysqld ?
2014-08-27 01:20:19
erkules
kolbe: even ther
2014-08-27 01:20:21
erkules
there
2014-08-27 01:20:42
erkules
1. you can have a more secure installation
2014-08-27 01:21:05
erkules
2. you can make sure you got always some resources left to at least log into the server
2014-08-27 01:22:02
kolbe
erkules: we should get you to write some blog posts on mariadb.com talking about all of this :)
2014-08-27 01:22:15
erkules
\o/
2014-08-27 01:22:17
seekwill
:)
2014-08-27 01:25:19
mgriffin_
<3 private tmp
2014-08-27 01:28:38
erkules
:)
2014-08-27 01:30:07
mgriffin_
an example where it helped was mounting an lvm snapshot of a datadir LV, without worrying that the backup software that copies flat files like /etc will see the mount
2014-08-27 01:30:20
kolbe
uhhh
2014-08-27 01:30:39
Jb-boin
well, i will try to give a shot to systemd on the next few weeks
2014-08-27 01:30:58
erkules
\o/
2014-08-27 01:31:32
erkules
but be aware the systemd-nspawn <-> machinectl part is still some kind of broken
2014-08-27 01:31:44
erkules
but we have lxc for that :)
2014-08-27 02:23:29
xbskid
Since I apparently have no idea how to add comments to the mariadb JIRA system... https://mariadb.atlassian.net/browse/MDEV-6503
2014-08-27 02:23:35
xbskid
I'm experiencing that error as well on sparc.
2014-08-27 02:23:55
xbskid
Gentoo, specifically.
2014-08-27 02:23:59
xbskid
Same version.
2014-08-27 02:25:20
seekwill
You need to create an account
2014-08-27 02:26:09
danblack
and then try the link https://mariadb.atlassian.net/secure/AddComment!default.jspa?id=42002
2014-08-27 02:28:02
xbskid
I tried creating an account, but it kept telling me my email addresses were at invalid domains.
2014-08-27 02:28:29
xbskid
I figured it was restricted to devs with @mariadb.org email accounts or something silly like that.
2014-08-27 02:28:53
seekwill
No, it should be open to anyone. Are you using free webmail accounts (@hotmail, @yahoo)?
2014-08-27 02:29:11
xbskid
Gmail.
2014-08-27 02:30:17
seekwill
Wow
2014-08-27 02:30:47
Jb-boin
strange, if you have another email you might try to use it
2014-08-27 02:30:49
seekwill
Well, yeah, that sucks
2014-08-27 02:30:57
xbskid
I've also tried email addresses at both of my personal domains, one of which is configured with Google Apps, and the other is using Gandi Webmail.
2014-08-27 02:31:05
xbskid
Nada
2014-08-27 02:31:59
Jb-boin
well, it seems that you are right
2014-08-27 02:32:01
Jb-boin
"Whoops!
2014-08-27 02:32:02
Jb-boin
The email address provided is not from an allowed domain."
2014-08-27 02:32:09
seekwill
Hmm... That needs to be fixed...
2014-08-27 02:33:41
danblack
where's the fix button?
2014-08-27 02:34:04
seekwill
It's a 1x1px transparent gif
2014-08-27 02:34:53
seekwill
xbskid: Check back tomorrow :)
2014-08-27 02:35:03
xbskid
Bit of a catch-22; I can't create an account to report a problem that I can't create an account.
2014-08-27 02:40:11
mgriffin_
can you create an account @mariadb.org or whatever, and then update your profile once logged in?
2014-08-27 02:42:42
xbskid
Doubtful; I believe JIRA wants email confirmation.
2014-08-27 02:42:52
xbskid
I can give it a shot.
2014-08-27 02:43:12
mgriffin_
or wait until tomorrow as was suggested (better idea)
2014-08-27 02:43:13
xbskid
Nope; even mariadb.org is disallowed.
2014-08-27 02:43:13
seekwill
Just wait till tomorrow :)
2014-08-27 02:43:29
xbskid
Will do
2014-08-27 03:14:35
Ancient
Could I ask someone to verify that I'm not an idiot, before I file a bug about this warning being invalid: http://pastebin.com/HPDypg6H INSERT ... SELECT should be binary log safe when selecting on a primary or unique key, shouldn't it?
2014-08-27 03:16:19
seekwill
Ancient: No, because you're using statement based replication
2014-08-27 03:16:24
seekwill
statement based binary loging
2014-08-27 03:17:03
seekwill
Ancient: The SELECT query can produce different results if the data is not the same
2014-08-27 03:17:30
seekwill
Also, the binary log does not know about primary/unique key
2014-08-27 03:26:24
Ancient
The SELECT can't produce different results because of the unique / primary key condition though, can it? Even if the binlog isn't aware of it, it's still in effect constraining the return of that select.
2014-08-27 03:29:13
seekwill
Ancient: You're assuming that the data is the same
2014-08-27 05:29:23
Ancient
Hmm, I suppose so. Okay.
2014-08-27 08:22:11
serg
xbskid: we use a hosted Jira instance, every now and then Atlassian developers break something there. I've reported a bug about it, usually this is resolved quickly
2014-08-27 08:23:05
serg
xbskid: alternatively, I can temporarily allow any domain you want to register from
2014-08-27 08:41:49
ws2k33
if i compile mariadb by source what is the best source to get an orginal my.cnf ?
2014-08-27 08:42:33
serg
ws2k3: what do you mean?
2014-08-27 08:46:14
ws2k33
of i compile mariadb from source where can i get the default my.cnf
2014-08-27 08:49:40
serg
default my.cnf that is used in debian packages is in debian/additions/my.cnf
2014-08-27 08:49:48
danblack
ws2k33: are you asking where your my.cnf should be placed on the filesystem ?
2014-08-27 08:50:14
serg
default my.cnf that is used in rpm packages is in support-files/rpm/my.cnf
2014-08-27 08:50:42
serg
binary tarballs don't install my.cnf by default
2014-08-27 08:50:56
serg
my.cnf examples (not installed) are in support-files/*.cnf
2014-08-27 08:51:08
ws2k33
thanks you serg
2014-08-27 12:20:56
rigid
is it generally a good idea to use mariadb with a huge table (~100GB ) on a single machine?
2014-08-27 12:21:32
rigid
and if yes, where are the major pitfalls with such sizes?
2014-08-27 12:24:03
spetrunia
rigid: big dataset per se is not the problem. The question is, how many queries will access the table.. and how many rows each query will read.
2014-08-27 12:24:25
spetrunia
innodb_buffer_pool_size is one of the important parameters
2014-08-27 12:25:32
rigid
spetrunia: ah ok, well. i guess the main problem will be the amount of data that has to be read for "WHERE ..." clauses
2014-08-27 12:25:44
rigid
i suppose InnoDB is the prefered engine then, not Aria
2014-08-27 12:25:46
rigid
?
2014-08-27 12:26:11
serg
yes. in fact, it'll be XtraDB
2014-08-27 12:58:33
rigid
ok, thank you
2014-08-27 13:00:22
Jb-boin
you might also consider tokudb as the engine
2014-08-27 13:09:37
rigid
Jb-boin: hm, it's hard to judge without benchmarks or knowing the code of both engines :-/
2014-08-27 13:14:28
erkules
Jb-boin: Do you have exp with tokudb?
2014-08-27 13:14:29
tanj
rigid: Jb-boin's point is that TokuDB uses compression, so it might be better than InnoDB on big datasets. However concurrency is not as good as on InnoDB.
2014-08-27 13:14:41
tanj
Jb-boin has lots of exp with tokudb :)
2014-08-27 13:14:44
erkules
erkules still misses tokudb exp
2014-08-27 13:14:59
erkules
tanj: So Jb-boin should blog about it :)
2014-08-27 13:15:12
tanj
erkules: nah, he's too lazy :)
2014-08-27 13:15:23
erkules
tanj: does he like sweets?
2014-08-27 13:15:48
Jb-boin
well, i dont have that much experience about it but yes it works well (at least) on environment without extensive concurrency
2014-08-27 13:15:56
erkules
erkules would also offer some offensive language
2014-08-27 13:16:03
tanj
tanj votes for that
2014-08-27 13:16:09
erkules
\o/
2014-08-27 13:16:17
tanj
erkules: there have been lots of blogs from our side on TokuDB last month
2014-08-27 13:16:21
erkules
Jb-boin: So what kind of workload you are using it for?
2014-08-27 13:16:22
Jb-boin
erkules, you cant, you signed an agreement to follow the netiquette
2014-08-27 13:16:33
tanj
erkules: Chris Calender
2014-08-27 13:16:43
erkules
tanj: Jb-boin I trust :)
2014-08-27 13:16:49
Jb-boin
mainly monitoring (zabbix) and data warehousing
2014-08-27 13:17:04
erkules
ok thats the kind of workload I assume too
2014-08-27 13:17:44
erkules
erkules also lacks infinidb knowhow
2014-08-27 13:17:53
tanj
erkules: I used it for analytic data (ex. processing of clicks on webpage, blah blah blah)
2014-08-27 13:18:09
erkules
tanj: s/blah/porn/ ?
2014-08-27 13:18:29
Jb-boin
i had some issues on a high concurrency workload but it was on tables with partitions and many performance issues have been fixed since on that kind of use
2014-08-27 13:18:42
tanj
erkules: ahah no it was really for marketing (logging of actions on FB)
2014-08-27 13:18:56
erkules
ok it is called marketing nowadays .. :)
2014-08-27 13:19:15
tanj
erkules: and InfiniDB, really good for all aggregate operations... AVG, SUM, COUNT, etc... but not really good for anything else
2014-08-27 13:19:21
erkules
great to see some tokudb adopion in the wild
2014-08-27 13:19:42
tanj
I am trying to make one of my customers adopt TokuDB
2014-08-27 13:19:49
tanj
they are a gaming shop which also log a lot of actions
2014-08-27 13:20:06
erkules
tanj: So regarding a DWH tokudb is something to look at?
2014-08-27 13:20:17
erkules
tanj++
2014-08-27 13:20:48
tanj
definitely
2014-08-27 13:20:53
rigid
hm, i'm not expecting lots of concurrency. I think i'll try TokuDB. I can alter the engine later anyway, i guess...
2014-08-27 13:21:10
tanj
rigid: yeah, it's painless to go frome on to the other. just alter table engine=...
2014-08-27 13:21:39
erkules
tanj: wasn't there a bug using alter regarding size?
2014-08-27 13:21:56
tanj
not afaik
2014-08-27 13:22:00
erkules
rigid: first define your workload
2014-08-27 13:22:09
rigid
erkules: define?
2014-08-27 13:22:09
erkules
great
2014-08-27 13:22:20
erkules
what kind of workload do you have
2014-08-27 13:22:30
erkules
Do you use i.e. FK
2014-08-27 13:22:44
erkules
afaik tokudb does not support FK
2014-08-27 13:22:55
rigid
one huge table in the 100G range, few queries but queries might need to walk the whole dataset
2014-08-27 13:23:13
rigid
what's FK?
2014-08-27 13:23:23
erkules
ForeignKey
2014-08-27 13:24:02
rigid
no foreign keys in this case
2014-08-27 13:24:23
erkules
ok and do you know how to backup tokudb tables?
2014-08-27 13:24:38
erkules
erkules loves to kill all the fun
2014-08-27 13:25:47
rigid
erkules: not using mysqldump? a backup is not planned :-P
2014-08-27 13:26:05
erkules
Jb-boin: tanj while on a full table scan file io is minimal compared to innodb. what about the cpu usage?
2014-08-27 13:26:17
erkules
rigid: mysqldump 100G? no way
2014-08-27 13:26:27
rigid
yeah, could be a problem...
2014-08-27 13:26:34
erkules
rigid: but if you don't need a backup thats fine
2014-08-27 13:27:04
erkules
rigid: to make it short you got to go back to filesystem snapshots
2014-08-27 13:27:14
rigid
well, it's a raid and currently it's just a test setup. but backup WILL be a problem in the future. maybe a 2nd mariadb machine
2014-08-27 13:27:27
thumbs
LVM snapshots for 100GB will be painful too
2014-08-27 13:27:44
rigid
ah ok, I guess duplicity could handle it then...
2014-08-27 13:27:55
erkules
LVM used to be painfull. modern lvm work like btrfs
2014-08-27 13:28:10
rigid
hm, do I use LVM? hmm
2014-08-27 13:28:34
tanj
thumbs: I did a 2TB LVM snapshot last week...
2014-08-27 13:28:49
rigid
the fs level is not optimized (yet). it's an ext4 raid5 anyway, so... not optimal
2014-08-27 13:28:51
thumbs
tanj: how long did it take?
2014-08-27 13:28:58
tanj
thumbs: 11 hours
2014-08-27 13:29:04
thumbs
tanj: yeah, painful.
2014-08-27 13:29:13
thumbs
:)
2014-08-27 13:29:19
erkules
tanj: what lvm version?
2014-08-27 13:29:21
tanj
thumbs: 100GB is a walk in the park :)
2014-08-27 13:29:48
tanj
erkules: I don't know. does it matter? OS was Debian 7
2014-08-27 13:30:12
tanj
erkules: I also used btrfs+lzo partition to store the snapshot... :)
2014-08-27 13:30:32
thumbs
erkules: fair point, yes.
2014-08-27 13:30:39
erkules
tanj: yes it does. Old lvm snaphot used to be quite expensive as every change was written into the snapshot. So even the snaphot could fail if changeset was to high
2014-08-27 13:30:58
erkules
erkules uses btrfs too :)
2014-08-27 13:30:58
tanj
ah. I thought we had been on LVM2 for years
2014-08-27 13:31:04
thumbs
erkules: I just have bad memories from that too.
2014-08-27 13:31:18
tanj
erkules: got to love this native compression, have you tried btrfs cow snapshots as well?
2014-08-27 13:31:30
erkules
tanj: yes
2014-08-27 13:31:52
erkules
tanj: a lot. I build my lxc/mariadb/galera cluster on that on a daily basis
2014-08-27 13:32:19
erkules
My talk on saturday was using galera/lxc/btrfs-snapshots/ansible \o/
2014-08-27 13:33:03
tanj
erkules: any slides?
2014-08-27 13:33:33
erkules
tanj: no I was on vacation the week before. I just dropped in and did the show
2014-08-27 13:33:54
tanj
erkules: do you use ansible?
2014-08-27 13:33:55
erkules
tanj: there is a recording but it is in german
2014-08-27 13:34:02
tanj
ach so
2014-08-27 13:34:12
erkules
tanj: hehe
2014-08-27 13:34:20
erkules
tanj: ansible and puppet
2014-08-27 13:34:35
erkules
erkules is a certified puppet idiot
2014-08-27 13:34:36
tanj
erkules: I have a great ansible playbook for galera... just installs and bootstrap a three node cluster
2014-08-27 13:34:44
erkules
tanj: me too
2014-08-27 13:34:50
tanj
tanj hates puppets
2014-08-27 13:34:54
erkules
tanj: thats what I was presenting
2014-08-27 13:35:13
tanj
cool stuff
2014-08-27 13:35:17
erkules
yupp
2014-08-27 13:35:18
tanj
mine is based on vagrant though
2014-08-27 13:35:22
erkules
ihh
2014-08-27 13:35:24
tanj
should move it to docker
2014-08-27 13:35:32
erkules
docker and galera?
2014-08-27 13:35:32
tanj
vagrant can use lxc anyway :)
2014-08-27 13:35:45
erkules
how should that work?
2014-08-27 13:35:56
tanj
that's the challenge :)
2014-08-27 13:36:08
erkules
it is
2014-08-27 13:36:44
erkules
at least all the blogpost we had using galera and docker are disgusting
2014-08-27 13:37:00
erkules
Only mine had been good :)
2014-08-27 13:37:32
erkules
but it was about criticizing the others \o/
2014-08-27 14:53:55
hieronemus001
I'm trying to pupetize galera mariadb 10.0. Anyone can help?
2014-08-27 14:54:22
thumbs
hieronemus001: ask erkules !
2014-08-27 14:56:29
tanj
death to puppet !
2014-08-27 14:56:57
tanj
hieronemus001: https://github.com/olindata/olindata-galera-puppet-demo
2014-08-27 14:57:37
tanj
or https://github.com/olindata/puppet-galera
2014-08-27 15:03:06
erkules
hieronemus001: just try not to /msg me
2014-08-27 15:03:11
erkules
s/try//
2014-08-27 15:03:19
erkules
s/to///
2014-08-27 15:04:28
erkules
to make it short I used environments for that. one inital (leader) and the default production
2014-08-27 15:04:36
erkules
the rest is easy
2014-08-27 15:04:56
erkules
tanj: puppet with hiera and an mysql enc rocks
2014-08-27 15:05:27
erkules
configuring machines via insert/update in a database is fun
2014-08-27 15:05:39
erkules
s/databas/mysql
2014-08-27 15:05:42
erkules
;)
2014-08-27 15:06:02
erkules
=> mysqle
2014-08-27 15:06:04
erkules
hmm
2014-08-27 15:38:00
hieronemus001
erkules: this module is tested only for debian
2014-08-27 15:52:09
f0ad
hey all, i have installed maridb 10.0.13 from the official yum repo, i started to load a fairly large dataset, all was going very smooth until....it just froze, it is no longer inserting records
2014-08-27 15:52:30
f0ad
i hooked up strace to it and all i get is:
2014-08-27 15:52:30
f0ad
restart_syscall(<... resuming interupted call ...>
2014-08-27 15:52:31
f0ad
nothing else
2014-08-27 15:53:56
serg
our yum repo? or fedora yum repo? what storage engine? load - how? what syscall?
2014-08-27 15:54:08
f0ad
the yum repo from the mariadb website
2014-08-27 15:54:16
f0ad
centos 6
2014-08-27 15:54:49
f0ad
all tables as innodb/xtradb
2014-08-27 15:56:07
f0ad
asking "show engines innodb status" just sits there, never returns
2014-08-27 15:56:17
f0ad
nothing in any logs that I can see
2014-08-27 15:57:26
serg
do you load with inserts or load data?
2014-08-27 15:57:38
f0ad
inserts, its a mysqldump
2014-08-27 15:57:43
serg
what exactly syscall is interrupted and resumed?
2014-08-27 15:57:46
hieronemus001
erkules:I'm using Hiera as well, do you have any resource related to your setup.
2014-08-27 15:58:05
serg
big multi-row inserts, I presume?
2014-08-27 15:58:23
f0ad
serg: this is the entire strace output:
2014-08-27 15:58:23
f0ad
restart_syscall(<... resuming interrupted call ...>
2014-08-27 15:58:27
f0ad
no other lines at all
2014-08-27 15:58:33
f0ad
and yes, large inserts
2014-08-27 15:59:37
serg
ok, restart_syscall(<... resuming interrupted call ...> is what strace always prints when attaching
2014-08-27 16:00:05
f0ad
ok, well, it never says anything else afterwards
2014-08-27 16:00:13
serg
f0ad: SHOW PROCESSLIST?
2014-08-27 16:02:02
f0ad
serg: 4 things
2014-08-27 16:02:02
f0ad
show engine innodb status (state init)
2014-08-27 16:02:02
f0ad
insert into table (state update)
2014-08-27 16:02:02
f0ad
show engine innodb status (State killed)
2014-08-27 16:02:02
f0ad
show processlist (state init)
2014-08-27 16:07:06
serg
hmm, I don't know. disk full? bad hardware? it couldn't have been a deadlock, as you're doing single-thread inserts
2014-08-27 16:07:12
serg
jplindst ^^^ any ideas?
2014-08-27 16:08:04
f0ad
disk isn't full, system doesn't indicate any failures of any kind, this system is not in use, the only thing running is the restore process
2014-08-27 16:09:19
hieronemus001
erkules: why r u pissed@me? I'm stuck
2014-08-27 16:10:20
f0ad
so far i'm had no luck with migrating to maria
2014-08-27 16:11:38
f0ad
any tips on migrating from mysql 5.0 to maria 10.0.13?
2014-08-27 16:12:29
serg
you could try: 1. use innodb, not xtradb (just in case it's xtradb-specific bug). 2. if you're migrating from mysql, you don't need to reload you data, you can simply copy the datadir
2014-08-27 16:12:47
f0ad
i tried taking a snapshot from the mysql 5.0 system, putting it on the maria system and letting xtradb do crash recovery, but that simply segfaults
2014-08-27 16:13:29
f0ad
it goes through, what seems like, the entire process and then crashes hard
2014-08-27 16:13:36
f0ad
start up again, and it'll repeat
2014-08-27 16:13:42
f0ad
so i gave up on that
2014-08-27 16:13:49
f0ad
and decided to do a dump/restore
2014-08-27 16:13:55
f0ad
but thats failing now in a different way
2014-08-27 16:14:46
ccalender
any clues on the crash? was it corruption-related?
2014-08-27 16:15:33
f0ad
ccalender: quite honestly, i dunno, i don't the logs handy, i didn't run a ST tho
2014-08-27 16:15:58
f0ad
without doing it again, i'm afraid i can't help there
2014-08-27 16:16:01
Jb-boin
f0ad, doing mysql5.0=>5.1=>maria5.5=>maria10 would be better
2014-08-27 16:16:57
f0ad
ugh, i guess i'm trying to do too much at once
2014-08-27 16:17:19
f0ad
i'm also trying to upgrade to barracuda format, and change the innodb page size
2014-08-27 16:17:57
f0ad
guess i'll do this one step at a time
2014-08-27 16:18:39
f0ad
i figured the dump/restore was the easiest in terms of not have leftover cruft from the old system
2014-08-27 16:18:49
f0ad
sort of a fresh start
2014-08-27 16:21:22
ccalender
dump/import is usually great, fwiw
2014-08-27 16:21:46
ccalender
fwiw, I've seen recent large binary upgrades from MySQL 5.1 direct to MariaDB 10.0 work just fine
2014-08-27 16:21:49
Jb-boin
f0ad, dump/import is good its not problematic to skip versions
2014-08-27 16:21:57
Jb-boin
its only if you want to migrate the whole datadir
2014-08-27 16:22:50
f0ad
ok, but the dump/import is what is now frozen, so...
2014-08-27 16:23:06
Jb-boin
and its mainly 5.0=>5.5+ that could be problematic, its better to upgrade 5.0 to 5.1 and run a mysql_upgrade before doing more
2014-08-27 16:23:22
f0ad
well, i can certainly do that
2014-08-27 16:23:29
Jb-boin
and running a mysql_upgrade before could also be of use if the datadir is older and the mysql_upgrade havent been ran
2014-08-27 16:25:48
f0ad
ok, i'll try this:
2014-08-27 16:25:48
f0ad
1) take snapshot, load on sysmte with mysql 5.0
2014-08-27 16:25:48
f0ad
2) start mysql, let crash recovery go
2014-08-27 16:25:48
f0ad
3) run upgrade_mysql
2014-08-27 16:25:48
f0ad
4) stop mysql
2014-08-27 16:25:48
f0ad
5) upgrade from 5.0 to 5.1
2014-08-27 16:25:48
f0ad
6) run upgrade_mysql
2014-08-27 16:25:49
f0ad
7) produce new mysqldump
2014-08-27 16:25:49
f0ad
8) mariadb ... reload ... upgrade_mysql ... yadda yadda
2014-08-27 16:25:51
f0ad
does that sound reasonable?
2014-08-27 16:27:46
Jb-boin
if you do a mysqldump from 5.0 and load it on 10 it will work as well :)
2014-08-27 16:27:59
f0ad
Jb-boin: but thats what i did, and that isn't working
2014-08-27 16:28:35
f0ad
Jb-boin: it got about 90% of the way complete, and then....froze
2014-08-27 16:29:11
f0ad
it's been sitting on one insert for 15+ hours when it normally takes 0-1s to complete
2014-08-27 16:29:35
Jb-boin
well, you dont need to dump/import if you do the upgrade process
2014-08-27 16:30:18
f0ad
i'd rather do the dump as that gives me the oportunity to upgrade innodb file format and page size
2014-08-27 16:30:32
f0ad
but...that clearly isn't working out
2014-08-27 16:31:26
Jb-boin
yes, its better to dump/reimport if possible
2014-08-27 16:31:58
Jb-boin
but i dont think that it would solve the issue as the dump produced from 5.0 and 5.5 might be the same
2014-08-27 16:32:35
Jb-boin
another possilbity is to import the dump on 5.5 as it could be a maria10 bug/limitation that you are hitting then upgrade to maria10
2014-08-27 16:35:15
f0ad
ok, i'll try a few different things
2014-08-27 16:44:13
Jb-boin
f0ad, just to have an idea, how big is the dump file?
2014-08-27 17:11:51
f0ad
Jb-boin: sorry, didn't see, it's 86G gzip compressed
2014-08-27 17:12:14
Jb-boin
ok
2014-08-27 17:52:04
nishttal2
where can I find documentation on how to write storage engines?