#maria Log v0.1

logs Catalogue

This page loads from the db, refresh as and when. It displays 24 hrs
For older data http://marialog.archivist.info/previous.php, logs go back to 2009-01-18

Table logs
date_time
user
message
2010-09-02 07:31:22
knielsen
anyone knows why we have 1G of rsync processes on hasky? (monitoring shows the box is stressed for memory)
2010-09-02 07:33:36
knielsen
looks like it's the remote backup that's not properly set up
2010-09-02 08:05:45
flytox
morning maria !
2010-09-02 08:19:14
hakank
Hi flytox
2010-09-02 08:20:34
bytee
hey hakank
2010-09-02 08:21:03
bytee
hakank: am finalising my turkey visit (re: flight), because i'm going to boston right after... so i might be changing arrival/departure dates soon
2010-09-02 08:22:39
hakank
ok
2010-09-02 08:22:46
hakank
bytee, good to know
2010-09-02 08:24:18
bytee
hakank: so whats on the wiki, is likely to change!
2010-09-02 08:24:29
bytee
hakank: good work on OSX installer; i will try it out now that I have OSX 10.6
2010-09-02 08:25:57
hakank
It should work on 10.5 and 10.6
2010-09-02 08:27:13
hakank
and I still have to script the packaging procedure, but I am learning while doing ;-)
2010-09-02 08:32:43
philip_stoev
spetrunia: ping
2010-09-02 08:33:16
spetrunia
philip_stoev: yes?
2010-09-02 08:34:49
philip_stoev
spetrunia: re: your latest code, do you think it is ready for benchmarking? Do you think it needs benchmarking. In other words, do you suspect a situation where performance may legitimately or illegitimately decrease? Does the code contain any rule-based decisions that may backfire under certain scenarios because they are not cost-based?
2010-09-02 08:35:38
spetrunia
yes, there is a possibility of decrease in performance
2010-09-02 08:36:01
spetrunia
the new strategy is used unconditionally over the old one
2010-09-02 08:36:28
spetrunia
it should make things better in most cases, but there can be cases where it makes things worse
2010-09-02 08:37:22
spetrunia
Igor was doing some benchmarking of this and according to him, in relevant queries there can be ~50% better performance.. he hasn't published the results, though
2010-09-02 08:37:45
spetrunia
philip_stoev: one concern that I have re benchmarking is that we have a known wrong-query-result bug
2010-09-02 08:37:51
philip_stoev
spetrunia: I see. By benchmarking I meant it in the QA sense, that is, looking for performance regressions, not improvements
2010-09-02 08:38:18
philip_stoev
spetrunia: so the question is, would you be interested to examine such performance regressions. Only queries that became significantly slower?
2010-09-02 08:38:37
philip_stoev
spetrunia: ok I can wait for the wrong-result bugs to be fixed first.
2010-09-02 08:38:53
spetrunia
I don't know what's the cause of the bug, but it theoretically could be that in some scenario execution goes totally wrong, so performance results will be invalid
2010-09-02 08:39:39
spetrunia
philip_stoev: re regressions - it would be interesting, I think.
2010-09-02 08:40:39
philip_stoev
spetrunia: ok let me know when you have fixed the more pressing bugs, and I will try to find some performance regressions
2010-09-02 08:40:51
spetrunia
ok
2010-09-02 08:47:30
bytee
hakank: scripting, let me send some docs over
2010-09-02 08:47:42
hakank
ok
2010-09-02 09:16:47
knielsen
knielsen needs cable for new phone, brb
2010-09-02 09:51:55
dreas
montywi: If you have time to have a thought about why my InnoDB tables didn't all properly converted from 5.0 to 5.1 that would be great :) I have to choose today to stick to 5.0 (OurDelta binaries) or upgrade to 5.1 (MariaDB OurDelta binaries)
2010-09-02 10:18:20
montywi
We are here testing our internet, so for some time access to hasky is unreliable
2010-09-02 10:18:42
montywi
dreas: give me a couple of hours to look at this (just woke up)
2010-09-02 10:18:58
dreas
montywi: Good morning! Sure :)
2010-09-02 10:19:14
dreas
montywi: I could also provide you access to the machine if required
2010-09-02 10:19:22
montywi
dreas: in principle it should be trivial to fix and find the issue
2010-09-02 10:20:02
montywi
dreas: is the problem with long table names solved ?
2010-09-02 10:20:34
montywi
If you could in a pastebin give me an exact short bug description of what you want to have tested/fixed it would help a lot
2010-09-02 10:20:54
montywi
(faster than reading this log and less chance that I miss anything important)
2010-09-02 10:21:35
dreas
montywi: No, the problem with the long table names is not solved. It's reported as a bug by knielsen (http://bugs.mysql.com/bug.php?id=56441). I would have to dump/drop the tables before update and reimport them afterwards I guess. Not a very nice solution, but there are not too many tables affected.
2010-09-02 10:22:26
dreas
montywi: The other issues that is more important to me is shown at http://paste-it.net/public/m5996d4/ . Several of my InnoDB tables are marked as "Table upgrade required." and it appears I have to rebuild them to get rid of that warning.
2010-09-02 10:23:17
knielsen
dreas: have you tried asking in #mysql ? There should be plenty people there more familiar with how to upgrade 5.0->5.1 (the process should be the same whether 5.1 is MySQL or MariaDB)
2010-09-02 10:23:37
knielsen
dreas: how to convert .frm from old to new version for innodb tables must be a very well-known issue
2010-09-02 10:23:53
mneptok
montywi: ah, that's what's happening. i am working on getting a reliable rsync to the backup drive here, and the commenction keeps failing.
2010-09-02 10:23:57
mneptok
*connection
2010-09-02 10:24:58
dreas
knielsen: I haven't asked there. Can try of course :) I need to be lucky that someone is helpful though
2010-09-02 10:25:41
bytee
mneptok: still awake!
2010-09-02 10:26:07
knielsen
montywi: do you know if there were .frm version changes from 5.0 to 5.1?
2010-09-02 10:26:09
mneptok
bytee: yeah. backups have not been happening in total, so i'm working on getting that fixed.
2010-09-02 10:26:23
bytee
mneptok: ok, good luck
2010-09-02 10:27:29
mneptok
bytee: with hasky's Internet being unreliable, i'll just reboot for a new kernel and go to bed
2010-09-02 10:28:42
bytee
mneptok: well then good night
2010-09-02 10:28:52
bytee
so far, hasky seems to be OK from here. no timeouts, nothing odd
2010-09-02 10:30:51
mneptok
bah, i'll reboot in the morning
2010-09-02 10:30:55
mneptok
mneptok goes to bed
2010-09-02 10:47:03
dreas
(11:13:51 AM) dreas: Naktibalda: So any idea why I have these problems? Is that a MySQL bug?
2010-09-02 10:47:03
dreas
(11:14:48 AM) Naktibalda: dreas, no idea, sorry
2010-09-02 10:47:13
dreas
From MySQL channel ;) Not very helpful so far :)
2010-09-02 11:06:25
philip_stoev
spetrunia: sorry, one more thing
2010-09-02 11:07:18
philip_stoev
spetrunia: did you see my email "Re: [Maria-developers] DS-MRR improvements patch ready for review" from "Tuesday, August 24, 2010 1:27 PM" . Your patch needs to include some way to destingush that the new optimizations have triggered , beyond eq_ref and 'Using join buffer'.
2010-09-02 11:07:51
philip_stoev
spetrunia: I just got a query that has 200 times performance difference between a patched and a non-patched server, and the EXPLAINs are virtually identical, with just the tables being permutated a bit
2010-09-02 11:37:09
spetrunia
philip_stoev: is it faster or slower?
2010-09-02 11:37:53
spetrunia
philip_stoev: and setting @@optimizer_switch='mrr_sort_keys=on|off' make a difference?
2010-09-02 11:38:20
spetrunia
philip_stoev: generally, I would not expect 200 times performance difference either way. this looks like you're hitting a bug...
2010-09-02 11:39:13
spetrunia
"EXPLAINs are virtually identical, with just the tables being permutated a bit" - well the fact that the server thinks query plans are nearly the same doesn't mean they actually are nearly the same
2010-09-02 11:56:08
montywi
knielsen: no frm changes that require upgrade; You can use .frm from 3.23 and they will work on MariaDB 5.1
2010-09-02 11:56:55
knielsen
montywi: hm, I see. Then I don't understand why dreas got the result that his innodb tables needed upgrade (MySQL manual says this means that .frm version is old)
2010-09-02 11:57:40
montywi
I can look into this after 1 hour; I have a couple of quick calls that I have to make first
2010-09-02 11:58:14
dreas
I'll be around for any tests / access / etc
2010-09-02 11:58:25
montywi
the problem with our internet seems to be our linksys router; I need to get in contact with Sanja or Jani to get this fixed
2010-09-02 12:01:20
knielsen
montywi: how did you determine this?
2010-09-02 12:02:17
knielsen
montywi: not saying it isn't so, but I remember once I visited we tested without the linksys router and still saw the problems, so just want to understand how the router was determined to be the problem?
2010-09-02 12:02:43
knielsen
(we already replaced the linksys once, btw)
2010-09-02 12:33:34
philip_stoev
spetrunia: it is slower. I will check that switch you mention, this is the first time I hear about it.
2010-09-02 12:33:45
philip_stoev
spetrunia: ok will file this as a bug
2010-09-02 12:34:31
philip_stoev
spetrunia: the problem is that the testing framework must have a visible indication as to whether any new 5.3 optimization actually kicked in. It is , I would say, imperative for good testing , automated and otherwise.
2010-09-02 13:38:03
montywi
knielsen: the guy put the cable that went to linxsys and put it into his laptop and tested that connection. according to him there was no packet loss then, but when he connected in the linksys then there was still packet loss to hasky
2010-09-02 13:38:31
montywi
The claim was that if you don't force the linksys port to 10M, then you get this kind of problems
2010-09-02 13:38:55
montywi
dreas: will start to look at your problem after 15 min
2010-09-02 13:39:04
knielsen
montywi: ok, that's the same test we did earlier, so if the test is ok then it sounds conclusive
2010-09-02 13:39:10
dreas
montywi: Awesome! I'll make sure I'm here
2010-09-02 13:39:42
knielsen
montywi: note however that I think there is a switch between the modem and the linksys, right? So it would be the switch nearest to the modem that needs to force to 10Mbit, not the linksys?
2010-09-02 13:40:52
montywi
he did a test this after the switch and said that the switch wasn't a problem
2010-09-02 13:41:10
montywi
he initially thought the switch would be the main problem.
2010-09-02 13:43:53
montywi
knielsen: could you copy the long table names files to hasky:/tmp so that I can take a look at this
2010-09-02 13:45:16
knielsen
montywi: I don't think you need to look at the long table name problem, it is understood. There is a way to reproduce in bug#56441, and bar suggested to me a possible way to fix (I can send you his suggestion if you want)
2010-09-02 13:45:40
knielsen
montywi: the issue that dreas still has is that he has some innodb tables that are marked as needing upgrade, and mysql_upgrade does not do this upgrade
2010-09-02 13:46:30
knielsen
montywi: I don't know if the problem is that the tables are not upgraded, or if the problem is that the tables should not be marked as needing upgrade, or even if there is a problem at all :-(
2010-09-02 13:46:58
hakank
time for a nap. I am up since 3am. See you later!
2010-09-02 13:47:16
hakank
montywi, knielsen, Mikhail is about to send the server(s) now
2010-09-02 13:47:26
hakank
He just asked me for postal address
2010-09-02 13:47:33
montywi
knielsen: Send me the suggestion
2010-09-02 13:47:41
montywi
hakank: you have it in the internal wiki
2010-09-02 13:48:23
montywi
knielsen: the main question is, will there be a fix in MySQL 5.1 soon or should we fix it ourselves?
2010-09-02 13:48:47
montywi
but will look start look at the innodb problem after a few minutes
2010-09-02 13:48:59
hakank
montywi, I got that already :-)
2010-09-02 13:49:11
hakank
I was just giving an update
2010-09-02 13:49:25
knielsen
montywi: yes, and I don't know the answer to that ... (the bug is still "open")
2010-09-02 13:50:24
knielsen
montywi: the log with innodb table errors has expired from the pastebin :-(
2010-09-02 13:52:38
montywi
dreas: does ALTER TABLE BRANDS engine=innodb work for you ?
2010-09-02 13:52:47
montywi
This is basicly identical to repair table
2010-09-02 13:52:57
dreas
montywi: Yes, that fixes the problem
2010-09-02 13:53:29
dreas
knielsen: I should still have that log. You mean with long tables?
2010-09-02 13:53:40
knielsen
montywi: it worked, but that is slow for big tables of course
2010-09-02 13:54:11
knielsen
dreas: no, the long table name problem is understood. I mean the log about the innodb tables that need upgrade, but upgrade says "storage engine does not support repair table" or some such
2010-09-02 13:56:19
dreas
knielsen: New post http://paste-it.net/public/p8bf8de/ .. that info?
2010-09-02 13:57:52
knielsen
dreas: yes, so it's this message: error : Table upgrade required. Please do "REPAIR TABLE `aliases`" or dump/reload to fix
2010-09-02 13:58:00
montywi
knielsen: for innodb, there is no other option than doing ALTER
2010-09-02 13:58:08
montywi
REPAIR would be exactly as slow
2010-09-02 13:58:17
montywi
but of course, mysql_upgrade should do this automaticly
2010-09-02 13:58:24
knielsen
montywi: do you know why the upgrade is required in the first place?
2010-09-02 13:59:18
montywi
the only reason is if there has been a change in collation
2010-09-02 13:59:34
montywi
in which case the index needs to be rebuild
2010-09-02 13:59:40
knielsen
right
2010-09-02 14:00:18
knielsen
dreas: so if that is the issue, the a table rebuild will be necessary
2010-09-02 14:00:38
dreas
Hmm. I don't understand that complete. Change in collation .. since when?
2010-09-02 14:01:41
montywi
between 5.0 and 5.1
2010-09-02 14:02:26
montywi
There was a lot of small changes in the collations between the version and if you happen to use anything in an index that would cause things to be sorted differently, you need to upgrade the table
2010-09-02 14:02:53
dreas
montywi .. ok fair enough. But the upgrade script should handle that right?
2010-09-02 14:03:06
montywi
the idea of mysql_upgrade is basicly just to check if you use a collation that has changed and force an repair or alter table on these just to be safe
2010-09-02 14:03:18
montywi
yes, and that is what I will check
2010-09-02 14:03:21
dreas
:) Clear!
2010-09-02 14:04:01
dreas
We could write a custom script of course. But 2 bugs when upgrading just 1 testing server does scare me a bit. Should I be scared and just wait for a few more months? Or is this somehow really exceptional?
2010-09-02 14:04:27
montywi
but the solutuion for you is still to do an alter table for all tables that are reported to require upgrade; I will today/tomorrow fix the upgrade script to do this automaticly
2010-09-02 14:05:11
dreas
montywi: Yes correct. We can write a script to handle that. And to exclude long tablenames before upgrading (and reimporting it afterwards)
2010-09-02 14:09:14
dreas
montywi: Do you expect to include a fix for the long tablename issue in there as well already or not yet? And do you guys know how long it would take for those fixes to make it in the OurDelta binaries?
2010-09-02 14:15:44
montywi
I plan to add a fix for long table names today/tomorrow
2010-09-02 14:16:18
montywi
I am just fixing a last bug needed for 5.1-release, after which we will do a release. I should be able to get this fix in there too
2010-09-02 14:19:04
montywi
(the table name length fix is trivial and fixing mysql_upgrade (actually mysql_check --check-upgrade) should not be hard either
2010-09-02 14:22:18
dreas
montywi: Awesome! Let me know if I can test it or if you want me to try some commands
2010-09-02 14:23:52
montywi
If you can use bzr to download mariadb-5.1 and compile it, you could test it (this after I have pushed my things)
2010-09-02 14:24:24
dreas
montywi: Shouldn't the ARCHIVE tables somehow also be automatically dealt with actually? I guess the upgrade script could/should handle the dump/reimport ?
2010-09-02 14:24:28
montywi
I just need to clearly see what going on.
2010-09-02 14:24:48
montywi
ALTER TABLE is same as dump/reimport (but faster)
2010-09-02 14:25:07
dreas
montywi: Hmm ok. Ideally that's fixed as well then I guess. That way there are no manual fixes required anymore
2010-09-02 14:25:40
montywi
so my work would be to run alter table instead of repair for engines that doesn't support repair.
2010-09-02 14:40:33
dreas
montywi: ALTER TABLE is not working for ARCHIVE tables
2010-09-02 14:41:22
dreas
http://paste-it.net/public/m07e8e1/
2010-09-02 14:41:28
dreas
I'
2010-09-02 14:41:37
dreas
I'd have to downgrade .. dump it .. upgrade again .. and then import it
2010-09-02 14:42:36
montywi
dreas: your paste only shows repair, not alter table
2010-09-02 14:43:00
montywi
sorry, my mistake
2010-09-02 14:43:42
montywi
I will check why this is the case. The 'alter table' should work.
2010-09-02 14:44:47
knielsen
montywi: http://dev.mysql.com/doc/refman/5.1/en/upgrading-from-previous-series.html , the section a bit down "known issue"
2010-09-02 14:46:48
montywi
looks like brian added an incompatibility between 5.0 and 5.1 :(
2010-09-02 14:49:38
igor
ssspetrunia: ping
2010-09-02 14:50:03
igor
spetrunia :ping
2010-09-02 14:51:25
tonu
!igor: DROP
2010-09-02 14:51:29
montywi
hm, it's not trivial to change mysql_check to use alter table the way it's done (as there is no error number to parse if engine doesn't support repair)
2010-09-02 14:51:43
montywi
and I can't parse the text, as it may be in another language
2010-09-02 14:52:34
knielsen
montywi: maybe its in any case a bit dangerous to have mysql_upgrade suddenly rebuilding an innodb table that could be terabytes in size, in a stable release ?
2010-09-02 14:52:59
knielsen
montywi: mysql_upgrade is run automatically by the .deb packages at every server start ...
2010-09-02 14:53:17
knielsen
(of course it only does anything when it's the first start after an upgrade)
2010-09-02 14:53:36
montywi
I got that and it's not optimal
2010-09-02 14:54:45
knielsen
montywi: what is it that's not optimal? (didn't understand)
2010-09-02 14:54:53
montywi
dreas: sorry, don't see a solution for the archive tables
2010-09-02 14:55:08
montywi
that it's run for each start
2010-09-02 14:55:32
montywi
but I assume there is no easy way to do it otherwise and it's safe because of the status file
2010-09-02 14:55:50
knielsen
montywi: well, that's how .deb MySQL packages worked for a long time I think
2010-09-02 14:55:51
dreas
montywi: I guess the package should simply refuse the upgrade if there are ARCHIVE tables? Or maybe you can somehow add some backwards compatibility which handles the conversion?
2010-09-02 14:56:36
montywi
That would require us to support the old obsolete archive table format in 5.1 which is a lot of work :(
2010-09-02 14:56:57
knielsen
montywi: I think the goal in Debian is to make package installation and upgrade as automatic as possible. And doing it each server start protects against it not completing during install for some reason (and yes, because of the status file the overhead should be minimal)
2010-09-02 14:57:03
montywi
The innodb issue, that I can fix
2010-09-02 14:57:05
knielsen
(but I'm only guessing)
2010-09-02 14:57:16
montywi
knielsen: agree with your guess
2010-09-02 15:01:49
montywi
knielsen: looks it would be trival to add 'repair ... for upgrade' which would for engines that doesn't support repair to instead use alter
2010-09-02 15:21:00
dreas
montywi: Should I ask the OpenQuery guys to include the archive dump/import in their package?
2010-09-02 15:21:40
montywi
dreas: as it's we who do their package, I don't think they can help you with doing automatic upgrade of archive tables
2010-09-02 15:22:15
dreas
montywi: Ah clear :)
2010-09-02 15:22:48
montywi
I found a reasonable good way to fix that mysql_upgrade will also work for innodb tables; An about 10 line fix
2010-09-02 15:23:09
montywi
it will just automaticly convert the repair to alter table
2010-09-02 15:26:12
montywi
will be able to push this tomorrow...
2010-09-02 15:26:28
montywi
dreas: ie, I will fix the long table names and automatic upgrade of innodb tables
2010-09-02 15:27:06
dreas
monywi: Awesome. I'll try a live server after that fix and will report new bugs ;)
2010-09-02 16:06:49
montywi
dreas: can you branch 5.1-release ? I should be able to push the fix there early tomorrow
2010-09-02 16:07:19
dreas
montywi: I'm not sure if I can :) If you can give me the right commands (Debian) I can try on our testing server
2010-09-02 17:00:11
dreas
Or I can provide someone access to test
2010-09-02 17:04:40
montywi
dreas: http://kb.askmonty.org/v/source-getting-the-mariadb-source-code
2010-09-02 17:05:48
montywi
the branch command is 'bzr branch lp:~maria-captains/maria/5.1-release maria-5.1'
2010-09-02 17:10:11
lchaplin
hey monty
2010-09-02 17:11:20
montywi
lchaplin: hej på dig
2010-09-02 18:10:26
dreas
montywi: Is there any chance you could build me a custom Debian package? Would make my life a lot easier. Installing from source would break the logic of this testing machine
2010-09-02 18:14:58
dreas
montywi: Not sure if that's just 1 command for you to build and package
2010-09-02 19:01:38
StylusEater
montywi: how's the sun fix going?
2010-09-02 19:03:57
montywi
StylusEater: I fixed the issue with federated tables
2010-09-02 19:04:24
montywi
however the other problem with func_gconcat is not solved
2010-09-02 19:04:43
montywi
Issue is that I get the error when compiling with -O3 but not with -O2
2010-09-02 19:05:10
montywi
Any chance you can install gdb so that I can find out what is causing the problem?
2010-09-02 19:05:12
StylusEater
montywi: I wonder if that's related to the gcc optimization bug knielsen and I found a while ago
2010-09-02 19:05:22
montywi
sorry, -O0 doesn't show the problem
2010-09-02 19:05:35
StylusEater
gdb
2010-09-02 19:05:43
StylusEater
one moment, let me login to that system
2010-09-02 19:08:00
StylusEater
6.8 ok?
2010-09-02 19:11:05
StylusEater
montywi: installed 6.8
2010-09-02 19:11:16
dreas
Sorry somehow I got kicked out
2010-09-02 19:12:16
dreas
montywi: Any chance you can provide me a Debian package to test? I need to go home now but will be back online from home (don't have IRC setup there but will try to connect here again). Otherwise I'm back tomorrow for sure.
2010-09-02 19:12:35
serg
dreas: just wanted to say that all pushes into lp:maria (and other maria branches) are automatically built by buildbot. You can find the sources, binaries, and debs here: http://knielsen-hq.org/archive/pack/
2010-09-02 19:12:55
StylusEater
buildbot ftw! :-)
2010-09-02 19:13:02
montywi
dreas: in other words, you can get it from the above after my push tomorrow
2010-09-02 19:13:23
dreas
montywi: Is it ok if I wait for that?
2010-09-02 19:16:27
dreas
Hope so :) Ping me in the morning if you want me to run the test before the build and I don't get online later tonight. Thnx for this awesome quick fix!
2010-09-02 19:22:11
montywi
StylusEater: thanks, continuing testing...
2010-09-02 19:32:09
montywi
StylusEater: found something. testing more...
2010-09-02 19:32:19
StylusEater
montywi: :-)
2010-09-02 19:40:34
montywi
StylusEater: bugs fixed. Will push tomorrow (with changes for dreas). Thanks for the help!
2010-09-02 19:40:51
StylusEater
montywi: glad to, thx for the fixes
2010-09-02 19:40:58
montywi
montywi going to make food
2010-09-02 19:55:32
Mchl
hello
2010-09-02 19:55:53
mneptok
ahoyhoy
2010-09-02 20:14:29
StylusEater
montywi: e-mail if you need something else on there... heading out
2010-09-02 20:14:31
StylusEater
later room
2010-09-02 20:39:07
serg
montywi ?
2010-09-02 20:40:47
serg
montywi: when are you going to push your merge and other changes to 5.1 ? They fix failures I see in my 5.2 tree, so I'm waiting for your push.
2010-09-02 21:47:32
serg
montywi ?
2010-09-02 21:59:49
mneptok
bbiab. need to run some errands.
2010-09-02 22:09:36
hakank
back