| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-30 | |||
| 10:52:43 | kashyap | (I often see "Related-Bug" on several patches, though | |
| 10:52:44 | kashyap | ) | |
| 10:53:10 | lyarwood | kashyap: I *think* that's okay yeah | |
| 10:53:23 | lyarwood | kashyap: maybe Related-Bug on the first? | |
| 10:53:45 | kashyap | lyarwood: You mean, add the Related-Bug on the casing patch? | |
| 10:54:07 | kashyap | (Which is the "first" patch in this 2-patch series) | |
| 10:54:24 | lyarwood | kashyap: yeah | |
| 10:54:32 | kashyap | lyarwood: Okido; I'll do that. Gracias | |
| 10:55:54 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565043 | |
| 10:55:55 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Lift the restriction of choices for `cpu_model_extra_flags` https://review.openstack.org/563926 | |
| 11:01:59 | kashyap | lyarwood: Done. Mind ACKing this, since you have context too: https://review.openstack.org/#/c/565043/ | |
| 12:24:42 | efried | ō/ | |
| 12:38:14 | bauzas | efried: you can remove your hat | |
| 12:38:26 | efried | bauzas: It's my crew cut | |
| 12:39:27 | bauzas | well, I'm 37 and my hairs become to leave me, so should I do something like °o/ ? | |
| 12:55:10 | zzzeek_ | jaypipes: mysql question. does SELECT * FROM (SELECT x, y FROM a) AS a_alias JOIN b ON a_alias.x=b.y FOR UPDATE , lock both "a", and "b" ? | |
| 12:58:37 | jaypipes | zzzeek_: it will lock all the rows in a and b that join on x = y, yes | |
| 12:58:52 | zzzeek_ | jaypipes: that's great news got a user claims it is not | |
| 12:59:08 | jaypipes | zzzeek_: what version of MySQL? | |
| 12:59:13 | zzzeek_ | jaypipes: hmm | |
| 12:59:31 | zzzeek_ | jaypipes: would have to ask | |
| 12:59:36 | jaypipes | zzzeek_: not that that should matter... I think that is the behaviour for at least since 4.0 | |
| 13:00:16 | zzzeek_ | jaypipes: OK he says "I noticed a different issue when using joinedload and with_for_update in Mysql with a REPEATABLE READ isolation level. In my case, it seems that the with_for_update only locked the joinedtable because the FOR UPDATE statement is only applied to the joinedtable - the root table is not locked because it is queried via the subselect. " but I'm going to ask him to further define how he knows this is the case | |
| 13:00:43 | zzzeek_ | jaypipes: if i had to move the FOR UPDATE around in the query it would really ruin my day :) | |
| 13:01:41 | jaypipes | zzzeek_: ask if both sides of the join condition are indexed. | |
| 13:01:47 | zzzeek_ | jaypipes: ok | |
| 13:01:58 | zzzeek_ | jaypipes: does it only lock indexed rows ? | |
| 13:02:24 | jaypipes | "For index records the search encounters, locks the rows and any associated index entries, the same as if you issued an UPDATE statement for those rows. Other transactions are blocked from updating those rows" | |
| 13:02:30 | jaypipes | zzzeek_: seems so, yes. | |
| 13:03:00 | jaypipes | zzzeek_: though that wording is a tad vague, would you agree? | |
| 13:03:16 | zzzeek_ | jaypipes: yes, what is an "index record"? does that mean the WHERE clause has to refer to an indexed column ? | |
| 13:03:58 | zzzeek_ | jaypipes: seems like it means, for any row that is part of an index. which is every row if you have a primary key | |
| 13:04:47 | jaypipes | zzzeek_: sounds to me like it means that the columns referenced in the JOIN condition (ON clause or WHERE <join table references>) need to be indexed. | |
| 13:05:02 | zzzeek_ | jaypipes: hmmmmm | |
| 13:05:24 | jaypipes | zzzeek_: but it's vague. it almost sounds like it's just saying that reads of both data (clustered index) and index pages (secondary indexes) are locked | |
| 13:05:35 | zzzeek_ | jaypipes: otherwise you think we are getting rows back where not all tables are locked? that seems so unintuitive b.c. you would think it locks everthing thats in the row | |
| 13:05:52 | zzzeek_ | guess we'd have to test it. blah | |
| 13:06:02 | jaypipes | :) | |
| 13:06:07 | zzzeek_ | how you do even test locking on MySQL :) it has such wierd behavior | |
| 13:06:26 | kashyap | zzzeek_: I think jaypipes just tries it on production users ;-) | |
| 13:06:37 | zzzeek_ | well, repeatable read. guess i can try a concurrent update against "a" selecvted alone, vs "a" selected in the subquery | |
| 13:06:56 | kashyap | But more seriously, /me is glad with all these *SQL experts, self can't do beyond 101 stuff | |
| 13:07:16 | zzzeek_ | can't beat a job that pays me to just sit around and know things | |
| 13:07:35 | jaypipes | zzzeek_: for the record, postgresql's select for update docs are virtually identical. | |
| 13:07:47 | zzzeek_ | jaypipes: yes, but w/ pg we dont get shenanigans like this | |
| 13:07:56 | jaypipes | zzzeek_: how do you know? :) | |
| 13:08:18 | zzzeek_ | jaypipes: its postgresql! it's a crystalline structure of timeless perfection! | |
| 13:08:43 | jaypipes | hehe | |
| 13:08:56 | zzzeek_ | when you find an ingenious feature of postgresql, and ask, "when was this developed?", the answer is: never. all PGs features exist outside of time-space | |
| 13:09:12 | jaypipes | lol :) | |
| 13:09:13 | zzzeek_ | like in "Interstellar" | |
| 13:09:52 | jaypipes | it does actually seem like PG has a similar behaviour with regards to indexing. | |
| 13:10:05 | zzzeek_ | jaypipes: meaning, you can't lock a row that doesnt refer to index columns? | |
| 13:10:30 | jaypipes | zzzeek_: "FOR UPDATE and FOR SHARE cannot be used in contexts where returned rows cannot be clearly identified with individual table rows;" | |
| 13:10:37 | jaypipes | zzzeek_: which makes sense, kinda. | |
| 13:10:58 | jaypipes | zzzeek_: if you can't identify a particular record with a condition, then you shouldn't be able to lock it. | |
| 13:11:01 | zzzeek_ | jaypipes: oh. that may be it then, is mysql making a virtual table out of the subquery? | |
| 13:11:27 | jaypipes | zzzeek_: could be? really would need to chat with this guy and find out what data is in the two tables | |
| 13:11:39 | jaypipes | zzzeek_: could be a weird edge case of some sort. | |
| 13:12:01 | zzzeek_ | jaypipes: https://bugs.mysql.com/bug.php?id=34182 | |
| 13:12:16 | zzzeek_ | jaypipes: oh this is falcon storage engine | |
| 13:12:21 | jaypipes | oh ffs | |
| 13:12:30 | zzzeek_ | jaypipes: that bug. let me ask user | |
| 13:15:04 | jaypipes | zzzeek_: note that that bug is describing a different behaviour. that bug is describing when the FOR UPDATE clause appears within the subquery. | |
| 13:15:26 | jaypipes | zzzeek_: and it's also 10 years old :) and the (now irrelevant) falcon storage engine | |
| 13:15:26 | zzzeek_ | jaypipes: ah | |
| 13:15:32 | zzzeek_ | jaypipes: yep | |
| 13:16:42 | jaypipes | zzzeek_: what's the bug link for SA? | |
| 13:17:50 | zzzeek_ | jaypipes: https://bitbucket.org/zzzeek/sqlalchemy/issues/4246/compound-query-select-apply-for-update-to | |
| 13:24:21 | jaypipes | zzzeek_: I wholeheartedly disagree with the Diggsey user in the original 4100 bug. | |
| 13:24:53 | zzzeek_ | jaypipes: :) so did I ! | |
| 13:25:37 | zzzeek_ | jaypipes: this is my life. non DB folks just complaining all day | |
| 13:26:04 | jaypipes | zzzeek_: heh. yeah, but you're used to it by now. | |
| 13:27:57 | openstack | Launchpad bug 1592015 in OpenStack Compute (nova) "libvirt: cleanup of a volume backed instance resize leaves behind the instance directory" [Medium,In progress] - Assigned to Feodor Tersin (ftersin) | |
| 13:27:57 | jmccarthy | Hi folks ! I believe I'm hitting this here .. but not at all sure where to go with it .. https://bugs.launchpad.net/nova/+bug/1592015 | |
| 13:28:12 | jmccarthy | Be nice ;) | |
| 13:29:49 | jmccarthy | In my case, after cold migration, a disk.info is left behind in old instance dir which doesn't get cleaned up (after verify resize) | |
| 13:30:14 | jaypipes | zzzeek_: aha! | |
| 13:30:35 | jaypipes | zzzeek_: I think I might have found a reason why Julien's particular query is doing what it's doing. | |
| 13:30:40 | zzzeek_ | jaypipes: oh yeah | |
| 13:35:02 | jaypipes | zzzeek_: added a comment on the bug above. | |
| 13:35:32 | mriedem | jmccarthy: if you want to take over https://review.openstack.org/#/c/414710/ i could restore it and you could rebase it | |
| 13:35:37 | mriedem | if it fixes the bug | |
| 13:35:47 | zzzeek_ | jaypipes: see that's some mysql guru shit right there | |
| 13:36:17 | mriedem | jmccarthy: although it looks like a series https://review.openstack.org/#/q/status:abandoned+topic:disk_snapshots | |
| 13:36:52 | jaypipes | zzzeek_: I believe postgresql will return an error when it sees an expression like that which it knows it cannot determine an effective join condition that can be applied in the read view protection. | |
| 13:37:10 | openstack | Launchpad bug 1666831 in OpenStack Compute (nova) ocata "Nova recreates instance directory after migration/resize" [Low,Fix committed] - Assigned to Lee Yarwood (lyarwood) | |
| 13:37:10 | mriedem | jmccarthy: comment 8 also says this was fixed with an alternative bug https://bugs.launchpad.net/nova/+bug/1666831 in ocata | |
| 13:37:15 | mriedem | jmccarthy: which release are you on? | |
| 13:37:45 | zzzeek_ | jaypipes: not sure about that. that's a standard SQLAlhcemy query | |
| 13:38:10 | jaypipes | zzzeek_: note that the way InnoDB works, the ORDER BY clause is required to generate the gap locks that would act as the read view protection for other transactions. | |
| 13:38:48 | zzzeek_ | jaypipes: ok....well if true this means we'd still need to change the query, but im doing some local testing and not sure if im replicating yet | |
| 13:39:01 | jaypipes | zzzeek_: we should change channel, though :) mriedem will surely be getting antsy :) | |
| 13:39:08 | zzzeek_ | jaypipes: sure. | |
| 13:41:37 | jmccarthy | mriedem: I'm using kolla images built off master | |
| 13:41:49 | zzzeek_ | jaypipes: the test script I'm running so far is confirming what he is seeing | |
| 13:42:44 | mriedem | jmccarthy: then that sounds like a regression and would be a new bug where you'd have to provide your details, and probably refer to that other bug for context | |
| 13:43:30 | jmccarthy | mriedem: Ok, I can look into raising a new bug, I'll check out that alternate - thanks ! | |
| 13:44:39 | jaypipes | zzzeek_: moved to #sqlalchemy | |
| 13:45:22 | jmccarthy | Ohh ok missed the # in comment 8 | |