| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-16 | |||
| 19:44:32 | mriedem | we've used nova-status upgrade check for that though | |
| 19:44:38 | mriedem | as in db sync fails | |
| 19:44:54 | mriedem | db sync in N fails b/c you didn't complete the online migrations in N-1 | |
| 19:45:11 | melwitt | mnaser: thanks. at the very least, we can add more info to the upgrade reno, I think. but it sounds like I messed up the [workarounds] option too much to fix | |
| 19:45:36 | mnaser | melwitt: nah, it's fine, i think the messaging needs to be more clear as in like | |
| 19:45:42 | mnaser | yo shit will be broken if you havent completed the upgrade | |
| 19:45:54 | mnaser | because to me i saw some stuff related to it but it didnt really feel like "that was my issue" | |
| 19:46:10 | dansmith | melwitt: just MHO of course. We've deviated from my initial proposal of workarounds in the past, but I do think that adding another possible "how it is if you didn't change it" case at this point is probably less helpful | |
| 19:46:22 | melwitt | well, I was thinking "if you're doing a rolling upgrade, set [workarounds]enable_consoleauth = True | |
| 19:46:50 | melwitt | mnaser: ^ rather than, yo shit will be broken | |
| 19:46:52 | mnaser | yeah i'd be in favour of enabling it given that it cant really do much | |
| 19:47:04 | mriedem | cdent: i couldn't do a block migration for this request spec thing, so added it to nova-status upgrade check https://review.openstack.org/#/c/581813/ | |
| 19:47:10 | mriedem | *blocker migration | |
| 19:47:14 | mnaser | melwitt: much better delivered. | |
| 19:47:15 | mnaser | :p | |
| 19:47:27 | openstack | Launchpad bug 1798188 in OpenStack Compute (nova) "VNC stops working in rolling upgrade by default" [Undecided,New] | |
| 19:47:27 | mnaser | melwitt: and my sh.... rolling upgrades were affected by https://bugs.launchpad.net/nova/+bug/1798188 | |
| 19:47:32 | melwitt | lol, no I mean it _won't_ be broken if you set it | |
| 19:47:49 | mnaser | oh | |
| 19:47:55 | mnaser | ah my brain has potato'd | |
| 19:48:00 | cdent | mriedem perhaps it would be good/ideal if we can instead of a suite of N blocker migrations we have a sanity check of some kind in a placement status check, which effectively does the same kind of "You're database isn't ready" thing. | |
| 19:48:09 | mnaser | i did ocata => pike => queens => rocky in 2 days | |
| 19:48:21 | mriedem | cdent: that's an optoin | |
| 19:48:22 | mriedem | *option | |
| 19:48:27 | melwitt | dansmith: I didn't understand the "how it is if you didn't change it" are you saying you think changing the default would be less helpful than adding more words to the upgrade reno? | |
| 19:48:29 | mriedem | for dropping create_incomplete_consumers | |
| 19:48:29 | mnaser | this one was by far the toughest but as expected i guess | |
| 19:49:06 | dansmith | melwitt: I'm saying if you change it now, then people reading the renos will see "This was deprecated. Oopps, undeprecated set this workaround, Oops Oops, nevermind, it's set by default now" | |
| 19:49:26 | dansmith | melwitt: and it just seems like we're piling on the confusion if we keep making that a moving target | |
| 19:49:49 | dansmith | melwitt: if anything, add something to nova-status and backport it to help make sure people are warned to pay attention to this | |
| 19:49:57 | dansmith | and get that released before people have a chance to stumble over this | |
| 19:50:56 | melwitt | dansmith: I see, yeah. that's true, if we change the default we have to change all of the words related to how the workaround works, that would be confusing if someone's seen it before | |
| 19:51:26 | mnaser | (but also how many people went through this document already given the issue i ran into today :p) | |
| 19:51:35 | dansmith | melwitt: and I don't think we get to alter the older renos, if I'm not mistaken, but even still it's out there so if someone is looking at X.1 docs and then they're similar but different in X.2... | |
| 19:52:14 | melwitt | argh, yeah. | |
| 19:52:19 | dansmith | mnaser: I'm talking about in a year when most people are deploying rocky and trying to figure out what the story is now | |
| 19:52:37 | mnaser | dansmith: makes sense | |
| 19:53:11 | melwitt | I have cursed nova-consoleauth :( | |
| 19:53:41 | melwitt | ok, so add something to nova-status. I hope everyone uses nova-status | |
| 19:55:20 | dansmith | of course not everyone does.. OSA does I think, and hopefully all the buzz around making this a generic thing will mean in a year people are looking at it | |
| 19:55:35 | dansmith | I thought you were also suggesting clarifying words in renos to help understand | |
| 19:55:56 | dansmith | I was just saying flipping the default behavior now and trying to document _that_ is the confusing part | |
| 19:55:59 | melwitt | yeah but IIUC that doesn't help someone upgrading to rocky if I can't backport those words | |
| 19:56:08 | melwitt | oh | |
| 19:56:12 | dansmith | you can backport the words, | |
| 19:56:33 | dansmith | I just don't think you should change the behavior and backport more words explaining how it's changed for the third time | |
| 19:56:45 | melwitt | got it, ok | |
| 19:57:30 | mriedem | what would the nova-status upgrade check look for? that [workarounds]/enable_consoleauth is False and return a warning? | |
| 19:58:02 | dansmith | yeah, and maybe check the services table or current tokens to see if you even use that stuff | |
| 19:58:09 | dansmith | if you don't use console, then you don't need to warn, | |
| 19:58:26 | dansmith | but if you do and you're rolling, pretty much should have that set right? | |
| 19:58:33 | melwitt | yeah | |
| 19:58:45 | mriedem | you can also tell if there are no console auth entries in the db right? | |
| 19:58:50 | dansmith | I said that | |
| 19:59:08 | mriedem | i said it with an accent | |
| 19:59:19 | dansmith | fancy | |
| 19:59:21 | mriedem | dansmith: btw, https://review.openstack.org/#/c/611094/ needs an assertion on it | |
| 19:59:23 | melwitt | there wouldn't be, before rocky though. they'd be in the nova-consoleauth service | |
| 19:59:31 | mriedem | melwitt: well, that's the point right? | |
| 19:59:45 | mriedem | if you're using nova-consoleauth in queens, and upgrading to rocky, you want the workaround enabled | |
| 20:00:05 | mriedem | and nova-consoleauth would show up in the services table in....one of the dbs | |
| 20:00:05 | dansmith | mriedem: ack, I have to run off for a bit but will hit that when I get back | |
| 20:00:08 | melwitt | yeah, I mean, if you are checking a queens deployment for whether they use consoles at all, you'd have to check the nova-consoleauth service | |
| 20:00:26 | mriedem | we don't want to make an rpc call from the status check | |
| 20:00:32 | mriedem | but we could check the services table to see if it's been started | |
| 20:00:35 | melwitt | oh, you're thinking if they don't use consoles they won't run the service at all. that makes sense too | |
| 20:00:37 | melwitt | yeah | |
| 20:00:51 | mriedem | i just don't know which db that'd be in | |
| 20:00:58 | mriedem | api? | |
| 20:01:00 | mriedem | no, | |
| 20:01:02 | mriedem | wrong schema | |
| 20:01:09 | mriedem | i guess just iterate the cell dbs | |
| 20:01:40 | mriedem | if you find a non-deleted nova-consoleauth service record in that db, but no console auth tokens in the db, and workarounds is false, then fail | |
| 20:02:56 | melwitt | yeah, or warn like dansmith said. only matters if you're rolling | |
| 20:03:33 | melwitt | i.e. it will only mess you up if you're rolling | |
| 20:04:43 | mriedem | i left a comment on the bug with the status ugprade check idea | |
| 20:05:01 | melwitt | thanks | |
| 20:05:29 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: Use tempfile for powervm config drive https://review.openstack.org/610174 | |
| 20:11:27 | mriedem | dansmith: np i got it, it was 1 line | |
| 20:11:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix formatting non-templated cell URLs with no config https://review.openstack.org/611094 | |
| 20:12:27 | mriedem | easy fix for another core ^ | |
| 20:17:22 | melwitt | +W | |
| 20:27:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Ignore uuid if already set in ComputeNode.update_from_virt_driver https://review.openstack.org/611162 | |
| 20:27:15 | mriedem | efried: jroll: ^ | |
| 20:30:07 | openstackgerrit | melanie witt proposed openstack/nova master: Bump os-brick version to 2.6.1 https://review.openstack.org/611109 | |
| 20:30:52 | efried | I don't get it. | |
| 20:32:25 | openstackgerrit | Sundar Nadathur proposed openstack/nova-specs master: Nova Cyborg interaction specification. https://review.openstack.org/603955 | |
| 20:33:58 | spatel | Folks! i have 64G compute node | |
| 20:34:09 | dansmith | spatel: it's not nice to brag | |
| 20:34:14 | dansmith | mriedem: thanks home skillet | |
| 20:34:16 | spatel | Should i go with 1G hugepage or 2M | |
| 20:34:46 | spatel | dansmith: i was going to write question but hit enter middle of i | |
| 20:34:49 | spatel | dansmith: i was going to write question but hit enter middle of it | |
| 20:34:55 | dansmith | spatel: I know, I'm just joking :P | |
| 20:35:00 | spatel | :) | |
| 20:35:49 | spatel | what do you recommend if that is the case | |
| 20:36:08 | spatel | Problem is if i launch application then it will be hard to adjust those value | |
| 20:36:36 | spatel | currently i have "hugepagesz=2M hugepages=27000 transparent_hugepage=never" | |
| 20:36:50 | dansmith | spatel: you probably want cfriesen | |
| 20:37:09 | spatel | cfriesen: ^^ | |