| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-16 | |||
| 15:05:53 | bauzas | mnaser: we allow one client per rootwrap config | |
| 15:06:07 | bauzas | if two configs, two clients | |
| 15:06:19 | bauzas | and since clients lazily load daemons if needed... | |
| 15:07:29 | bauzas | and I guess https://github.com/openstack/nova/blob/master/nova/utils.py#L123 is your PITA | |
| 15:09:39 | mnaser | bauzas: I’m investigating on how to pull this out and seeing what breaks terribly with multiple clients | |
| 15:13:53 | bauzas | mnaser: could you test something ? what if you have two distinct services running different config files, with each of them differencing by the rootwrap_config option value | |
| 15:14:17 | bauzas | mnaser: in this case, I guess we would automatically create two clients and two daemons | |
| 15:27:24 | openstackgerrit | Sundar Nadathur proposed openstack/nova-specs master: Nova Cyborg interaction specification. https://review.openstack.org/603955 | |
| 15:29:47 | kashyap | mriedem: Hi, I saw a ping fly by last night on 'hpet' and libvirt. I wonder if it's resolved | |
| 15:34:42 | mriedem | https://review.openstack.org/#/c/607989/ | |
| 15:35:05 | mriedem | lots of chatter yesterday about whether or not libvirt exposed hpet capability as a clock source for the host caps | |
| 15:35:09 | mriedem | it doesn't look like it does | |
| 15:36:50 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Do not call linux_net.delete_net_dev on Windows https://review.openstack.org/610916 | |
| 15:36:51 | openstackgerrit | Jan Gutter proposed openstack/os-vif master: Fix random test_unplug_ovs failures https://review.openstack.org/611017 | |
| 15:36:59 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: OVA and StrOpt images as VM templates https://review.openstack.org/609736 | |
| 15:50:25 | mriedem | gerritbot must be dead | |
| 15:50:32 | mriedem | imacdonn: +W on https://review.openstack.org/#/c/608091/ | |
| 15:51:13 | imacdonn | mriedem: OK, thanks .... I need to get a new rev in to fix a typo on in the release note | |
| 15:51:34 | mriedem | i fixed it | |
| 15:51:50 | imacdonn | mriedem: ok, cool, thanks .. just got email from review too | |
| 16:00:31 | kashyap | mriedem: So, from a quick chat w/ the QEMU & libvirt folks -- | |
| 16:01:06 | kashyap | They say: "I'd would not do that" (configuring 'hpet') | |
| 16:01:12 | kashyap | As it's super expensive compared to other timer sources | |
| 16:01:18 | kashyap | And even worse for virtual machines | |
| 16:01:50 | kashyap | "HPET access involves a context switch to QEMU userspace, where as TSC is handled by KVM natively" | |
| 16:02:06 | kashyap | mriedem: But to your original question -- no I don't see it in libvirt's host capabilities either. | |
| 16:04:40 | kashyap | Probably because "no one has asked for it before" | |
| 16:12:29 | kashyap | mriedem: Responded on the review with details. | |
| 16:14:35 | efried | hah, so after all of that, we may wind up not doing this thing at all? | |
| 16:15:35 | cdent | rad | |
| 16:15:59 | mriedem | well it was explicitly disabled in libvirt guests originally for a reason | |
| 16:16:11 | mriedem | b/c apparently at least for windows images it can skew the clock in the guest | |
| 16:16:25 | mriedem | kashyap: thanks for investigating | |
| 16:16:34 | mriedem | artom: +2 on your live migration cleanup thing https://review.openstack.org/#/c/609517/ | |
| 16:16:48 | artom | mriedem, thanks for the thorough reviewing :) | |
| 16:17:17 | artom | dansmith, feel like hitting up ^^ ? | |
| 16:19:44 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add native implementation OVSDB API https://review.openstack.org/482226 | |
| 16:19:54 | kashyap | mriedem: Yeah, also RHEL disables it completely even now | |
| 16:35:17 | dansmith | artom: man that's a lot of derping | |
| 16:35:41 | artom | dansmith, herp | |
| 16:36:12 | kashyap | Hey folks, a random question -- does anyone came across upstream bugs asking for CPU hotplug in Nova? | |
| 16:37:03 | kashyap | Okay, I see a few blueprints, old and new | |
| 16:41:49 | mnaser | before i start diving | |
| 16:42:15 | mnaser | really old environment: juno-era, upgraded all the way up to rocky (no ffus) .. i'm seeing exceptions in placement once i hit rocky (around _create_incomplete_consumers_for_provider) | |
| 16:42:36 | mnaser | wit DBDuplicateEntry exceptions for unique consumer uuid | |
| 16:43:16 | mnaser | i'm guessing that it's trying to create incomplete consumers but they're there, or something. | |
| 16:43:18 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Handle volume API failure in _post_live_migration https://review.openstack.org/611083 | |
| 16:44:01 | mriedem | mnaser: traceback in a paste? | |
| 16:44:41 | tssurya | dansmith: had a question about the cell templating stuff, | |
| 16:44:43 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: Handle volume API failure in _post_live_migration https://review.openstack.org/611084 | |
| 16:44:51 | dansmith | tssurya: yah? | |
| 16:45:00 | mnaser | mriedem: http://paste.openstack.org/show/732260/ | |
| 16:45:02 | tssurya | shouldn't we consider the cell0's transport_url here : https://github.com/openstack/nova/blob/a53e46a75936b55c93face840764a67f2186cb11/nova/objects/cell_mapping.py#L144 ? | |
| 16:45:05 | mnaser | OH also fun little thing i found out about today | |
| 16:45:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Fail to live migration if instance has a NUMA topology https://review.openstack.org/611088 | |
| 16:45:14 | stephenfin | artom: Thoughts? https://review.openstack.org/611088 | |
| 16:45:22 | mnaser | Q=>R upgrades requires you to run api_db sync first then db sync after (i dont think this is documented) | |
| 16:45:26 | tssurya | right now running db sync without local_cell parameter gives out errors | |
| 16:45:43 | mnaser | because cell disabled field is missing from api database so the db sync fails | |
| 16:45:46 | stephenfin | artom: I'd personally like to backport that as far as we can go. I'm kind of sick of explaining how broken this is to people | |
| 16:46:00 | mnaser | api_db sync first adds that field, which then lets db sync do it after | |
| 16:46:03 | mriedem | mnaser: i think it's ordered that way in the upgrade docs | |
| 16:46:11 | melwitt | ||
| 16:46:12 | mnaser | really, let me double check | |
| 16:46:24 | dansmith | tssurya: not sure what you mean.. pastebin an error? | |
| 16:46:25 | mriedem | https://docs.openstack.org/nova/latest/user/upgrade.html#rolling-upgrade-process | |
| 16:46:28 | tssurya | mnaser, mriedem: yea someone ran into the same issue and we changed the order | |
| 16:46:31 | mriedem | "Using the newly installed nova code, run the DB sync. (nova-manage api_db sync; nova-manage db sync). These schema change operations should have minimal or no effect on performance, and should not cause any operations to fail." | |
| 16:46:32 | tssurya | dansmith: ok | |
| 16:46:37 | artom | stephenfin, I don't know the full history, but I feel like it's opening a can of worms | |
| 16:47:00 | mnaser | mriedem: serves me right for looking at the queens docs thinking it hasnt change because it "looks" the same | |
| 16:47:06 | artom | stephenfin, also, I could imagine a scenario where an operator really pinky swears the destination host is fine, and wants to live migrate regardless | |
| 16:47:10 | mnaser | you're right, the order was swapped in rocky, my bad | |
| 16:47:23 | artom | stephenfin, so I'm not sure I'm comfortable with such a heavy handed approach | |
| 16:47:31 | mnaser | but anyways, back to that gigantic traceback | |
| 16:47:32 | mriedem | mnaser: i think grenade was doing it the right way before that docs change, | |
| 16:47:34 | mriedem | our docs were just old | |
| 16:47:37 | tssurya | dansmith: https://pastebin.com/7cQKv0fz | |
| 16:47:41 | artom | stephenfin, totally get where you're coming from though :) | |
| 16:47:44 | stephenfin | artom: Fair point. Wanna stick your thoughts in that review? | |
| 16:48:01 | dansmith | tssurya: oh because cell0's transport_url column can be NULL ? | |
| 16:48:03 | artom | stephenfin, yep, will do | |
| 16:48:15 | stephenfin | ta | |
| 16:48:25 | tssurya | yea | |
| 16:48:34 | dansmith | tssurya: add "and val" here: https://github.com/openstack/nova/blob/a53e46a75936b55c93face840764a67f2186cb11/nova/objects/cell_mapping.py#L162 | |
| 16:48:51 | dansmith | tssurya: you gonna cook up a patch or do you want me to? | |
| 16:49:09 | dansmith | tssurya: I wonder why/how we're not hitting that in the gate? | |
| 16:49:15 | dansmith | do we set it to something bogus? | |
| 16:49:29 | tssurya | dansmith: would be nice if you do it.. | |
| 16:49:33 | dansmith | tssurya: sure | |
| 16:49:46 | tssurya | we were just doing the upgrade checks for rocky and saw this | |
| 16:49:46 | mriedem | mnaser: so this is being triggered when listing resource providers, what is doing that? | |
| 16:50:03 | mnaser | mriedem: i'm assuming nova-scheduler? | |
| 16:50:26 | mnaser | https://github.com/openstack/nova/blob/377921103121bc62a3f7fce60c63e30815406851/nova/api/openstack/placement/objects/resource_provider.py#L1923-L1965 | |
| 16:50:27 | mnaser | this is interesting | |
| 16:50:56 | dansmith | tssurya: hmm, actually, it's not nullable on the object, so I'm not sure how it'd be doing the right thing if you have NULL in the db | |
| 16:51:16 | tssurya | dansmith: but we can have NULL on the conf file | |
| 16:51:38 | tssurya | technically there is no requirement to produce a transport_url for db sync right ? | |
| 16:52:15 | tssurya | s/to produce/to supply | |
| 16:52:16 | dansmith | tssurya: ohh, I see, I thought you were saying it was NULL in the database | |