Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-09-18
15:30:10 mriedem so like the day of RC1
15:30:12 mriedem got it
15:30:13 kashyap (Due to conflicts, etc.)
15:30:38 kashyap mriedem: :D I mentioned it exactly for that -- in case it doesn't merge, that typo shouldn't fall through the cracks
15:30:49 kashyap (And you've helpfully fixed it, by the tag)
15:30:55 mriedem i sure hope everyone rechecking the vpmem and pcpu series is actually looking at every failure to make sure we're not recheck grinding in some regression
15:31:33 stephenfin I am. We're seeing mirror issues, InnoDB issues, and something with devstack_gate and null bytes
15:31:38 mriedem kashyap: i don't see you or aspiers as +1 on that bug fix
15:31:40 stephenfin None of which I know how to fix :(
15:32:00 kashyap mriedem: Damn, I have it sit in a tab, let me do that
15:36:15 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: Add a placement audit command https://review.opendev.org/670112
15:41:29 shilpasd dansmith: i have addressed your minor nit isolated aggregate spec on https://review.opendev.org/#/c/675384/
15:44:44 mriedem i guess that wasn't related to aspiers so ignore the ping
15:47:22 dansmith shilpasd: you only changed it in that one place
15:47:36 dansmith shilpasd: it should be changed elsewhere too, except maybe in work items..
15:47:49 shilpasd dansmith: okay, will do that
15:53:27 mloza Hello, I'm getting tons of 'Unexpected exception in API method: MigrationNotFound:' in nova-api. Here is the full log http://paste.openstack.org/show/777427/
15:54:35 mloza I don't see pending migrations in `nova migration-list`
15:54:59 mloza I'm using stable/stein branch
15:55:40 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: Get the CPU model, not 'arch' from get_capabilities() https://review.opendev.org/682267
15:58:24 openstackgerrit sean mooney proposed openstack/nova master: make config drives sticky bug 1835822 https://review.opendev.org/669738
15:58:25 openstack bug 1835822 in OpenStack Compute (nova) "vms loose acess to config drive with CONF.force_config_drive=True after hard reboot" [Medium,In progress] https://launchpad.net/bugs/1835822 - Assigned to Matt Riedemann (mriedem)
15:58:36 sean-k-mooney rebase to kick it down the queue
15:58:39 kashyap mriedem: Thanks for updating
16:00:19 openstackgerrit Merged openstack/nova master: Remove an unused file and a related description https://review.opendev.org/681955
16:01:59 openstackgerrit Shilpa Devharakar proposed openstack/nova-specs master: Update spec: filtering of alloc candidates by forbidden aggregates https://review.opendev.org/675384
16:02:09 openstackgerrit Artom Lifshitz proposed openstack/nova master: Add func test for 'required' PCI NUMA policy https://review.opendev.org/682941
16:02:45 artom sean-k-mooney, stephenfin ^^ Dunno if it's that useful given that we test the 'legacy' policy which essentially does the same, but since I noticed it was missing while doing downstream reviews...
16:03:24 sean-k-mooney the legacy policy works differently if you have no device numa info on the host
16:03:44 mriedem artom: is that from you investigating that numa live migration testing thread in the ML?
16:04:06 artom mriedem, nope, from reviewing downstream backports
16:04:19 artom mriedem, he still hasn't posted logs from that ML post, btw
16:04:30 artom Makes it kinda hard to RCA :)
16:04:31 sean-k-mooney mriedem: its from https://review.opendev.org/#/c/674072/ we have a test backport for the customer to try to see if that actully works for them
16:05:15 sean-k-mooney i was wrigitn the spec for this but im fixing the config drive test currently
16:06:55 sean-k-mooney mriedem: we need to backport https://review.opendev.org/#/c/624444/ first and artom noticed that require did not have test coverage
16:07:23 artom Not in func tests, anyways
16:07:40 sean-k-mooney right there are unit tests
16:07:42 efried dansmith: The isolated aggs spec update lgtm https://review.opendev.org/#/c/675384/ want me to fast approve?
16:07:50 mriedem unit tests are generally insufficient for any of these kinds of features
16:08:02 mriedem at this point in nova, unit tests are generally insufficient for just about anything
16:08:02 artom mriedem, the same could be said for func tests ;)
16:08:18 mriedem artom: for low level hardware stuff sure
16:08:21 mriedem i wouldn't see that in general
16:08:34 artom mriedem, I mean low level hardware stuff, like PCI or NUMA
16:09:05 mriedem so the broken feature that landed in queens is just getting rolled out to the customer that wanted it now?
16:09:19 mriedem 18 months later...right on time
16:09:26 sean-k-mooney yes
16:09:32 mriedem and queens is your current LTS?
16:09:51 artom Long live the queen, indeed
16:09:59 sean-k-mooney newton will be end of line in decmeber and queens is our next LTS release downstream
16:10:17 sean-k-mooney so people are upgrading
16:11:13 mnaser this maybe more of a libvirt question, but i'll ask.. `cpu_mode` defaults to `host-model` in nova when using kvm, but the libvirt definition seems to be `<cpu mode='custom' match='exact' check='full'> ... </cpu>` (this is what i gather from virsh dumpxml)
16:11:29 mriedem mnaser: meet kashyap
16:11:38 mriedem <3
16:11:48 mnaser reading the nova code, it seems that nova is doing the right thing(tm)
16:12:16 sean-k-mooney mnaser: libvirt is also ment to default to host model if we say nothing for what its worth
16:12:33 mnaser right, im just checkin an issue where nested virt is available, virsh capabilities shows vmx
16:12:36 mnaser but vms dont spawn with it
16:12:39 sean-k-mooney but i dont think it always did which is why nova does.
16:12:57 efried I'm going to be taking most of the rest of the week off (family in town). Will check in periodically. Email if you need to reach me. o/
16:13:01 mnaser so i can add it to the list of extra flags (or try and solve it upstream :])
16:13:19 sean-k-mooney do you have a warning about svm not abliable in teh qemu instance log
16:13:31 mriedem mnaser: there are a bunch of recently updated docs here as well https://docs.openstack.org/nova/stein/admin/configuration/hypervisor-kvm.html#specify-the-cpu-model-of-kvm-guests
16:13:33 mriedem if that helps
16:13:52 mnaser ya i went over those, i can totally do passthrough or the other options, but right now my combination should just work ideally
16:14:10 mnaser man it would be nice if we split otu some of the libvirt driver code
16:14:13 mnaser so github search indexes it
16:14:13 mnaser :p
16:14:16 sean-k-mooney qemu does not always detect the host model correctly and somethime expose amd version of vmx (svm) if you use host model
16:14:30 sean-k-mooney and nested virt is enabled
16:14:50 mnaser yeah but the libvirt definition i see here clearly says use a custom set of stuff forbidding callback too
16:15:13 mnaser with a bunch of "requires" too
16:15:30 sean-k-mooney can you paste the full xml somewhere
16:15:32 mnaser i dont know if virsh dumpxml shows the thing that virsh determines or the thing that it was fed
16:15:48 sean-k-mooney i dont think we should be listing a lot of features in the xml
16:15:52 mnaser http://paste.openstack.org/show/777429/
16:16:35 sean-k-mooney do you have the extra cpu flags config set out of interest
16:16:48 mnaser https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10171-L10174
16:16:50 mnaser nope none are set
16:16:51 mnaser but that..
16:16:52 mriedem stephenfin: https://review.opendev.org/#/c/681938/ now depends on your fix so once that passes i'll review your fix
16:17:05 mnaser that doesnt look right
16:17:29 stephenfin mriedem: You saw https://review.opendev.org/#/c/682111/, I assume?
16:17:42 mriedem i didn't dig into it
16:17:42 mnaser https://github.com/openstack/nova/blob/master/nova/virt/libvirt/utils.py#L533-L539
16:17:46 mnaser so tohat gives it back qemu64
16:17:57 sean-k-mooney mnaser: can you set cpu-mode="host-model" explcitly?
16:18:02 mnaser then we start with https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10158-L10166
16:18:03 mriedem stephenfin: artom said i can't trust functional tests for numa stuff so...
16:18:11 mnaser sean-k-mooney: yeah i think i might try that next
16:18:22 mnaser i think the config docs aren't up to date in that case
16:18:28 mnaser unset/none != host-model
16:18:34 sean-k-mooney mnaser: that code path is ment to be equavlent but i think its actullly not
16:18:42 stephenfin it's not NUMA stuff
16:18:45 mriedem HA
16:18:46 stephenfin https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_ebb/682111/1/check/nova-tox-functional/ebbc041/testr_results.html.gz
16:19:04 stephenfin Also the reason that patch is WIP (the test doesn't belong in that file)
16:19:11 sean-k-mooney mnaser: we would bail out here normally https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L10152-L10156
16:19:47 mnaser except if its not set, which the config docs say: If ``virt_type="kvm|qemu"``, it will default to ``host-model``, otherwise it will default to ``none``.

Earlier   Later