Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-06-08
07:27:38 opendevreview Yongli He proposed openstack/nova master: smartnic support - reject server move and suspend https://review.opendev.org/c/openstack/nova/+/779913
07:27:45 opendevreview Yongli He proposed openstack/nova master: smartnic support - functional tests https://review.opendev.org/c/openstack/nova/+/780147
09:04:19 slaweq hi nova team
09:04:34 slaweq in one of neutron jobs I have errors in nova-compute, like https://df08b3f27c09a85d3fe8-f65209ceca8d59b8b2f1e727ebf5e4dd.ssl.cf2.rackcdn.com/periodic/opendev.org/openstack/neutron/master/devstack-tobiko-faults-centos/d402aac/compute1/logs/screen-n-cpu.txt
09:04:47 slaweq do You maybe know what can be the reason of such issue?
09:04:55 slaweq I have virt_type=qemu set there
09:05:01 slaweq and cpu_mode=None
09:31:00 lyarwood stephenfin: morning, if you get a chance today can you hit https://review.opendev.org/c/openstack/nova/+/794639 please?
09:31:40 lyarwood https://review.opendev.org/q/topic:%2522bug/1905701%2522+status:open+branch:master is also ready for review if anyone has time
10:00:57 kashyap gibi: Hey, today's the upstream call, yeah? I'd like to bring this (fairly mechanical thing) up today - https://blueprints.launchpad.net/nova/+spec/add-bochs-display-device
10:02:06 kashyap gibi: WIP here: https://review.opendev.org/q/topic:%22bp%252Fadd-bochs-display-device%22+(status:open%20OR%20status:merged)
10:27:45 sean-k-mooney slaweq: it looks like that job is trying to enable sev es to be availe in the guest without kvm
10:28:10 sean-k-mooney libvirt.libvirtError: internal error: unknown feature amd-sev-es
10:29:07 slaweq sean-k-mooney yeah, but there is nothing like that configured in the job
10:29:18 slaweq and that's actually weird for me :)
10:29:57 slaweq sean-k-mooney I think we may be hitting https://bugzilla.redhat.com/show_bug.cgi?id=1961562 there
10:29:58 opendevmeet bugzilla.redhat.com bug 1961562 in libvirt "vm can not start with error as "internal error: unknown feature amd-sev-es"" [Urgent,Verified] - Assigned to phrdina
10:30:14 sean-k-mooney slaweq: ya its not a nova bug
10:30:21 slaweq but would be great if You could confirm that :)
10:30:37 sean-k-mooney we are not requesting it in the xml so this a libvirt /qemu issue on the amd host
10:31:01 sean-k-mooney ah ok
10:32:53 sean-k-mooney so that looks like the exact same issue
10:32:55 sean-k-mooney Error from libvirt when retrieving domain capabilities for arch i686 / virt_type qemu / machine_type q35: [Error Code 1]: internal error: unknown feature amd-sev-es {{(pid=75619) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:982}}
10:33:40 sean-k-mooney slaweq: as a workaround you could likely define a cpu model that did not support sev-es
10:34:22 slaweq sean-k-mooney thx, I will try that
10:35:03 sean-k-mooney cpu_mode=custom cpu_model=EPYC-IBPB
10:35:12 sean-k-mooney in the libvirt section
10:35:48 sean-k-mooney cpu_mode=none in the libvirt driver on x86 is equvalent to cpu_mode=host-model
10:36:52 sean-k-mooney although it is actully generating <cpu match="exact"> with qemu instead of kvm
10:37:15 sean-k-mooney which i tought was the same as host-passthough not host-model since there is no model specified
10:37:53 slaweq ok, so setting cpu_mode=custom cpu_model=EPYC-IBPB in the job should helps to workaround that issue, correct?
10:41:05 sean-k-mooney yes i think so but only if we know that all the host support that
10:41:18 sean-k-mooney if we dont know that we have other options
10:42:29 sean-k-mooney we could go with cpu_model=kvm64 or proably Nehalem-IBRS
10:43:07 slaweq sean-k-mooney that job is running in the u/s ci so I can't be sure that all providers will support that EPYC-IBPB
10:43:10 sean-k-mooney cpu_model=Nehalem-IBRS while an intel plathform is very clouse to the new x86 featrel level that is being used to compile rhel 9 with
10:44:09 slaweq and also tests in that job don't need any advanced cpu features so I think that I would try with as much "generic" type as possible
10:44:11 sean-k-mooney slaweq: i would try cpu_model=Nehalem-IBRS then and if that dows not work try falling back to qemu64
10:44:29 slaweq sean-k-mooney ok, thx a lot
10:44:32 slaweq I will try that
10:44:46 kashyap slaweq: The EPYC-IBPB model will only work if ... obviously your host supports it
10:44:56 kashyap You can check it via `virsh cpu-models x86_64`
10:45:21 kashyap ("it" == supported models by your libvirt+QEMU)
10:45:47 sean-k-mooney kashyap: this is running on upshift which is hosted by vexhost using i would guess there new amd eypc zen2 servers
10:45:47 slaweq kashyap I actually can't check it like that in the zuul ci jobs
10:47:29 sean-k-mooney assuming they are all the same then they are second gen epyc cpus from 2 years ago but nehalem is ~8-11 years old so it should be emulatable on anything that is still in production today
10:47:43 kashyap I see
10:53:50 stephenfin lyarwood: took me a bit but +2 on https://review.opendev.org/c/openstack/nova/+/794639 now. sean-k-mooney might be interested if they've time
10:55:47 sean-k-mooney i can re review it now
10:56:50 stephenfin lyarwood: also, can you send an email to openstack-discuss RE: https://review.opendev.org/c/openstack/nova/+/794252. They're likely totally useless but we've always done it historically
10:57:04 sean-k-mooney stephenfin: well
10:57:12 sean-k-mooney for PCI passthrough attach, from '_attach_direct_passthrough_ports': not relevant since these clearly can't be virtio devices
10:57:17 sean-k-mooney they can
10:57:28 sean-k-mooney oh _attach_direct_passthrough_ports
10:57:50 stephenfin Really? That doesn't make much sense to me
10:57:58 sean-k-mooney i was going to say macvtap and vdpa sriov interface will be virtio
10:58:03 stephenfin Other types of "passthrough", sure
10:58:04 stephenfin Yeah
10:58:18 sean-k-mooney if that is just direct then ya
10:58:46 stephenfin Would be good to double check. I think it is but I might have missed something
10:59:27 sean-k-mooney im checking now but it shoudl be fine
10:59:45 stephenfin in other news, /me has a briar thorn stuck in his fingertip that he can't remove which is making typing extremely uncomfortable today :-D
10:59:53 sean-k-mooney attach_interface is the entrypoint that would handel all sriov attach anyway
11:00:14 sean-k-mooney ouch
11:01:13 sean-k-mooney stephenfin: so right now it should be ok https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4821-L4840
11:01:43 sean-k-mooney i have worked with prototype nics that expsoed virtio-net-pci devices as VFsd though in the past
11:08:23 sean-k-mooney with the caveat the i woudl silll like the sev atibetes fo the nices to be handeled in vif.py eventualy im +1 on that change
11:13:08 opendevreview Stephen Finucane proposed openstack/nova master: docs: Replace 'nova boot' with 'openstack server create' https://review.opendev.org/c/openstack/nova/+/794007
11:13:14 gibi kashyap: ack, I've added to the agenda
11:13:59 stephenfin gibi: lyarwood: if you've time this week, could I get your eyes on this follow-up series for the RPC 6.0 change from last cycle. bauzas is +2 on pretty much all of it now https://review.opendev.org/q/topic:"compute_rpc_6.0"+status:open
11:20:13 sean-k-mooney stephenfin: is that removing the proxy?
11:20:35 stephenfin no, just removing references to instance_type (in favour of flavour)
11:20:51 sean-k-mooney ah ok
12:10:21 gibi stephenfin: queued up
14:47:53 sean-k-mooney lyarwood: so looking at the patch to not destroy secrets on hard reboot i still dont see a test asserting that wehn we do hard reboot that we then pass destroy_secrets=False
14:47:59 sean-k-mooney you added https://review.opendev.org/c/openstack/nova/+/793463/4..5/nova/tests/unit/virt/libvirt/test_driver.py#19352
14:48:04 sean-k-mooney in the most recent version
14:48:12 sean-k-mooney but that is calling clean up directly
14:48:46 sean-k-mooney i was asking you to call hard reboot and assert it called cleanup with destroy_secrets=False
15:00:22 sean-k-mooney when is the nova team meeting is it now or in an hour?
15:02:06 sean-k-mooney lyarwood: never mind it is caugt by nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_hard_reboot
15:02:21 gibi sean-k-mooney: https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
15:02:26 gibi sean-k-mooney: 16:00 UTC
15:02:33 gibi sean-k-mooney: wich is 1 hour from now
15:02:45 sean-k-mooney yep i need to add that to my calandar
15:03:26 sean-k-mooney thanks
15:05:06 sean-k-mooney got the ical form here https://meetings.opendev.org/#Nova_Team_Meeting
15:06:52 sean-k-mooney stephenfin: https://review.opendev.org/c/openstack/nova/+/793463 contains a virt driver api change im not sure we are allwowed to backport that
15:07:03 sean-k-mooney stephenfin: at least if we wanted too we would have to mofify the patch
15:07:19 sean-k-mooney so that if the virt driver did not supprot it we would invoke the api without the new parmater
15:07:50 sean-k-mooney sorry https://review.opendev.org/c/openstack/nova/+/794252/1
15:10:31 stephenfin yeah, we need to discuss that. On one hand, it's breaking a stable API. On the other hand, we've clearly said the API isn't actually stable :)
15:11:12 sean-k-mooney well we basically said out of tree driver are not supproted
15:11:27 sean-k-mooney ane the patch fixes the in tree ones.
15:11:41 sean-k-mooney but we know out of tree driver are used by vmware for one
15:13:46 sean-k-mooney stephenfin: i guess if thes apis are only called in the virt driver it would be ok
15:14:19 sean-k-mooney it would only be an issue if destroy and cleanup are called in the compute manager
15:15:10 sean-k-mooney we would be overloading on the number of keyword arguments
15:15:21 sean-k-mooney i cant remember if python allows that now or not
15:16:00 kashyap gibi: Afraid, I might not be able to stay for the full meeting - if it lasts longer than 15 min, as I need to run an errand around 18:30 CET...

Earlier   Later